Object detection device

Abstract

To appropriately detect an object having a small number of measurement points.SOLUTION: An object detection device 50 includes a separation unit 113 that separates point cloud data into moving point cloud data and still point cloud data other than the moving point cloud data; a calculation unit 114 that, for each measurement point constituting the separated still point cloud data, calculates a required multiple necessary for the overlay process on data of each measurement point based on a distance from a moving body to each measurement point and adds the required multiple to the data; a memory unit 12 that stores, for each frame, the still point cloud data composed of the data of each measurement point to which the required multiple has been added; and a data processing unit 115 that, when the still point cloud data of a new frame is separated and the calculation unit 114 adds the required multiple to the data of each measurement point, overlays, based on the required multiple, the data of the corresponding measurement point from the still point cloud data of a past frame stored in the memory unit 12 onto the still point cloud data of the new frame.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an object detection device that detects objects around a vehicle. [Background technology]

[0002] As this type of device, a technique is known in which an object ahead of a vehicle is detected by using point cloud data indicating three-dimensional positions acquired by a lidar (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-57399 Summary of the Invention [Problem to be solved by the invention]

[0004] In general, it is difficult to detect distant objects measured by a lidar because the number of points measured (the number of measurement points) is small. Detecting objects around a vehicle enables smooth vehicle movement, improving traffic convenience and safety, and thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0005] An object detection device according to one aspect of the invention includes a detector mounted on a moving body, which irradiates electromagnetic waves into a three-dimensional space around the moving body and detects the external environment around the moving body based on the reflected waves; a point cloud data acquisition unit which acquires point cloud data for each frame from the detector, the point cloud data including three-dimensional position information of measurement points on the surface of the object from which the reflected waves are obtained; a separation unit which separates the point cloud data acquired by the point cloud data acquisition unit into moving point cloud data corresponding to the measurement points on the surface of the moving object and static point cloud data other than the moving point cloud data; and a data acquisition unit which, for each measurement point constituting the static point cloud data separated by the separation unit, calculates data for each measurement point based on the distance from the moving body to each measurement point. a storage unit that stores, for each frame, still point cloud data consisting of data of each measurement point to which the required multiple has been added; a data processing unit that, when the still point cloud data of a new frame is separated by the separation unit and the required multiple is added to the data of each measurement point, superimposes data of the corresponding measurement point from the still point cloud data of a past frame stored in the storage unit onto the still point cloud data of the new frame based on the required multiple; and a detection unit that detects a still object based on the still point cloud data after superimposition by the data processing unit. [Effects of the Invention]

[0006] According to the present invention, it is possible to appropriately detect a stationary object having a small number of measurement points, for example, because it is located far away. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a main part of a vehicle control device including an object detection device. [Figure 2] FIG. 2 is a schematic diagram illustrating the light emitted by the lidar to the space ahead of the host vehicle. [Figure 3A] Schematic diagram illustrating the number of measurement points scanned and irradiated on an object located 50 m from the vehicle. [Figure 3B] Schematic diagram illustrating the number of measurement points scanned and irradiated on an object located 100 m away from the vehicle. [Figure 3C]Schematic diagram illustrating the number of measurement points scanned and irradiated on an object located 150 m from the vehicle. [Figure 4A] FIG. 10 is a schematic diagram showing measurement points in space corresponding to an object in the first frame most recently acquired. [Figure 4B] FIG. 4 is a schematic diagram showing measurement points in space corresponding to an object in a second frame acquired as the frame immediately before the first frame. [Figure 4C] FIG. 10 is a schematic diagram showing measurement points in space corresponding to an object in a third frame acquired as the frame two frames before the first frame. [Figure 5] FIG. 10 is a schematic diagram showing measurement points in a space corresponding to an object in the first frame after superposition. [Figure 6A] FIG. 10 is a diagram showing an example of the number of measurement points per distance calculated based on the size of an object and the illumination angular resolution of a lidar. [Figure 6B] FIG. 6B is a diagram showing an example of a required multiple for each distance calculated based on the number of measurement points in FIG. 6A. [Figure 6C] A diagram illustrating the distance range for each required multiple, created based on Figure 6B. [Figure 7A] FIG. 10 is a diagram illustrating still point cloud data of a second frame stored in a storage unit as past data. [Figure 7B] FIG. 10 is a diagram illustrating still point cloud data of a third frame stored in a storage unit as past data. [Figure 7C] FIG. 10 is a diagram illustrating still point cloud data of an eighth frame stored in a storage unit as past data. [Figure 8A] 10 is a flowchart showing an example of an object detection process executed by a calculation unit based on a program. [Figure 8B] 10 is a flowchart showing an example of an object detection process executed by a calculation unit based on a program. DETAILED DESCRIPTION OF THE INVENTION

[0008] The object detection device according to the embodiment of the present invention can be applied to a vehicle having an automatic driving function, i.e., an automatic driving vehicle. Note that the vehicle to which the object detection device according to the embodiment is applied may be referred to as the subject vehicle to distinguish it from other vehicles. The host vehicle may be an engine vehicle that has an internal combustion engine (engine) as a driving source, an electric vehicle that has a traction motor as a driving source, or a hybrid vehicle that has an engine and a traction motor as driving sources. The host vehicle can run not only in an automatic driving mode that does not require driving operation by the driver, but also in a manual driving mode that is operated by the driver.

[0009] When a self-driving vehicle is driving in self-driving mode (hereinafter referred to as self-driving or autonomous driving), it recognizes the external environment around the vehicle based on detection data from on-board detectors such as cameras and LiDAR (Light Detection and Ranging).Based on the recognition results, the self-driving vehicle generates a driving trajectory (target trajectory) for a predetermined time from the current time point, and controls the driving actuators so that the vehicle drives along the target trajectory.

[0010] <Vehicle control device> 1 is a block diagram illustrating the configuration of a main part of a vehicle control device 100 that is mounted on a host vehicle and includes an object detection device. The vehicle control device 100 has a controller 10, a communication unit 1, a positioning unit 2, an internal sensor group 3, a camera 4, a LIDAR 5, and a driving actuator AC. The vehicle control device 100 also has an object detection device 50 that forms part of the vehicle control device 100. The object detection device 50 detects objects around the vehicle based on detection data from the LIDAR 5.

[0011] The communication unit 1 communicates with various servers (not shown) via networks including wireless communication networks such as the Internet and mobile phone networks, and acquires map information, driving history information, traffic information, and the like from the servers periodically or at any timing. Networks include not only public wireless communication networks but also closed communication networks established for each predetermined management area, such as wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like. The acquired map information is output to the storage unit 12, and the map information stored in the storage unit 12 is updated.

[0012] The positioning unit (GNSS unit) 2 has a positioning sensor that receives positioning signals transmitted from positioning satellites. The positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 2 measures the current position (latitude, longitude, altitude) of the vehicle using the positioning information received by the positioning sensor.

[0013] The internal sensor group 3 is a collective term for a plurality of sensors (internal sensors) that detect the driving state of the host vehicle. For example, the internal sensor group 3 includes a vehicle speed sensor that detects the vehicle speed of the host vehicle, an acceleration sensor that detects the longitudinal acceleration and the lateral acceleration (lateral acceleration) of the host vehicle, a rotation speed sensor that detects the rotation speed of the driving source, a yaw rate sensor that detects the rotation angular velocity around the vertical axis of the center of gravity of the host vehicle, etc. The internal sensor group 3 also includes sensors that detect the driving operations of the driver in manual driving mode, such as operation of the accelerator pedal, operation of the brake pedal, operation of the steering wheel, etc.

[0014] The camera 4 has an imaging element such as a CCD or CMOS sensor, and captures images of the surroundings (front, rear, and sides) of the vehicle.

[0015] The LIDAR 5 irradiates electromagnetic waves (laser light, etc.) into a three-dimensional space around the vehicle and detects the external environment around the vehicle based on the waves reflected from the object. More specifically, the laser light, etc. irradiated by the LIDAR 5 is reflected at a point on the surface of the object (which may be referred to as a measurement point) and returns to the LIDAR 5. Therefore, in the case of a LIDAR 5 using an FMCW (frequency continuously modulated wave) method, the LIDAR 5 measures the distance from the light source of the LIDAR 5 to that point, the intensity of the reflected and returned laser light, the relative speed of the object located at that measurement point, etc. Furthermore, in the case of a LIDAR 5 using a ToF (time of flight) method, the LIDAR 5 measures the distance from the light source of the LIDAR 5 to that point and the intensity of the reflected and returned laser light. Either an FMCW method or a ToF method may be used for the LIDAR 5 in implementing the present invention. That is, it is sufficient if the position and shape of at least objects ahead of the vehicle (moving objects such as other vehicles, and stationary objects such as the road surface and structures) can be detected by scanning and irradiating laser light or the like from the lidar 5 attached to a predetermined position (front) of the vehicle in both horizontal and vertical directions around the vehicle (forward). Objects detected by the lidar 5 include people as well. Therefore, moving objects include vehicles such as moving cars and bicycles as well as moving people (pedestrians, etc.), and stationary objects include road structures, parked vehicles, people stationary on the road, fallen objects, etc. In the following description, the three-dimensional space is represented by an X axis along the traveling direction of the vehicle (which may also be called the depth direction), a Y axis along the width direction of the vehicle (corresponding to the horizontal direction), and a Z axis along the height direction of the vehicle (corresponding to the vertical direction). Therefore, the three-dimensional space may be referred to as an XYZ space.

[0016] Actuators AC are driving actuators for controlling the driving of the host vehicle. When the driving source is an engine, actuators AC include a throttle actuator that adjusts the opening of the engine's throttle valve (throttle opening). When the driving source is a driving motor, actuators AC include the driving motor. Actuators AC also include a brake actuator that operates the host vehicle's braking device and a steering actuator that drives the steering device.

[0017] The controller 10 is configured by an electronic control unit (ECU). More specifically, the controller 10 includes a computer having a calculation unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM and a RAM, and other peripheral circuits (not shown) such as an I / O interface. It should be noted that although a plurality of ECUs with different functions, such as an engine control ECU, a traction motor control ECU, and a braking device ECU, can be provided separately, for the sake of convenience, in FIG. 1 the controller 10 is shown as a collection of these ECUs.

[0018] The storage unit 12 stores highly accurate, detailed map information (referred to as high-accuracy map information). The high-accuracy map information includes road position information, road shape (curvature, etc.) information, road gradient information, intersection and branch point position information, number of lanes (driving lanes), lane width and position information for each lane (information on lane center positions and lane boundary lines), position information of landmarks (traffic lights, signs, buildings, etc.) on the map, and road surface profile information such as road surface irregularities. The storage unit 12 also stores programs for various controls, information such as thresholds used in the programs, setting information for on-board detectors such as the LIDAR 5, and the like. Furthermore, the storage unit 12 can also store data acquired by the lidar 5 as past data.

[0019] <Configuration of the calculation unit> The calculation unit 11 has, as its functional configuration, a point cloud data acquisition unit 111, an information acquisition unit 112, a separation unit 113, a calculation unit 114, a processing unit 115, an object detection unit 116 (hereinafter simply referred to as the detection unit), an integration unit 117, and a driving control unit 118. 1, the point cloud data acquisition unit 111, the information acquisition unit 112, the separation unit 113, the calculation unit 114, the processing unit 115, the detection unit 116, and the integration unit 117 are included in the object detection device 50. Details of each unit included in the object detection device 50 will be described later.

[0020] In the autonomous driving mode, the driving control unit 118 generates a target trajectory based on the external conditions around the vehicle, including the size, position, relative movement speed, etc. of an object detected by the object detection device 50. Specifically, the driving control unit 118 generates a target trajectory based on the size, position, relative movement speed, etc. of the object detected by the object detection device 50, so as to avoid collision or contact with the object or to follow the object. The driving control unit 118 controls the actuators AC so that the host vehicle travels along the target trajectory. Specifically, the driving control unit 118 controls the actuators AC along the target trajectory to adjust the accelerator opening and drive the braking device and the steering device. In the manual driving mode, the driving control unit 118 controls the actuators AC in accordance with a driving command (such as a steering operation) from the driver acquired by the internal sensor group 3.

[0021] <Light emitted by lidar> 2 is a schematic diagram illustrating the light emitted by the LIDAR 5 onto the space ahead of the vehicle 101. In this embodiment, a laser beam or the like is sequentially scanned and emitted onto a plurality of measurement points set in advance within the field of view (hereinafter referred to as FOV) of the LIDAR 5, and detection data based on the reflected waves from each measurement point is acquired. Information based on the reflected waves from the plurality of measurement points within the FOV is referred to as one frame's worth of detection data. The LIDAR 5 repeats, on a frame-by-frame basis, the irradiation of the laser beam or the like onto the plurality of measurement points within the FOV and the reception of the reflected waves from the plurality of measurement points.

[0022] As shown in Figure 2, when the same object OBT exists at different distances from the vehicle 101, the shorter the distance to the object OBT, the greater the number of measurement points on the object surface of the object OBT, and the longer the distance to the object OBT, the fewer the number of measurement points on the object surface. 2, the number of measurement points scanned and irradiated onto the object OBT located 100 m from the host vehicle 101 is smaller than the number of measurement points scanned and irradiated onto the object OBT located 50 m from the host vehicle 101. Moreover, the number of measurement points scanned and irradiated onto the object OBT located 150 m from the host vehicle 101 is even smaller than the number of measurement points scanned and irradiated onto the object OBT located 100 m from the host vehicle 101.

[0023] 3A, 3B, and 3C are schematic diagrams illustrating the number of measurement points scanned and irradiated on an object OBT located 50 m, 100 m, and 150 m from the host vehicle 101 in FIG. 2, respectively. In the embodiment, as an example, the irradiation direction of a laser beam or the like is controlled to scan at 0.1-degree intervals in both the vertical and horizontal directions, and the size of the object OBT to be detected is 0.3 m horizontally and 0.8 m vertically. That is, the numbers of measurement points illustrated in FIGS. 3A, 3B, and 3C indicate that the number of measurement points in the space (0.3 m × 0.8 m in this example) corresponding to the object OBT located 50 m, 100 m, and 150 m from the host vehicle 101 is 31, 7, and 4, respectively.

[0024] In order to detect an object OBT, the object detection device 50 counts the number of measurement points (the number of measurement points constituting a group) that have approximately the same distance value to the measurement point, or the XYZ coordinate value of the measurement point, or the relative velocity value of the measurement point, or the absolute velocity value of the measurement point, in the detection data of the most recently acquired frame (referred to as the first frame for convenience), based on the policy of ensuring that the number of measurement points on the object surface is at least a predetermined number (for example, 10 points) or more. "Approximately the same values" means that the values ​​are close enough to be considered as measurement points on the same object surface. Next, the size of the object to be measured (here, 0.3 m × 0.8 m) and the maximum detection distance (e.g., 150 m) are specified, and data from the measurement points is selectively superimposed in the hope of increasing the number of measurement points constituting the group to the predetermined number (here, 10) at any distance up to the maximum detection distance. Specifically, from the data of the measurement points in past frames (for convenience, referred to as the second frame, the third frame, etc.) acquired before the first frame, the measurement points to be retained are determined and extracted based on the past frame in which the corresponding measurement point is located and the distance to the corresponding measurement point, and the extracted data of the measurement points in the past frames are superimposed on the detection data of the first frame. The data of the measurement points in the past frames is the data stored as past data in the memory unit 12 described above.

[0025] 3A illustrates a case where an object OBT is present at a position 50 m from the host vehicle 101. The number of measurement points in the space corresponding to the object OBT is 31, which is more than the predetermined number (here, 10), and therefore, this is an example in which overlapping of data from the measurement points is not necessary.

[0026] 3B illustrates a case where an object OBT is present at a position 100 m from the host vehicle 101. The number of measurement points in the space corresponding to the object OBT is seven, which is less than the predetermined number (10 in this case), and therefore, it is necessary to superimpose the data of the measurement points.

[0027] The schematic diagram shown in Fig. 3C illustrates a case where an object OBT (Fig. 3C) is present at a position 150 m from the host vehicle 101. The number of measurement points in the space corresponding to the object OBT is four, and as in Fig. 3B, this is an example in which the data of the measurement points must be superimposed because the number of measurement points is not equal to or greater than the predetermined number (10 in this case).

[0028] 4A, 4B, and 4C, an example of superimposing data of measurement points in space corresponding to an object OBT located 150 m from the host vehicle 101 will be described. Fig. 4A is a schematic diagram showing measurement points (four points) in space corresponding to the object OBT in the first frame acquired most recently, and is similar to the case of Fig. 3C.

[0029] FIG. 4B is a schematic diagram showing measurement points (four points) in space corresponding to the object OBT in the second frame acquired as the frame immediately before the first frame. FIG. 4C is a schematic diagram showing measurement points (four points) in space corresponding to the object OBT in a past frame (the third frame) acquired as the frame two frames before the first frame.

[0030] The object detection device 50 superimposes, for each measurement point, data of the corresponding measurement points (four points) from the second frame one frame before the object detection device 50 (FIG. 4B) on the data of the measurement points (four points) from the first frame shown in FIG. 4A. Although the number of measurement points in the space corresponding to the object OBT in the first frame after the superposition increases to 4 + 4 = 8 points, this does not exceed the predetermined number (here, 10). Next, the object detection device 50 superimposes, for each measurement point, data of the corresponding measurement points (four points) from the third frame one frame before the object detection device 50 (FIG. 4C) on the data of the measurement points (eight points) in the space corresponding to the object OBT in the first frame after the superposition.

[0031] Fig. 5 is a schematic diagram showing the measurement points (12 points) in the space corresponding to the object OBT in the first frame after superposition. In Fig. 5, the number of measurement points in the space corresponding to the object OBT has increased to 8 + 4 = 12 points, which is more than the predetermined number (here, 10). The object detection device 50 sequentially overlays data of measurement points from previous frames in the hope of increasing the number of measurement points in the space corresponding to the object OBT up to the above-mentioned predetermined number (here, 10), and then detects the presence of the object OBT based on the still point cloud data. Such an object detection device 50 will now be described in more detail.

[0032] <Details of the object detection device> The object detection device 50 includes a LIDAR 5 in addition to the point cloud data acquisition unit 111, information acquisition unit 112, separation unit 113, calculation unit 114, processing unit 115, detection unit 116, and integration unit 117 described above.

[0033] The point cloud data acquisition unit 111 acquires point cloud data, including position information indicating the three-dimensional position coordinates of measurement points on the surface of the object OBT obtained by the LIDAR 5, and velocity information indicating the relative movement velocity of the measurement points, as detection data of the LIDAR 5. The point cloud data acquisition unit 111 acquires the point cloud data in frame units at predetermined time intervals. The predetermined time corresponds to the time it takes for the LIDAR 5 to scan and irradiate a laser beam or the like onto multiple measurement points set within the FOV of the LIDAR 5 at the above-mentioned 0.1 degree intervals, and for one frame of detection data based on reflected waves from each measurement point within the FOV to be output from the LIDAR 5. The number of frames of point cloud data acquired per second by the point cloud data acquisition unit 111 may be referred to as the frame rate of the point cloud data.

[0034] The information acquisition unit 112 acquires required multiple information that indicates the relationship between the distance to a measurement point on the surface of the object OBT to be detected by the lidar 5 and the number of times that the data of the measurement points needs to be overlapped (hereinafter referred to as the required multiple). The necessary multiple information acquired by the information acquisition unit 112 is information calculated as follows. First, based on the size of the object OBT predetermined as the detection target and the irradiation angle resolution (for example, the above-mentioned 0.1 degree interval) of the laser light or the like scanned and irradiated from the lidar 5, the number of measurement points (FIGS. 3A, 3B, and 3C) in the space corresponding to the object OBT (0.3 m × 0.8 m in this example) is calculated for each distance from the host vehicle 101. Next, the number of frames required for superposition to ensure that the number of measurement points in the space corresponding to the object OBT at each distance is equal to or greater than a predetermined number (10 in this case) is calculated as the above-mentioned necessary multiple of the measurement point data.

[0035] 6A is a diagram showing an example of the number of measurement points for each distance calculated based on the size of the object OBT and the irradiation angular resolution of the LIDAR 5. The horizontal axis indicates the distance to the space corresponding to the object OBT, and the vertical axis indicates the number of measurement points. The graph in FIG. 6A shows the number of measurement points in the space corresponding to the object OBT to be detected by the LIDAR 5 for each distance from the host vehicle 101.

[0036] FIG. 6B is a diagram showing an example of the required multiple for each distance calculated based on the number of measurement points in FIG. 6A. The horizontal axis indicates the distance to the space corresponding to the object OBT, and the vertical axis indicates the required multiple. The graph in FIG. 6B shows the required multiple of the measurement point data required to ensure a predetermined number (10 in this case) or more of measurement points in the space corresponding to the object OBT, for each distance from the host vehicle 101. A required multiple of 1 indicates that superimposition using measurement point data from past frames is not necessary (in other words, the number of frames required for superimposition is 0), and required multiples of 2, 3, ..., 7, and 8 indicate that the number of frames required for superimposition using measurement point data from past frames is 1, 2, ..., 6, and 7, respectively.

[0037] Fig. 6C is a diagram illustrating distance ranges for each required multiple, created based on Fig. 6B. As an example, when the distance from the host vehicle 101 to the space corresponding to the object OBT is 150 m, the required multiple is theoretically 3.

[0038] The information acquired by the information acquisition unit 112 is the required multiple information corresponding to Fig. 6C. When the required multiple information is stored in advance in a predetermined area of ​​the storage unit 12, the information acquisition unit 112 reads and acquires the required multiple information from the storage unit 12. The information acquisition unit 112 may also receive the above information from an external device via the communication unit 1.

[0039] The separation unit 113 separates the point cloud data acquired by the point cloud data acquisition unit 111 into moving point cloud data corresponding to measurement points on the surface of a moving object and static point cloud data other than the moving point cloud data (point cloud data with a sufficiently slow moving speed). The reason for this separation is to apply the above-mentioned overlay process using data on measurement points from past frames only to the static point cloud data. More specifically, since the position of a moving object differs for each frame of point cloud data, even if overlay process using data on measurement points from past frames is applied to the moving point cloud data, the overlay process may not be able to sufficiently increase the number of measurement points in the space corresponding to the object OBT. Furthermore, measurement points that do not actually exist may be generated around the space corresponding to the object OBT, which may result in the object being detected at a position where it does not actually exist. In contrast, the positions of stationary objects are approximately the same in each frame of the point cloud data (there is only a slight shift due to the swaying of the vehicle 101, etc.), so by applying an overlay process using measurement point data from past frames to the stationary point cloud data, it is expected that the number of apparent measurement points in the space corresponding to the object OBT can be increased by the overlay process.

[0040] When the separation unit 113 separates the still point cloud data of a new frame, the calculation unit 114 calculates and adds a required multiple for the data of each measurement point using the distance from the vehicle 101 to the measurement point, based on the required multiple information ( FIG. 6C ) acquired by the information acquisition unit 112. The point cloud data made up of the data of the measurement points to which the required multiple has been added is recorded in the storage unit 12 as point cloud data of the first frame.

[0041] In addition, the calculation unit 114 extracts measurement points required for overlay from the point cloud data of the second frame based on the value of the required multiple of each measurement point that constitutes the point cloud data of the second frame stored in the memory unit 12 as the previous frame immediately before the first frame (the previous frame ID is 1) and the value of the previous frame ID. More specifically, the calculation unit 114 determines that the measurement points constituting the point cloud data of the second frame have a required multiple greater than the previous frame ID (in this case, a number greater than 2) as measurement points to be used for overlay, and leaves the data of these measurement points in the storage unit 12. In other words, data of measurement points whose required multiple is the same as the frame ID or smaller than the value of the frame ID is deleted from the storage unit 12 as data not required for overlay. Similarly, for the point cloud data of the third frame stored in the memory unit 12 as the frame two frames before the first frame (the past frame ID is 2), a determination is made based on the required multiple and the past frame ID (2 in this case) as to whether or not to keep the data in the memory unit 12 as a measurement point to be used for overlay, and the data of the measurement points not required for overlay is deleted from the memory unit 12. The same process as above is repeated until the number of overlaps reaches the maximum required multiple (8 in this case) in Fig. 6C. In other words, the process is repeated up to the seventh frame stored in storage unit 12 as the seventh frame before the first frame (the past frame ID is 7).

[0042] The above process of extracting measurement points to be used for overlay from the point cloud data of past frames will be described in detail with reference to Figures 7A, 7B, and 7C. Figures 7A, 7B, and 7C are all diagrams explaining the still point cloud data of past frames stored in storage unit 12 at the time when the still point cloud data of the most recent first frame was separated. Figure 7A is a diagram explaining the still point cloud data of the second frame (past frame ID is 1), which was acquired, separated, and the required multiple calculated one frame before the first frame, and is stored in the memory unit 12 as past data. Figure 7B is a diagram explaining the still point cloud data of the third frame (past frame ID is 2), which was acquired, separated, and the required multiple calculated two frames before the first frame, and is stored in the memory unit 12 as past data. Figure 7C is a diagram illustrating the still point cloud data of the eighth frame (past frame ID is 7), which was acquired, separated, and the required multiple calculated seven frames before the first frame (in other words, six frames before the second frame) and stored as past data in memory unit 12.

[0043] In Fig. 7A (still point cloud data from one frame before the first frame, i.e., with a previous frame ID of 1), data of measurement points whose required multiple has a value equal to or smaller than the previous frame ID (here, 1) is enclosed in a thick frame. The area enclosed in this thick frame is data of measurement points that do not require overlay using data of measurement points from previous frames. In other words, it may be deleted from the storage unit 12 (or excluded from overlay targets). In Figure 7B (still point cloud data from the frame two frames before the first, i.e., with a previous frame ID of 2), data of measurement points whose required multiple has a value equal to or smaller than the previous frame ID (here, 2) is enclosed in a thick frame. The area enclosed in this thick frame is data of measurement points that do not need to be superimposed using data of measurement points from previous frames. In other words, it may be deleted from the storage unit 12 (or excluded from being superimposed). In Figure 7C (still point cloud data from the seventh frame before the first frame, i.e., with a previous frame ID of 7), data of measurement points whose required multiple has a value equal to or smaller than the previous frame ID (7 in this case) is surrounded by a thick frame. The area surrounded by this thick frame is data of measurement points that do not need to be overlaid using data of measurement points from previous frames. In other words, it may be deleted from the storage unit 12 (or excluded from being overlaid).

[0044] As described above, by extracting measurement points to be used for overlay from the point cloud data of past frames and leaving only the data of the extracted measurement points in the memory unit 12 as still point cloud data of the past frames, it is possible to reduce the amount of storage capacity of the memory unit 12 reserved for still point cloud data of past frames compared to when data of all measurement points is left as still point cloud data of past frames.

[0045] The processing unit 115 superimposes the data of each measurement point constituting the still point cloud data of the first frame to which the required multiple for each measurement point has been added by the calculation unit 114 with the data of the corresponding measurement point (in other words, the extracted measurement point) from the still point cloud data of past frames (the second frame, the third frame, etc.) that were acquired and separated before the first frame and stored in the memory unit 12.

[0046] The detection unit 116 performs object detection on both the still point cloud data and the moving point cloud data. <Static point cloud data> The maximum size (Xmax and Ymax) in the XY directions of the object OBT can be recognized even without information on the height direction (Z direction) of the object OBT. Therefore, the detection unit 116 projects each measurement point constituting the still point cloud data onto the XY plane so as to remove information on the height direction from the position information of each measurement point corresponding to the still point cloud data, and converts the position information of each measurement point from three dimensions to two dimensions. Specifically, when the position coordinates of each measurement point are expressed in the XYZ coordinate system, the detection unit 116 projects each measurement point corresponding to the still point cloud data onto the XY plane, and converts the still point cloud data into two-dimensional data expressed in the XY coordinate system. The detection unit 116 detects stationary objects around the vehicle 101 based on the converted XY data. As an example, the detection unit 116 executes a clustering process on the two-dimensional data on the XY plane to detect a bounding box, which is a circumscribing area of ​​the stationary object, from the XY plane. Information indicating the detection result of the stationary object is recorded in the storage unit 12, for example. As another example of detecting stationary objects around the vehicle 101, the following method may be adopted. That is, the detection unit 116 divides the XY plane into a grid of a predetermined size, and extracts only the difference between the maximum and minimum values ​​in the height direction of each measurement point of the stationary point cloud data present in each grid. After that, only grids whose difference values ​​exceed a predetermined threshold are detected as three-dimensional objects.

[0047] <Moving point cloud data> As in the case of the still point cloud data, the detection unit 116 detects moving objects from the moving point cloud data. Information indicating the detection result of the moving object is recorded in the storage unit 12, for example.

[0048] The integration unit 117 integrates, on a two-dimensional coordinate map of the XY plane, the object information derived from the stationary point cloud detected by the detection unit 116 and the object information derived from the moving point cloud detected by the detection unit 116. Information indicating the integration result is recorded in the storage unit 12, for example.

[0049] <Explanation of the flowchart> 8A and 8B are flowcharts showing an example of an object detection process executed by the calculation unit 11 of the controller 10 in Fig. 1 based on a predetermined program. The process shown in the flowcharts in Fig. 8A and 8B is repeated at predetermined intervals while the host vehicle 101 is traveling in the autonomous driving mode, for example.

[0050] First, in step S10 of FIG. 8A, the calculation unit 11 acquires three-dimensional point cloud data from the LIDAR 5 using the point cloud data acquisition unit 111, and the process proceeds to step S20. In step S20, the calculation unit 11 performs a separation process using the separation unit 113 to separate the point cloud data acquired by the point cloud data acquisition unit 111 into moving point cloud data corresponding to measurement points on the surface of a moving object and static point cloud data other than the moving point cloud data, and then proceeds to step S30.

[0051] An example of the separation process will be described. The separation unit 113 estimates the absolute movement speed of the host vehicle 101 based on position information indicating the three-dimensional position coordinates of measurement points on the surface of the object OBT included in the point cloud data acquired by the LIDAR 5 and speed information indicating the relative movement speed of the measurement points. First, point cloud data is extracted from the point cloud data, excluding information on measurement points corresponding to the object OBT, i.e., point cloud data corresponding to the road surface around the vehicle 101 (hereinafter referred to as road surface point cloud data).Based on the position coordinates (three-dimensional positions) included in the extracted road surface point cloud data, a unit vector indicating the direction of the relative movement speed is calculated. Next, a conversion equation for converting the relative movement speed of the measurement point corresponding to the road surface into absolute movement speed is set as an objective function, and the movement speed (absolute movement speed) of the vehicle 101 is estimated by solving an optimization problem that optimizes the objective function so as to approach zero. Furthermore, based on the relative movement speed of the measurement points and the estimated absolute movement speed of the vehicle 101, the absolute movement speed of each of the plurality of measurement points corresponding to the point cloud data is calculated. Finally, the point cloud data is divided into moving point cloud data corresponding to measurement points whose absolute values ​​of absolute moving speeds are equal to or greater than a predetermined speed, and stationary point cloud data other than the moving point cloud data.

[0052] In step S30, the calculation unit 11 calculates the required multiple for each measurement point for the separated still point cloud data using the calculation unit 114, and then the process proceeds to step S40. In step S40, the calculation unit 11 adds the required multiple to the data of the measurement point by the calculation unit 114, and the process proceeds to step S50.

[0053] In step S50, the calculation unit 11 records the still point cloud data of the first frame to which the required multiple has been added in the storage unit 12, and then proceeds to step S60. Through the processing of steps S10 to S50, each time point cloud data of a new frame is acquired, separation, calculation of the required multiple, and recording of the still point cloud data to which the required multiple has been added are repeated. When point cloud data of a new frame is acquired, the still point cloud data stored in the storage unit 12 at that time becomes the still point cloud data of the previous frame.

[0054] In step S60, the calculation unit 11 performs processing to extract measurement points required for overlay from the point cloud data of the past frame based on the value of the required multiple of each measurement point constituting the point cloud data stored in the storage unit 12 as a past frame and the value of the past frame ID, using the calculation unit 114, and then proceeds to step S70. Details of the processing of step S60 will be described later with reference to FIG. 8B.

[0055] In step S70, the processing unit 115 of the calculation unit 11 superimposes the data of each measurement point constituting the still point cloud data of the first frame with the data of the corresponding measurement point from the still point cloud data of past frames (the second frame, the third frame, etc.) that were acquired and separated before the first frame and stored in the memory unit 12, and then proceeds to step S80.

[0056] In step S80, the calculation unit 11 records the still point cloud data of the frame after superimposition in the storage unit 12, and the process proceeds to step S90.

[0057] In step S90, the calculation unit 11 detects stationary objects around the vehicle 101 based on the superimposed stationary point cloud data using the detection unit 116, and then the process proceeds to step S100.

[0058] In step S100, the calculation unit 11 detects moving objects around the vehicle 101 based on the moving point cloud data using the detection unit 116, and then the process proceeds to step S110.

[0059] In step S110, the calculation unit 11 integrates, via the integration unit 117, the object information derived from the stationary point cloud detected by the detection unit 116 and the object information derived from the moving point cloud detected by the detection unit 116 onto a two-dimensional coordinate map on the XY plane, and then proceeds to step S120.

[0060] In step S120, the calculation unit 11 determines whether or not to end the processing. If the host vehicle 101 continues traveling in the autonomous driving mode, the calculation unit 11 makes a negative determination in step S120, returns to step S10 in FIG. 8A, and repeats the above-described processing. By returning to step S10, detection of objects, etc. based on point cloud data is periodically and repeatedly performed while the host vehicle 101 is traveling. On the other hand, if the host vehicle 101 has finished traveling in the autonomous driving mode, the calculation unit 11 makes a positive determination in step S120 and ends the processing in FIG. 8A.

[0061] FIG. 8B is a flowchart illustrating the details of the process of step S60 (FIG. 8A) executed by the calculation unit 11.

[0062] In step S61, the calculation unit 11 reads out still point cloud data of a past frame from the storage unit 12 using the calculation unit 114, and the process proceeds to step S62. The initial value of the past frame ID of the past frame to be read out is 1.

[0063] In step S62, the calculation unit 11 determines whether or not "required multiple > past frame ID" is true for each measurement point of the still point cloud data of the read past frame by the calculation unit 114. If "required multiple > past frame ID" is true, the calculation unit 114 makes an affirmative judgment in step S62 and proceeds to step S63, where the measurement point is determined to be a measurement point required for overlay. On the other hand, if "required multiple>past frame ID" does not hold, the calculation unit 114 makes a negative decision in step S62 and proceeds to step S64, where it determines that the measurement point is not required for superposition.

[0064] In step S65, the calculation unit 11 determines whether or not "past frame ID = maximum required multiple - 1" is true, using the calculation unit 114. If true, the calculation unit 114 makes an affirmative decision in step S65, terminates the processing in Fig. 8B, and returns to Fig. 8A to proceed to step S70. If false, the calculation unit 114 makes a negative decision in step S65, increases the past frame ID by one, and returns to step S61 in Fig. 8B. The above-described process is repeated until the value of the past frame ID becomes the maximum required multiple -1.

[0065] According to the embodiment described above, the following advantageous effects are achieved. (1) The object detection device 50 is mounted on the host vehicle 101 as a moving body, and includes a lidar 5 as a detector that irradiates a three-dimensional space around the host vehicle 101 with electromagnetic waves such as laser light to detect the external environment around the host vehicle 101 based on the reflected waves; a point cloud data acquisition unit 111 that acquires point cloud data including three-dimensional position information of measurement points on the surface of the object OBT from which the reflected waves are obtained, from the lidar 5 for each frame; a separation unit 113 that separates the point cloud data acquired by the point cloud data acquisition unit 111 into moving point cloud data corresponding to the measurement points on the surface of the moving object OBT and static point cloud data other than the moving point cloud data; and a detection unit 114 that detects each measurement point from the host vehicle 101 for each measurement point constituting the static point cloud data separated by the separation unit 113. a storage unit (12) that stores, for each frame, still point cloud data made up of the data of each measurement point to which the necessary multiple has been added; a processing unit (115) that, when the still point cloud data of a new frame is separated by a separation unit (113) and the calculation unit (114) adds the necessary multiple to the data of each measurement point, superimposes the data of the corresponding measurement point from the still point cloud data of a past frame stored in the storage unit (12) onto the still point cloud data of the new frame based on the necessary multiple; and a detection unit (116) that detects a still object based on the still point cloud data after superimposition by the processing unit (115). This configuration makes it possible to properly detect stationary objects that are located far away or that are small in size even if they are located near the vehicle 101, and therefore have a small number of measurement points on their surface. The moving point cloud data, which may become erroneous due to the overlay process, is separated, and the overlay process is performed only on the data of the measurement points that make up the stationary point cloud data, thereby achieving an appropriate overlay effect.

[0066] (2) The object detection device 50 of (1) above further includes an information acquisition unit 112 that acquires required multiple information that defines a predetermined relationship between the distance from the vehicle 101 to the measurement point and the required multiple, and the calculation unit 114 calculates the required multiple for each measurement point that constitutes the still point cloud data separated by the separation unit 113, based on the distance from the vehicle 101 to each measurement point and the required multiple information acquired by the information acquisition unit 112. With this configuration, it is possible to properly calculate the required multiple required for the overlay process.

[0067] (3) In the object detection device 50 described in (2) above, the calculation unit 114 determines and extracts data of measurement points of past frames to be superimposed on the still point cloud data of a new frame, based on which past frame the corresponding measurement points belong to and the distance to the corresponding measurement points. With this configuration, it is possible to appropriately superimpose the data of each measurement point constituting the still point cloud data of the first frame, which is a new frame, with the data of the corresponding measurement point in the still point cloud data of past frames (e.g., the second frame, the third frame, etc.) stored in the memory unit 12. In other words, compared to overlaying the data of all measurement points that make up the still point cloud data of past frames, it is possible to reduce the processing load and the amount of memory used in the processing.

[0068] (4) In the object detection device 50 described in (3) above, the calculation unit 114 removes data that is not extracted in (3) above from the still point cloud data of past frames stored in the storage unit 12. This configuration makes it possible to reduce the storage capacity of the storage unit 12 compared to when data not required for overlay is left stored in the storage unit 12.

[0069] (5) In the object detection device 50 described in (1) above, the information acquisition unit 112 acquires, as required multiple information, information that determines the required multiple for each distance from the vehicle 101 to a measurement point based on the predetermined size of the stationary object, the scanning angle resolution at which the lidar 5 emits laser light or the like, and the number of measurement points predetermined to detect the stationary object. With this configuration, it becomes possible to appropriately acquire the necessary multiple information required for the overlapping process performed by the processing unit 115.

[0070] (6) In the object detection device 50 of (1) above, the detection unit 116 further includes an integration unit 117 that detects moving objects based on the moving point cloud data and integrates the position information of stationary objects and the position information of moving objects. This configuration makes it possible to obtain position information for both stationary and moving objects.

[0071] The above embodiment can be modified in various ways, and modifications will be described below. (Variation 1) In the above explanation, the detection of a stationary object is described based on the size of one object OBT (0.3 m horizontally x 0.8 m vertically), but it may also be configured to detect a stationary object based on the sizes of multiple objects OBT1, OBT2 that are different from each other. Specifically, the size of the first object OBT1 is set to the above-mentioned size (0.3 m horizontally x 0.8 m vertically), and the size of the second object OBT2 is set to, for example, 0.7 m horizontally x 0.15 m vertically, and stationary objects are detected based on the sizes of these two objects OBT1 and OBT2.

[0072] When objects OBT1 and OBT2 of different sizes are present at equal distances (e.g., 100 m) from the vehicle 101, the larger the size of the objects OBT1 and OBT2 (in other words, the larger the surface area), the greater the number of measurement points on the object surface, and the smaller the size of the objects OBT1 and OBT2, the fewer the number of measurement points on the object surface.

[0073] Therefore, the information acquisition unit 112 acquires two types of required multiple information corresponding to the sizes of the two objects OBT1 and OBT2, respectively. The calculation unit 114 calculates the required multiples required to detect the two objects OBT1 and OBT2 for each measurement point constituting the still point cloud data separated by the separation unit 113, based on the distance from the vehicle 101 to each measurement point and the two types of required multiple information acquired by the information acquisition unit 112. Furthermore, when the required multiples for detecting the two objects OBT1 and OBT2 at the same distance are different, the calculation unit 114 adopts the required multiple with the larger value at that distance.

[0074] According to variant example 1, even when the number of measurement points on the surface of multiple objects OBT1 and OBT2 of different sizes is small, for example because these objects are stationary at a distance, it is possible to properly detect each individual stationary object.

[0075] (Variation 2) When a ToF (Time of Flight) LIDAR 5 is used, unlike the FMCW method, the point cloud data does not include velocity information indicating the relative moving velocity of each measurement point. Therefore, when the LIDAR 5 is a ToF method, the following method may be used as the separation process for separating the moving point cloud data and the stationary point cloud data in the above-mentioned step S20.

[0076] Based on the three-dimensional point cloud data of multiple frames obtained consecutively, the separation unit 113 projects each measurement point onto an XY plane so as to remove information in the height direction from the position information of each measurement point, thereby converting the position information of each measurement point from three-dimensional to two-dimensional. That is, each measurement point corresponding to the point cloud data is projected onto an XY plane, and the point cloud data is converted into two-dimensional data represented in an XY coordinate system. Next, the positions of each measurement point converted into two-dimensional data are offset based on the traveling vector of the vehicle 101. In other words, when the relative coordinate position of the most recent frame is taken as the absolute coordinate, the positions on the XY plane corresponding to each measurement point of the multiple frames obtained consecutively are offset so that the relative coordinate position of the previous frame matches the absolute coordinate position. After the offset processing, the positions of each measurement point on the XY plane overlap between the most recent frame and the previous frame for measurement points on stationary objects (in other words, the deviation in the position of corresponding measurement points between the most recent frame and the previous frame is within a specified distance), but do not overlap between the most recent frame and the previous frame for measurement points on moving objects.

[0077] The separation unit 113 separates the point cloud data of the most recent frame into moving point cloud data composed of measurement points whose positions do not overlap with corresponding measurement points of past frames on the XY plane after the offset processing, and stationary point cloud data other than the moving point cloud data. In this way, even when the LIDAR 5 is a ToF type, it is possible to separate the moving point cloud data and the stationary point cloud data by a simple calculation.

[0078] The above description is merely an example, and the present invention is not limited to the above-described embodiment and modifications, as long as the features of the present invention are not impaired. One or more of the above-described embodiment and modifications can be arbitrarily combined. [Explanation of symbols]

[0079] 1 Communication unit, 2 Positioning unit, 3 Internal sensor group, 4 Camera, 5 Lidar, 10 Controller, 11 Calculation unit, 12 Memory unit, 50 Object detection device, 100 Vehicle control device, 101 Vehicle, 111 Point cloud data acquisition unit, 112 Information acquisition unit, 113 Separation unit, 114 Calculation unit, 115 Processing unit, 116 Detection unit, 117 Integration unit, 118 Travel control unit, AC actuator

Claims

1. a detector mounted on the moving body, which irradiates electromagnetic waves into a three-dimensional space around the moving body and detects an external environment around the moving body based on reflected waves; a point cloud data acquisition unit that acquires, for each frame, point cloud data from the detector, the point cloud data including three-dimensional position information of measurement points on the surface of the object from which the reflected waves are obtained; a separation unit that separates the point cloud data acquired by the point cloud data acquisition unit into moving point cloud data corresponding to measurement points on the surface of a moving object and stationary point cloud data other than the moving point cloud data; a calculation unit that calculates a required multiple for a superimposition process for data of each measurement point based on a distance from the moving object to each measurement point for each measurement point constituting the still point cloud data separated by the separation unit, and adds the required multiple to the data; a storage unit that stores the still point cloud data, which is composed of data of each measurement point to which the required multiple has been added, for each frame; a data processing unit that, when the separation unit separates the still point cloud data of a new frame and the calculation unit adds the required multiple to the data of each measurement point, superimposes data of the corresponding measurement point among the still point cloud data of the past frame stored in the storage unit onto the still point cloud data of the new frame based on the required multiple; a detection unit that detects a stationary object based on the stationary point cloud data after superposition by the data processing unit; An object detection device comprising:

2. 2. The object detection device according to claim 1, an information acquisition unit that acquires required multiple information that defines a relationship between the distance from the moving object to the measurement point and the required multiple; the calculation unit calculates the required multiple for each measurement point constituting the still point cloud data separated by the separation unit, based on the distance from the moving object to each measurement point and the required multiple information acquired by the information acquisition unit. An object detection device characterized by:

3. 3. The object detection device according to claim 2, the calculation unit determines and extracts data of the measurement points of the past frames to be superimposed on the still point cloud data of the new frame, based on which past frame the corresponding measurement points belong to and the distance to the corresponding measurement points; An object detection device characterized by:

4. The object detection device according to claim 3, the calculation unit removes data that is not to be extracted from the still point cloud data of the past frames stored in the storage unit. An object detection device characterized by:

5. 3. The object detection device according to claim 2, the information acquisition unit acquires, as the required multiple information, information that defines the required multiple for each distance from the moving body to the measurement point based on a predetermined size of the stationary object, a scanning angle resolution at which the detector irradiates the electromagnetic wave, and a predetermined number of the measurement points for detecting the stationary object. An object detection device characterized by:

6. 2. The object detection device according to claim 1, The detection unit further detects a moving object based on the moving point cloud data, an integration unit that integrates the position information of the stationary object and the position information of the moving object; An object detection device characterized by:

7. The object detection device according to any one of claims 1 to 6, An object detection device, characterized in that the detector is a lidar.

Images