Information processing device, control method, program, and storage medium

Abstract

A controller 13 of an information processing device 1 mainly functions as a first acquisition means, a second acquisition means, and an object detection means. The first acquisition means acquires point cloud data that is a set of data for each point measured by a measurement device. The second acquisition means acquires movement area information relating to the movement area of an object on a horizontal plane within the measurement range of the measurement device. The object detection means detects a data cluster for each object from the point cloud data on the basis of the movement area information.

Description

Technical Field

[0001] This disclosure relates to the processing of measured data.

Background Art

[0002] Conventionally, a lidar device that irradiates a pulse of laser light onto a detection space and detects an object in the detection space based on the level of the reflected light has been known. For example, in Patent Document 1, a lidar that scans the surrounding space by appropriately controlling the emission direction (scanning direction) of repeatedly emitted light pulses and observes the return light to generate point cloud data representing information such as distance and reflectance, which are information regarding objects existing in the surroundings, is disclosed. Further, in Patent Document 2, a technique for detecting an object frame surrounding an object based on point cloud data and performing object tracking processing by associating object frames in a time series is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] When detecting an object from point cloud data obtained by measuring an object, in reality, data obtained by measuring different objects may be erroneously associated as data obtained by measuring the same object.

[0005] This disclosure has been made to solve the above problems, and a main object thereof is to provide an information processing apparatus capable of appropriately performing the association between measured data and an object.

Means for Solving the Problems

[0006] The invention described in the claims is, A first acquisition means for acquiring point cloud data, which is a collection of data for each point measured by a fixedly installed measuring device, Includes multiple lanes On the horizontal plane within the measurement range of the measuring device This is a map showing the location of the lane and the direction of movement within the lane. A second acquisition means for acquiring movement area information, Based on the aforementioned movement region information, from the point cloud data On the aforementioned multiple lanes A vehicle detection means for detecting data clusters for each vehicle, Equipped with 、 The vehicle detection means prohibits detecting data existing on lanes with different directions of movement as clusters representing the same vehicle. It is an information processing device.

[0007] Furthermore, the invention described in the claims is, Computers A fixed measuring device acquires point cloud data, which is a collection of data for each individual point measured by the device. Includes multiple lanes On the horizontal plane within the measurement range of the measuring device This is a map showing the location of the lane and the direction of movement within the lane. Obtain information on the area of ​​movement, Based on the aforementioned movement region information, from the point cloud data On the aforementioned multiple lanes Discover data clusters for each vehicle. death, The detection of data existing on lanes with different directions of movement as the same vehicle cluster is prohibited. This is a control method.

[0008] Furthermore, the invention described in the claims is, A fixed measuring device acquires point cloud data, which is a collection of data for each individual point measured by the device. Includes multiple lanes On the horizontal plane within the measurement range of the measuring device This is a map showing the location of the lane and the direction of movement within the lane. Obtain information on the area of ​​movement, Based on the aforementioned movement region information, from the point cloud data On the aforementioned multiple lanes Discover data clusters for each vehicle. death, The detection of data existing on lanes with different directions of movement as a cluster representing the same vehicle is prohibited. It is a program that instructs a computer to perform a process. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic configuration of the lidar unit common to each embodiment. [Figure 2] An example of the installation location of the lidar is shown. [Figure 3] This is a block diagram showing an example of the hardware configuration of the information processing device. [Figure 4] This is an example of a flowchart showing an overview of the processing executed by the controller in the first embodiment. [Figure 5] This is a diagram showing a lane area based on a lane map and segments detected on the lane. [Figure 6] This is an example of a flowchart showing a detailed procedure of the tracking object detection process. [Figure 7] This is a diagram showing a lane area based on a lane map and segments detected on the lane. [Figure 8] This is an example of a flowchart showing a detailed procedure of the new object detection process. [Figure 9] This is a block diagram showing an example of the hardware configuration of the information processing device according to the second embodiment. [Figure 10] In a two-dimensional space looking directly down at the field of view, the movement trajectory of a certain tracking object is indicated by an arrow on a virtual grid. [Figure 11] (A) An enlarged view of a grid cell showing the movement direction of the tracking object by an arrow. (B) A histogram showing the aggregated movement directions in the grid cells. [Figure 12] This is a diagram showing the movement direction and its frequency in each grid cell within the field of view. [Figure 13] (A) A diagram showing an overview of the first method for determining the boundary of a lane. (B) A diagram showing an overview of the second method for determining the boundary of a lane.

Embodiments of the Invention

[0010] According to a preferred embodiment of the present invention, the information processing device includes: a first acquisition means for acquiring point cloud data, which is a collection of point-by-point data measured by a measuring device; a second acquisition means for acquiring movement area information relating to the movement area of ​​an object on a horizontal plane within the measurement range of the measuring device; and an object detection means for detecting clusters of object-specific data from the point cloud data based on the movement area information. According to this embodiment, the information processing device can accurately detect clusters of object-specific data from the point cloud data based on the movement area information.

[0011] In one embodiment of the above-described information processing device, the object detection means prohibits detecting data existing on different moving regions as clusters representing the same object. This embodiment allows the information processing device to suitably prevent detecting data of different objects as clusters of data representing a single object.

[0012] In another embodiment of the information processing device described above, the moving area is a lane, and the object detection means prohibits detecting the data existing on different lanes as clusters representing the same object. In this embodiment, the information processing device can suitably prevent detecting data existing across lanes as clusters of data representing a single object.

[0013] In another embodiment of the information processing device described above, the object detection means prohibits detecting the data on the first lane and the data on the second lane as clusters representing the same object when the directions of movement of the first lane and the second lane indicated by the movement area information are different. This embodiment allows the information processing device to suitably prevent detecting data existing across opposite lanes as clusters of data representing a single object.

[0014] In another embodiment of the above-described information processing device, when the object detection means detects a cluster corresponding to the tracked object at the current processing time based on the predicted position of the tracked object detected at a past processing time, it detects the cluster by excluding the data that exists in a movement area different from the predicted position. In this embodiment, the information processing device prevents detecting data that exists in a movement area different from the predicted position of the tracked object as the tracked object, thereby enabling highly accurate object tracking.

[0015] In another embodiment of the information processing device described above, when the object detection means detects a cluster corresponding to an object newly detected at the current processing time based on data that does not correspond to an object detected at a past processing time, it prohibits detecting data with different movement regions as clusters representing the same object. In this embodiment, the information processing device can appropriately determine the clusters of data corresponding to objects that were not detected at past processing times.

[0016] In another embodiment of the above-described information processing device, the information processing device further comprises a segment extraction means for extracting segments representing the data clusters, and the object detection means detects one or more of the segments for each object as clusters. In this embodiment, the information processing device can suitably detect clusters of data for each object from point cloud data.

[0017] According to another preferred embodiment of the present invention, the information processing device includes: acquisition means for acquiring point cloud data, which is a collection of point-by-point data measured by a measuring device; first generation means for generating movement history information representing the direction of movement of an object that has passed through a position on a horizontal plane within the measurement range of the measuring device; and second generation means for generating movement area information according to any of the above based on the movement history information. In this embodiment, the information processing device can suitably generate movement history information useful for detecting clusters of data representing objects.

[0018] According to another preferred embodiment of the present invention, there is a control method performed by a computer, wherein the computer acquires point cloud data, which is a collection of point-by-point data measured by a measuring device, acquires movement area information relating to the movement area of ​​an object on a horizontal plane within the measurement range of the measuring device, and detects clusters of data for each object from the point cloud data based on the movement area information. By performing this control method, the computer can accurately detect clusters of data for each object from the point cloud data.

[0019] According to another preferred embodiment of the present invention, a program is provided that causes a computer to perform a process of acquiring point cloud data, which is a collection of data for each point measured by a measuring device, acquiring movement area information relating to the movement area of ​​an object on a horizontal plane within the measurement range of the measuring device, and detecting clusters of data for each object from the point cloud data based on the movement area information. By executing this program, the computer can accurately detect clusters of data for each object from the point cloud data. Preferably, the above program is stored on a storage medium. [Examples]

[0020] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0021] <First Example> (1) Rider Unit Overview Figure 1 shows a schematic configuration of the Lidar unit 100 common to each embodiment. The Lidar unit 100 includes an information processing device 1 that processes data generated by the sensor group 2, and a sensor group 2 that includes at least a Lidar (Light Detection and Ranging, or Laser Illuminated Detection And Ranging) 3. Figure 2 shows an example of Lidar 3 installation. The Lidar 3 shown in Figure 2 is installed so that the road is included within the field of view "Rv", which is the range in which the Lidar 3 can measure distance. In reality, the field of view Rv will have a shape corresponding to the maximum measuring distance of the Lidar 3. The Lidar unit 100 then detects an object and outputs information regarding the object detection result.

[0022] The information processing device 1 is electrically connected to the sensor group 2 and processes the data output by the various sensors included in the sensor group 2. In this embodiment, the information processing device 1 performs object detection based on the point cloud data output by the lidar 3. The object detection process includes estimating the state of the object, such as estimating the object's position, and processing related to tracking the object. The information processing device 1 is fixedly installed, for example, housed together with the lidar 3 in a housing. The information processing device 1 may be provided as an electronic control device for the lidar 3 and integrated with the lidar 3, or it may be provided at a location separate from the lidar 3 while being able to communicate with the lidar 3 for data.

[0023] LIDA 3 discretely measures the distance to objects in the external environment by emitting a pulsed infrared laser while changing its angle within a predetermined angular range in the horizontal and vertical directions. In this case, LIDA 3 has an irradiation unit that irradiates laser light while changing the irradiation direction (i.e., scanning direction), a light receiving unit that receives reflected light (scattered light) of the irradiated laser light, and an output unit that outputs data based on the received signal output by the light receiving unit. LIDA 3 then generates point cloud data that shows the distance to the object irradiated by the pulsed laser (measured distance) and the received intensity of the reflected light (reflection intensity value) for each measurement direction (i.e., the direction of pulsed laser emission), with LIDA 3 as the reference point. In this case, LIDA 3 calculates the time from the emission of the pulsed laser until the light receiving unit detects the reflected light as the time of flight of light, and determines the measured distance corresponding to the calculated time of flight. Hereafter, the point cloud data obtained from one measurement across the entire field of view Rv will be considered as one frame of point cloud data.

[0024] Here, the point cloud data can be considered as an image where each measurement direction is represented by a pixel, and the measured distance and reflectance value for each measurement direction are represented by the pixel value. In this case, the pulse laser emission direction differs in elevation and depression angles for the vertical arrangement of pixels, and the pulse laser emission direction differs in horizontal angles for the horizontal arrangement of pixels. For each pixel, a coordinate value in a three-dimensional coordinate system based on the lidar 3 is determined based on the corresponding pair of emission direction and measured distance. Hereafter, this coordinate value will be denoted as (x, y, z), where the pair of x and y coordinates represents the horizontal position, and the z coordinate represents the vertical position.

[0025] Note that LIDA 3 is not limited to the scanning type LIDA described above, but may also be a flash type LIDA that generates 3D data by diffusing laser light into the field of view of a 2D array sensor. Hereafter, the point measured by the pulsed laser emitted from the irradiation unit (and its measurement data) will also be referred to as the "measured point". LIDA 3 is an example of a "measurement device" in the present invention.

[0026] Sensor group 2 may include various external and / or internal sensors in addition to the lidar 3. For example, sensor group 2 may include a GNSS (Global Navigation Satellite System) receiver necessary for generating location information.

[0027] (2) Configuration of an information processing device Figure 3 is a block diagram showing an example of the hardware configuration of the information processing device 1 according to the first embodiment. The information processing device 1 mainly comprises an interface 11, a memory 12, and a controller 13. These elements are interconnected via a bus line.

[0028] Interface 11 performs interface operations related to the exchange of data between the information processing device 1 and external devices. In this embodiment, interface 11 acquires output data from the sensor group 2, such as the lidar 3, and supplies it to the controller 13. Interface 11 may be a wireless interface such as a network adapter for wireless communication, or it may be a hardware interface for connecting to external devices via cables, etc. Interface 11 may also perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0029] Memory 12 is composed of various volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, and flash memory. Memory 12 stores programs for the controller 13 to execute predetermined processes. Note that the programs executed by the controller 13 may be stored in storage media other than memory 12.

[0030] Furthermore, memory 12 stores information necessary for the controller 13 to perform predetermined processing. For example, in this embodiment, memory 12 stores lane map LM and tracked object information IT.

[0031] The lane map LM is a map that represents the position of lanes and the direction of movement within those lanes. For example, the lane map LM includes information indicating whether or not a grid cell is a lane for each cell in a virtual grid (also called a "grid cell") defined by meshing a two-dimensional space on a horizontal plane at predetermined intervals in each dimension. Furthermore, grid cells that are lanes also contain lane identification information (lane ID) and information indicating the direction of movement within the lane map LM. In this case, if there are multiple lanes on one side, a common lane ID may be assigned to lanes going in the same direction, or different lane IDs may be assigned to different lanes going in the same direction. The lane map LM may be part of the map data, or it may be generated based on the second embodiment described later.

[0032] The tracked object information IT is information indicating the results of object tracking by the information processing device 1, and is generated or updated with each frame cycle. For example, in the tracked object information IT, identification information (tracked object ID) is assigned to objects that the information processing device 1 has tracked or is currently tracking (also called "tracked objects"). Here, if the same tracked object exists at different processing times, the tracked object is managed by the same tracked object ID. In addition to the tracked object ID mentioned above, the tracked object information IT includes, for example, the time information when the tracked object was detected, the location information of the tracked object, and the classification information of the tracked object.

[0033] The controller 13 includes one or more processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit), and controls the entire information processing device 1. In this case, the controller 13 executes various processes described later by running programs stored in memory 12, etc. Functionally, the controller 13 has a segment detection unit 15, an object detection unit 16, a prediction unit 17, and an information output unit 18.

[0034] The segment detection unit 15 extracts adjacent data sets (also called "segments") from the point cloud data generated in the frame period corresponding to the current processing time. In this case, the segment detection unit 15 performs processing to extract one or more segments from the point cloud data based on any clustering technique, such as Euclidean distance clustering. In addition, as a preprocessing step before the above-mentioned segment extraction, the segment detection unit 15 performs processing to remove data of measured points corresponding to stationary objects (so-called background points or stationary points) from the point cloud data, and processing to remove data of measured points corresponding to noise (so-called noise points) from the point cloud data.

[0035] The object detection unit 16 groups the segments detected by the segment detection unit 15 into one or more segments for each object, and detects each group's segments as clusters representing tracked objects. In this case, the object detection unit 16 refers to the lane map LM and performs the following processes: detecting clusters of segments corresponding to tracked objects already detected at past processing times (also called "tracked object detection process"), and detecting clusters of segments corresponding to objects not detected at past processing times as new tracked objects (also called "new object detection process"). The object detection unit 16 then updates the tracked object information IT based on the tracked object detection process and the new object detection process.

[0036] The object detection unit 16 may classify the detected tracked objects and determine the class of each tracked object. Here, the class is the type of object that may be detected by the Rider 3, such as a vehicle (including a bicycle), a pedestrian, or a sign. In this case, the object detection unit 16 performs the above classification based, for example, on the size and / or shape of the segment of the tracked object. In this case, prior information regarding the size and / or shape of each class is stored in the memory 12 or the like, and the object detection unit 16 performs the classification based on the size and / or shape of each segment by referring to this prior information. The object detection unit 16 may also classify tracked objects using a model trained based on deep learning (neural network). In this case, the above model is trained in advance to output the classified class of the tracked object when point cloud data representing the segment of the tracked object is input. Furthermore, the object detection unit 16 may detect only objects of a specific class (e.g., automobiles) as tracked objects. In this case, the object detection unit 16 may detect segments representing objects belonging to a specific class as segments representing tracked objects, and remove segments representing objects that do not belong to that class.

[0037] The prediction unit 17 predicts the position and velocity of the tracked object in the tracked object information IT. In this case, for example, the prediction unit 17 inputs the representative position of the object being tracked at the current processing time into an arbitrary object tracking model, and obtains the prediction results of the position and velocity of the target object from the model. The object tracking model may be a model based on a Kalman filter or a model based on deep learning. The representative position in this case may be the centroid position of all segments of the target object, or it may be the position of a specific part of the object identified after estimating the entire area of ​​the target object using an object model or the like. The prediction unit 17 may also set a bounding box surrounding the segments of the object and perform tracking based on the bounding box to predict the position and velocity of the object. The prediction unit 17 may also assume that any object newly detected by the object detection unit 16 at the current processing time is moving at a predetermined speed in the direction of movement of the lane in which the object is located.

[0038] The information output unit 18 outputs information about objects detected by the lidar 3. In this case, the information output unit 18 may output, for example, information about the number of objects detected within a predetermined period, or information about the position and / or speed of each object. In this case, as one form of output, the information output unit 18 may store the above information in the memory 12, or transmit it via the interface 11 to a server device that collects information about road conditions.

[0039] The controller 13 functions as a "first acquisition means," a "second acquisition means," a "segment extraction means," an "object detection means," and a computer that executes a program.

[0040] Figure 4 is an example of a flowchart showing an overview of the processes executed by the controller 13 in the first embodiment. The controller 13 repeatedly executes the processes in this flowchart at predetermined processing cycles (for example, the frame cycle of the rider 3).

[0041] The controller 13 acquires point cloud data measured by the lidar 3 via the interface 11 (step S11). Then, the segment detection unit 15 of the controller 13 detects segments based on the point cloud data acquired in step S11 (step S12). In this case, the segment detection unit 15 also performs noise point and background point removal processing.

[0042] Next, the object detection unit 16 performs tracking object detection processing based on the lane map LM (step S13). This allows the object detection unit 16 to detect clusters of segments corresponding to the object being tracked. Furthermore, if the object detection unit 16 cannot detect a tracked object detected at a previous processing time within the segments obtained in step S12, it considers that the tracked object has disappeared. Details of the tracking object detection processing will be explained in detail with reference to Figures 5 and 6. In addition, the object detection unit 16 performs new object detection processing based on the lane map LM (step S14). This allows the object detection unit 16 to detect clusters of segments corresponding to objects that were not detected at previous processing times. Details of this process will be explained in detail with reference to Figures 7 and 8. Through the tracking object detection processing and new object detection processing, segments are clustered (i.e., grouped) for each object. Note that the object detection unit 16 may perform tracking processing only for objects belonging to a specific class (e.g., automobiles).

[0043] Next, the prediction unit 17 predicts the position and velocity of the object being tracked based on the cluster of segments for each object being tracked (step S15). After that, the information output unit 18 outputs information related to the prediction results from the prediction unit 17.

[0044] Furthermore, the processing performed by the controller 13 is not limited to being implemented by software through a program, but may also be implemented by a combination of hardware, firmware, and software. Additionally, the processing performed by the controller 13 may be implemented using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the program executed by the controller 13 in this embodiment may be implemented using this integrated circuit.

[0045] (3) Tracking object detection process Next, the details of the tracking object detection process will be described. In the tracking object detection process, the object detection unit 16 detects a cluster of segments corresponding to the tracking object at the current processing time, based on the position of the tracking object predicted by the prediction unit 17 at the previous processing time. In this case, the object detection unit 16 prohibits detecting segments that exist across lanes different from the predicted position of the tracking object as part of the tracking object, based on the lane map LM. This effectively suppresses the detection of segments detected on the opposite lane of the lane where the tracking object is located as part of the tracking object.

[0046] Figure 5 shows the lane region based on the lane map LM and the segments s1 to s5 detected on that lane. Here, the region within the field of view Rv of lane A and the region within the field of view Rv of the opposite lane (i.e., the lane with the opposite direction of movement) lane B are shown. In addition, there is a vehicle 7A bordered by a rectangular frame in lane A, and there is a vehicle 7B bordered by a rectangular frame in lane B. Here, vehicle 7A is a tracked object that was already detected in a previous processing time and is moving from bottom to top, while vehicle 7B is a new object that was not detected because it was outside the field of view Rv in a previous processing time and is moving from top to bottom. Mark t1 represents the predicted position of vehicle 7A (here, a representative position defined as the center of the front of the vehicle) at the current processing time, as predicted in the previous processing time.

[0047] Here, in the tracking object detection process, the object detection unit 16 detects segments corresponding to the vehicle 7A, which is the tracking object detected at the previous processing time, based on the lane map LM. In this case, for example, the object detection unit 16 detects segments as the tracking object, excluding segments belonging to lanes in different directions of travel from segments that exist within a predetermined distance from the predicted position of the tracking object. In the example in Figure 5, first, the object detection unit 16 detects segments s1 to s5 that exist within a predetermined distance from the predicted position of the vehicle 7A (see mark t1). Next, the object detection unit 16 detects segment s5, which is located on lane B in a different direction of travel from lane A, from among the detected segments s1 to s5, and excludes segment s5 to detect a cluster of segments representing the vehicle 7A. In this case, the object detection unit 16 detects segments s1 to s4 as a cluster of segments representing the vehicle 7A.

[0048] Furthermore, when determining segments to exclude based on the lane map LM, the object detection unit 16 may, instead of using the lane where the predicted position exists as a reference, exclude segments located in lanes with a different direction of travel than the lane where the segment closest to the predicted position (in this case, segment s1) exists. In addition, the object detection unit 16 may prohibit detecting segments located in different lanes as the same tracked object, regardless of whether the lane directions of travel are the same or not.

[0049] In this way, the object detection unit 16 prevents the detection of segments that span across lanes as the same tracked object, and accurately detects clusters of segments that represent tracked objects.

[0050] Here, let's consider the case where we select a cluster of segments representing the tracked object based solely on the distance between the predicted position of the tracked object and each segment. In this case, since segment s4 is farther from the predicted position than segment s5, it is not possible to detect segment s4 as a segment representing the tracked object while excluding segment s5, regardless of how the predetermined distances mentioned above are set.

[0051] Figure 6 is an example of a flowchart showing the detailed steps of the tracking object detection process performed by the object detection unit 16. In step S13 of the flowchart in Figure 4, the object detection unit 16 executes the flowchart shown in Figure 6.

[0052] First, the object detection unit 16 calculates the distance for all combinations of the predicted position of a tracked object that was detected as a tracked object in a past processing time and whose predicted position at the current processing time has been calculated, and the segment detected by the segment detection unit 15 based on the point cloud data at the current processing time (step S21). For example, if there are 3 tracked objects and 10 segments, there are 30 (=3 × 10) pairs of tracked objects and segments, and the distance between the predicted position and the segment corresponding to each pair is calculated. In this case, for example, the object detection unit 16 calculates the distance based on the centroid position of the segment.

[0053] Next, the object detection unit 16 overwrites the distance between a tracked object and a segment pair that crosses different lanes with a second predetermined value (step S22). In this case, for example, based on the lane map LM, the object detection unit 16 sets the distance of all combinations of tracked objects and segments that correspond to segments located in lanes different from the predicted position of the tracked object as the second predetermined value. In this case, the second predetermined value will be greater than or equal to the first predetermined value used in step S25, which will be described later. This prevents the object detection unit 16 from detecting a segment located in a lane different from the predicted position as a tracked object. Note that, taking into account lane changes by vehicles, the object detection unit 16 does not need to perform the above overwrite process if a segment located in a lane different from the predicted position of the tracked object is in the same lane as the predicted position of the tracked object and is moving in the same direction.

[0054] The object detection unit 16 then performs the following steps S23 to S29 for each tracked object. First, the object detection unit 16 selects the pair of tracked objects and segments with the shortest distance (step S23). In this case, the object detection unit 16 determines that a pair to select exists (step S24; No) and proceeds to step S25. The object detection unit 16 then determines whether the distance of the selected pair is greater than or equal to a first predetermined value (step S25). Here, the first predetermined value is determined by considering, for example, the size of the tracked object (e.g., a car) and error information of the predicted position of the tracked object output by a Kalman filter, etc. If the distance of the selected pair is greater than or equal to the first predetermined value (step S25; Yes), the object detection unit 16 completes the correspondence between the target tracked object and the segment (step S29). This determines the cluster of segments corresponding to the target tracked object at the current processing time.

[0055] On the other hand, if the distance between the selected pairs is less than a first predetermined value (step S25; No), the object detection unit 16 determines whether the segments of the selected pairs have already been associated with another tracked object (step S26). If the segments of the selected pairs have already been associated with another tracked object (step S26; Yes), the object detection unit 16 determines that the selected pairs should not be associated and proceeds to step S28. On the other hand, if the segments of the selected pairs have not been associated with any other tracked object (step S26; No), the object detection unit 16 associates the segments with the tracked object of the selected pairs (step S27). That is, the object detection unit 16 detects the segments of the selected pairs as segments representing the tracked object of the selected pairs.

[0056] Then, in step S28, the object detection unit 16 selects the next shortest distance pair (step S28). If there are no pairs to select (step S24; Yes), the object detection unit 16 completes the correspondence between the target tracked object and the segment (step S29). This determines the cluster of segments corresponding to the target tracked object at the current processing time.

[0057] (4) Novel object detection process Next, the details of the new object detection process will be described. In the new object detection process, the object detection unit 16 clusters segments that could not be associated with a tracked object in the tracked object detection process, and detects the clusters of segments formed by the clustering as new tracked objects. In this case, the object detection unit 16 prohibits detecting segments that span different lanes as the same object, based on the lane map LM. This effectively suppresses the detection of clusters of segments that span opposite lanes, etc., as the same object.

[0058] Figure 7 shows the lane region based on the lane map LM and the segments s11 to s17 detected on that lane. Here, the region within the field of view Rv of lane A and the region within the field of view Rv of the opposite lane (i.e., the lane with the opposite direction of movement) lane B are shown. In addition, there is a vehicle 7C bordered by a rectangular frame in lane A, and there is a vehicle 7D bordered by a rectangular frame in lane B. For the sake of explanation, both vehicle 7C and vehicle 7D are assumed to be objects that were not detected at previous processing times.

[0059] In this case, segment s14 is further away from segments s11-s13 than from segments s15-s17. Therefore, when performing any clustering process, it is difficult to cluster segments s11-s14, which correspond to vehicle 7C, and segments s15-s17, which correspond to vehicle D, separately.

[0060] Taking the above into consideration, the object detection unit 16 prohibits detecting segments on lane A and segments on lane B, which is the opposite lane of lane A, as the same object based on the lane map LM. This makes it possible for the object detection unit 16 to cluster segments s11 to s14 corresponding to vehicle 7C and segments s15 to s17 corresponding to vehicle D separately.

[0061] Figure 8 is an example of a flowchart showing the detailed steps of the new object detection process performed by the object detection unit 16. In step S14 of the flowchart in Figure 4, the object detection unit 16 executes the flowchart shown in Figure 8.

[0062] First, the object detection unit 16 excludes from the current processing time the segments generated by the segment detection unit 15 that were associated with a tracked object in the previously executed tracked object detection process (step S31). Next, the object detection unit 16 generates any two pairs of segments to be processed and calculates the distance for each pair (step S32). In this case, if the number of segments to be processed is N (where N is an integer of 2 or more), N C calculates the distance between pairs of two segments. If there is one or fewer segments to be processed, the process proceeds to step S39. The object detection unit 16 may also detect segments that could not be associated in either the tracking object detection process or the new object detection process as segments representing new tracking objects.

[0063] Next, the object detection unit 16 overwrites the distance between pairs of segments that span different lanes with a second predetermined value (step S33). This prevents the object detection unit 16 from detecting segments located on different lanes than the predicted position as the same object. However, taking into account lane changes by vehicles, the object detection unit 16 does not need to perform the above overwrite process on pairs of segments located on lanes that are moving in the same direction, even if those segments are located on different lanes than the predicted position of the tracked object.

[0064] Next, the object detection unit 16 selects the pair of segments with the shortest distance (step S34). In this case, the object detection unit 16 determines that a pair to be selected exists (step S35; No) and proceeds to step S36. The object detection unit 16 then determines whether the distance of the selected pair is greater than or equal to a first predetermined value (step S36). Here, the first predetermined value is determined, for example, by taking into account the size of the tracked object (e.g., a car). If the distance of the selected pair is greater than or equal to the first predetermined value (step S36; Yes), the object detection unit 16 completes the association between the newly detected object and the segment from the current processing time (step S39). This determines the cluster of segments at the current processing time corresponding to the new tracked object.

[0065] On the other hand, if the distance between the selected pairs is less than a first predetermined value (step S36; No), the segments of the selected pairs are set as segments representing the same object (step S37). That is, the object detection unit 16 considers the selected pairs of segments to be the same cluster representing the same object.

[0066] Then, in step S38, the object detection unit 16 selects the next shortest distance pair (step S38). If there is no pair to select (step S35; Yes), the object detection unit 16 performs a process to determine the set of segments corresponding to an object from the pairs of segments set as the same object and the remaining individual segments (step S39). Step S39 corresponds to the process of determining which segments (one or more) correspond to each of the several objects that have not been tracked so far. For example, suppose there are segments "s1" to "s5" that have not been associated with the tracked object, and in the process prior to step S39, the pairs "s1 and s2", "s2 and s3", and "s1 and s3" have been set as segments of the same object. In this case, the object detection unit 16 determines that "s1, s2, and s3", "s4", and "s5" each correspond to different objects (i.e., there are three newly detected objects).

[0067] (5) Variation The rider unit 100 may be mounted on a vehicle. In this case, the rider 3 is provided on the vehicle, and the information processing device 1 is an on-board device of the vehicle or an electronic control unit (ECU) built into the vehicle. In this case, the information processing device 1 detects objects around the vehicle on which the rider unit 100 is mounted (for example, surrounding vehicles) based on point cloud data generated by the rider 3. Even in this case, the information processing device 1 can accurately perform tracking of objects around the vehicle on which the rider unit 100 is mounted.

[0068] Furthermore, the lane map LM may not be limited to the lanes through which vehicles travel, but may also be a map showing movement areas such as bicycle paths and sidewalks (and the direction of movement if a direction of movement is specified). Even in this case, the information processing device 1 can perform highly accurate object tracking by prohibiting segments existing in different movement areas from being considered the same object based on the lane map LM. The lane map LM is an example of "movement area information".

[0069] As described above, the controller 13 of the information processing device 1 according to this embodiment mainly functions as a first acquisition means, a second acquisition means, and an object detection means. The first acquisition means acquires point cloud data, which is a collection of data for each point measured by the measuring device. The second acquisition means acquires movement area information regarding the movement area of ​​an object on a horizontal plane within the measurement range of the measuring device. Based on the movement area information, the object detection means detects clusters of data for each object from the point cloud data. As a result, the information processing device 1 can accurately track objects.

[0070] <Second Example> In the second embodiment, an information processing device 1A that generates the tracking object information IT described in the first embodiment will be described. The information processing device 1A may be the same device as the information processing device 1 in the first embodiment, or it may be a different device from the information processing device 1.

[0071] (1) Configuration of an information processing device Figure 9 is a block diagram showing an example of the hardware configuration of the information processing device 1A according to the second embodiment. The information processing device 1 mainly comprises an interface 11A, a memory 12A, and a controller 13A. These elements are interconnected via a bus line.

[0072] Interface 11A performs interface operations related to the exchange of data between the information processing device 1A and external devices. In this embodiment, interface 11A acquires output data from sensors such as the lidar 3A and supplies it to the controller 13A. Interface 11A may also perform interface operations with various peripheral devices such as input devices, display devices, and sound output devices.

[0073] Memory 12A is composed of various volatile and non-volatile memories such as RAM, ROM, hard disk drive, and flash memory. Memory 12A stores a program for controller 13A to perform predetermined processing.

[0074] Furthermore, memory 12A stores information necessary for controller 13A to perform predetermined processing. For example, in the second embodiment, memory 12A stores movement history information IM.

[0075] Movement history information (IM) is information about the movement of tracked objects and is generated based on the object tracking results. For example, movement history information (IM) is information that at least indicates the position and direction of movement of each tracked object in the time series for each frame period in which point cloud data is obtained. In addition, movement history information (IM) may further include information indicating the classification (class) of each tracked object.

[0076] The controller 13A includes one or more processors such as a CPU, GPU, and TPU, and controls the entire information processing device 1. In this case, the controller 13A executes various processes described later by running programs stored in memory 12A, etc. Functionally, the controller 13A has a segment detection unit 15A, a tracking unit 17A, and a lane map generation unit 19A.

[0077] The segment detection unit 15A extracts segments that are adjacent to each other from the point cloud data generated in the frame period corresponding to the current processing time. The processing performed by the segment detection unit 15A is the same as the processing performed by the segment detection unit 15 in the first embodiment.

[0078] The tracking unit 17A performs object tracking based on time-series segments. In this case, the tracking unit 17A uses time-series segments and an arbitrary object tracking model to determine whether object segments detected in consecutive frame periods represent the same object. The object tracking model may be a Kalman filter-based model or a deep learning-based model. In this case, the tracking unit 17A may determine a representative point from the segments and perform tracking based on the representative point. In this case, the representative point may be the centroid of the segment or a measurement point corresponding to a specific part of the object. In another example, the tracking unit 17A may set a bounding box for the segments and perform tracking based on the bounding box.

[0079] Furthermore, the tracking unit 17A generates movement history information IM based on the object tracking results. In this case, for example, the tracking unit 17A generates information (records) indicating at least the position and direction of movement of each tracked object identified by the object tracking results for each frame period, and stores them in memory 12A as movement history information IM. The tracking unit 17A may further include classification information of the tracked objects in the movement history information IM.

[0080] Furthermore, the tracking unit 17A may generate clusters of one or more segments based on the distance between segments, similar to the object detection unit 16 in the first embodiment, and detect each generated cluster as a tracked object. Also, the tracking unit 17A may classify the detected tracked objects and determine the class of each tracked object, similar to the object detection unit 16 in the first embodiment.

[0081] The lane map generation unit 19A generates a lane map based on the movement history information IM. In this case, the lane map generation unit 19A considers the field of view Rv as a two-dimensional space viewed from directly above, and defines a virtual grid by meshing the two-dimensional space at predetermined intervals in each dimension. The lane map generation unit 19A then aggregates the movement direction of the tracked object based on the movement history information IM for each grid cell, and estimates the position of the lane and the direction of movement on the lane based on the aggregation results. The lane map generation unit 19A then stores the estimated results of the position of the lane and the direction of movement on the lane for each grid cell as a lane map in the memory 12A. This lane map is used as the lane map LM in the first embodiment.

[0082] The controller 13A functions as an "acquisition means," a "first generation means," a "second generation means," and a computer or the like that executes the program.

[0083] (2) Lane map generation Next, we will explain a specific example of a method for estimating the position of each lane and the direction of movement within the lane for each grid cell.

[0084] Figure 10 shows the movement trajectory of a tracked object on a virtually defined grid in a two-dimensional space viewed from directly above the field of view Rv, indicated by arrows 81.

[0085] The lane map generation unit 19A identifies the grid squares that the tracked object has passed through and the direction of movement within those grid squares, based on the movement history information IM, and performs aggregation of the movement direction of the tracked object for each grid square for all tracked objects. For example, in the example in Figure 10, the lane map generation unit 19A detects each grid square that overlaps with the arrow 81 as a grid square that the tracked object has passed through, and counts that there was movement in the direction of movement indicated by the arrow 81 within that grid square. In this case, the lane map generation unit 19A may, for example, refer to the classification information of the movement history information IM and perform the above aggregation based only on the movement history of a specific class (e.g., automobiles).

[0086] Figure 11(A) is an enlarged view of the grid cell 80, where the direction of movement of the tracked object is indicated by arrow 81A. Figure 11(B) is a histogram summarizing the direction of movement in the grid cell 80.

[0087] In the case of the tracked object shown in Figure 10, as shown in Figure 11(A), the lane map generation unit 19A counts that the tracked object has moved once in the direction of movement of 350 degrees in the grid cell 80. Then, based on the movement history information IM, the lane map generation unit 19A aggregates the direction of movement of the tracked object that has passed through the grid cell 80 and generates the histogram shown in Figure 11(B). The lane map generation unit 19A then considers 310 degrees, which is the most frequent, as the direction of movement on the grid cell 80. Alternatively, instead of determining the most frequent angle as the direction of movement, the lane map generation unit 19A may determine a representative direction of movement in the grid cell 80 based on any statistical method (for example, the median after excluding outliers, the mean, etc.). Furthermore, the lane map generation unit 19A determines that grid cells where the number of moved objects is less than a predetermined number are not lanes.

[0088] Figure 12 is a diagram that clearly shows the direction and frequency of movement in each grid cell within the field of view Rv. In Figure 12, the direction of the arrow indicates the most frequently occurring direction of movement, and the thickness of the arrow increases with the number of occurrences. Also, arrows are not represented in grid cells where the number of times an object has passed through is less than a predetermined number. Furthermore, in Figure 12, the actual areas of existence of lanes X and Y are indicated by dashed lines. Note that lanes X and Y are assumed to be opposite lanes.

[0089] Generally, when a grid is located on a roadway (excluding intersections), it is assumed that movement directions corresponding to the actual lanes will appear frequently. Furthermore, it is assumed that objects pass more frequently in the center of the actual lanes, and less frequently between lanes (for example, directly above the center line). These characteristics are then thought to manifest on the grid in two-dimensional space according to the actual arrangement of the roadway. In the example in Figure 12, thick arrows appear in the center of both lane X and lane Y, indicating a high frequency of object passage, while no arrows appear between lane X and lane Y due to the low frequency of object passage.

[0090] Taking the above into consideration, the lane map generation unit 19A generates a lane map based on the direction of movement calculated for each grid cell. In this case, the lane map generation unit 19A performs clustering based on a representative direction of movement and the position of the grid cell for grid cells where the number of moved objects exceeds a predetermined number. In this case, for example, the lane map generation unit 19A performs clustering where adjacent grid cells with similar directions of movement are grouped into the same cluster. The lane map generation unit 19A then recognizes the clusters of grid cells formed by the clustering as lanes and determines a lane ID and direction of movement for each grid cell forming a cluster. The lane map generation unit 19A then generates a lane map showing the lane ID and direction of movement for each grid cell determined to be a lane.

[0091] Here, the method for determining lane boundaries will be explained in detail with reference to Figures 13(A) and 13(B).

[0092] Figure 13(A) shows an overview of the first method for determining lane boundaries. In the first method, the lane map generation unit 19A determines that grid squares where frequently occurring directions of movement are opposite each other correspond to lane boundaries. In Figure 13(A), the directions of movement of the grid squares to the left of line 90 all roughly indicate the first direction (approximately 330 degrees), and the directions of movement of the grid squares to the right of line 90 all roughly indicate the second direction (approximately 150 degrees). Thus, the directions of movement of objects are opposite with line 90 as the boundary. Therefore, the lane map generation unit 19A estimates that the area near line 90 (grid squares overlapping with line 90) corresponds to lane boundaries.

[0093] Figure 13(B) shows an overview of the second method for determining lane boundaries. In the second method, the lane map generation unit 19A determines that a grid cell with a minimum frequency of passage, or a grid cell located between grid cells with a maximum frequency of passage, corresponds to a lane boundary. As shown in Figure 13(B) by the presence or absence and thickness of the arrows, the grid cell enclosed by dashed line frames 91A and 91C has a passage frequency higher than a predetermined degree, while the grid cell enclosed by dashed line frame 91B, which lies between them, has a passage frequency lower than a predetermined degree. Therefore, in this case, the lane map generation unit 19A estimates that the grid cell enclosed by dashed line frame 91B corresponds to a lane boundary. In this way, the lane map generation unit 19A can estimate that a lane boundary exists between high-frequency and low-frequency regions based on the distribution of high-frequency and low-frequency regions, even by looking at the frequency of object passage (i.e., without looking at the direction of movement).

[0094] Furthermore, the lane map generation unit 19A may use constraints such as lanes being straight lines or smooth curves to equalize lane information between grid cells, or to interpolate information in grid cells with a small amount of data (number of passes) using adjacent grid cells.

[0095] (3) Variation In addition to generating a lane map relating to the lanes through which vehicles travel, the information processing device 1A may also generate a map showing movement areas (and directions of movement) such as sidewalks and bicycle paths. Note that in cases where the direction of movement of an object is not fixed to a single direction, such as on a sidewalk, there will be multiple peaks in the histogram representing different directions of movement. In this case, the information processing device 1A may, for example, not define a representative direction of movement for grid cells in the histogram where multiple directions of movement occur at a frequency exceeding a predetermined degree, and may define multiple representative directions of movement corresponding to multiple peaks.

[0096] In the above-described embodiment, the program can be stored using various types of non-transitory computer-readable medium and supplied to a computer, such as a controller. Non-transitory computer-readable medium includes various types of tangible storage medium. Examples of non-transitory computer-readable medium include magnetic storage medium (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage medium (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).

[0097] The present invention has been described above with reference to the examples, but the present invention is not limited to the above examples. Various modifications to the structure and details of the present invention can be made as understood by those skilled in the art within the scope of the present invention. That is, the present invention includes the full disclosure, including the claims, and of course, various modifications and alterations that those skilled in the art could make in accordance with the technical idea. Furthermore, the above-mentioned patent and non-patent documents and other disclosures cited herein are incorporated by reference. [Explanation of Symbols]

[0098] 1. 1A Information Processing Device 2 Sensor Groups 3 Riders 100 Rider Unit

Claims

1. A first acquisition means for acquiring point cloud data, which is a collection of data for each point measured by a fixedly installed measuring device, A second acquisition means for acquiring movement area information, which is a map representing the position of the lane on a horizontal plane and the direction of movement on the lane within the measurement range of the measuring device, which includes multiple lanes. A vehicle detection means that detects clusters of data for each vehicle on the multiple lanes from the point cloud data based on the aforementioned movement area information, Equipped with, The vehicle detection means prohibits the detection of data existing on lanes with different directions of movement as clusters representing the same vehicle.

2. The information processing apparatus according to claim 1, wherein the vehicle detection means prohibits detecting the data existing on different lanes as clusters representing the same vehicle.

3. The information processing apparatus according to claim 2, wherein the vehicle detection means prohibits detecting the data on the first lane and the data on the second lane as clusters representing the same vehicle when the direction of movement of the first lane and the second lane indicated by the movement area information are different.

4. The information processing apparatus according to any one of claims 1 to 3, wherein the vehicle detection means detects the cluster corresponding to the tracked vehicle at the current processing time based on the predicted position of the tracked vehicle detected at a past processing time, and detects the cluster by excluding the data that is located on a lane different from the predicted position.

5. The information processing apparatus according to claim 4, wherein when the vehicle detection means detects a cluster corresponding to a vehicle newly detected at the current processing time based on the data which does not correspond to a vehicle detected at a past processing time, it prohibits detecting data with different lanes as clusters representing the same vehicle.

6. The system further includes a segment extraction means for extracting segments that represent the aforementioned data sets, The information processing apparatus according to any one of claims 1 to 5, wherein the vehicle detection means detects one or more segments for each vehicle as the cluster.

7. An acquisition means for acquiring point cloud data, which is a collection of data for each point measured by a fixedly installed measuring device, A first generation means generates movement history information representing the direction of movement of a vehicle that passed through a position on the horizontal plane within the measurement range of the measuring device, A second generation means for generating movement area information according to any one of claims 1 to 6 based on the aforementioned movement history information, An information processing device having

8. Computers A fixed measuring device acquires point cloud data, which is a collection of data for each individual point measured by the device. The measurement device acquires movement area information, which is a map representing the position of the lane on a horizontal plane and the direction of movement on the lane within the measurement range of the measuring device, which includes multiple lanes. Based on the aforementioned movement area information, clusters of data for each vehicle on the multiple lanes are detected from the point cloud data. The detection of data existing on lanes with different directions of movement as the same vehicle cluster is prohibited. Control method.

9. A fixed measuring device acquires point cloud data, which is a collection of data for each individual point measured by the device. The measurement device acquires movement area information, which is a map representing the position of the lane on a horizontal plane and the direction of movement on the lane within the measurement range of the measuring device, which includes multiple lanes. Based on the aforementioned movement area information, clusters of data for each vehicle on the multiple lanes are detected from the point cloud data. A program that causes a computer to perform a process that prohibits detecting the data existing on lanes with different directions of movement as the cluster representing the same vehicle.

10. A storage medium storing the program described in claim 9.

Images