Vehicle estimation device

The vehicle estimation device uses LIDAR to generate and superimpose three-dimensional point cloud data for accurate vehicle type classification, addressing inaccuracies in existing systems by enhancing dimension-based estimation.

JP2024157889AInactive Publication Date: 2024-11-08今井 龙一 +3
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023072523
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing vehicle classification systems struggle to accurately determine whether vehicles are large or small, especially when using two-dimensional images or three-dimensional point cloud data with potential data loss, leading to inaccuracies in vehicle type estimation.

Method used

A vehicle estimation device utilizing LIDAR to generate three-dimensional point cloud data, superimpose multiple sets to enhance accuracy, and employ an estimation model learned from rotated superimposed data to determine vehicle type based on dimensions.

Benefits of technology

Accurately distinguishes between large and small vehicles, improving estimation accuracy even in dark conditions by complementing data deficiencies through superimposition and using appropriate learning data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157889000001_ABST
    Figure 2024157889000001_ABST
Patent Text Reader

Abstract

To provide a vehicle estimation device capable of accurately determining a type of a vehicle.SOLUTION: A measuring instrument 2 measures a distance to each one of points on a front surface of a vehicle during traveling, so as to generate three-dimensional point group data of the vehicle based on the distances. In three-dimensional point group data superimposition processing 4, superimposition three-dimensional point group data is generated by superimposing a plurality of elements of three-dimensional point group data of the vehicle measured during traveling. In vehicle type estimation processing 6, the type of the vehicle is estimated based on the generated superimposition three-dimensional point group data. Since the type of the vehicle is estimated based on the superimposition three-dimensional point group data, compensation can be performed by superimposition even when there is loss in each element of the three-dimensional point group data, so as to enable more correct estimation.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for estimating the type of a vehicle. [Background technology]

[0002] For traffic volume surveys and facility management, it is necessary to determine the type of automobile. For example, in traffic volume surveys, investigators visually classify passing vehicles by type at intersections and count the traffic volume for each classification.

[0003] In recent years, deep learning has been used to distinguish the type of object captured in an image (for example, the YOLO model). This object detection model can be trained on various vehicles, and a device that can distinguish the type of vehicle using the trained model can be realized.

[0004] However, when a device based on the above-mentioned trained model is applied to traffic volume surveys, it may be possible to estimate the type of vehicle, but it is not possible to estimate whether the vehicle is large or small. In particular, it is difficult to distinguish between large and small vehicles (e.g., large trucks and small trucks) of the same type of vehicle (e.g., trucks).

[0005] Therefore, the inventors have already proposed a system that can estimate the type of vehicle, including large and small vehicles (Patent Document 1). This system can conduct traffic volume surveys based on captured images, which can greatly improve the efficiency of traffic volume surveys. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 2022-163387 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the system of Patent Document 1, since the judgment is based on a two-dimensional image, it is not possible to judge the size accurately, and there is room for improvement in the accuracy of judging whether the vehicle is a large vehicle or a small vehicle.

[0008] On the other hand, even if three-dimensional point cloud data obtained by laser ranging such as LIDAR is used, there is a risk that missing points will be unavoidable, making it impossible to achieve sufficient determination accuracy.

[0009] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide a vehicle estimation device that can solve the above problems and determine the type of a vehicle with high accuracy. [Means for solving the problem]

[0010] Independently applicable features of the present invention are listed below.

[0011] (1)-(3) A vehicle estimation device according to the present invention includes a measuring instrument that measures distances to each point on the surface of a moving vehicle to generate three-dimensional point cloud data, obtains three-dimensional point cloud data at multiple positions for the moving vehicle from the measuring instrument, overlays the multiple sets of three-dimensional point cloud data to generate superimposed three-dimensional point cloud data, and estimates the type of vehicle based on the superimposed three-dimensional point cloud data.

[0012] Therefore, even if the three-dimensional point cloud data has defects, the accuracy of vehicle estimation can be improved.

[0013] (4) The vehicle estimation device according to the present invention is characterized in that the measuring instrument is a LIDAR.

[0014] Therefore, vehicle location estimation can be performed even in dark places such as at night.

[0015] (5) The vehicle estimation device according to the present invention is characterized in that, in generating the superimposed three-dimensional point cloud data, multiple sets of three-dimensional point cloud data are superimposed so that the surface shapes on the side closer to the measuring instrument match.

[0016] Therefore, the overlapping can be performed with high accuracy.

[0017] (6) The vehicle estimation device of the present invention is characterized in that the type of vehicle is estimated by using at least any one of the length, width, and height dimensions of the overlaid 3D point cloud data to determine whether the vehicle is at least a large vehicle or a small vehicle.

[0018] Therefore, it is possible to accurately determine whether the object is large or small.

[0019] (7) The vehicle estimation device of the present invention is characterized in that the vehicle type is estimated by providing the overlaid three-dimensional point group data to an estimation model trained on the basis of the overlaid three-dimensional point cloud data.

[0020] Therefore, it is possible to infer the type of vehicle, which cannot be determined from dimensions alone.

[0021] (8) The vehicle estimation device of the present invention is characterized in that learning is performed using learning data generated from a plurality of superimposed three-dimensional point cloud data obtained by rotating superimposed three-dimensional point cloud data obtained by actual measurements.

[0022] Therefore, appropriate learning data can be obtained efficiently.

[0023] The term "apparatus" is a concept that includes not only one consisting of one computer, but also one consisting of multiple computers connected via a network, etc. Therefore, when the means of the present invention (or a part of the means) is distributed among multiple computers, these multiple computers correspond to the apparatus.

[0024] The term "program" is a concept that includes not only programs that can be executed directly by a CPU, but also programs in source format, compressed programs, encrypted programs, programs that work in conjunction with an operating system to perform their functions, and the like. [Brief description of the drawings]

[0025] [Figure 1] 1 illustrates an overall configuration of a vehicle estimation device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing the installation status of LIDAR 28. [Diagram 3] FIG. 1 is a diagram showing a hardware configuration. [Figure 4] 4 is a flowchart of a vehicle estimation program 26. [Diagram 5] FIG. 1 is a diagram showing three-dimensional point cloud data and a boundary box. [Figure 6] FIG. 2 is a diagram showing three-dimensional point cloud data of a vehicle part. [Figure 7] 4 is a flowchart of a vehicle estimation program 26. [Figure 8] FIG. 13 is a diagram showing superimposed three-dimensional point cloud data and a boundary box 80. [Figure 9] FIG. 2 is a diagram showing types of vehicles. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] 1. Overall process flow 1 shows the overall configuration of a vehicle estimation device according to an embodiment of the present invention. A measuring instrument 2 measures the distance to each point on the surface of a traveling vehicle, and generates three-dimensional point cloud data of the vehicle based on the distance. In a three-dimensional point cloud data superposition process 4, multiple three-dimensional point cloud data of the vehicle measured while traveling are superposed to generate superposed three-dimensional point cloud data.

[0027] In the vehicle type estimation process 6, the vehicle type is estimated based on the generated superimposed three-dimensional point cloud data.

[0028] As described above, the type of vehicle is estimated based on the overlaid three-dimensional point cloud data. Therefore, even if there are gaps in each piece of three-dimensional point cloud data, the data can be complemented by overlaying, allowing for a more accurate estimation.

[0029] 2. Hardware configuration 2 shows the installation state of a measuring instrument, a LIDAR (Laser Imaging Detection and Ranging) 28. In this embodiment, the LIDAR 28 is installed on a utility pole or the like on a road.

[0030] The LIDAR 28 irradiates a laser in all directions up, down, left and right within a range of a predetermined angle θ around a central axis 34, which is the installation direction of the device, and receives the reflected laser. The LIDAR 28 measures the time from irradiation to reflection and calculates the distance to the object. Based on this, the coordinates of each point on the surface of the object are calculated and output as three-dimensional point cloud data. The LIDAR 28 outputs the three-dimensional point cloud data in coordinates in which the central axis 34 is the Y axis, an axis 40 perpendicular to the central axis 34 in the up-down direction is the Z axis, and an axis perpendicular to the central axis 34 in the left-right direction (direction perpendicular to the paper surface) is the X axis.

[0031] The hardware configuration of the vehicle estimation device is shown in Fig. 3. The CPU 10 is connected to a memory 12, a display 14, a communication circuit 16, an SSD 18, a DVD-ROM drive 20, a keyboard / mouse 22, and a LIDAR 28. The communication circuit 16 is a circuit for connecting to the Internet.

[0032] The LIDAR 28 shines laser light onto an object such as a vehicle, measures the distance to the surface of the object based on the time it takes to receive the reflected light, and generates three-dimensional point cloud data indicating the three-dimensional coordinates of each point on the object's surface based on this.

[0033] The SSD 18 stores an operating system 24 and a vehicle estimation program 26. The vehicle estimation program 26 performs its functions in cooperation with the operating system 24. These programs were recorded on a DVD-ROM 30 and installed in the SSD 18 via the DVD-ROM drive 20.

[0034] 3. Vehicle estimation processing 4 shows a flowchart of the vehicle estimation program 26. The CPU 10 acquires one frame of three-dimensional point cloud data from the LIDAR 28 (step S2). An example of the acquired three-dimensional point cloud data is shown in FIG. 5A. As shown in the figure, three-dimensional point cloud data including not only the vehicle but also the road surface and the like is acquired. Note that the coordinates of this three-dimensional point cloud data are three-dimensional coordinates with the central axis 34, which is the installation direction of the LIDAR 28, as the Y axis, as shown in FIG.

[0035] Next, the CPU 10 extracts three-dimensional point cloud data of the vehicle portion from the three-dimensional point cloud data (step S3). As shown in Fig. 5B, a boundary box 42 surrounding the vehicle is obtained to extract the vehicle.

[0036] In this embodiment, a trained estimation model that has been trained to extract vehicle parts from three-dimensional point cloud data by surrounding them with a boundary box is used. As the training data, a set of three-dimensional point cloud data as shown in FIG. 5A and coordinates that specify the boundary box of the vehicle included therein is used. As the estimation model, OpenPC Det, which is YOLO adapted to three dimensions, can be used.

[0037] Next, as shown in Fig. 5B, the center 44 (center as coordinates) of the boundary box is calculated (step S5). The CPU 10 determines whether the center 44 is closer to the LIDAR 28 than a preset first plane 36 (see Fig. 2) (step S6). The first plane 36 is a virtual plane that is approximately perpendicular to the road surface.

[0038] If it is on the far side, the boundary box is not processed. If it is on the close side, as shown in Fig. 5C, the center 44 of the boundary box 42 and the center 44P of the boundary box 42P in the previous frame are associated with each other as the same vehicle based on the distance between them (step S8). Since the vehicle generally travels at a speed within a predetermined range, it is possible to associate them based on the distance traveled since the previous frame.

[0039] When the vehicle travels and the center 44 of the boundary box 42 is located closer to the LIDAR 28 than the second surface 38 (step S7), the boundary boxes thereafter are not processed. In addition, the CPU 10 overlays multiple pieces of 3D point cloud data of the same vehicle tracked in step S8 (step S9).

[0040] In measurements using the LIDAR 28, it is not possible to obtain 3D point cloud data for areas shaded by the laser. Therefore, as the vehicle approaches, the density of the point cloud and the areas where 3D point cloud data can and cannot be obtained change in each frame depending on the distance and angle between the LIDAR 28 and the vehicle, as shown in Figures 6A and 6B.

[0041] In this embodiment, the missing parts are complemented by superimposing the 3D point cloud data of each frame to generate superimposed 3D point cloud data as shown in Fig. 6C, thereby improving the accuracy of vehicle type estimation in the next step.

[0042] When the three-dimensional point cloud data of each frame is superimposed, the data is superimposed so that the shapes of the parts of each frame where the laser is irradiated and there are no defects (the front part for an approaching vehicle, and the back part for a receding vehicle) match.

[0043] Next, the CPU 10 estimates the type of the vehicle based on the superimposed three-dimensional point cloud data (step S10).

[0044] A detailed flowchart of vehicle type estimation is shown in Fig. 7. As shown in Fig. 8, the CPU 10 generates a minimum boundary box 50 that contains the superimposed three-dimensional point cloud data. The CPU 10 calculates the length L of the generated boundary box 50 (step S21). Based on the calculated length L, it is determined whether the vehicle is a large vehicle, a small vehicle, or a two-wheeled vehicle (step S22).

[0045] In the traffic volume survey established by the Ministry of Land, Infrastructure, Transport and Tourism, vehicle types are classified as shown in Figure 9. First, to distinguish between large vehicles, small vehicles, and motorcycles, vehicles with length L greater than 4.7 m are classified as large vehicles. Vehicles with length L less than 4.7 m are classified as small vehicles or motorcycles. Although not shown in Figure 9, in general, most motorcycles have length L less than 2 m, so vehicles with length L greater than 2 m are classified as small vehicles, and vehicles with length L less than 2 m are classified as motorcycles.

[0046] As described above, when determining whether the vehicle is a large vehicle, a small vehicle, or a two-wheeled vehicle, the CPU 10 estimates whether the large vehicle is a bus or a regular freight vehicle based on the superimposed three-dimensional point cloud data (step S24), and estimates whether the small vehicle is a passenger car or a small freight vehicle (step S25). In this embodiment, the estimation is performed based on the superimposed three-dimensional point cloud data with missing parts complemented, so that the accuracy of the estimation can be improved.

[0047] The estimation in step S24 uses an estimation model (neural network) trained to estimate whether the object is a bus or a regular freight vehicle. As training data, a set of superimposed 3D point cloud data and a label indicating whether the superimposed 3D point cloud data is a bus or a regular freight vehicle is used.

[0048] The estimation in step S25 uses an estimation model (neural network) trained to estimate whether the vehicle is a passenger car or a small freight vehicle. As training data, a set of superimposed 3D point cloud data and a label indicating whether the 3D point cloud data is a passenger car or a small freight vehicle is used.

[0049] Steps S2 to S10 are repeated for each frame, and steps S5 to S10 are repeated for each vehicle shown in the frame.

[0050] In this manner, it is possible to discriminate between passenger cars, small freight vehicles, buses, regular freight vehicles, and motorcycles.

[0051] In this embodiment, since a laser is used, estimation can be performed with high accuracy even at night.

[0052] 4. Other Modifications (1) In the above embodiment, after determining whether the vehicle is large or small, the vehicle type is estimated and discriminated between passenger cars, small freight vehicles, buses, ordinary freight vehicles, and motorcycles. However, only the determination of whether the vehicle is large or small may be performed. Also, the vehicle type may be estimated without determining whether the vehicle is large or small.

[0053] (2) In the above embodiment, estimation is performed for passenger cars, small freight vehicles, buses, regular freight vehicles, and motorcycles. However, more detailed estimation may be performed for sedans, SUVs, etc., or for manufacturer names and vehicle types (Celsior (trademark), Crown (trademark), 10-ton trucks, 20-ton trucks, etc.).

[0054] (3) In the above embodiment, the extraction process in step S3 and the estimation process in step S10 are performed using coordinates in which the central axis 34 in the installation direction of the LIDAR 28 is the Y-axis. In either process, the coordinates of the three-dimensional point cloud data will differ depending on the installation direction of the LIDAR 28. Therefore, it is preferable to prepare a large number of pieces of the same three-dimensional point cloud data with different coordinates as learning data for training an estimation model used in the above estimation process.

[0055] Therefore, when using the measured three-dimensional point cloud data as learning data, it is preferable to generate a large amount of learning data by rotating the data in increments of 2 degrees (predetermined angles) in the range of -10 degrees to +10 degrees (predetermined angle range) about the X-axis, Y-axis, and Z-axis. In this way, a large amount of learning data can be easily obtained.

[0056] Furthermore, instead of using the central axis 34 of the installation direction of the LIDAR 28 as the Y-axis as described above, the three-dimensional point cloud data may be expressed using coordinates in which a plane parallel to the road is a plane defined by the XY axes, and a direction perpendicular to this is the Z-axis. In this way, three-dimensional point cloud data can be obtained without being affected by the installation direction of the LIDAR 28, eliminating the need to generate learning data by performing rotation as described above (or the amount of learning data generated by rotation can be reduced).

[0057] (4) In the above embodiment, the center point 44 of the boundary box 42 is used to track the same vehicle in each frame. However, a point representative of the boundary box 42 other than the center point 44 (such as any corner point, the center point of any face, or the center of gravity of the three-dimensional point cloud data) may be used.

[0058] Also, shape characteristics may be calculated from the three-dimensional point cloud data, and the same vehicle may be linked to each frame based on the characteristics.

[0059] (5) In the above embodiment, the operator sets the first plane 36 and the second plane 38. However, the CPU 10 may set the first plane 36 and the second plane 38 at a predetermined interval in a direction perpendicular to the road surface projected in the three-dimensional point cloud data.

[0060] (6) In the above embodiment, the 3D point cloud data of each frame between the point where the center 44 of the boundary box passes beyond the first surface 36 and the point where it passes beyond the second surface 38 is overlaid to generate superimposed 3D point cloud data.

[0061] However, it is also possible to use a point that represents the boundary box 42 other than the center 44. Also, different representative points may be used on the first surface 36 and the second surface 38.

[0062] (7) In the above embodiment, the type of vehicle is determined based on an approaching vehicle. However, the type of vehicle may be determined based on a receding vehicle.

[0063] (8) In the above embodiment, the type of vehicle is estimated. However, instead of or in addition to this, the number of passing vehicles may be counted.

[0064] In this case, it can be determined that one vehicle has passed if the center 44 of the boundary box passes over the first plane 36 and then the second plane 38. By taking into account the above estimation of the type of vehicle, it is possible to measure how many of each type of vehicle have passed.

[0065] As for a vehicle in the opposite lane, it can be determined that it has passed if the center 44 of the boundary box passes over the second plane 38 and then the first plane 36. In this way, it can be determined whether a vehicle is approaching or receding depending on the order in which it passes over the first plane 36 and then the second plane 38, so that both the upper and lower lanes can be imaged by the LIDAR 28 and measurements can be taken simultaneously.

[0066] (9) In the above embodiment, whether a vehicle is large or small is determined based on the length L of the boundary box shown in Fig. 8. However, the determination may also be based on the width or height of the boundary box, or by taking the width and height into consideration. Furthermore, the type of vehicle (truck, bus, etc.) other than large or small may also be determined based on these dimensions.

[0067] (10) In the above embodiment, the processing in Fig. 1 is implemented using a CPU. However, a part or the whole of the processing may be implemented using a logic circuit.

[0068] (11) The above modifications can be implemented in combination with each other.

Claims

1. A measuring device is provided for measuring the distance to each point on the surface of a moving vehicle and generating three-dimensional point cloud data. Acquire 3D point cloud data from a measuring instrument at multiple positions for a moving vehicle; superimposing the plurality of sets of three-dimensional point cloud data to generate superimposed three-dimensional point cloud data; A vehicle estimation device that estimates the type of vehicle based on superimposed three-dimensional point cloud data.

2. A measuring device is provided for measuring the distance to each point on the surface of a moving vehicle and generating three-dimensional point cloud data. At least one processor; a memory storing at least one program configured to be executed by said at least one processor; A vehicle estimation device comprising: The at least one program comprises: Acquire 3D point cloud data from a measuring instrument at multiple positions for a moving vehicle; superimposing the plurality of sets of three-dimensional point cloud data to generate superimposed three-dimensional point cloud data; A vehicle estimation device having an instruction to estimate a type of vehicle based on the superimposed three-dimensional point cloud data.

3. A vehicle estimation program for implementing a vehicle estimation device by a computer, Acquire 3D point cloud data from a measuring instrument at multiple positions for a moving vehicle; superimposing the plurality of sets of three-dimensional point cloud data to generate superimposed three-dimensional point cloud data; A vehicle estimation program comprising instructions for estimating a type of vehicle based on the overlaid three-dimensional point cloud data.

4. In any one of the devices or programs according to claims 1 to 3, The apparatus or program, wherein the measuring instrument is a LIDAR.

5. In any one of the devices or programs according to claims 1 to 3, In generating the superimposed three-dimensional point cloud data, the device or program is characterized in that multiple sets of three-dimensional point cloud data are superimposed so that the surface shapes of the sides closer to the measuring instrument match.

6. In any one of the devices or programs according to claims 1 to 3, The device or program is characterized in that the vehicle type estimation uses at least one of the length, width, and height dimensions of the overlaid 3D point cloud data to determine whether the vehicle is at least large or small.

7. In any one of the devices or programs according to claims 1 to 3, The device or program is characterized in that the vehicle type is estimated by providing the superimposed 3D point group data to an estimation model trained based on superimposed 3D point cloud data.

8. The device or program of claim 7, The device or program is characterized in that the learning is performed using learning data generated from a plurality of superimposed 3D point cloud data obtained by rotating superimposed 3D point cloud data obtained by actual measurement.

Citation Information

Patent Citations

  • Vehicle estimation device

    JP2022163387A