Information processing device, information processing method, and program

Abstract

To provide an information processing unit capable of improving the determination accuracy of the information concerning an object detected by a radar unit, and an information processing method.SOLUTION: The information processing unit includes: an associating unit that associates multiple pieces of detection information detected at a certain time as detection information of a specific detection target based on time-series changes in information detected by the radar unit; a determination unit that determines the attributes of an object to be detected based on the associated detection information; and an output unit that outputs the determination result of the attributes.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Vehicle detection systems are being considered that use radar equipment to detect vehicles on the road, measure their speed, or classify their vehicle types. These vehicle detection systems can be used for speed enforcement, traffic volume counters, and vehicle classification at highway toll booths. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-163317 Summary of the Invention [Problem to be solved by the invention]

[0004] There is room for improvement in the accuracy of determining information such as the number, size, shape, and type of targets detected using radar equipment.

[0005] Non-limiting examples of the present disclosure contribute to providing an information processing device and an information processing method that can improve the accuracy of determining information related to a detection target by a radar device. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present disclosure includes a combining unit that combines multiple pieces of detection information detected at a certain time based on time-series changes in the detection information from the radar device into detection information for a specific detection target, a discrimination unit that discriminates attributes of the detection target based on the combined detection information, and an output unit that outputs the discrimination results of the attributes.

[0007] In an information processing method according to one embodiment of the present disclosure, an information processing device combines multiple pieces of detection information detected at a certain time based on time-series changes in the detection information from a radar device into detection information for a specific detection target, determines the attributes of the detection target based on the combined detection information, and outputs the attribute determination results.

[0008] A program according to one embodiment of the present disclosure causes an information processing device to execute a process of combining multiple pieces of detection information detected at a certain time based on time-series changes in detection information from a radar device as detection information for a specific detection target, determining attributes of the detection target based on the combined detection information, and outputting the attribute determination results.

[0009] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0010] According to an embodiment of the present disclosure, it is possible to improve the accuracy of determining information related to a detection target by a radar device.

[0011] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing a first example of vehicle detection using a radar device. [Figure 2] FIG. 2 shows a second example of vehicle detection using a radar device. [Figure 3] FIG. 1 is a diagram illustrating an example of a configuration of a vehicle detection system according to an embodiment. [Figure 4] A flowchart showing an example of signal processing in one embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a result obtained by signal processing in one embodiment. [Figure 6] FIG. 3 shows a third example of vehicle detection using a radar device. [Figure 7A] A diagram showing an example of the positional relationship between a radar device and a detection target. [Figure 7B] A diagram showing an example of the positional relationship between a radar device and a detection target. [Figure 7C] A diagram showing an example of the positional relationship between a radar device and a detection target. [Figure 8] Diagram showing an example of a cluster [Figure 9] Flowchart showing an example of primary determination in cluster combining processing [Figure 10] FIG. 10 is a diagram showing an example of a processing procedure based on FIG. 9. [Figure 11] A diagram showing a first example of join processing based on a cluster join table. [Figure 12] A diagram showing a second example of join processing based on a cluster join table. [Figure 13] FIG. 10 is a diagram showing an example of determination when cluster combining processing is not performed. [Figure 14] FIG. 10 is a diagram showing an example of determination when performing cluster combining processing. [Figure 15] FIG. 10 is a diagram showing a first example of vehicle type classification based on type information and likelihood information of multiple frames. [Figure 16] FIG. 10 is a diagram showing a first example of vehicle type classification based on type information and likelihood information of multiple frames. [Figure 17] FIG. 10 is a diagram showing a first example of vehicle type classification based on type information and likelihood information of multiple frames. [Figure 18] Diagram showing an example of TSF (Time-Spatial Feature) [Figure 19] Flowchart showing an example of time-series information processing DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions are designated by the same reference numerals, and redundant description will be omitted.

[0014] (One embodiment) <Knowledge that led to this disclosure> For example, a vehicle detection system is being considered in which a radar device attached to a structure such as a utility pole or a footbridge detects vehicles on the road, measures the speed of the detected vehicles, or classifies the vehicle type of the detected vehicles. This vehicle detection system can be used for speed enforcement, traffic volume counters, and vehicle type classification at highway toll booths, for example.

[0015] A radar device of a vehicle detection system, for example, transmits radio waves (transmission waves) and receives reflected waves that are reflected by a detection target (e.g., a vehicle). The radar device or a control device that controls the radar device generates information (hereinafter referred to as point cloud information) about a set of reflection points (hereinafter referred to as point cloud) that correspond to the detection target, based on the received reflection waves, and outputs the generated point cloud information to an information processing device. The point cloud represents, for example, the locations of reflection points corresponding to the detection target in a detection area whose origin is the position of the radar device, as well as the shape and size of the detection target.

[0016] In point cloud information, the number of point clouds corresponding to one detection target is not limited to one. When the detection target is a large vehicle such as a truck or bus, two or more point clouds may appear for one large vehicle.

[0017] FIG. 1 is a diagram showing a first example of vehicle detection by a radar device 100. FIG. 1 shows the radar device 100 and a truck T that is the detection target of the radar device 100. In the example of FIG. 1, a transmission wave transmitted from the radar device 100 is reflected at two locations: in front of the driver's seat of the truck and above the bed of the truck, and the radar device 100 receives the reflected waves reflected at the two locations. Since the two reflection locations shown in FIG. 1 are relatively far apart, two point clouds appear in the point cloud information from one truck.

[0018] When two or more point clouds are obtained for a large vehicle, there is a possibility that there will be a discrepancy in the distance between the point clouds in the detection area. In such a case, it may be difficult to determine that two or more point clouds correspond to one detection target (e.g., a large vehicle). For example, in such a case, there is a possibility that each of the two or more point clouds may be erroneously determined to correspond to a detection target (e.g., a standard car or a compact car) smaller than the large vehicle.

[0019] Furthermore, when the detection target is a moving object (for example, a vehicle), the reflection point of the detection target changes as the position of the detection target as seen from the radar device 100 changes over time. For example, even if radio waves are reflected at a reflection point on the top of the vehicle at one point in time, the radio waves may be reflected at a reflection point on the bottom of the vehicle at another point in time. In this way, when the reflection point changes over time, variations may occur in the feature amounts obtained from the point cloud.

[0020] FIG. 2 is a diagram illustrating a second example of vehicle detection using a radar device. FIG. 2 illustrates a radar device 100, a vehicle traveling in a direction approaching the radar device 100, a point cloud generated based on waves reflected by the vehicle, and a classification result of vehicle types classified based on feature amounts obtained from the point cloud. Note that FIG. 2 exemplarily illustrates the vehicle positions, point clouds, and vehicle type classification results at five time points from time points t1 to t5. In the example of FIG. 2, the feature amount obtained at time point t4 differs from the feature amounts obtained at other time points. For example, as the positional relationship between the radar device and the vehicle changes, the reflection points of the vehicle at time point t4 may differ from the reflection points of the vehicle at time points t1 to t3 and t5. In this case, the feature amount at time point t4 may differ from the feature amounts at time points t1 to t3 and t5.

[0021] In the example of Figure 2, the classification results at t1 to t3 and t5 are "standard-sized vehicles," but the classification result at t4 is "large vehicles." Since the detection target in the example of Figure 2 is a "standard-sized vehicle," the rate at which it is correctly determined to be a "standard-sized vehicle" (recall rate) is 80%. As illustrated in Figure 2, if there is variation in the feature values obtained from the point cloud, errors may occur in the classification (determination) of vehicle types based on the point cloud.

[0022] This disclosure describes an example of a configuration and operation that can improve the accuracy of detection (or determination) in a vehicle detection system using a radar device. Note that "detection" may be read as "detection." "Determination" may be read as "discrimination," "identification," or "recognition." Furthermore, in the following description, vehicle type classification may be read as vehicle type determination.

[0023] <Example of system configuration and processing procedure> Fig. 3 is a diagram showing an example of the configuration of vehicle detection system 1 according to this embodiment. Fig. 4 is a flowchart showing an example of signal processing in this embodiment. Below, with reference to Figs. 3 and 4, the vehicle detection system 1 according to this embodiment and an example of signal processing in vehicle detection system 1 will be described.

[0024] The vehicle detection system 1 according to this embodiment includes, for example, a radar device 100, a radar control unit 200, a setting unit 300, a Doppler velocity correction unit 400, a pre-processing unit 500, a clustering unit 600, a feature creation unit 700, a classification unit 800, a learning information database (DB) 900, a discrimination information learning unit 1000, a cluster combining unit 1100, a tracking unit 1200, a time series information accumulation unit 1300, a time series information storage unit 1400, a time series information processing unit 1500, and a vehicle recognition information output unit 1600.

[0025] Each of the components shown in FIG. 3 may have the form of a signal processing device (or an information processing device), or two or more of the components shown in FIG. 3 may be included in one signal processing device (or an information processing device). For example, the components shown in FIG. 3 except for the radar device 100 may be included in one signal processing device (or an information processing device), and this signal processing device may be connected to the radar device 100 wirelessly or via a wire. Furthermore, the components shown in FIG. 3 may be distributed and arranged in multiple signal processing devices (or information processing devices). For example, all of the components shown in FIG. 3, including the radar device 100, may be included in one signal processing device (or an information processing device).

[0026] Furthermore, the processes corresponding to the setting unit 300, the Doppler velocity correction unit 400, the preprocessing unit 500, the clustering unit 600, the feature creation unit 700, the classification unit 800, the learning information DB 900, the discrimination information learning unit 1000, the cluster combining unit 1100, the tracking unit 1200, the time-series information accumulation unit 1300, the time-series information storage unit 1400, and the time-series information processing unit 1500 may be performed by a single piece of software. In this case, the software that executes the process corresponding to the radar control unit 200 and the software that executes the process corresponding to the vehicle recognition information output unit 1600 may be separate pieces of software.

[0027] The radar device 100, for example, transmits a transmission wave and receives a reflected wave that is the transmission wave reflected by a detection target.

[0028] The setting unit 300 sets the installation conditions and road information (S100 in FIG. 4). The installation conditions may be, for example, conditions related to the location where the radar device 100 is installed. The road information may include, for example, information related to roads present in the detection range of the radar device 100. For example, the road information may include information related to at least one of the road width, the direction in which the road extends, and the traveling direction of vehicles traveling on the road. The installation conditions and road information may be corrected based on time-series information. For example, the setting unit 300 may estimate the orientation of the radar device 100 based on the trajectory of vehicle movement indicated by the time-series information, and correct the difference from the orientation indicated by the installation conditions. This correction can eliminate or reduce the discrepancy between the area design information for the area where the radar device is to be installed and the actual installation information.

[0029] The radar control unit 200, for example, controls the radar device 100 and performs detection of a detection target by the radar device 100 (hereinafter, may be referred to as "radar detection") (S200 in FIG. 4). The radar control unit 200 may perform control based on, for example, differences in the performance of the radar device 100. For example, the performance of the radar device 100 may be represented by at least one of the detection range, detection cycle, and detection accuracy of the radar device 100. For example, the radar control unit 200 acquires reflected waves from the radar device 100 and generates point cloud information based on information such as the reception timing and reception strength of the reflected waves. The point cloud information may include, for example, the position of the point cloud and the Doppler velocity of the detection target corresponding to the point cloud.

[0030] The Doppler velocity correction unit 400 corrects the detected Doppler velocity by referring to, for example, the installation conditions of the radar device 100 and road information (S300 in FIG. 4). The Doppler velocity correction process in S300 will be described later.

[0031] The preprocessing unit 500 performs preprocessing of the point cloud information by, for example, referring to the installation conditions of the radar device 100 and road information (S400 in FIG. 4). The preprocessing may include, for example, a process of generating point cloud information to be output to the clustering unit 600 based on the point cloud information acquired from the radar device 100. The preprocessing may also include, for example, processes such as noise removal, filtering, and coordinate conversion. For example, the point cloud information acquired from the radar device 100 may include point cloud information in a polar coordinate system defined by the distance from the radar device 100 as the starting point and the angle (elevation angle and azimuth angle) as seen from the radar device 100. The preprocessing may also include a process of increasing the point cloud information using information from several frames prior to the current time, and a process of keeping the height information constant in the point cloud information to be output to the clustering unit 600 so that clusters do not split in the height direction when clustering is performed. The preprocessing unit 500 may convert this point cloud information into point cloud information in a Cartesian coordinate system based on the position of the radar device 100.

[0032] The Cartesian coordinate system may be expressed by X, Y, and Z coordinates. For example, the plane on which the vehicle travels may be the XY plane where Z=0, and the point directly below the position of the radar device 100 on the XY plane where Z=0 may be the origin (i.e., the point where X=Y=Z=0). The Y axis may be an axis along a direction perpendicular to the radar substrate. For example, the smaller the Y coordinate, the closer the point is to the radar device 100.

[0033] The clustering unit 600 performs, for example, clustering processing on the point cloud information (S500 in FIG. 4). For example, the clustering processing may use DBSACN (Density-based spatial clustering of applications with noise), which clusters the point cloud based on the point cloud information obtained by the above-described preprocessing, to generate clusters. Note that the algorithm used for the clustering processing is not limited to DBSCAN. Furthermore, the clustering unit 600 assigns identification information (e.g., ID) to each of the generated clusters to identify them.

[0034] The feature creation unit 700 creates features (S600 in FIG. 4). For example, a feature may be created for each cluster. The feature may include at least one of the following eleven parameters: The radius of the smallest circle that encompasses the points in the cluster, the number of points in the cluster, the proportion of core points in a cluster, Cluster covariance, which indicates the variation in point cloud positions within a cluster. The width of the X coordinates of the points in the cluster, The width of the Y coordinates of the points in the cluster, The width of the Z coordinates of the points in the cluster, The mean Doppler velocity of the points in the cluster, The variance of the Doppler velocity of the points in the cluster, The average SNR (Signal to Noise Ratio) of the points in the cluster, Variance of SNR of point clouds within a cluster.

[0035] Here, the ratio of core points in a cluster may be, for example, a feature value when DBSCAN or Grid-Based DBSCAN is used in the clustering unit 600. The width of the X coordinate may be, for example, the difference between the maximum and minimum values of the X coordinate of the point cloud. The width of the Y coordinate and the width of the Z coordinate may also be the same as the width of the X coordinate.

[0036] The classification unit 800 classifies the type (model) of an object (e.g., a vehicle) detected by the radar device 100 based on, for example, the feature created by the feature creation unit 700 (S700 in FIG. 4). For example, the classification unit 800 classifies the model of a cluster based on the feature created for each cluster, a machine learning model called SVM (Support Vector Machine), and pre-stored learning information. Note that the "type" of the detection object, such as "model," may be interpreted as the "attribute" of the detection object. The classification unit 800 outputs information associating a cluster with the model determined for the cluster.

[0037] The learning information database (DB) 900 stores, for example, learning information that is referred to in classification by the classification unit 800.

[0038] The discrimination information learning unit 1000 performs a learning process to generate learning information used to classify vehicle types, for example.

[0039] The cluster combining unit 1100 performs, for example, a cluster combining process (S800 in FIG. 4). The cluster combining process in S800 will be described later.

[0040] The tracking unit 1200, for example, tracks clusters in chronological order (S900 in FIG. 4). Note that the clusters tracked in chronological order by the tracking unit 1200 may be clusters that have been subjected to combining processing in the cluster combining unit 1100, or may be clusters that have not been subjected to combining processing. Also, in each configuration subsequent to the tracking unit 1200, the clusters to be processed may be clusters that have been subjected to combining processing in the cluster combining unit 1100, or may be clusters that have not been subjected to combining processing.

[0041] For example, the tracking unit 1200 performs tracking in a time series using a Kalman filter and JPDA (Joint Probabilistic Data Association). By performing tracking, the tracking unit 1200 determines clusters corresponding to the same detection target at different points in time. The tracking unit 1200 assigns the same identification information (ID) to clusters corresponding to the same detection target at different points in time.

[0042] The time-series information accumulation unit 1300 accumulates the time-series information in the time-series information storage unit 1400 (S1000 in FIG. 4 ), for example. The time-series information includes, for example, information about a current cluster and clusters from before the current time. In the information about the current cluster and clusters from before the current time, the same identification information is assigned to clusters corresponding to the same detection target at different times. Furthermore, in the time-series information, each cluster at each time point is associated with the classification result of the vehicle type corresponding to the cluster.

[0043] The time-series information processing unit 1500 performs time-series information processing, for example, based on the time-series information stored in the time-series information storage unit 1400 (S1100 in FIG. 4). The time-series information processing in S1100 will be described later.

[0044] The vehicle recognition information output unit 1600 outputs vehicle recognition information obtained, for example, by time-series information processing (S1200 in FIG. 4). The vehicle recognition information may include at least a portion of information such as the vehicle type, the vehicle speed, identification information attached to the vehicle, and vehicle location information.

[0045] The vehicle detection system 1 may, for example, periodically execute the process shown in Fig. 4, or may execute it non-periodically (for example, in response to an instruction from an external device). For example, the process shown in Fig. 4 may be executed at a period corresponding to the detection period of the radar device 100.

[0046] Fig. 5 is a diagram showing an example of the results obtained by signal processing in this embodiment. Fig. 5 shows example point cloud information generated at three points #1 to #3 and the results of processing the point cloud information.

[0047] For example, the clustering unit 600 in FIG. 3 clusters the point cloud information to generate clusters as shown in FIG.

[0048] Then, features are created for the clusters, and the classification unit 800 in Fig. 3 classifies the clusters into vehicle types as shown in Fig. 5. In the example in Fig. 5, the clusters at time points #1 and #3 are classified into vehicle types of "standard cars," and the cluster at time point #2 is classified into vehicle types of "large cars."

[0049] Next, after the clustering combining process is performed, the tracking unit 1200 tracks the clusters in chronological order. In the example of Fig. 5, by performing tracking, it is determined that the clusters from time point #1 to time point #3 correspond to the same detection target. In this case, as shown in Fig. 5, the same ID may be assigned to clusters corresponding to the same detection target.

[0050] The time-series information processing unit 1500 performs time-series information processing on the tracking results. In the case of Fig. 5, as a result of the time-series information processing, the classification result of the vehicle type at time point #2 is changed from "large vehicle" to "standard vehicle" (in other words, the classification is corrected).

[0051] Next, an example of the Doppler velocity correction in the Doppler velocity correction unit 400, the cluster combining process in the cluster combining unit 1100, and the time-series information processing in the time-series information processing unit 1500 will be described.

[0052] <Doppler velocity correction processing> An example of the Doppler velocity correction process in the Doppler velocity correction unit 400 will be described.

[0053] The information output by the radar device 100 includes Doppler velocity. The Doppler velocity corresponds to, for example, the moving speed of the detection target. The Doppler velocity is determined based on, for example, the change in distance between the detection target and the radar device 100.

[0054] Fig. 6 is a diagram showing a third example of vehicle detection using a radar device. Fig. 3 shows the radar device 100 and a vehicle traveling at a constant speed V in a direction approaching the radar device 100. Fig. 3 also shows, by way of example, a Doppler speed Vd1 based on the movement of the vehicle from time t1 to time t2 and a Doppler speed Vd2 based on the movement of the vehicle from time t3 to time t4. Note that the time interval between time t1 and time t2 may be the same as the time interval between time t3 and time t4.

[0055] The Doppler velocity Vd1 is determined based on the amount of change Dd1 between the distance D1 between the vehicle and the radar device 100 at time t1 and the distance D2 between the vehicle and the radar device 100 at time t2. The Doppler velocity Vd2 is determined based on the amount of change Dd2 between the distance D3 between the vehicle and the radar device 100 at time t3 and the distance D4 between the vehicle and the radar device 100 at time t4.

[0056] In the case of FIG. 6, the amount of change Dd2 is smaller than the amount of change Dd1, and therefore the Doppler velocity Vd2 is smaller than the Doppler velocity Vd1.

[0057] 6, a difference occurs between the Doppler velocity that can be detected by the radar device 100 and the vehicle's speed depending on the positional relationship between the radar device 100 and the vehicle. For example, even if the vehicle is traveling at a constant speed, the closer the vehicle is to the radar device 100, the smaller the change in the distance between the vehicle and the radar device 100, and therefore the detected Doppler velocity is smaller than the vehicle's speed.

[0058] The Doppler velocity correction unit 400 corrects the Doppler velocity based on, for example, the positional relationship between the radar device 100 and the vehicle. By correcting the Doppler velocity, the vehicle speed can be estimated more accurately.

[0059] 7A, 7B, and 7C are diagrams showing an example of the positional relationship between the radar device 100 and the detection target. 7A, 7B, and 7C show an example of the positional relationship between the radar device 100 and the detection target in XYZ space. In this example, the detection target travels straight on a plane. The following shows an example of Doppler velocity correction when a radio wave (transmitted wave) is reflected at a reflection point P on the detection target and the reflected wave is received by the radar device 100.

[0060] 7A, 7B, and 7C, the X-axis and Y-axis are defined as being parallel to the plane (road surface) on which the detection object moves. In other words, the XY plane is parallel to the surface on which the detection object moves. The Z-axis is defined as being perpendicular to the plane on which the detection object moves. Illustratively, the XY plane where Z=0 is defined as the plane on which the detection object moves. The Y-axis is defined along the direction in which the detection object moves. In this example, the detection object moves straight in the negative direction of the Y-axis.

[0061] 7A, 7B, and 7C, X=0 and Y=0 are defined relative to the position where the radar device 100 is installed. For example, as shown in FIG. 7A, the height from the plane where the detection target moves (the XY plane where Z=0) to the position where the radar device 100 is installed is h radar In this case, the XYZ coordinates indicating the location where the radar device 100 is installed are (X, Y, Z) = (0, 0, h radar )

[0062] Point Q in Figure 7A indicates the intersection point between a line that passes through reflection point P and is parallel to the Z axis and the XY plane where Z = 0. The α axis shown in Figure 7A is the axis of a line that extends from the origin O of the XYZ space toward point Q. The angle between the Y axis and the α axis is represented as φ.

[0063] Fig. 7B is a diagram showing a plane (α-Z plane) along the α axis and Z axis in Fig. 7A, and Fig. 7C is a diagram showing the XY plane where Z=0 in Fig. A.

[0064] The line segment L1 shown in FIGS. 7A and 7B is parallel to the α axis and extends from the Z axis to the reflection point P. The line segment L2 is a line segment between the reflection point P and the point R where the radar device 100 is installed. The angle formed by the lines L1 and L2 is represented as θ. The XYZ coordinates indicating the position of the reflection point P are (X, Y, Z) = (Px, Py, h reflect The position of the reflection point P (for example, XYZ coordinates) is calculated based on the reflected wave received by the radar device 100.

[0065] The target velocity V indicates the moving speed of the detection target. The velocity V' indicates the velocity component along the α axis of the target velocity V. The Doppler velocity Vd is calculated based on the reflected wave reflected at the reflection point P.

[0066] As shown in Fig. 7B, the relationship between velocity V' and Doppler velocity Vd is Vd = V' cos θ. Also, as shown in Fig. 7C, the relationship between target velocity V and velocity V' is V' = V cos φ. Therefore, target velocity V is expressed as equation (1) using θ, φ, and Doppler velocity Vd.

number

[0067] Here, as shown in FIG. 7B, θ is expressed as equation (2) based on the position of the reflection point P and the position (height) of the radar device 100.

number

[0068] Furthermore, as shown in FIG. 7C, φ can be expressed as equation (3) based on the position of the reflection point P.

number

[0069] The Doppler velocity Vd is corrected based on θ calculated by equation (2), φ calculated by equation (3), and equation (1). By this correction, the target velocity V is estimated.

[0070] <Cluster joining process> Next, we will explain the cluster combining process. In the cluster combining process, even if a cluster corresponding to one detection target (e.g., a vehicle) is separated into multiple clusters, the trajectories of the time changes of the separated multiple clusters overlap, and past cluster information is used to determine whether the clusters can be combined.

[0071] FIG. 8 is a diagram showing examples of clusters. FIG. 8 illustrates the positions of clusters when a vehicle to be detected is viewed from above. FIG. 8 shows Example 1 in which two clusters are detected separately from one vehicle, and Example 2 in which two clusters in total, one from each vehicle, are detected. In each example, #n-3 to #n, in the frame at each time point, the cluster at that time point (current cluster) and the cluster at an earlier time point than that time point (past cluster) are superimposed and shown. The time interval between frames is, for example, 50 ms, but the present disclosure is not limited to this.

[0072] For example, in frame #n-3, two clusters at time point #n-3 are shown, and in frame #n-2, two clusters at time point #n-3 and two clusters at time point #n-2 are shown. For example, in frame #n-2, the two clusters at time point #n-3 are "past clusters." The same is true for other time points. For example, in frame #n, two current clusters at time point #n and past clusters at time points #n-1 to #n-3 are shown.

[0073] As shown in Example 1 of Figure 8, two current clusters corresponding to one vehicle form a single trajectory when overlapped with a past cluster. In other words, the trajectories traced by the past cluster and the two current clusters overlap over time (in other words, the trajectory deviation is minimal). This trajectory corresponds to the trajectory traveled by one vehicle corresponding to the cluster. Therefore, the correlation between the positions of the clusters in each frame is relatively high. On the other hand, as shown in Example 2 of Figure 8, the trajectories traced by the current clusters corresponding to two vehicles do not overlap over time when overlapped with a past cluster. Therefore, the correlation between the positions of the clusters decreases as the number of overlapping clusters increases. Therefore, the case where the trajectories traced by the clusters overlap can be said to be the case where the correlation between the clusters in the time-series changes (or changes over time) of the positions where the clusters occurred is relatively high.

[0074] Whether or not to combine multiple clusters, in other words, whether or not two clusters correspond to the same detection target, can be determined based on the degree of overlap of the trajectories when superimposed with past clusters, for example.

[0075] The cluster combining unit 1100 determines, for example, whether to combine multiple clusters detected at a certain time point based on predetermined conditions (cluster combining conditions). In the following, for example, in the cluster combining process, a primary determination of whether to combine clusters is made based on four conditions: the distance between clusters, the positional relationship with previous clusters, the correlation coefficient between clusters, and the vehicle types classified based on the characteristics of the clusters.

[0076] The above four conditions are merely examples, and the present disclosure is not limited to these. For example, some of the four conditions may be omitted, or other conditions may be added in addition to the four conditions. Furthermore, the distance between clusters may be expressed by, for example, Euclidean distance or Manhattan distance.

[0077] The cluster combining unit 1100 creates, for example, a cluster combining table based on the results of the primary determination, and uses the cluster combining table to perform a secondary determination of clusters to be combined.

[0078] Fig. 9 is a flowchart showing an example of the primary determination of the cluster combining process. The primary determination of the cluster combining process shown in Fig. 9 starts, for example, when the cluster combining unit 1100 acquires the processing result from the classification unit 800 (S701).

[0079] The cluster combining unit 1100 extracts (or selects), for example, two clusters from among the clusters detected in the frame to be processed (S702). Note that identification information (e.g., an ID) for identifying each cluster in the frame may be assigned to each cluster.

[0080] The cluster combining unit 1100 determines whether the distance between the two extracted clusters is equal to or less than a threshold (S703), for example. The threshold for the distance is 10 m, for example.

[0081] If the distance between the clusters is not equal to or less than the threshold (NO in S703), the primary determination of the joining process for the two extracted clusters ends (S709).

[0082] If the distance between the clusters is equal to or less than the threshold (YES in S703), the cluster combining unit 1100 extracts, for example, for each of the two clusters, past clusters that exist within a specified radius r from the center of the cluster by overlapping information about the past clusters (S704). The information about the past clusters includes past clusters that have been detected 50 frames in the past from the current time. The specified radius r is, for example, 7.5 m. Note that although the information about the past clusters is described as being for 50 frames, the present disclosure is not limited to this. The number of frames of information about the past clusters may be changed. For example, it may be dynamically changed based on another parameter (e.g., the speed of the detection target) or may be changed according to a user setting.

[0083] The cluster combining unit 1100 determines whether the number of accumulated clusters is equal to or greater than a threshold (S705). The accumulated clusters may include past clusters existing within the specified radius r of each of the two clusters in S704 described above, and the cluster of the current frame. The threshold for the number of clusters may be, for example, 25.

[0084] If the number of accumulated clusters is not equal to or greater than the threshold value (NO in S705), the primary determination of the combining process for the two extracted clusters ends (S709).

[0085] If the number of accumulated clusters is equal to or greater than the threshold (YES in S705), the cluster combining unit 1100 determines whether the correlation coefficient is equal to or greater than the threshold (S706). Here, the correlation coefficient is a coefficient that indicates the correlation of the positional relationship between the accumulated clusters, and |r xy |. The correlation coefficient is a value that indicates a high correlation when the positions of the accumulated clusters exist along the same trajectory, and a low correlation when the positions of the accumulated clusters vary. For example, if the correlation coefficient indicating the highest correlation is 1 and the correlation coefficient indicating the lowest correlation is 0, the threshold for the correlation coefficient may be 0.95.

[0086] If the correlation coefficient is not equal to or greater than the threshold (NO in S706), the primary determination of the joining process for the extracted two clusters is completed (S709).

[0087] If the correlation coefficient is equal to or greater than the threshold (YES in S706), the cluster combining unit 1100 determines whether the proportion of a specific vehicle type (e.g., large vehicle) in the classification results of each accumulated cluster is equal to or greater than the threshold (S707). For example, if the proportion is expressed as a percentage, the threshold for the proportion is 50%.

[0088] If the proportion of large vehicles is not equal to or greater than the threshold (NO in S707), the primary determination of the joining process for the extracted two clusters is completed (S709).

[0089] If the proportion of large vehicles is equal to or greater than the threshold (YES in S707), the cluster combining unit 1100 determines that the two clusters extracted in S703 are to be combined, and reflects the determination result in the cluster combining table (S708).Then, the primary determination of the cluster combining process for the two extracted clusters ends (S709).

[0090] Furthermore, after S709, if there is a pair of two clusters among the clusters detected in the frame to be processed that have not yet undergone merging processing, processing from S702 onwards may be performed on the two clusters that have not yet undergone merging processing.

[0091] After performing the primary determination of the cluster combining process for each pair of two clusters detected in the frame to be processed (after S709), the cluster combining unit 1100 performs the secondary determination process (S710) based on, for example, the cluster combining table. In this secondary determination process, the vehicle type corresponding to the combined cluster (combined cluster) may be determined to be a large vehicle.

[0092] Fig. 10 is a diagram showing an example of a processing procedure based on Fig. 9. Fig. 10 illustrates clusters when a vehicle is viewed from above. On the left side of Fig. 10, two clusters (current clusters) included in a frame at a certain point in time are shown.

[0093] The distance between the current clusters shown on the left in Fig. 10 is equal to or greater than the threshold (YES in S703 in Fig. 9). In this case, for each of the two clusters, a past cluster existing within a radius r from the center of the cluster is extracted (S704 in Fig. 9).

[0094] On the right side of Fig. 10, clusters of frames earlier than the frame shown on the left side of Fig. 10 (past clusters) are superimposed on the current cluster. On the right side of Fig. 10, clusters contained within a circle of radius r are extracted.

[0095] In the example on the right of Fig. 10, the number of extracted clusters is equal to or greater than the threshold (YES in S705), the correlation coefficient is equal to or greater than the threshold (YES in S706), and the proportion of large vehicles is equal to or greater than the threshold (YES in S707). In this case, the two clusters shown on the left of Fig. 10 are determined to be candidates for merging. If they are candidates for merging, this is reflected in the cluster merging table.

[0096] Next, an example of the secondary determination process of the cluster combining process based on the cluster combining table will be described. The cluster combining table shows, in tabular form, pairs of clusters that have been determined to satisfy the cluster combining condition in the primary determination illustrated in FIG. 9, and pairs of clusters that have been determined not to satisfy the cluster combining condition. In other words, the cluster combining table visually shows whether the correspondence between two clusters has a relationship that satisfies the cluster combining condition or a relationship that does not satisfy the cluster combining condition. The correspondence between two clusters does not have to be processed in tabular form.

[0097] Fig. 11 is a diagram showing a first example of a merging process based on a cluster merging table. The left side of Fig. 11 shows a cluster merging table generated for clusters assigned six IDs "1" to "6". In the following description, a cluster assigned ID:i will be referred to as cluster #i. In the example of Fig. 9, i is an integer between 0 and 6. The center of Fig. 11 shows an example of a procedure for confirming merging targets in the cluster merging table. The right side of Fig. 11 shows an example of a cluster merging mask including the secondary determination result of whether or not to merge clusters.

[0098] In the row and column of the number "i" in the cluster joining table in Figure 11, "●" indicates a cluster that was determined to join with cluster #i in the primary judgment, and "-" indicates a cluster that was determined not to join with cluster #i in the primary judgment.

[0099] The row and column of "0" in the cluster combination table of FIG. 11 indicates that in the primary determination, cluster #0 was determined to be combined with clusters #2, #3, and #4.

[0100] If it is determined that cluster #0 will be joined with clusters #2, #3, and #4, the cluster joining table is checked to see if clusters #2, #3, and #4 will be joined with each other.

[0101] As shown in the row and column "2" of the cluster combination table in Fig. 11, in the primary determination, it is determined that cluster #2 will combine with clusters #3 and #4. Also, as shown in the row and column "3" of the cluster combination table in Fig. 11, it is determined that cluster #3 will combine with cluster #4.

[0102] In this case, the secondary determination determines that clusters #0, #2, #3, and #4 will be combined, as indicated by the confirmation result "1." Because it is determined that clusters #0, #2, #3, and #4 will be combined, a new ID: 7 is assigned to clusters #0, #2, #3, and #4, as indicated by the "1" in the cluster combination mask in Figure 11.

[0103] The row and column "1" in the cluster merging table in Figure 11 indicates that the primary determination determined that cluster #1 should be merged with cluster #6 (see "2" in the confirmation result). In this case, because there are no other clusters than clusters #1 and #6, there is no need to perform a subsequent check in the cluster merging table.

[0104] In this case, as indicated by the "2" in the cluster combination mask in FIG. 11, a new ID: 8 is assigned to clusters #1 and #6.

[0105] The row and column "5" in the cluster merging table in Figure 11 indicates that the primary determination determined that there was no cluster to merge with cluster #5 (see "3" in the confirmation result). In this case, ID:5 remains unchanged, as indicated by "3" in the cluster merging mask.

[0106] Fig. 12 is a diagram showing a second example of a joining process based on a cluster joining table. Similar to Fig. 11, Fig. 12 shows a cluster joining table, a joining confirmation procedure, and a cluster joining mask. The difference between Fig. 11 and Fig. 12 is that Fig. 11 shows an example in which it is determined that clusters #2 and #3 will join, while Fig. 12 shows an example in which it is determined that clusters #2 and #3 will not join.

[0107] The row and column of "0" in the cluster coupling table of FIG. 12 indicates that, similarly to FIG. 11, the primary determination determined that cluster #0 should be coupled with clusters #2, #3, and #4.

[0108] If it is determined that cluster #0 is to be combined with clusters #2, #3, and #4, the cluster combination table is checked to see if clusters #2, #3, and #4 are to be combined with each other.

[0109] As shown in the row and column "2" in the cluster merging table of Fig. 12, the primary determination determines that cluster #2 will merge with cluster #4 and will not merge with cluster #3. If the primary determination determines that one or more pairs among clusters #0, #2, #3, and #4 will not merge, the secondary determination determines that clusters #0, #2, #3, and #4 will not merge, as shown by "1" in the confirmation result. In this case, no new IDs are assigned to clusters #0, #2, #3, and #4, as shown by "1" in the cluster merging mask of Fig. 12.

[0110] The row and column "1" in the cluster merging table in Fig. 12 indicate that, similar to Fig. 11, the primary determination determined that there was no cluster to merge with cluster #5 (see "2" in the confirmation result). Therefore, a new ID: 7 is assigned to clusters #1 and #6 (see "2" in the cluster merging mask).

[0111] The IDs assigned to the combined clusters in the secondary determination are not particularly limited. For example, the IDs of the combined clusters may be distinguished from the IDs of the uncombined clusters. For example, the number of digits assigned to the IDs of the combined clusters may be different from the number of digits assigned to the IDs of the uncombined clusters.

[0112] Furthermore, a new feature may be set for the combined cluster. For example, the position (X, Y, Z coordinates) of the combined cluster may be newly set. For example, the Y coordinate of the combined cluster may be the smallest Y coordinate among the multiple clusters before being combined. In this case, the X coordinate of the combined cluster may be the X coordinate of the cluster corresponding to the Y coordinate. Furthermore, the Z coordinate of the combined cluster may be the average of the Z coordinates of the multiple clusters before being combined. Furthermore, the feature of the combined cluster may be the average of the feature values of the multiple clusters before being combined.

[0113] Fig. 13 is a diagram showing a determination example when cluster combining processing is not performed. Fig. 14 is a diagram showing a determination example when cluster combining processing is performed. Figs. 13 and 14 show the positions of point clouds when the vehicle is viewed from above, two clusters obtained from the point clouds, and the determination results of the detection target. Note that the point clouds and the two clusters obtained from the point clouds are the same between Figs. 13 and 14.

[0114] In the case of Fig. 13, the two clusters are not combined, and therefore it is determined that the two clusters correspond to different detection targets. On the other hand, in the case of Fig. 14, the two clusters are combined, and it is determined that the two clusters should be combined, and therefore it is determined that the one combined cluster including the two clusters corresponds to one detection target.

[0115] As can be seen from a comparison between FIGS. 13 and 14, the execution of the cluster combining process can reduce the detection target determination error rate.

[0116] <Time series information processing> Next, the time-series information processing will be described. In the time-series information processing, the vehicle type of the detection target is classified in consideration of the classification results of the vehicle type at multiple points in time (multiple frames).

[0117] 15 is a diagram showing a first example of vehicle type classification based on type information and likelihood information of a plurality of frames. In the following explanation, vehicle type classification for one detection target will be taken as an example.

[0118] FIG. 15 shows the types classified using a single frame (single frame) (hereinafter referred to as "single frame types") and the types classified using multiple frames (hereinafter referred to as "multiple frame types") for each of the frames numbered "1" to "11" (frames #1 to #11). The single frame types shown in FIG. 15 correspond to the classification results of vehicle types in clusters determined to be the same detection target in time series by tracking. In the example of FIG. 15, frame #1 is the first frame that includes a detection target. Furthermore, likelihood information and the frame number used when calculating the TSF (Time-Spatial Feature) for each frame are shown.

[0119] The likelihood information indicates the proportion of frames classified into each type based on the classification results of "people, bicycles, motorcycles, standard vehicles, large vehicles, and others." In other words, the likelihood information indicates the likelihood that the cluster of the detection target can be determined to be each of "people, bicycles, motorcycles, standard vehicles, large vehicles, and others." Note that "other" indicates that the cluster of the frame does not fit into any of "people, bicycles, motorcycles, standard vehicles, and large vehicles." "Unassigned" indicates that the frame does not contain a cluster. For example, a case in which a cluster is not contained in a frame corresponds to any of the following cases: the radar device 100 did not detect an object (did not receive a reflected wave), point cloud information was not obtained, or point cloud information was obtained but the point cloud information did not contain sufficient information (e.g., a sufficient number of point clouds) to generate a cluster.

[0120] In the example of FIG. 15, the specified number of frames is 5. For example, in the case of frame #5 in FIG. 15, of the five frames from frame #1 to frame #5, the single-frame type is "large vehicle" for four frames from frame #1 to frame #4, and the single-frame type is "standard vehicle" for frame #5. In this case, the likelihood information for "large vehicle" in frame #5 is 80%, and the likelihood information for "standard vehicle" is 20%. For example, in the case of frame #7, of the five frames from frame #3 to frame #7, the single-frame type is "large vehicle" for frames #3, #4, and #6, and the single-frame type is "standard vehicle" for frames #5 and #7. In this case, the likelihood information for "large vehicle" in frame #7 is 60%, and the likelihood information for "standard vehicle" is 40%.

[0121] For example, in the example of FIG. 15, the vehicle type corresponding to a proportion of the classification results indicated by the likelihood information that is equal to or greater than a threshold is determined to be the type of the multiple frames (the vehicle type to be detected). For example, the threshold may be 50%. In frames #5 and #7, the vehicle type corresponding to a proportion of the classification results indicated by the likelihood information that is equal to or greater than the threshold is a "large vehicle," so the type of the multiple frames (the vehicle type to be detected) in frames #5 and #7 is a "large vehicle."

[0122] As shown in Figure 15, by processing past frames and the current frame in chronological order, the type of a single frame can be corrected if it is out of sync with other past frames, thereby reducing erroneous judgments.

[0123] Fig. 16 is a diagram showing a second example of type information and likelihood information for frames at multiple time points. As in Fig. 15, Fig. 16 shows the type of single frame and the type of multiple frames for each of frames #1 to #11. In the example of Fig. 16, frame #1 is the first frame that includes a detection target. Also shown for each frame is likelihood information and the frame number used when calculating the TSF for that frame.

[0124] In the example of FIG. 16, the type of single frame is "other" in frames #3 and #6, and the type of single frame is "unassigned" in frames #4, #5, and #7 to #11.

[0125] For example, if the types of a single frame include “unallocated” and “other,” the type of the detection target may be determined based on the likelihood information of the types excluding “unallocated” and “other.” For example, in FIG. 16, the likelihood information shown in parentheses corresponds to the likelihood information of the types excluding “unallocated” and “other.”

[0126] For example, in the case of frame #5, the type of frame #5 is determined based on frames #1 and #2 out of the five frames from frame #1 to frame #5, excluding frame #3, which is "other," and frames #4 and #5, which are "unassigned." For example, in frames #1 and #2, the type of the single frame is "large vehicle," so the likelihood information for "large vehicle" in frame #5 is 100%. In this case, the vehicle type corresponding to the proportion of each classification result indicated by the likelihood information that is equal to or greater than the threshold is "large vehicle," so in frame #5, the type of multiple frames (vehicle type to be detected) is "large vehicle."

[0127] The likelihood information may be included in the determination result and output to an external device. In this case, the output likelihood information may be likelihood information of a type excluding "unallocated" and "other" (likelihood information in parentheses in FIG. 16), or may be likelihood information of a type including "unallocated" and "other", or may be both of these.

[0128] Also, for example, in a certain frame, if each of the vehicle types classified within a specified number of frames is "other" or "unassigned," the type of the frame prior to that frame may be reflected. For example, in Fig. 16, in frame #7, the type classified using each of frames #3 to #7 alone is "other" or "unassigned," so in this case, the type classified in frame #6, "large vehicle," is reflected in frame #7.

[0129] Note that if a predetermined number of frames are consecutively classified as "other" or "unassigned" for each of the vehicle types within the designated number of frames, the classification may be stopped. For example, in the example of FIG. 16, frames #7 to #11 are consecutively classified as "other" or "unassigned" for each of the vehicle types within the designated number of frames. If such frames are consecutively classified, the classification may be stopped.

[0130] Fig. 17 is a diagram showing a third example of type information and likelihood information of frames at multiple time points. As in Fig. 15, Fig. 17 shows the type of single frame and the type of multiple frames for each of frames #1 to #11. In the example of Fig. 16, frame #1 is the first frame that includes a detection target. Also shown for each frame is likelihood information and the frame number used when calculating the TSF for that frame.

[0131] In the example of FIG. 17, likelihood information for types excluding "unassigned" and "other" is shown in parentheses, as in FIG. 16. Also, in the example of FIG. 17, "to be corrected" indicates that the type classification is to be corrected because there is no vehicle type corresponding to a percentage of the respective classification results indicated by the likelihood information that is equal to or greater than the threshold value of 50%. For example, in frame #5, the likelihood information for large vehicles is 33%, the likelihood information for standard vehicles is 33%, and the likelihood information for motorcycles is 33%. In this case, there is no vehicle type corresponding to a percentage of the respective classification results indicated by the likelihood information that is equal to or greater than the threshold value of 50%, and so the type falls under "to be corrected." In this way, when the vehicle type cannot be determined by comparing the likelihood information with a threshold value, the vehicle type may be determined based on the TSF. An example of the TSF will be described below.

[0132] Fig. 18 is a diagram showing an example of a TSF, which exemplarily shows examples of features and TSFs for frames #1 to #6.

[0133] 18, frames #1 to #3, #5, and #6 include clusters. These clusters are assigned the same ID of 1. Frame #4 does not include a cluster (that is, it is unallocated).

[0134] Here, the TSF based on frames #1 to #3 and #5 is generated based on, for example, features obtained from clusters assigned ID=1 for each of frames #1 to #3 and #5. For example, the TSF based on frames #1 to #3 and #5 may be one or more of the maximum, average, minimum and variance of certain features obtained from clusters assigned ID=1 for each of frames #1 to #3 and #5. When there are multiple features obtained from a cluster, the number of TSFs may be the same as or different from the number of features obtained from the clusters.

[0135] Furthermore, the TSF based on frames #2, #3, #5, and #6 is generated based on, for example, features obtained from clusters assigned ID=1 in each of frames #2, #3, #5, and #6. For example, the TSF based on frames #1 to #3, and #5 may be one or more of the maximum, average, minimum, and variance of a certain feature obtained from clusters assigned ID=1 in each of frames #1 to #3, and #5.

[0136] For example, if there are N types of features obtained from one cluster, TSF creates the maximum, average, minimum, and variance for each of the N types, resulting in four times the number of features obtained from one cluster.

[0137] By classifying the vehicle type using the TSF calculated as described above, the vehicle type can be determined based on the TSF even when the vehicle type cannot be determined by comparing the likelihood information with a threshold value.

[0138] Next, the processing procedure of the time-series information processing will be described. Fig. 19 is a flowchart showing an example of the time-series information processing. This time-series information processing starts, for example, when the time-series information processing unit 1500 acquires the processing result from the tracking unit 1200 via the time-series information accumulation unit 1300 (S1001).

[0139] The time-series information processing unit 1500 acquires information on past clusters having the same ID in time series as the cluster to be determined in the current frame (hereinafter, past target information) for a predetermined number of frames from the time-series information storage unit 1400 (S1002). For example, in the examples of FIGS. 15 to 17, the specified number of frames is 5.

[0140] The time-series information processing unit 1500 determines whether vehicle model information is present in any of the acquired frames (S1003). The vehicle model information is, for example, the result of vehicle model classification performed on past clusters in the past target information for the acquired frames. For example, the case where vehicle model information is not present may be the case of "other" or "unassigned" as exemplified in FIGS. 15 to 17.

[0141] If vehicle type information exists (YES in S1003), the time-series information processing unit 1500 determines whether the proportion of the vehicle type with the highest frequency is equal to or greater than a threshold (S1004). For example, in the example of frame #5 in Fig. 15, among the types of single frames from frame #1 to frame #5, the vehicle type with the highest frequency is "large vehicle," and the proportion is 80%.

[0142] If the proportion of the most common car model is equal to or greater than the threshold (YES in S1004), the time-series information processing unit 1500 determines that the car model corresponding to the cluster indicated by the same ID in the time series is the car model with the most common proportion (S1005), and the flow then ends.

[0143] If the percentage of the most common vehicle type is not equal to or greater than the threshold (NO in S1004), the time-series information processing unit 1500 creates a time-series feature (TSF) (S1006). For example, in the case of frame #5 in Fig. 17, there is no vehicle type with a percentage equal to or greater than the threshold (50%), so the TSF is calculated.

[0144] The time-series information processing unit 1500 determines the vehicle type from the time-series feature quantities (S1007). For example, the time-series information processing unit 1500 may perform machine learning processing based on the time-series feature quantities, create a machine learning model, and use the machine learning model to determine the vehicle type. Note that the machine learning model here may be different from the machine learning model in the classification unit 800. For example, the machine learning model in the time-series information processing unit 1500 may be created using the machine learning model in the classification unit 800. Then, the flow ends.

[0145] If no vehicle model information is available (NO in S1003), the time-series information processing unit 1500 determines the vehicle model based on the time-series information in the frame prior to the current frame (S1008). In other words, the determination result of the previous frame is used in this case. Then, the flow ends.

[0146] As described above, the time-series feature amounts have more types than the feature amounts of a cluster of a single frame. Therefore, by using the time-series feature amounts in S1007 of Fig. 19, the accuracy of determining the vehicle type can be improved.

[0147] As described above, the vehicle detection system 1 in this embodiment has at least one information processing device. The information processing device includes at least a cluster combining unit 1100 (an example of a combining unit) that combines multiple pieces of detection information detected at a certain time as detection information of a specific detection target based on time-series changes in the detection information (e.g., clusters) by the radar device 100, a classification unit 800 (an example of a discrimination unit) that determines the attributes of the detection target based on the combined detection information, and a vehicle recognition information output unit (an example of an output unit) that outputs the attribute discrimination results. This configuration can improve the accuracy of determining information about the detection target by the radar device 100.

[0148] For example, because the cluster combining unit 1100 combines multiple clusters corresponding to the same detection target into one cluster, it is possible to avoid erroneous determination that multiple clusters corresponding to the same detection target correspond to multiple detection targets.

[0149] Furthermore, even if there is variation in the feature amounts of a cluster indicated by the detection results of a plurality of frames, the time-series information processing unit 1500 can improve the accuracy of determining the type of detection target by referring to information from past points in time.

[0150] Furthermore, by performing the cluster combining process in the cluster combining section 1100 and the time-series information processing in the time-series information processing section 1500, the number of detection targets and the type of each detection target can be determined more accurately.

[0151] Note that some of the processes described in the above-described embodiments may be omitted (skip). For example, the Doppler velocity correction process may be omitted. Also, the cluster combining process and one of the time-series information processes may be omitted.

[0152] For example, in the case where vehicle classification is not performed, the time-series information processing may be omitted. Also, in the case where large vehicles are not detected (for example, in the case where the system is applied to a road where the passage of large vehicles is restricted), the cluster combining processing may be omitted.

[0153] In the present embodiment, an example is shown in which the detection target is a vehicle and the type of the vehicle is determined, but the detection target of the present disclosure is not limited to a vehicle.

[0154] The present disclosure can be realized in software, hardware, or software in conjunction with hardware.

[0155] Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0156] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0157] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility.

[0158] The present disclosure may be implemented in any type of apparatus, device, or system (collectively referred to as a communications apparatus) that has a communications function. The communications apparatus may include a wireless transceiver and processing / control circuitry. The wireless transceiver may include a receiver and a transmitter, or both functions. The wireless transceiver (transmitter and receiver) may include a radio frequency (RF) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of communication devices include telephones (e.g., cell phones, smartphones), tablets, personal computers (PCs) (e.g., laptops, desktops, notebooks), cameras (e.g., digital still / video cameras), digital players (e.g., digital audio / video players), wearable devices (e.g., wearable cameras, smartwatches, tracking devices), game consoles, digital book readers, telehealth / telemedicine devices, communication-enabled vehicles or mobile transportation (e.g., cars, airplanes, ships), and combinations of the above devices.

[0159] Communications equipment is not limited to portable or mobile equipment, but also includes non-portable or fixed equipment, devices, and systems of any kind, such as smart home devices (such as appliances, lighting equipment, smart meters or metering devices, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0160] Furthermore, in recent years, in the field of IoT (Internet of Things) technology, CPS (Cyber Physical Systems) has been attracting attention as a new concept that creates new added value by linking information between physical space and cyberspace. This CPS concept can also be adopted in the above-mentioned embodiments.

[0161] That is, as a basic configuration of a CPS, for example, an edge server located in physical space and a cloud server located in cyberspace can be connected via a network, and processing can be distributed and performed by processors installed on both servers. Here, it is preferable that each piece of processing data generated on the edge server or cloud server is generated on a standardized platform, and the use of such a standardized platform can improve the efficiency of building a system that includes a variety of sensor groups and IoT application software.

[0162] Communications include data communications via cellular systems, wireless LAN systems, communications satellite systems, etc., as well as data communications via combinations of these.

[0163] A communications apparatus also includes devices such as controllers and sensors connected or coupled to a communications device that performs the communications functions described in this disclosure, such as controllers and sensors that generate control and data signals used by the communications device to perform the communications functions of the communications apparatus.

[0164] The communication apparatus also includes infrastructure facilities, such as base stations, access points, and any other apparatus, device, or system that communicates with or controls the various apparatuses listed above, but are not limited to these.

[0165] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the disclosure.

[0166] Although specific examples of the present disclosure have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. [Industrial Applicability]

[0167] An embodiment of the present disclosure is suitable for a radar system. [Explanation of symbols]

[0168] 100 Radar Equipment 200 Radar control unit 300 Setting Department 400 Doppler velocity correction unit 500 Pretreatment section 600 Clustering Department 700 Feature Creation Unit 800 Classification Department 900 Learning Information Database (DB) 1000 Discrimination Information Learning Unit 1100 Cluster connection part 1200 Tracking Unit 1300 Time Series Information Storage Unit 1400 Time series information storage unit 1500 Time Series Information Processing Unit 1600 Vehicle recognition information output unit

Claims

1. a combining unit that combines multiple pieces of detection information simultaneously detected at a certain time based on a time-series change in the detection information obtained by the radar device as detection information of a specific detection target; a discrimination unit that classifies attributes of the detection target based on the combined detection information, and discriminates the attributes of the detection target based on likelihood information for each attribute candidate obtained by integrating the classification results at multiple times; an output unit that outputs the attribute determination result; Equipped with the determination unit determines the first attribute as the attribute of the detection target when likelihood information of the highest first attribute in the likelihood information for each of the attribute candidates exceeds a threshold, and determines the attribute of the detection target based on a time-spatial feature (TSF), which is a statistical value of a feature amount obtained from the combined detection information of the detection target at the multiple times; Information processing device.

2. the combining unit generates a third frame by superimposing a first frame indicating a positional relationship of first detection information detected at a first time and a second frame indicating a positional relationship of second detection information detected at a second time among a plurality of frames corresponding to the plurality of times, the first frame indicating a positional relationship of first detection information detected at a first time, and a second frame indicating a positional relationship of second detection information detected at a second time, and when it is determined that a trajectory indicated by a change in the first detection information overlaps a trajectory indicated by a change in the second detection information in the third frame, the combining unit combines the first detection information and the second detection information. The information processing device according to claim 1 .

3. the combining unit determines whether the trajectories overlap based on a distance between the first detection information and the second detection information and past detection information that is at a predetermined distance from each of the first detection information and the second detection information. The information processing device according to claim 2 .

4. the determination unit does not change the determined first attribute when the attribute of the detection target is not determined after the time at which the first attribute is determined among the plurality of times. The information processing device according to claim 1 .

5. The TSF is at least one of a maximum, an average, a minimum, and a variance of a feature obtained from the combined detection information at the multiple times. The information processing device according to claim 1 .

6. The information processing device Based on the time-series changes in the detection information from the radar device, multiple pieces of detection information detected simultaneously at a certain time are combined as detection information for a specific detection target, classifying attributes of the detection target based on the combined detection information, and determining the attributes of the detection target based on likelihood information for each attribute candidate obtained by integrating the classification results at multiple times; If likelihood information of a first attribute that is highest in the likelihood information for each of the attribute candidates exceeds a threshold, the first attribute is determined to be an attribute of the detection target; if the likelihood information of the first attribute does not exceed the threshold, the attribute of the detection target is determined based on a time-spatial feature (TSF), which is a statistical value of a feature amount obtained from the combined detection information of the detection target at the multiple times; outputting the attribute determination result; Information processing methods.

7. In the information processing device, Based on the time-series changes in the detection information from the radar device, multiple pieces of detection information detected simultaneously at a certain time are combined as detection information for a specific detection target, classifying attributes of the detection target based on the combined detection information, and determining the attributes of the detection target based on likelihood information for each attribute candidate obtained by integrating the classification results at multiple times; If likelihood information of a first attribute that is highest in the likelihood information for each of the attribute candidates exceeds a threshold, the first attribute is determined to be an attribute of the detection target; if the likelihood information of the first attribute does not exceed the threshold, the attribute of the detection target is determined based on a time-spatial feature (TSF), which is a statistical value of a feature amount obtained from the combined detection information of the detection target at the multiple times; outputting the attribute determination result; A program that executes a process.

8. a combining unit that combines multiple pieces of detection information simultaneously detected at a certain time based on a time-series change in the detection information obtained by the radar device as detection information of a specific detection target; a discrimination unit that discriminates attributes of the detection target based on the combined detection information; an output unit that outputs the attribute determination result; Equipped with the determination unit determines an attribute of the detection target based on a time-spatial feature (TSF), which is a statistical value of a feature amount obtained from the combined detection information of the detection target at the multiple times, in the detection information at the multiple times; The TSF is at least one of a maximum, an average, a minimum, and a variance of a feature obtained from the combined detection information at the multiple times. Information processing device.

Images