Sensor synthesis method and sensor synthesis device

Abstract

To accurately acquire the position and speed of an object while suppressing the influences received from an environment condition.SOLUTION: A sensor synthesizing method according to the present disclosure performs: acquiring a first plot that has at least the position information of an object, and is mapped to a two-dimensional map; acquiring a cluster that is constituted by a plurality of clustered second plots having the position and speed information of the object, and mapped to the two-dimensional map; based on the positional relations of the first plot and cluster, classifying the first plot, with the cluster existing in its vicinity, as high accuracy data (1), the first plot, with the cluster not existing in its vicinity, as uncertain data (2), and the cluster, with the first plot nonexistent in its vicinity, as low certainty data (3).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a sensor merging method and device for determining the position and velocity of an object using an image recognition sensor and a complementary sensor. [Background technology]

[0002] There is known a technology for recognizing objects using an image recognition sensor that uses a camera (see, for example, Patent Document 1). An image recognition sensor has the advantage of being able to classify attributes based on the appearance of an object. There is also known a technology for measuring distance using a distance measurement sensor that uses radar (see, for example, Patent Document 2). A distance measurement sensor that uses radar has the advantage of being accurate in distance measurement and being less affected by environmental conditions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-97557 [Patent Document 2] Patent No. 6109998 [Non-patent literature]

[0004] [Non-Patent Document 1] “YOLO9000: Better, Faster, Stronger”, Joseph Redmon, Ali Farhadi, University of Washington, Allen Institute for AI, http: / / pjreddie.com / yolo9000 / , retrieved March 25, 2021 Summary of the Invention [Problem to be solved by the invention]

[0005] However, camera-based image recognition sensors have the problem of being easily affected by environmental conditions such as weather and brightness, and measuring object position relative to surrounding objects can be inaccurate. Furthermore, radar-based distance measurement sensors use radio wave reflection, so they generally have difficulty detecting smaller or non-metallic objects compared to image recognition sensors. Furthermore, lowering the radar filtering threshold to unnecessarily increase detection sensitivity also makes it more susceptible to picking up noise.

[0006] In order to solve the above problem, an object of the present invention is to accurately acquire the position and velocity of an object while suppressing the influence of environmental conditions. [Means for solving the problem]

[0007] To achieve the above object, the sensor compositing method and sensor compositing device disclosed herein compare and classify object position information acquired by two different means on a two-dimensional map.

[0008] Specifically, the sensor synthesis method according to the present disclosure includes: obtaining a first plot having at least position information of the object and mapped onto a two-dimensional map; obtaining clusters composed of a plurality of second plots having clustered object position information and velocity information, the clusters being mapped onto the two-dimensional map; and According to the positional relationship between the first plot and the cluster, (1) The first plot in which the clusters exist nearby is called high accuracy data; (2) the first plot where the cluster does not exist nearby is called uncertain data; and (3) The clusters not present in the first plot are low-confidence data; To classify into Do the following.

[0009] Specifically, the sensor synthesis device according to the present disclosure includes: A sensor synthesis device including a data processing unit that processes data from two types of information acquisition devices, The data processing unit acquiring, from one of the information acquisition devices, a first plot having at least position information of an object and mapped onto a two-dimensional map; acquiring, from another of the information acquisition devices, clusters composed of a plurality of second plots having clustered position information and velocity information of objects, the clusters being mapped onto the two-dimensional map; and According to the positional relationship between the first plot and the cluster, (1) The first plot in which the clusters exist nearby is called high accuracy data; (2) the first plot where the cluster does not exist nearby is called uncertain data; and (3) The clusters not present in the first plot are low-confidence data; To classify into Do the following.

[0010] The sensor synthesis method and the sensor synthesis device according to the present disclosure include: The neighboring plot may be a plot where the distance between the first plot and any of the second plots included in the cluster is equal to or less than a predetermined value.

[0011] The data processing unit of the sensor combining method and the sensor combining device according to the present disclosure includes: If there is the first plot classified as uncertain data, Performing the classification based on the positional relationship again on the second plot obtained by changing the parameters; and discarding the first plot that has been reclassified as indeterminate data; may be performed.

[0012] The data processing unit of the sensor combining method and the sensor combining device according to the present disclosure includes: With respect to the first plot classified as the high accuracy data, the position of the second plot that is closest to the first plot among the second plots included in the cluster that exists in the vicinity is determined as the position of the object represented by the first plot, and the average speed of the second plots included in the cluster that exists in the vicinity is determined as the speed of the object represented by the first plot; For the clusters classified as the low-probability data, the center position of the cluster may be taken as the position of the object represented by the cluster, and the average speed of the second plots included in the cluster may be taken as the speed of the object represented by the cluster.

[0013] The sensor synthesis method according to the present disclosure includes: position information of the first plot is acquired by a camera; The position and velocity information of the second plot may be obtained by radar.

[0014] The sensor synthesis device according to the present disclosure comprises: one of the information acquisition devices is a camera; The other of the information acquisition devices may be a radar.

[0015] The sensor synthesis method and sensor synthesis device compare and classify object position information obtained by two different means on a two-dimensional map, thereby making it possible to accurately obtain the object's position and velocity while minimizing the effects of environmental conditions.

[0016] The above inventions can be combined as much as possible. [Effects of the Invention]

[0017] According to the present disclosure, it is possible to accurately acquire the position and velocity of an object while suppressing the influence of environmental conditions. [Brief explanation of the drawings]

[0018] [Figure 1] 1 shows an example of the procedure of a sensor synthesis method according to the present invention. [Figure 2] 3 is a diagram illustrating a process of representing a vehicle position on a two-dimensional map according to the present invention. FIG. [Figure 3] 1 is a diagram illustrating the position calculation and velocity calculation of a radar cluster according to the present invention. FIG. [Figure 4] FIG. 1 is a diagram illustrating the positional relationship between an image recognition plot and a radar cluster according to the present invention. [Figure 5] 10A and 10B are diagrams illustrating the deletion of radar clusters according to the present invention. [Figure 6] FIG. 2 is a diagram illustrating the position and velocity of an image recognition plot according to the present invention. [Figure 7] FIG. 2 is a diagram illustrating pedestrian information according to the present invention. [Figure 8] FIG. 2 is a diagram illustrating the position and velocity of an image recognition plot according to the present invention. [Figure 9] FIG. 2 is a diagram illustrating the position and velocity of an image recognition plot according to the present invention. [Figure 10] FIG. 2 is a diagram illustrating pedestrian information according to the present invention. [Figure 11] FIG. 2 is a diagram illustrating a crosswalk and a Y axis according to the present invention. [Figure 12] FIG. 2 is a diagram illustrating pedestrian information according to the present invention. [Figure 13] FIG. 2 is a diagram illustrating pedestrian information according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings indicate the same components.

[0020] (Embodiment) The procedure of the sensor combining method according to this embodiment is shown in FIG. The sensor synthesis method according to this embodiment includes the following steps: Obtaining a first plot (41) having at least position information of an object and mapped onto a two-dimensional map (step S101); Obtaining a cluster (20, 30) composed of a plurality of second plots (21, 31) having clustered position information and velocity information of the object and mapped to a two-dimensional map (step S102); and Due to the positional relationship between the first plot (41) and the cluster (20, 30), (1) The first plot (41) where clusters (20, 30) are located nearby is considered to be high accuracy data. (2) The first plot, where no clusters are nearby, is called uncertain data, and (3) The first plot (41) shows clusters that do not have neighbors as low probability data. (Steps S103 to S108), The sensor combining device may be a sensor combining device including a data processing unit that processes data from two types of information acquisition devices, and the data processing unit may perform the sensor combining method. Each step will be explained below. Note that, in the following, a pedestrian will be used as an example of an object, but the object is not limited to this.

[0021] The first plot (41) may be considered to be neighboring when the distance between the first plot (41) and any of the second plots (21, 31) included in the cluster (20, 30) is equal to or less than a predetermined value σ.

[0022] Here, the sensor compositing method according to this embodiment receives as input pedestrian information acquired by two types of means and a two-dimensional (hereinafter, "two-dimensional" will be abbreviated as "2D") display of the pedestrian's position. The pedestrian information includes the type, position, speed, etc. of the pedestrian, as will be described later. In this embodiment, a case will be described in which a camera functioning as an image recognition sensor is used to acquire pedestrian information for the first plot (41), and a radar functioning as a complementary sensor is used to acquire pedestrian information for the second plot (21, 31, 51). However, this is not limiting. Furthermore, in the sensor compositing device, one of the two types of information acquisition devices may be a camera, and the other of the two types of information acquisition devices may be a radar. A known technique may be used to map the pedestrian positions acquired by the camera and the radar onto a 2D map (hereinafter, "mapping onto a 2D map" will be abbreviated as "2D mapping") and display them on the 2D map.

[0023] (Step S101) The process of 2D mapping the pedestrian positions acquired by the camera may be performed as follows: Image recognition AI inference is performed on images acquired at predetermined intervals (frames) to acquire pedestrian information (Non-Patent Document 1). Then, the pedestrian positions are 2D mapped. Hereinafter, the pedestrian positions acquired by the camera and 2D mapped are referred to as "image recognition plots." Note that the image recognition plots are linked to the pedestrian information that is the basis of the plots.

[0024] Similarly, the area where the vehicle exists may be represented on a 2D map. 2D mapping of a vehicle will be described with reference to FIG. 2. FIG. 2(a) is a diagram showing an image of a road including a pedestrian crossing captured by a camera, and FIG. 2(b) is a diagram showing the positions of a pedestrian 13 and a vehicle 11 within the image recognition range 10 of FIG. 2(a) on a 2D map. On the 2D map of FIG. 2(b), points L and R at both ends of the bottom of the bounding box 12 of FIG. 2(a) are 2D converted, and a rectangular area with a depth of 2 m, whose base is the edge connecting the 2D converted points L and R, is treated as the vehicle body area 14. Since the radar plot described below is generally strongly affected by the reflective surface of the vehicle body, the exact depth of the vehicle body and the influence of the vehicle body orientation are not considered. Hereinafter, in this embodiment, as shown in FIG. 2(b), the crossing direction of the pedestrian crossing is defined as the Y-axis direction, and the direction perpendicular to the Y-axis direction is defined as the X-axis direction, and the position and velocity are represented by these two components.

[0025] (Step S102) The process of 2D mapping the positions of pedestrians acquired by radar may be performed as follows. Radar data for a specified number of scans is acquired from the time the image frame was captured, and radar output corrections are made, including the Y-axis position based on the camera installation height, speed correction calculations, and reflection cross-section offsets. Then, plots are selected by filtering based on the X-value range, Y-value range, mask range, crosswalk area setting range, and reflection cross-section value. Here, the mask range filter is used to remove unnecessary plots that occur fixedly, such as reflections from stationary objects. The mask range is a rectangular area that the user specifies and registers in advance on a 2D map; approximately 10 rectangular areas can be set, or it can be set automatically.

[0026] As a speed filter, for example, the speed in the Y-axis direction (Y speed) may be filtered into multiple ranges, such as -10 km / h to -0.1 km / h and +0.1 km / h to +10 km / h. Speed ​​filtering is performed for each speed filter, and clustering is performed using DBScan. By dividing the pedestrian speed into several stages using the speed filter and performing clustering for each stage, it is possible to prevent plots of pedestrian speeds that differ significantly from one another from forming clusters. Then, the pedestrian positions obtained by radar and each clustering result are 2D mapped. Hereinafter, the pedestrian positions obtained by radar and 2D mapped will be referred to as "radar plots." Clusters included in the 2D mapped clustering results will be referred to as "radar clusters."

[0027] From the clustering results of 2D mapping, radar cluster information such as the radar cluster speed, radar cluster cross section, and radar cluster position (coordinates) is generated. The calculation of the radar cluster speed and position is explained using Figure 3.

[0028] 3(a) shows radar plots 21-1 to 21-6 that make up radar cluster 20 (not shown), and indicates the speed of each. Specifically, the speed of radar plot 21-1 is −1.0 km / h, the speed of radar plot 21-2 is −2.0 km / h, the speed of radar plot 21-3 is −0.9 km / h, the speed of radar plot 21-4 is −1.2 km / h, the speed of radar plot 21-5 is +2.0 km / h, and the speed of radar plot 21-6 is −3.5 km / h. Note that FIG. 3(a) only considers the speed in the Y-axis direction of FIG. 2(b), and indicates speed in the positive direction of the Y-axis with + and speed in the negative direction of the Y-axis with −.

[0029] Regarding the velocity of a radar cluster, the average value of the Y-axis direction velocities of all radar plots included in one radar cluster (hereinafter, "Y-axis direction velocity" will be abbreviated as "Y velocity") is calculated, discarding the maximum and minimum values, and this is the Y velocity of the radar cluster. Specifically, in Figure 3(a), of the Y velocities of radar plots 21-1 to 21-6, the maximum value is +2.0 km / h for radar plot 21-5, and the minimum value is -3.5 km / h for radar plot 21-6. Therefore, the Y velocity of radar cluster 20 is the average value of the Y velocities of radar plots 21-1 to 21-4, which is approximately -1.3 km / h. Similarly, the cross section is calculated as the average value excluding the maximum and minimum values. Note that, in the case of DBScan, minPts must be 3 or greater. Furthermore, when calculating the velocity and cross section of a radar cluster, if there are two or more identical maximum values, only one data point is discarded, and the other maximum values ​​with the same value are used to calculate the average. The same applies to the minimum value.

[0030] To determine the location of a radar cluster, all radar plots included in a radar cluster, excluding those for which speed has been discarded, are enclosed in a rectangle, and the coordinates of the center point of this rectangle are used as the location of the radar cluster. Note that radar plots with speeds that differ from those around them are considered to be radar errors or areas where the target's limbs are extended, and are not used in position calculations.

[0031] Specifically, as shown in FIG. 3B, the radar cluster 20 ignores the radar plots 21-5 and 21-6, and surrounds the radar plots 21-1 to 21-4 with a rectangle, and the coordinates of the center point of the rectangle are the cluster position 22.

[0032] (Step S103) The positions of the image recognition plots and radar clusters are compared, and for each image recognition plot, it is confirmed whether there are any clustered radar plots within a radius σ from the image recognition plot.The image recognition plot is then linked to the radar cluster containing the radar plots that exist within a radius σ from the image recognition plot.Unclustered radar plots are treated as noise here, so they are ignored even if they are nearby. A specific example of this step is shown in FIG. 4. FIG. 4 is a diagram showing an image recognition plot and a radar plot represented on a 2D map. In FIG. 4, a radar clustered radar plot 21-1 exists within the radius σ of the image recognition plot 41-1, so the image recognition plot 41-1 and the radar cluster 20 are linked. Here, in FIG. 4, a radar plot 51 is a radar plot that has not been radar clustered. Therefore, even though the radar plot 51 exists within the radius σ of the image recognition plot 41-1, it is ignored as noise and is not linked. As shown in FIG. 5, if the cluster position 22 of a radar cluster is within the vehicle body area 14 on the 2D map, the radar cluster is deleted.

[0033] (Step S104) Each image recognition plot and radar cluster is classified as high confidence data, uncertain data, or low confidence data. Specifically, image recognition plots that are confirmed to have radar clusters in the vicinity (within a radius σ) are classified as high-accuracy data. Image recognition plots for which there were no radar clusters in the vicinity (within a radius σ) were classified as uncertain data. Radar clusters that are not associated with any image recognition plots are classified as low-confidence data.

[0034] (Step S105) It is checked whether or not there is uncertain data in the classification result in step S104. If there is uncertain data, the process proceeds to step S106, and if there is no uncertain data, the process proceeds to step S109.

[0035] (Step S106) Crossing person information is reacquired by radar, and the crossing person's position is 2D mapped, and a radar plot and radar cluster are reacquired, similar to the method described above. When reacquiring crossing person information by radar, the radar parameter settings used in the initial acquisition may be changed. For example, parameter tables may be prepared for the initial acquisition and reacquisition, and these may be switched. Examples of parameters include the radar scan period, Y-axis position, and cross section offset, and these may be changed. Furthermore, when filtering is performed, the range may be changed. Then, similar to the method described above, the speed, cross section, and position of the radar cluster are calculated from the reacquired radar cluster. Note that, because reacquisition of crossing person information by radar is not performed more than twice in a row, after calculating the speed, cross section, and position of the radar cluster from the reacquired radar cluster, it is preferable to restore the radar parameters used in the initial acquisition from the parameters used in the reacquisition. Then, the image recognition plot, which is uncertain data, is compared with the reacquired radar plot and radar cluster in the same manner as in step S103. Then, again, the image recognition plot and radar cluster are classified as high-accuracy data, uncertain data, or low-accuracy data in the same manner as in step S104.

[0036] (Step S107) It is checked whether or not there is uncertain data in the classification result in step S106. If there is uncertain data, the process proceeds to step S108, and if there is no uncertain data, the process proceeds to step S109.

[0037] (Step S108) Image recognition plots that are deemed to be inconclusive data are discarded because they are likely to be false positives in image recognition.

[0038] (Step S109) Crossing person information is extracted from the high-accuracy data and the low-accuracy data. Specifically, for the image recognition plot, which is the high-accuracy data, the position of the closest radar plot in the radar cluster closest to the image recognition plot is taken as the crossing person position, and the speed of the closest radar cluster is taken as the crossing person speed. In addition, the crossing person type is taken from the crossing person type in the crossing person information linked to the image recognition plot.

[0039] A case where there is one radar cluster close to the image recognition plot is shown in Figure 6. The position and velocity of the image recognition plot 41-1 are the position (x, y) and velocity s of the radar cluster 20 of the closest radar plot 21-1. As a result, the pedestrian information based on the image recognition plot 41-1 has a pedestrian type of adult, a pedestrian position of (x, y), and a pedestrian speed of s, as shown in Fig. 7. In Fig. 7, the pedestrian type of the pedestrian information linked to the image recognition plot 41-1 is assumed to be adult.

[0040] A case where there are multiple radar clusters close to the image recognition plot 41-1 is shown in Figure 8. For the image recognition plot 41-1, the position of the closest radar plot 21-1 in the radar cluster 20 closest to the image recognition plot 41-1 is taken as the pedestrian position, and the speed of the closest radar cluster 20 is taken as the pedestrian speed.

[0041] FIG. 9 shows a case where multiple image recognition plots are linked to the same radar cluster. In FIG. 9(a), image recognition plots 41-1 and 41-2 are linked to the radar cluster 20. In this case, as shown in FIG. 9(b), the image recognition plot 41-1 sets the position (x1, y1) of the radar plot 21-1 as the pedestrian position and the speed s of the radar cluster as the pedestrian speed. Also, as shown in FIG. 9(c), the image recognition plot 41-2 sets the position (x2, y2) of the radar plot 21-2 as the pedestrian position and the speed s of the radar cluster as the pedestrian speed. Note that in FIG. 9, the pedestrian type in the pedestrian information linked to the image recognition plot 41-1 is assumed to be an adult, and the pedestrian type in the pedestrian information linked to the image recognition plot 41-2 is assumed to be a child.

[0042] Figure 10 shows pedestrian information based on radar clusters, which are low-accuracy data. For radar clusters, which are low-accuracy data, the position of the radar cluster is taken as the pedestrian position, and the speed of the radar cluster is taken as the pedestrian speed. In Figure 10, the position of the radar cluster is taken as (x, y), and the speed of the radar cluster is taken as s. Since low-accuracy data does not allow image recognition, the type of pedestrian is treated as unknown. When counting the number of people crossing the street, they are initially counted as one person, but as there is a possibility that multiple people are lined up, this is treated as a conditional count.

[0043] (Step S110) The predicted crossing completion time and average crossing speed are calculated from the crossing person information extracted in step S109. The predicted crossing completion time is the time remaining until there are no more crossing people on the crosswalk. Specifically, the remaining time until each crossing person finishes crossing is calculated from the Y-axis speed and Y-axis position of each crossing person, and the time of the crossing person with the longest remaining time is output.

[0044] The relationship between the crosswalk and the Y-axis is shown in Figure 11. As shown in Figure 11, the crossing direction of crosswalk 15 is the direction of the Y-axis, and the ends of the crosswalk are 12 m and 27 m on the Y-axis. The positive direction of the Y-axis represents the pedestrian speed as +speed, and the negative direction of the Y-axis represents -speed. For example, if a pedestrian is crossing crosswalk 15 at +speed, this means that they are entering crosswalk 15 from Y-axis position 12 m and crossing toward Y-axis position 27 m.

[0045] The predicted crossing completion time for four pedestrians shown in Figure 12 will be described below when they cross the crosswalk shown in Figure 11. Note that in the calculation of the predicted crossing completion time below, decimals are rounded up, but this is not limited to this. Since pedestrian 1's speed is +5.0 km / h, he is crossing in the positive direction of the Y-axis (from Y-axis position 12 m toward Y-axis position 27 m in Figure 11), and his current Y-axis position is 12.0 m. Therefore, for pedestrian 1, the remaining distance of the crosswalk 15 is 27 m - 12 m = 15 m. Therefore, since pedestrian 1's speed is +5.0 km / h and the remaining distance of the crosswalk 15 is 15 m, it can be predicted that it will take approximately 11 seconds for pedestrian 1 to complete crossing. Calculating the same as for pedestrian 1, it can be predicted that it will take approximately 15 seconds for pedestrian 2 to complete crossing.

[0046] Since pedestrian 3's speed is -5.0 km / h, he is crossing in the negative direction of the Y axis (from Y-axis position 27 m to Y-axis position 12 m in Figure 11), and his current Y-axis position is 18.0 m. Therefore, the remaining distance of crosswalk 15 for pedestrian 3 is 12 m - 18 m = -6 m. Therefore, since pedestrian 3's speed is -5.0 km / h and the remaining distance of crosswalk 15 is -6 m, it can be predicted that it will take approximately 5 seconds for pedestrian 3 to complete crossing. Calculating the same as for pedestrian 3, it can be predicted that it will take 3 seconds for pedestrian 4 to complete crossing.

[0047] The time it takes for pedestrian 1 to complete crossing is approximately 11 seconds, for pedestrian 2 it is approximately 15 seconds, for pedestrian 3 it is 5 seconds, and for pedestrian 4 it is 3 seconds, so taking the maximum of these, the predicted time to complete crossing is 15 seconds.

[0048] The average tram speed will be specifically calculated using the tram information shown in Figure 13. The speeds can be averaged using either the root mean square or the absolute value mean. In Figure 13, the speed of tram 1 is +5.0 km / h, the speed of tram 2 is +3.0 km / h, the speed of tram 3 is -5.0 km / h, and the speed of tram 4 is -5.0 km / h, so the root mean square is approximately 4.6 km / h and the absolute value mean is 4.5 km / h.

[0049] The data processing unit of the sensor combining method and sensor combining device according to this embodiment may output data after performing step S110. Examples of the output data include the number of pedestrians, predicted crossing completion time, average pedestrian speed, and each piece of pedestrian information (type of pedestrian, pedestrian position, pedestrian speed, etc.). The number of digits for each piece of data is arbitrary; for example, the predicted crossing completion time may be rounded up, the average pedestrian speed may be output to one decimal place, and the position and speed of each piece of pedestrian information may be output to one decimal place.

[0050] As described above, the sensor synthesis method and sensor synthesis device compare and classify the position information of an object obtained by two types of means on a two-dimensional map, thereby making it possible to accurately obtain the position and velocity of an object while minimizing the effects of environmental conditions.

[0051] The above inventions can be combined as much as possible. [Industrial Applicability]

[0052] The sensor synthesis method and sensor synthesis device according to the present disclosure can be applied to the sensing technology industry. [Explanation of symbols]

[0053] 10: Image recognition range 11: Vehicle 12: Bounding box 13: Crossing 14: Body area 15: Crosswalk 20, 30: Radar cluster 21, 31, 51: Radar plot 22: Cluster position 41: Image Recognition Plot

Claims

1. obtaining a first plot having at least position information of the object and mapped onto a two-dimensional map; obtaining clusters composed of a plurality of second plots having clustered object position information and velocity information, the clusters being mapped onto the two-dimensional map; and According to the positional relationship between the first plot and the cluster, (1) The first plot in which the clusters exist nearby is high-accuracy data; (2) the first plot in which the clusters are not nearby is called uncertain data; and (3) The clusters that are not in the vicinity of the first plot are low-confidence data; To classify into A sensor synthesis method for performing If there is the first plot classified as uncertain data, Performing the classification based on the positional relationship again on the second plot obtained by changing the parameters; and discarding the first plot that has again been classified as uncertain data; A sensor synthesis method characterized by:

2. 2. The sensor composition method according to claim 1, wherein the first plot is considered to be close to the second plot when the distance between the first plot and any of the second plots included in the cluster is equal to or less than a predetermined value.

3. With respect to the first plot classified as the high accuracy data, the position of the second plot that is closest to the first plot among the second plots included in the cluster existing in the vicinity is determined as the position of the object represented by the first plot, and the average speed of the second plots included in the cluster existing in the vicinity is determined as the speed of the object represented by the first plot; For the clusters classified as the low-probability data, the center position of the cluster is set as the position of the object represented by the cluster, and the average speed of the second plots included in the cluster is set as the speed of the object represented by the cluster.

3. The sensor combining method according to claim 1 or 2.

4. The position information of the first plot is acquired by a camera; The position information and velocity information of the second plot are obtained by radar.

4. The sensor combining method according to claim 1, wherein:

5. A sensor synthesis device including a data processing unit that processes data from two types of information acquisition devices, The data processing unit acquiring, from one of the information acquisition devices, a first plot having at least position information of an object and mapped onto a two-dimensional map; acquiring, from another of the information acquisition devices, clusters composed of a plurality of second plots having clustered position information and velocity information of objects, the clusters being mapped onto the two-dimensional map; and According to the positional relationship between the first plot and the cluster, (1) The first plot in which the clusters exist nearby is high-accuracy data; (2) the first plot in which the clusters are not nearby is called uncertain data; and (3) The clusters that are not in the vicinity of the first plot are low-confidence data; To classify into The present invention is characterized by carrying out the following: Furthermore, if the first plot classified as uncertain data exists, Performing the classification based on the positional relationship again on the second plot obtained by changing the parameters; and discarding the first plot that has again been classified as uncertain data; A sensor synthesis device characterized by performing the above.

6. The sensor composition device according to claim 5 , wherein the first plot is considered to be close to the second plot when the distance between the first plot and any of the second plots included in the cluster is equal to or less than a predetermined value.

7. The data processing unit With respect to the first plot classified as the high accuracy data, the position of the second plot that is closest to the first plot among the second plots included in the cluster existing in the vicinity is determined as the position of the object represented by the first plot, and the average speed of the second plots included in the cluster existing in the vicinity is determined as the speed of the object represented by the first plot; For the clusters classified as the low-probability data, the center position of the cluster is set as the position of the object represented by the cluster, and the average speed of the second plots included in the cluster is set as the speed of the object represented by the cluster.

7. The sensor combining device according to claim 5 or 6.

8. one of the information acquisition devices is a camera; The other of the information acquisition devices is a radar.

8. The sensor combining device according to claim 5, wherein the sensor combining device is a sensor combining device.

Images