Data processing device, data processing method and data processing program

Abstract

The noise determination unit (106) analyzes LiDAR data obtained by the LiDAR (200) scanning a space and millimeter-wave data in which detection points detected by the millimeter-wave radar (300) scanning the space are plotted, and determines point data that is noise from a plurality of point data included in the LiDAR data. The noise removal unit (107) removes point data that is determined to be noise by the noise determination unit (106) from the LiDAR data.

Description

[Technical Field]

[0001] The present disclosure relates to sensor data. [Background technology]

[0002] Some processes that use sensor data require the acquired sensor data in its entirety. For example, a process that uses image data acquired from a camera requires the acquired image data in its entirety. In such processing, there is a problem that even if the acquired sensor data is unreliable due to noise, the sensor data must be used as is.

[0003] Patent Document 1 discloses a sensor noise removal device that can convert sensor data whose reliability has been reduced due to noise into sensor data in a state where no noise is present. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2024-107047 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology of Patent Document 1 uses image processing technology to determine blurred areas as noise areas in image data, and then determines the presence or absence of obstacles in the noise areas from sensor data other than the camera, and replaces the noise areas. However, there are situations where blurred areas cannot be determined as noise using only data from a single sensor. For example, in poor environments such as fog, it is difficult to determine blurred areas as noise using only point cloud data acquired by LiDAR. LiDAR stands for Light Detection and Ranging.

[0006] In view of the above circumstances, the present disclosure has as its main object to accurately remove noise contained in data acquired by a sensor. [Means for solving the problem]

[0007] The data processing device according to the present disclosure comprises: a noise determination unit that analyzes point cloud data obtained by scanning a space with a first sensor and plot data in which sensed points sensed by a second sensor, which is a different type of sensor from the first sensor, scanning the space from a scanning position adjacent to the scanning position of the first sensor, are plotted, and determines point data that is noise among a plurality of point data included in the point cloud data; The noise removal unit removes point data determined to be noise by the noise determination unit from the point cloud data. [Effects of the Invention]

[0008] According to the present disclosure, noise contained in data acquired by a sensor can be removed with high accuracy. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram showing an example of the functional configuration of a filtering processing device according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the hardware configuration of the filtering processing device according to the first embodiment. [Figure 3] 4 is a flowchart showing an example of the operation of the filtering processing device according to the first embodiment. [Figure 4] 5A to 5C are diagrams for explaining the operation of extracting a free space according to the first embodiment. [Figure 5] FIG. 3 is a diagram illustrating a noise determination operation according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of the functional configuration of a filtering processing device according to a second embodiment. [Figure 7] 10 is a flowchart showing an example of the operation of the filtering processing device according to the second embodiment. [Figure 8] 10 is a flowchart showing an example of the operation of the filtering processing device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described with reference to the drawings. In the following description of the embodiments and the drawings, the same reference numerals denote the same or corresponding parts.

[0011] Embodiment 1 ***Configuration Description*** FIG. 1 shows an example of the functional configuration of a filtering processing device 100 according to this embodiment. FIG. 2 shows an example of the hardware configuration of the filtering processing device 100 according to this embodiment. The filtering processing device 100 is a data processing device that processes data acquired by a sensor. The operation procedure of the filtering processing device 100 corresponds to a data processing method. Furthermore, a program that realizes the operation of the filtering processing device 100 corresponds to a data processing program.

[0012] The filtering processing device 100 is connected to the LiDAR 200 and the millimeter-wave radar 300. The LiDAR 200 scans a space with a light beam and obtains point cloud data indicating the distance measurement results as LiDAR data. In the LiDAR data, the position (sensing point) of a reflective object sensed by the LiDAR 200 receiving a reflected wave of a light beam is indicated as point data. The LiDAR 200 corresponds to the first sensor.

[0013] The millimeter-wave radar 300 is a different type of sensor from the LiDAR 200. The millimeter-wave radar 300 also scans the same space as the LiDAR 200 using millimeter waves, and obtains plot data indicating the distance measurement results as millimeter-wave data. The millimeter-wave radar 300 is a radar in the frequency band of 30 GHz to 300 GHz. In the millimeter wave data, the positions (detection points) of reflecting objects detected by the millimeter wave radar 300 receiving reflected millimeter waves are plotted. The millimeter wave radar 300 corresponds to the second sensor.

[0014] The scanning position of the LiDAR 200 (the installation position of the LiDAR 200) and the scanning position of the millimeter-wave radar 300 (the installation position of the millimeter-wave radar 300) are adjacent to each other. It is assumed that the LiDAR 200 and the millimeter-wave radar 300 are mounted on the same moving body. In this embodiment, it is assumed that the LiDAR 200 and the millimeter-wave radar 300 are mounted on the same vehicle. That is, in this embodiment, the distance between the scanning position of the LiDAR 200 and the scanning position of the millimeter-wave radar 300 is within the range of the size of the vehicle. Therefore, the scanning position of the LiDAR 200 and the scanning position of the millimeter-wave radar 300 are almost the same position. The LiDAR 200 and the millimeter wave radar 300 scan, for example, the space ahead of the vehicle.

[0015] The filtering processing device 100 is a computer, and includes, for example, the hardware shown in Fig. 2. Specifically, the filtering processing device 100 includes a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication device 904. The filtering processing device 100 also includes the functional components shown in Fig. 1. The functional components in Fig. 1 (except for the conversion reference information storage unit 105, the same applies below) are realized by, for example, a program. The auxiliary storage device 903 stores programs that realize the functions of these functional components. These programs are loaded from the auxiliary storage device 903 into the main storage device 902. The processor 901 then executes these programs to perform the operations of the functional components in FIG. FIG. 2 shows a schematic diagram of a state in which a processor 901 is executing a program that implements the functions of the functional components of FIG. The conversion reference information storage unit 105 in FIG.

[0016] In FIG. 1, a LiDAR data acquisition unit 101 performs LiDAR data acquisition processing. More specifically, the LiDAR data acquisition unit 101 acquires LiDAR data from the LiDAR 200. In this embodiment, the LiDAR data acquisition unit 101 acquires, as the LiDAR data, point cloud data obtained by the LiDAR 200 scanning the space ahead of the vehicle.

[0017] The millimeter wave data acquisition unit 102 performs millimeter wave data acquisition processing. More specifically, the millimeter-wave data acquiring unit 102 acquires millimeter-wave data from the millimeter-wave radar 300. In the present embodiment, the millimeter-wave data acquiring unit 102 acquires, as the millimeter-wave data, plot data obtained by the millimeter-wave radar 300 scanning the space ahead of the vehicle.

[0018] The LiDAR data coordinate conversion unit 103 performs LiDAR data coordinate conversion processing. More specifically, the LiDAR data coordinate conversion unit 103 performs coordinate conversion of the LiDAR data acquired from the LiDAR data acquisition unit 101.

[0019] The millimeter wave data coordinate conversion unit 104 performs millimeter wave data coordinate conversion processing. More specifically, the millimeter wave data coordinate conversion unit 104 performs coordinate conversion on the millimeter wave data acquired from the millimeter wave data acquisition unit 102 .

[0020] The LiDAR data coordinate conversion unit 103 and the millimeter wave data coordinate conversion unit 104 perform coordinate conversion corresponding to the positional relationship between the scanning position of the LiDAR 200 and the scanning position of the millimeter wave radar 300. In other words, the LiDAR data coordinate conversion unit 103 and the millimeter wave data coordinate conversion unit 104 perform coordinate conversion so that the coordinate system of the LiDAR data after the coordinate conversion and the coordinate system of the millimeter wave data after the coordinate conversion are the same. For example, the LiDAR data coordinate conversion unit 103 and the millimeter wave data coordinate conversion unit 104 perform coordinate conversion of the LiDAR data and the millimeter wave data so that they are in a coordinate system based on a specific position of the vehicle. It should be noted that if the millimeter-wave data coordinate conversion unit 104 performs coordinate conversion of the millimeter-wave data in accordance with the coordinate system of the LiDAR data, it is possible to omit the LiDAR data coordinate conversion unit 103. Conversely, if the LiDAR data coordinate conversion unit 103 performs coordinate conversion of the LiDAR data in accordance with the coordinate system of the millimeter-wave data, it is possible to omit the millimeter-wave data coordinate conversion unit 104.

[0021] The conversion reference information storage unit 105 stores the conversion reference information. The conversion reference information is information that the LiDAR data coordinate conversion unit 103 and the millimeter wave data coordinate conversion unit 104 refer to when performing coordinate conversion. The transformation reference information indicates, for example, a specific position of the vehicle that serves as a reference for coordinate transformation, the scanning position and direction of the LiDAR 200, the scanning position and direction of the millimeter-wave radar 300, and the like. Note that the conversion reference information may be any information that can be used for coordinate conversion by the LiDAR data coordinate conversion unit 103 and the millimeter wave data coordinate conversion unit 104.

[0022] The noise determination unit 106 performs noise determination processing. More specifically, the noise determination unit 106 analyzes the LiDAR data after the coordinate transformation and the millimeter-wave data after the coordinate transformation, and determines point data that is noise among the plurality of point data included in the LiDAR data after the coordinate transformation.

[0023] The noise removal unit 107 performs noise removal processing. More specifically, the noise removal unit 107 removes point data determined to be noise by the noise determination unit 106 from the LiDAR data after coordinate transformation.

[0024] The output unit 108 performs output processing. More specifically, the output unit 108 outputs the coordinate-converted LiDAR data from which the noise removal unit 107 has removed the point data (point data determined as noise by the noise determination unit 106).

[0025] ***Explanation of Operation*** FIG. 3 is a flowchart showing an example of the operation of the filtering processing device 100 according to this embodiment.

[0026] In step S101 , the LiDAR data acquisition unit 101 acquires LiDAR data from the LiDAR 200 . The LiDAR data acquisition unit 101 outputs the LiDAR data to the LiDAR data coordinate conversion unit 103.

[0027] In parallel, in step S102, the millimeter wave data acquisition unit 102 acquires millimeter wave data from the millimeter wave radar 300. The millimeter wave data acquisition unit 102 outputs the millimeter wave data to the millimeter wave data coordinate conversion unit 104 .

[0028] Note that steps S101 and S102 are performed synchronously, that is, the LiDAR data acquired in step S101 and the millimeter-wave data acquired in step S102 are synchronized.

[0029] In step S103, the LiDAR data coordinate conversion unit 103 acquires the LiDAR data from the LiDAR data acquisition unit 101. Then, the LiDAR data coordinate conversion unit 103 performs coordinate conversion of the LiDAR data by referring to the conversion reference information. The LiDAR data coordinate conversion unit 103 outputs the LiDAR data after the coordinate conversion to the noise determination unit 106. In the following description of the processing, unless otherwise noted, "LiDAR data" means the LiDAR data after the coordinate conversion.

[0030] In step S104, the millimeter-wave data coordinate conversion unit 104 acquires millimeter-wave data from the millimeter-wave data acquisition unit 102. Then, the millimeter-wave data acquisition unit 102 performs coordinate conversion of the millimeter-wave data by referring to the conversion reference information. The millimeter-wave data coordinate conversion unit 104 outputs the millimeter-wave data after the coordinate conversion to the noise determination unit 106. In the following description of the processing, unless otherwise noted, "millimeter-wave data" means the millimeter-wave data after the coordinate conversion.

[0031] In step S105, the noise determination unit 106 analyzes the millimeter wave data and extracts free spaces. The free space is a subspace in front of the vehicle where no objects exist. The process of step S105 will be described in detail later.

[0032] In step S106, the noise determination unit 106 determines that point data present in the area corresponding to the free space extracted in step S105 in the LiDAR data is noise. The noise determination unit 106 outputs the LiDAR data and noise information notifying the point data determined to be noise to the noise removal unit 107. The process of step S106 will be described in detail later.

[0033] In step S107, the noise removal unit 107 removes the point data determined as noise by the noise determination unit 106 from the LiDAR data. That is, the noise removal unit 107 removes the point data notified by the noise information from the LiDAR data. The noise removal unit 107 outputs the LiDAR data after removing the point data to the output unit 108. The noise removal unit 107 may re-transform the coordinate-transformed LiDAR data after removing the point data into the original coordinate system, and output the re-transformed LiDAR data to the output unit 108.

[0034] In step S108, the output unit 108 outputs the LiDAR data from which the point data determined to be noise has been removed to a predetermined output destination. For example, the output unit 108 outputs the LiDAR data after the point data determined to be noise has been removed to a subsequent stage of processing that performs subsequent data processing using the LiDAR data.

[0035] Next, the process of step S105 will be described in detail.

[0036] FIG. 4 shows an example of millimeter wave data after coordinate transformation. In FIG. 4, the millimeter-wave radar origin corresponds to the scanning position of the millimeter-wave radar 300 (the installation position of the millimeter-wave radar 300). A sector-shaped area 310 centered on the millimeter-wave radar origin shown in FIG. 4 represents the space ahead of the vehicle. Nine divided areas 340 within the sector-shaped area 310 correspond to divided spaces obtained by virtually dividing the space ahead of the vehicle. Although there are nine divided areas 340 in FIG. 4, the number of divided areas 340 is any number equal to or greater than two. In other words, the number of divisions when virtually dividing the space is any number equal to or greater than two. The points 320 (black circles) and the points 330 (x) in FIG. 4 respectively represent sensing points obtained by the millimeter wave radar 300 scanning the space ahead of the vehicle. The millimeter-wave radar 300 cannot detect particles such as fog, rain, smoke, and aerosols. Therefore, points 320 and 330 are detection points obtained by detecting objects in the space ahead of the vehicle that are much larger than fog, rain, smoke, aerosols, etc. Point 330 represents the detection point closest to the millimeter-wave radar origin in each divided area 340 (hereinafter referred to as the nearest point). Point 320 represents a detection point other than the nearest point. The noise determination unit 106 extracts the nearest point 330 for each divided area 340. Then, for each divided area 340, the noise determination unit 106 extracts a partial space in the space in front of the vehicle that is closer to the millimeter-wave radar origin than the nearest point 330 as a free space. In the region 340a, the partial space from the millimeter wave radar origin to the nearest point 330a is free space.

[0037] If the millimeter-wave data is two-dimensional data, the noise determination unit 106 may calculate the range of the free space in the height direction using the FOV of the millimeter-wave radar 300, and define a three-dimensional free space. FOV is an abbreviation for Field of View.

[0038] Next, the process of step S106 will be described in detail.

[0039] FIG. 5 shows the nearest neighbor point 330 from FIG. 4 superimposed on the LiDAR data. 5, the LiDAR origin corresponds to the scanning position of the LiDAR 200 (the installation position of the LiDAR 200). A sector-shaped region 210 centered on the LiDAR origin represents the space in front of the vehicle. In other words, the sector-shaped region 210 corresponds to the sector-shaped region 310 shown in FIG. 4. Furthermore, the nine divided regions 240 in the sector-shaped region 210 correspond to the nine divided regions 340 shown in FIG. 4, respectively. 5, the points (black squares) and points (black triangles) denoted by reference numerals 220 and 230 represent sensing points (sensing positions) obtained by scanning the space ahead of the vehicle by the LiDAR 200. In other words, the points 220 and 230 are point data included in the LiDAR data. The LiDAR 200 can also detect particles such as fog, rain, smoke, and aerosols. Therefore, the points 220 and 230 may be detection points obtained by detecting fog, rain, smoke, aerosols, etc. On the other hand, no objects sufficiently larger than particles exist in the subspace identified as free space in the analysis of millimeter-wave data. Therefore, detection points existing in the region corresponding to free space are likely to be detection points for fog, rain, smoke, aerosols, etc. The point with reference numeral 230 is point data that exists in the area of ​​the LiDAR data that corresponds to the free space in Fig. 4. The point with reference numeral 220 is point data that exists outside the area that corresponds to the free space. The noise determination unit 106 determines that the point data of the code 230 existing in the region corresponding to the free space is noise. In the divided region 240a on the right side of Fig. 5, the noise determination unit 106 determines that the point data 230a existing in the region from the LiDAR origin to the nearest point 330a is noise. On the other hand, since the point data 220a is located farther from the LiDAR origin than the nearest point 330a, the noise determination unit 106 does not determine that the point data 220a is noise.

[0040] The noise determination unit 106 generates noise information that notifies the point data determined to be noise, and outputs the generated noise information to the noise removal unit 107 together with the LiDAR data. The noise information indicates, for example, the identifier of the point data that the noise determination unit 106 has determined to be noise and the position in the LiDAR data. As described above, the noise removal unit 107 removes the point data notified by the noise information from the LiDAR data.

[0041] ***Explanation of the effect of the embodiment*** As described above, according to this embodiment, noise contained in LiDAR data can be removed with high accuracy. According to this embodiment, particularly when point data of fog, rain, smoke, aerosols, etc. is included in the LiDAR data, this point data can be removed with high accuracy. Furthermore, since the LiDAR data after noise removal can be provided to subsequent processing, malfunctions in the subsequent processing can be reduced.

[0042] Embodiment 2 In this embodiment, an example will be described in which it is verified whether or not point data determined to be noise by the noise determination unit 106 is noise. In this embodiment, differences from the first embodiment will be mainly described. The matters not explained below are the same as those in the first embodiment.

[0043] ***Configuration Description*** FIG. 6 shows an example of the functional configuration of the filtering processing device 100 according to this embodiment. 6, compared to FIG. 1, a feature amount calculation unit 111, an area division unit 112, and a noise verification unit 113 are added. The functions of the feature amount calculation unit 111, the region partitioning unit 112, and the noise verification unit 113 are also realized by a program, similar to the LiDAR data acquisition unit 101, etc. The program that realizes the functions of the feature amount calculation unit 111, the region partitioning unit 112, and the noise verification unit 113 is executed by the processor 901.

[0044] In this embodiment, the LiDAR data coordinate conversion unit 103 outputs the LiDAR data after the coordinate conversion to the noise determination unit 106 as well as to the feature amount calculation unit 111.

[0045] The feature amount calculation unit 111 performs feature amount calculation processing. More specifically, the feature amount calculation unit 111 acquires the LiDAR data after coordinate transformation from the LiDAR data coordinate transformation unit 103. Then, the feature amount calculation unit 111 calculates a feature amount for each point data included in the acquired LiDAR data after coordinate transformation.

[0046] The region dividing unit 112 performs region dividing processing. More specifically, the area dividing unit 112 virtually divides the space in front of the vehicle into a non-verification area and a verification area. In this embodiment, the area division unit 112 designates the area of ​​the road on which the vehicle is traveling in the space ahead of the vehicle as the verification non-target area. Also, the area division unit 112 designates the area outside the road in the space ahead of the vehicle as the verification target area. In this way, the area division unit 112 divides the space ahead of the vehicle into the verification non-target area and the verification target area. The region dividing unit 112 outputs region range definition information that defines the range of the region not to be verified and the range of the region to be verified in the LiDAR data after the coordinate transformation to the noise verification unit 113.

[0047] In this embodiment, it is assumed that the subsequent processing is vehicle control using LiDAR data. In such vehicle control, it is desirable to remove noise as much as possible within the roadway in order to detect obstacles. Noise increases the possibility of false detection, so it is desirable to remove noise. For this reason, it is desirable not to restore point data that has been determined to be noise within the roadway. On the other hand, outside the road, it is desirable to secure as many point data as possible for vehicle self-localization. Therefore, it is desirable to verify point data determined to be noise and restore point data that were removed due to erroneous determination. From this perspective, in this embodiment, the area division unit 112 designates the area of ​​the road as a non-verification area, and designates the area outside the road as a verification area. Note that the region partitioning unit 112 may specify the non-verification target region and the verification target region using a different method.

[0048] In this embodiment, the noise removal unit 107 outputs the noise information acquired from the noise determination unit 106 to the noise verification unit 113 .

[0049] The noise verification unit 113 performs noise verification processing. More specifically, the noise verification unit 113 verifies whether or not the point data determined to be noise by the noise determination unit 106 is noise. The noise verification unit 113 can identify the point data determined to be noise by the noise determination unit 106 and removed by the noise removal unit 107 by referring to the noise information. In this embodiment, the noise verification unit 113 follows the determination result by the noise determination unit 106 for point data that exists in a non-verification target area in the LiDAR data after coordinate transformation. That is, the noise verification unit 113 does not verify whether or not the point data that exists in a non-verification target area in the LiDAR data after coordinate transformation is noise. On the other hand, for point data that exists in a verification target area in the LiDAR data after coordinate transformation, the noise verification unit 113 verifies whether or not the point data that has been determined to be noise by the noise determination unit 106 is noise. The noise verification unit 113 evaluates the feature amount of the point data determined to be noise by the noise determination unit 106, and verifies whether the point data determined to be noise by the noise determination unit 106 is noise or not. This feature amount is the feature amount calculated by the feature amount calculation unit 111. Furthermore, if the noise verification unit 113 determines as a result of the verification that the point data determined to be noise by the noise determination unit 106 is not noise, the noise verification unit 113 restores the point data determined to be noise by the noise determination unit 106 and removed by the noise removal unit 107. The noise verification unit 113 outputs the LiDAR data after restoring the point data to the output unit 108.

[0050] The output unit 108 outputs the LiDAR data after the point data has been restored by the noise verification unit 113.

[0051] The elements other than the LiDAR data coordinate conversion unit 103, the noise removal unit 107, the output unit 108, the feature amount calculation unit 111, the region partitioning unit 112, and the noise verification unit 113 are the same as those shown in Fig. 1. Therefore, detailed description of these elements will be omitted.

[0052] ***Explanation of Operation*** FIG. 7 is a flowchart showing an example of the operation of the filtering processing device 100 according to this embodiment.

[0053] In parallel with steps 101 to S107 described in the first embodiment, in step S111, the feature amount calculation unit 111 calculates the feature amounts of the point data of the LiDAR data. The feature amount calculation unit 111 acquires the LiDAR data after the coordinate transformation from the LiDAR data coordinate transformation unit 103. Then, the feature amount calculation unit 111 calculates a feature amount for each point data of the LiDAR data after the coordinate transformation. For example, the feature amount calculation unit 111 calculates, as the feature amount, statistical values ​​(maximum value, variance, average) of reflection intensity, tendencies of primary reflection and secondary reflection, slice feature amount, and the like. The feature amount calculation unit 111 outputs the calculated feature amount for each point data to the noise verification unit 113 . In the following description of the processing, unless otherwise noted, "LiDAR data" refers to the LiDAR data after coordinate transformation.

[0054] In step S112, the noise verification unit 113 verifies the point data that has been determined to be noise by the noise determination unit . Details of step S112 will be described later with reference to FIG.

[0055] In step S113, the noise verification unit 113 determines whether or not there is restoration target point data. The restoration target point data is point data that the noise verification unit 113 has determined to be restored as a result of the verification in step S112. If there is restoration target point data, the process proceeds to step S 114. On the other hand, if there is no restoration target point data, the process proceeds to step S 115. Note that if there is no restoration target point data, the noise verification unit 113 outputs the LiDAR data after the point data has been deleted by the noise removal unit 107 to the output unit 108 as is.

[0056] In step S114, 116 restores the restoration target point data in the LiDAR data. As long as the restoration target point data can be restored to the LiDAR data at the same position, 116 may restore the restoration target point data in any manner. 116 outputs the LiDAR data after restoring the restoration target point data to the output unit 108.

[0057] If there are multiple restoration target point data, steps S113 and S114 are performed for each restoration target point data.

[0058] In step S115, the output unit 108 outputs the LiDAR data to a predetermined output destination. When step S114 is performed, the output unit 108 outputs the LiDAR data after the restoration target point data has been restored. On the other hand, if step S114 is not performed, the LiDAR data after the point data has been deleted by the noise removal unit 107 is output.

[0059] FIG. 8 is a flowchart showing the details of step S112 in FIG.

[0060] In step S1121, the noise verification unit 113 selects any one of the point data determined to be noise. The noise verification unit 113 refers to the noise information acquired from the noise removal unit 107 and identifies the point data determined to be noise.

[0061] Next, in step S1122, the noise verification unit 113 determines whether or not the point data selected in step S1121 is in the verification target area. The noise verification unit 113 compares the range of the verification target area indicated in the area range definition information acquired from the feature calculation unit 111 with the position of the point data indicated in the noise information, and determines whether the point data selected in step S1121 is within the verification target area. If the point data selected in step S1121 is in the verification target area, the process proceeds to step S1123. On the other hand, if the point data selected in step S1121 is in the non-verification target area, the process ends.

[0062] In step S1123, the noise verification unit 113 determines whether or not the feature amount of the point data selected in step S1121 matches the restoration condition. The restoration condition is, for example, that the feature amount exceeds a threshold value. If the feature amount of the point data selected in step S1121 exceeds the threshold, the noise verification unit 113 determines that the feature amount satisfies the restoration condition. The noise verification unit 113 compares the feature amount for each point data notified by the feature amount calculation unit 111 with a threshold value. If the feature amount of the point data selected in step S1121 matches the restoration condition, the process proceeds to step S1124. On the other hand, if the feature amount of the point data selected in step S1121 does not match the restoration condition, the process ends.

[0063] In step S1124, the noise verification unit 113 designates the point data selected in step S1121 as restoration target point data.

[0064] If there are multiple pieces of point data determined to be noise, the noise verification unit 113 performs the processes of steps S1121 to S1124 for each piece of point data. When the noise verification unit 113 has performed the processes of steps S1121 to S1124 for all of the point data determined to be noise, the flow in Fig. 8 is completed. Thereafter, the processes from step S113 onwards in Fig. 7 are performed.

[0065] ***Explanation of the effect of the embodiment*** In this embodiment, it is possible to set the filtering strength of the point cloud data according to the subsequent processing, and therefore, according to this embodiment, it is possible to extract point cloud data suitable for the subsequent processing.

[0066] Although the first and second embodiments have been described above, these two embodiments may be combined and implemented. Alternatively, one of these two embodiments may be partially implemented. Alternatively, these two embodiments may be partially combined and implemented. Furthermore, the configurations and procedures described in these two embodiments may be modified as necessary.

[0067] ***Additional hardware configuration information*** Here, a supplementary explanation of the hardware configuration of the filtering processing device 100 will be given. The processor 901 is a CPU, a DSP, etc. CPU stands for Central Processing Unit, and DSP stands for Digital Signal Processor. The main memory device 902 is a RAM, which stands for Random Access Memory. The auxiliary storage device 903 is a ROM, flash memory, HDD, etc. ROM stands for Read Only Memory. HDD stands for Hard Disk Drive. The communication device 904 is an electronic circuit that performs the communication processing of data. The communication device 904 is, for example, a communication chip or a NIC, which stands for Network Interface Card.

[0068] The auxiliary storage device 903 also stores an OS, which stands for Operating System. At least a part of the OS is executed by the processor 901 . The processor 901 executes at least a part of the OS, and also executes a program that implements the functions of the functional components shown in FIG. The processor 901 executes the OS, which performs task management, memory management, file management, communication control, and the like. In addition, at least one of information, data, signal values, and variable values ​​indicating the results of processing of the functional components shown in Figure 1 is stored in at least one of the main memory device 902, the auxiliary memory device 903, and the register and cache memory within the processor 901. 1 may be stored on a portable recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, a DVD, etc. The portable recording medium on which the program for realizing the functions of the functional components shown in FIG. 1 is stored may be distributed.

[0069] Furthermore, at least one "unit" of the functional components shown in FIG. 1 may be read as a "circuit," a "step," a "procedure," a "process," or a "circuitry." The filtering processing device 100 may also be realized by a processing circuit. The processing circuit is, for example, a logic IC, a GA, an ASIC, or an FPGA. IC stands for Integrated Circuit. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. In this case, the functional components shown in FIG. 1 are each implemented as part of a processing circuit. In this specification, the term "processing circuitry" refers to a generic concept that encompasses a processor and a processing circuit. That is, a processor and a processing circuit are each specific examples of "processing circuitry."

[0070] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) a noise determination unit that analyzes point cloud data obtained by scanning a space with a first sensor and plot data in which sensed points sensed by a second sensor, which is a different type of sensor from the first sensor, scanning the space from a scanning position adjacent to the scanning position of the first sensor, are plotted, and determines point data that is noise among a plurality of point data included in the point cloud data; a noise removal unit that removes point data determined to be noise by the noise determination unit from the point cloud data. (Appendix 2) The noise determination unit A data processing device as described in Appendix 1, which extracts a free space, which is a subspace within the space where no objects exist, by analyzing the plot data, and determines that point data existing in an area corresponding to the free space in the point cloud data is noise. (Appendix 3) The noise determination unit 3. The data processing device according to claim 2, wherein for each divided space of a plurality of divided spaces obtained by virtually dividing the space, a sensing point closest to a scanning position of the second sensor is extracted as a nearest point, and a partial space within the space that is closer to the scanning position of the second sensor than the nearest point for each of the extracted divided spaces is extracted as the free space. (Appendix 4) The noise determination unit 2. The data processing device according to claim 1, further comprising: analyzing a sensing result of the second sensor after performing a coordinate transformation corresponding to a positional relationship between a scanning position of the first sensor and a scanning position of the second sensor. (Appendix 5) The data processing device further comprises: 2. The data processing device according to claim 1, further comprising a noise verification unit that verifies whether or not point data determined by the noise determination unit to be noise is noise. (Appendix 6) The data processing device further comprises: a region dividing unit that virtually divides the space into a non-verification region and a verification region; The noise verification unit Regarding point data that exists in the non-verification target area in the point cloud data, the point data that exists in the verification target area in the point cloud data is determined to be noise by the noise determination unit in accordance with the determination result by the noise determination unit. 6. The data processing device of claim 5, wherein the data processing device verifies whether the (Appendix 7) The noise verification unit 7. The data processing device according to claim 5, wherein, when it is determined as a result of the verification that the point data determined to be noise by the noise determination unit is not noise, the point data determined to be noise by the noise determination unit and removed by the noise removal unit is restored. (Appendix 8) The noise verification unit A data processing device according to any one of appendices 5 to 7, which evaluates the features of point data determined to be noise by the noise determination unit and verifies whether the point data determined to be noise by the noise determination unit is noise or not. (Appendix 9) the first sensor and the second sensor are mounted on a vehicle traveling on a road; The region division unit A data processing device as described in Appendix 6, which designates an area of ​​the space in front of the vehicle that is on the road as the non-verification area, and designates an area of ​​the space in front of the vehicle outside the road as the verification area, thereby dividing the space in front of the vehicle into the non-verification area and the verification area. (Appendix 10) The noise determination unit A data processing device according to any one of appendices 1 to 8, which analyzes point cloud data obtained by the first sensor being a LiDAR (Light Detection and Ranging) scanning the space and plot data in which detection points detected by the second sensor being a millimeter-wave radar scanning the space are plotted. (Appendix 11) a computer analyzes point cloud data obtained by scanning a space with a first sensor and plot data in which sensed points sensed by a second sensor, which is a sensor of a different type from the first sensor, scanning the space from a scanning position adjacent to the scanning position of the first sensor are plotted, and determines point data that is noise from among a plurality of point data included in the point cloud data; A data processing method in which the computer removes point data determined to be noise from the point cloud data. (Appendix 12) a noise determination process for analyzing point cloud data obtained by scanning a space with a first sensor and plot data in which sensed points sensed by a second sensor, which is a different type of sensor from the first sensor, scanning the space from a scanning position adjacent to the scanning position of the first sensor are plotted, and determining point data that is noise among a plurality of point data included in the point cloud data; a noise removal process for removing point data determined to be noise by the noise determination process from the point cloud data. [Explanation of symbols]

[0071] 100 filtering processing device, 101 LiDAR data acquisition unit, 102 millimeter wave data acquisition unit, 103 LiDAR data coordinate conversion unit, 104 millimeter wave data coordinate conversion unit, 105 conversion reference information storage unit, 106 noise determination unit, 107 noise removal unit, 108 output unit, 111 feature calculation unit, 112 area division unit, 113 noise verification unit, 200 LiDAR, 300 millimeter wave radar, 901 processor, 902 main memory device, 903 auxiliary memory device, 904 communication device.

Claims

1. a noise determination unit that analyzes point cloud data obtained by scanning a space with a first sensor that is a LiDAR (Light Detection and Ranging) and plot data in which sensed points sensed by a second sensor that is a millimeter wave radar scanning the space from a scanning position adjacent to the scanning position of the first sensor are plotted, and determines point data that is noise among a plurality of point data included in the point cloud data; a noise removal unit that removes point data determined to be noise by the noise determination unit from the point cloud data; The noise determination unit a plurality of sensing points are arranged in a second sector area, which is a sector area representing the space and has an origin at the scanning position of the second sensor obtained by the coordinate transformation; for each divided area obtained by dividing the second sector area by a line segment extending from the origin of the second sector area toward an arc of the second sector area, a sensing point closest to the origin of the second sector area is extracted as a nearest point; and for each divided area, a partial area closer to the origin of the second sector area than the nearest point is extracted as a second extracted partial area; a data processing device that arranges the plurality of point data in a first sector area, which is a sector area obtained by coordinate transformation and represents the space with the scanning position of the first sensor as its origin, and which is coincident with the second sector area, and is divided into a plurality of divided areas that coincide with the plurality of divided areas of the second sector area; extracts a partial area corresponding to the second extracted partial area in each divided area of ​​the first sector area as a first extracted partial area; extracts point data present in the first extracted partial area; and determines only the extracted point data to be noise.

2. The data processing device further comprises:

2. The data processing device according to claim 1, further comprising a noise verification unit that evaluates a feature amount of point data determined to be noise by the noise determination unit and verifies whether the point data determined to be noise by the noise determination unit is noise or not.

3. The data processing device further comprises: a region dividing unit that virtually divides the space into a non-verification region and a verification region; The noise verification unit 3. The data processing device according to claim 2, wherein for point data in the point cloud data that exists in the non-verification target area, the noise determination unit determines whether the point data that exists in the verification target area in the point cloud data and that is determined to be noise by the noise determination unit is noise or not, in accordance with the determination result by the noise determination unit.

4. The noise verification unit 3. The data processing device according to claim 2, wherein, if the verification result determines that the point data determined to be noise by the noise determination unit is not noise, the point data determined to be noise by the noise determination unit and removed by the noise removal unit is restored.

5. the first sensor and the second sensor are mounted on a vehicle traveling on a road; The region division unit 4. A data processing device as described in claim 3, wherein an area of ​​the space in front of the vehicle that is on the road is designated as the non-verification area, and an area of ​​the space in front of the vehicle outside the road is designated as the verification area, thereby dividing the space in front of the vehicle into the non-verification area and the verification area.

6. A computer analyzes point cloud data obtained by scanning a space with a first sensor that is a LiDAR (Light Detection and Ranging) and plot data in which sensed points sensed by a second sensor that is a millimeter wave radar scanning the space from a scanning position adjacent to the scanning position of the first sensor are plotted, and determines point data that is noise among a plurality of point data included in the point cloud data; In the data processing method, the computer removes point data determined to be noise from the point cloud data, The computer a plurality of sensing points are arranged in a second sector area, which is a sector area representing the space and has an origin at the scanning position of the second sensor obtained by the coordinate transformation; for each divided area obtained by dividing the second sector area by a line segment extending from the origin of the second sector area toward an arc of the second sector area, a sensing point closest to the origin of the second sector area is extracted as a nearest point; and for each divided area, a partial area closer to the origin of the second sector area than the nearest point is extracted as a second extracted partial area; a first sector area that is a sector area that represents the space and has the scanning position of the first sensor as its origin, obtained by coordinate transformation, and that coincides with the second sector area; the first sector area is divided into a plurality of divided areas that coincide with the plurality of divided areas of the second sector area; a partial area that corresponds to the second extracted partial area in each divided area of ​​the first sector area is extracted as a first extracted partial area; point data that exists in the first extracted partial area is extracted; and only the extracted point data is determined to be noise.

7. a noise determination process that analyzes point cloud data obtained by scanning a space with a first sensor that is a LiDAR (Light Detection and Ranging) and plot data in which sensed points sensed by a second sensor that is a millimeter-wave radar scanning the space from a scanning position adjacent to the scanning position of the first sensor are plotted, and determines point data that is noise among a plurality of point data included in the point cloud data; a noise removal process for removing point data determined to be noise by the noise determination process from the point cloud data, The noise determination process includes: The computer, a plurality of sensing points are arranged in a second sector-shaped region that is a sector-shaped region representing the space and has an origin at the scanning position of the second sensor, obtained by the coordinate transformation; for each divided region obtained by dividing the second sector-shaped region by a line segment extending from the origin of the second sector-shaped region toward an arc of the second sector-shaped region, a sensing point that is closest to the origin of the second sector-shaped region is extracted as a nearest point; and for each divided region, a partial region that is closer to the origin of the second sector-shaped region than the nearest point is extracted as a second extracted partial region; a data processing program that arranges the plurality of point data in a first sector area, which is a sector area obtained by coordinate transformation and represents the space with the scanning position of the first sensor as its origin, and which is coincident with the second sector area and is divided into a plurality of divided areas that coincide with the plurality of divided areas of the second sector area; extracts a partial area corresponding to the second extracted partial area in each divided area of ​​the first sector area as a first extracted partial area; extracts point data present in the first extracted partial area; and determines only the extracted point data to be noise.

Images