Map generation device, map generation method, and map generation system

Abstract

This aims to minimize the degradation of map generation accuracy even when 3D point cloud data acquired from sensors includes data relating to 3D points of objects located far from the sensors. [Solution] The map generation device 3 is configured to include a data acquisition unit 11 that acquires 3D point cloud data including distance data indicating the distance to each 3D point from a sensor 1 that measures the distance to one or more 3D points on an object present in the observation area; a data extraction unit 12 that extracts 3D point cloud data from the 3D point cloud data acquired by the data acquisition unit 11 in which the distance to the 3D point is within a first threshold; and a map generation unit 13 that generates a map of the observation area based on the map data extracted by the data extraction unit 12.

Description

Technical Field

[0001] The present disclosure relates to a map generation device, a map generation method, and a map generation system.

Background Art

[0002] There is a map generation device that generates a map of an observation area. As such a map generation device, for example, Patent Document 1 discloses a map generation device that acquires 3D point cloud data indicating distances to one or more 3D points on an object existing in an observation area from a sensor, and generates a map of the observation area based on the 3D point cloud data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The map generation device disclosed in Patent Document 1 generates a map of an observation area based on 3D point cloud data acquired from a sensor. Among the 3D point cloud data, there may be data indicating distances to 3D points of an object existing at a position far from the sensor. Since the data indicating distances to 3D points of an object existing at a far position has lower distance accuracy compared to the data indicating distances to 3D points of an object existing at a near position, there is a problem that the generation accuracy of the map may deteriorate.

[0005] This disclosure was made to solve the above-mentioned problems, and aims to provide a map generation device that can suppress the deterioration of map generation accuracy even when the 3D point cloud data acquired from the sensor includes distance data indicating the distance to the 3D points of objects located far from the sensor. [Means for solving the problem]

[0006] The map generation device according to this disclosure includes a data acquisition unit that acquires 3D point cloud data including distance data indicating the distance to each 3D point from a sensor that measures the distance to one or more 3D points on an object present in an observation area; a data extraction unit that extracts distance data from the 3D point cloud data acquired by the data acquisition unit in which the distance to the 3D point is within a first threshold; and a map generation unit that generates a map of the observation area based on the distance data extracted by the data extraction unit. [Effects of the Invention]

[0007] According to this disclosure, even if the 3D point cloud data acquired from the sensor includes distance data indicating the distance to the 3D points of objects located far from the sensor, the degradation of map generation accuracy can be suppressed. [Brief explanation of the drawing]

[0008] [Figure 1] This is a configuration diagram showing a map generation system including a map generation device 3 according to Embodiment 1. [Figure 2] This is a hardware configuration diagram showing the hardware of the map generation device 3 according to Embodiment 1. [Figure 3] This is a hardware configuration diagram of a computer when the map generation device 3 is implemented by software or firmware, etc. [Figure 4] This is a flowchart showing the map generation method, which is the processing procedure of the map generation device 3. [Figure 5] This is an explanatory diagram showing the process of extracting distance data by the data extraction unit 12. [Figure 6] This is an explanatory diagram showing a captured image containing pixels corresponding to each three-dimensional point. [Figure 7] This is a diagram showing a map generation system including a map generation device 3 according to Embodiment 2. [Figure 8] This is a hardware configuration diagram showing the hardware of the map generation device 3 according to Embodiment 2. [Figure 9] This flowchart shows the processing procedure of the data extraction unit 14. [Figure 10] This is a configuration diagram showing a map generation system including a map generation device 3 according to Embodiment 3. [Figure 11] This is a hardware configuration diagram showing the hardware of the map generation device 3 according to Embodiment 3. [Modes for carrying out the invention]

[0009] To provide a more detailed explanation of this disclosure, the forms for implementing this disclosure will be described below with reference to the attached drawings.

[0010] Embodiment 1. Figure 1 is a diagram showing a map generation system including a map generation device 3 according to Embodiment 1. Figure 2 is a hardware configuration diagram showing the hardware of the map generation device 3 according to Embodiment 1. The map generation system shown in Figure 1 includes a sensor 1, a camera 2, and a map generation device 3.

[0011] Sensor 1 is implemented, for example, by LiDAR (Light Detection and Ranging). Sensor 1 measures the distance to one or more three-dimensional points on an object present within the observation area. Sensor 1 outputs 3D point cloud data, including distance data indicating the distance to each 3D point, to the map generation device 3. Camera 2 will photograph the observation area described above. Camera 2 outputs the image data of the observed area to the map generation device 3. The map generation device 3 includes a data acquisition unit 11, a data extraction unit 12, and a map generation unit 13.

[0012] The data acquisition unit 11 is realized by, for example, the data acquisition circuit 21 shown in FIG. 2. The data acquisition unit 11 acquires three-dimensional point cloud data from the sensor 1 and acquisition data from the camera 2. The data acquisition unit 11 outputs the three-dimensional point cloud data to the data extraction unit 12 and outputs the acquisition data to the map generation unit 13.

[0013] The data extraction unit 12 is realized by, for example, the data extraction circuit 22 shown in FIG. 2. The data extraction unit 12 acquires three-dimensional point cloud data from the data acquisition unit 11. The data extraction unit 12 extracts distance data within a first threshold from the three-dimensional point cloud data acquired by the data acquisition unit 11. The first threshold may be stored in the internal memory of the data extraction unit 12 or may be provided from outside the map generation device 3. The data extraction unit 12 outputs the extracted distance data to the map generation unit 13.

[0014] The map generation unit 13 is realized by, for example, the map generation circuit 23 shown in FIG. 2. The map generation unit 13 acquires distance data from the data extraction unit 12 and acquisition data from the data acquisition unit 11. The map generation unit 13 adds acquisition data to the acquired distance data and generates a map of the observation area based on the distance data to which the acquisition data has been added. The generated data of the map generated by the map generation unit 13 may be output to, for example, a radar device not shown or a display device not shown.

[0015] Here, the map generation unit 13 adds photographic data to the distance data and generates a map of the observation area based on the distance data with the added photographic data. However, this is just one example, and the map generation unit 13 may also generate a map of the observation area based on the distance data without adding photographic data to the distance data. Adding photographic data to distance data can make it easier to distinguish between multiple 3D points for each object, potentially improving the accuracy of map generation. On the other hand, even when generating a map of an observation area based on distance data without added photographic data, the accuracy of map generation will not deteriorate if it is easy to distinguish between multiple 3D points for each object.

[0016] In Figure 1, the data acquisition unit 11, data extraction unit 12, and map generation unit 13, which are components of the map generation device 3, are assumed to be implemented by dedicated hardware as shown in Figure 2. That is, the map generation device 3 is assumed to be implemented by a data acquisition circuit 21, a data extraction circuit 22, and a map generation circuit 23. Each of the data acquisition circuit 21, data extraction circuit 22, and map generation circuit 23 can be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0017] The components of the map generation device 3 are not limited to those implemented by dedicated hardware; the map generation device 3 may also be implemented by software, firmware, or a combination of software and firmware. Software or firmware is stored as a program in the computer's memory. A computer refers to the hardware that executes programs, and includes, for example, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor).

[0018] Figure 3 is a hardware configuration diagram of a computer when the map generation device 3 is implemented by software or firmware, etc. If the map generation device 3 is implemented by software or firmware, a program that causes the computer to execute the respective processing procedures in the data acquisition unit 11, the data extraction unit 12, and the map generation unit 13 is stored in the memory 31. The computer's processor 32 then executes the program stored in the memory 31.

[0019] Furthermore, Figure 2 shows an example in which each component of the map generation device 3 is implemented by dedicated hardware, and Figure 3 shows an example in which the map generation device 3 is implemented by software or firmware, etc. However, this is only one example, and some components of the map generation device 3 may be implemented by dedicated hardware, while the remaining components may be implemented by software or firmware, etc.

[0020] Next, we will explain the operation of the map generation system shown in Figure 1. Sensor 1 measures the distance to one or more 3D points on objects present within the observation area. If there are two or more objects within the observation area, Sensor 1 measures the distance to one or more 3D points on each object. If sensor 1 is implemented using LiDAR, sensor 1 receives reflected light from a laser pulse irradiated onto the observation area at a predetermined measurement cycle. Sensor 1 calculates the distance to the 3D point from which the laser pulse was irradiated based on the time from when the laser pulse was irradiated until when the reflected light was received. Sensor 1 outputs 3D point cloud data, which includes distance data indicating the distance to one or more 3D points, to the map generation device 3. Camera 2 captures the same observation area as sensor 1. Camera 2 outputs the image data of the observed area to the map generation device 3.

[0021] Figure 4 is a flowchart showing the map generation method, which is the processing procedure of the map generation device 3. The data acquisition unit 11 acquires 3D point cloud data from sensor 1 (step ST1 in Figure 4) and captures image data from camera 2 (step ST2 in Figure 4). The data acquisition unit 11 outputs 3D point cloud data to the data extraction unit 12 and the image data to the map generation unit 13.

[0022] The data extraction unit 12 acquires 3D point cloud data from the data acquisition unit 11. As shown in Figure 5, the data extraction unit 12 extracts distance data from the 3D point cloud data where the distance to a 3D point is within a first threshold (step ST3 in Figure 4). Figure 5 is an explanatory diagram showing the distance data extraction process by the data extraction unit 12. In Figure 5, the circles represent the three-dimensional points of the object. Distance data indicating the distance to a 3D point of an object located far away has lower distance accuracy compared to distance data indicating the distance to a 3D point of an object located nearby. Therefore, the data extraction unit 12 extracts distance data from the 3D point cloud data where the distance to the 3D point is within a first threshold. The data extraction unit 12 outputs the extracted distance data to the map generation unit 13.

[0023] The map generation unit 13 acquires distance data from the data extraction unit 12 and image acquisition data from the data acquisition unit 11. The map generation unit 13 adds the image data to the acquired distance data (step ST4 in Figure 4). Specifically, as shown in Figure 6, the map generation unit 13 extracts color information of pixels corresponding to 3D points related to acquired distance data from the captured data. The process of extracting color information of pixels corresponding to 3D points is a well-known technique, so a detailed explanation is omitted. Figure 6 is an explanatory diagram showing an image containing pixels corresponding to each three-dimensional point.

[0024] The map generation unit 13 adds photographic data to distance data, which indicates the distance to each three-dimensional point, by adding color information to the distance data. Distance data indicates the distance from sensor 1 to a 3D point, and color information indicates the color of the 3D point. Therefore, distance data with added color information indicates both the distance to the 3D point and the color of the 3D point. The map generation unit 13 generates a map of the observation area based on distance data with added image data (step ST5 in Figure 4). The process of generating a map based on distance data is a well-known technique, so a detailed explanation is omitted.

[0025] In the above embodiment 1, the map generation device 3 is configured to include a data acquisition unit 11 that acquires 3D point cloud data including distance data indicating the distance to each 3D point from a sensor 1 that measures the distance to one or more 3D points on an object present in the observation area; a data extraction unit 12 that extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11 in which the distance to the 3D point is within a first threshold; and a map generation unit 13 that generates a map of the observation area based on the distance data extracted by the data extraction unit 12. Therefore, the map generation device 3 can suppress the deterioration of map generation accuracy even if the 3D point cloud data acquired from the sensor 1 includes distance data indicating the distance to a 3D point of an object located far from the sensor.

[0026] Embodiment 2. Embodiment 2 describes a map generation device 3 that performs a threshold reduction process to lower the first threshold, or a threshold increase process to raise the first threshold.

[0027] Figure 7 is a configuration diagram showing a map generation system including a map generation device 3 according to Embodiment 2. In Figure 7, the same reference numerals as in Figure 1 indicate the same or corresponding parts, so a detailed explanation is omitted. Figure 8 is a hardware configuration diagram showing the hardware of the map generation device 3 according to Embodiment 2. In Figure 8, the same reference numerals as in Figure 2 indicate the same or corresponding parts, so a detailed explanation is omitted. The map generation device 3 shown in Figure 7 comprises a data acquisition unit 11, a data extraction unit 14, and a map generation unit 13.

[0028] The data extraction unit 14 is implemented, for example, by the data extraction circuit 24 shown in Figure 8. The data extraction unit 14 acquires 3D point cloud data from the data acquisition unit 11, similar to the data extraction unit 12 shown in Figure 1. The data extraction unit 14, similar to the data extraction unit 12 shown in Figure 1, extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11, where the distance to the 3D point is within a first threshold. The data extraction unit 14 outputs the extracted distance data to the map generation unit 13 if the number of extracted distance data is less than or equal to the second threshold, and the number of extracted distance data is greater than or equal to the third threshold. The second threshold may be stored in the internal memory of the data extraction unit 14, or it may be provided from outside the map generation device 3. The third threshold is smaller than the second threshold. The third threshold may be stored in the internal memory of the data extraction unit 14, or it may be provided from outside the map generation device 3.

[0029] If the number of extracted distance data points is greater than the second threshold, the data extraction unit 14 performs a threshold reduction process to lower the first threshold. The data extraction unit 14 then extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11, where the distance to the 3D point is within the first threshold after threshold reduction processing. If the number of extracted distance data points is less than the third threshold, the data extraction unit 14 performs a threshold increase process to raise the first threshold. The data extraction unit 14 then extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11, where the distance to the 3D point is within the first threshold after the threshold increase process.

[0030] In Figure 7, the data acquisition unit 11, data extraction unit 14, and map generation unit 13, which are components of the map generation device 3, are assumed to be implemented by dedicated hardware as shown in Figure 8. That is, the map generation device 3 is assumed to be implemented by a data acquisition circuit 21, a data extraction circuit 24, and a map generation circuit 23. Each of the data acquisition circuit 21, data extraction circuit 24, and map generation circuit 23 can be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0031] The components of the map generation device 3 are not limited to those implemented by dedicated hardware; the map generation device 3 may also be implemented by software, firmware, or a combination of software and firmware. If the map generation device 3 is implemented by software or firmware, a program that causes a computer to execute the respective processing procedures in the data acquisition unit 11, data extraction unit 14, and map generation unit 13 is stored in the memory 31 shown in Figure 3. Then, the processor 32 shown in Figure 3 executes the program stored in the memory 31.

[0032] Furthermore, Figure 8 shows an example in which each component of the map generation device 3 is implemented by dedicated hardware, while Figure 3 shows an example in which the map generation device 3 is implemented by software or firmware, etc. However, this is merely one example, and some components of the map generation device 3 may be implemented by dedicated hardware, while the remaining components may be implemented by software or firmware, etc.

[0033] Next, the operation of the map generation system shown in Figure 7 will be described. However, it is the same as the map generation system shown in Figure 1, except for the data extraction unit 14. Therefore, only the operation of the data extraction unit 14 will be described here. Figure 9 is a flowchart showing the processing procedure of the data extraction unit 14.

[0034] The data extraction unit 14 acquires 3D point cloud data from the data acquisition unit 11. The data extraction unit 14 extracts distance data from the 3D point cloud data where the distance to a 3D point is within a first threshold (step ST11 in Figure 9). The data extraction unit 14 compares the number of extracted distance data with a second threshold, and then compares the number of extracted distance data with a third threshold (step ST12 in Figure 9). If the number of extracted distance data is less than or equal to the second threshold (step ST13 in Figure 9: YES), and the number of extracted distance data is greater than or equal to the third threshold (step ST14 in Figure 9: YES), the data extraction unit 14 outputs the extracted distance data to the map generation unit 13 in the same manner as the data extraction unit 12 shown in Figure 1 (step ST15 in Figure 9).

[0035] If the number of extracted distance data points is greater than the second threshold (step ST13: NO in Figure 9), the data extraction unit 14 performs a threshold reduction process to lower the first threshold (step ST16 in Figure 9). The threshold reduction process involves lowering the first threshold from X to, for example, Xn, where n is an integer between 1 and X. When threshold reduction processing is performed, the data extraction unit 14 extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11, where the distance to the 3D point is within the first threshold after threshold reduction processing (step ST11 in Figure 9). The data extraction unit 14 compares the number of extracted distance data with a second threshold, and then compares the number of extracted distance data with a third threshold (step ST12 in Figure 9). If the number of extracted distance data is less than or equal to the second threshold (step ST13 in Figure 9: YES), and the number of extracted distance data is greater than or equal to the third threshold (step ST14 in Figure 9: YES), the data extraction unit 14 outputs the extracted distance data to the map generation unit 13 in the same manner as the data extraction unit 12 shown in Figure 1 (step ST15 in Figure 9).

[0036] If the number of extracted distance data points is less than the third threshold (step ST14: NO in Figure 9), the data extraction unit 14 performs a threshold increase process to raise the first threshold (step ST17 in Figure 9). The threshold increase process involves raising the first threshold from X to, for example, X+m, where m is an integer greater than or equal to 1. When threshold increment processing is performed, the data extraction unit 14 extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11, where the distance to the 3D point is within the first threshold after the threshold increment processing (step ST11 in Figure 9). The data extraction unit 14 compares the number of extracted distance data with a second threshold, and then compares the number of extracted distance data with a third threshold (step ST12 in Figure 9). If the number of extracted distance data is less than or equal to the second threshold (step ST13 in Figure 9: YES), and the number of extracted distance data is greater than or equal to the third threshold (step ST14 in Figure 9: YES), the data extraction unit 14 outputs the extracted distance data to the map generation unit 13 in the same manner as the data extraction unit 12 shown in Figure 1 (step ST15 in Figure 9).

[0037] In the above embodiment 2, the map generation device 3 shown in Figure 7 is configured such that the data extraction unit 14 extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11 where the distance to the 3D point is within a first threshold, and if the number of extracted distance data is greater than a second threshold, it performs a threshold reduction process to lower the first threshold, and extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11 where the distance to the 3D point is within the first threshold after the threshold reduction process. Therefore, the map generation device 3 shown in Figure 7 can remove distance data indicating the distance to the 3D point of an object located farther from the sensor 1 than the map generation device 3 shown in Figure 1, and as a result, the deterioration of the map generation accuracy can be suppressed.

[0038] Furthermore, in Embodiment 2, the map generation device 3 is configured such that the data extraction unit 14 extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11 where the distance to the 3D point is within a first threshold. If the number of extracted distance data is less than a third threshold, the data acquisition unit 11 performs a threshold increase process to raise the first threshold, and then extracts distance data from the 3D point cloud data acquired by the data acquisition unit 11 where the distance to the 3D point is within the first threshold after the threshold increase process. Therefore, the map generation device 3 can generate a map even when the number of distance data extracted from the 3D point cloud data is small.

[0039] Embodiment 3. Embodiment 3 describes a map generation device 3 in which the map generation unit 15 performs calibration to correct the position of the 3D points related to the shooting data so that the positions of the 3D points related to the distance data and the positions of the 3D points related to the shooting data are aligned if they are misaligned.

[0040] Figure 10 is a configuration diagram showing a map generation system including a map generation device 3 according to Embodiment 3. In Figure 10, the same reference numerals as in Figures 1 and 7 indicate the same or corresponding parts, so a detailed explanation is omitted. Figure 11 is a hardware configuration diagram showing the hardware of the map generation device 3 according to Embodiment 3. In Figure 11, the same reference numerals as in Figures 2 and 8 indicate the same or corresponding parts, so a detailed explanation is omitted. The map generation device 3 shown in Figure 10 comprises a data acquisition unit 11, a data extraction unit 12, and a map generation unit 15.

[0041] The map generation unit 15 is implemented, for example, by the map generation circuit 25 shown in Figure 11. The map generation unit 15 acquires distance data from the data extraction unit 12 and image acquisition data from the data acquisition unit 11. Before adding the image data to the distance data, the map generation unit 15 performs calibration to correct the position of the 3D points related to the image data if there is a discrepancy between the positions of the 3D points related to the distance data and the positions of the 3D points related to the image data, so that the positions of both are aligned. The map generation unit 15 adds corrected image data to the distance data and generates a map of the observation area based on the distance data to which the corrected image data has been added. The map generation data generated by the map generation unit 15 may be output to, for example, a radar device (not shown) or a display device (not shown).

[0042] In the map generation device 3 shown in Figure 10, the map generation unit 15 is applied to the map generation device 3 shown in Figure 1. However, this is merely one example, and the map generation unit 15 may also be applied to the map generation device 3 shown in Figure 7.

[0043] In Figure 10, the data acquisition unit 11, data extraction unit 12, and map generation unit 15, which are components of the map generation device 3, are assumed to be implemented by dedicated hardware as shown in Figure 11. That is, the map generation device 3 is assumed to be implemented by a data acquisition circuit 21, a data extraction circuit 22, and a map generation circuit 25. Each of the data acquisition circuit 21, data extraction circuit 22, and map generation circuit 25 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0044] The components of the map generation device 3 are not limited to those implemented by dedicated hardware; the map generation device 3 may also be implemented by software, firmware, or a combination of software and firmware. If the map generation device 3 is implemented by software or firmware, a program that causes a computer to execute the respective processing procedures in the data acquisition unit 11, data extraction unit 12, and map generation unit 15 is stored in the memory 31 shown in Figure 3. Then, the processor 32 shown in Figure 3 executes the program stored in the memory 31.

[0045] Furthermore, Figure 11 shows an example in which each component of the map generation device 3 is implemented by dedicated hardware, while Figure 3 shows an example in which the map generation device 3 is implemented by software or firmware. However, this is merely one example, and some components of the map generation device 3 may be implemented by dedicated hardware, while the remaining components may be implemented by software or firmware.

[0046] Next, the operation of the map generation system shown in Figure 10 will be described. However, all parts except the map generation unit 15 are the same as those in the map generation system shown in Figure 1. Therefore, only the operation of the map generation unit 15 will be described here.

[0047] The map generation unit 15 acquires distance data from the data extraction unit 12 and image acquisition data from the data acquisition unit 11. The map generation unit 15 determines the positional relationship between the distance data and the photographic data before adding the photographic data to the distance data. The following describes in detail the process by which the map generation unit 15 determines the positional relationship.

[0048] The map generation unit 15 identifies pixels corresponding to 3D points related to distance data in the captured data acquired by the data acquisition unit 11. The map generation unit 15 performs calibration to correct the position of the 3D points related to the image data if there is a discrepancy between the position of the 3D points related to the distance data and the position of the 3D points related to the image data, so that the positions of both are aligned. The calibration to correct the position of the 3D points related to the image data is a process to correct the distortion of the image, and since the correction process itself is a well-known technique, a detailed explanation will be omitted.

[0049] Here, when the map generation unit 15 performs calibration to correct the positions of the 3D points related to the captured data, it identifies the pixels corresponding to all the 3D points. In other words, the map generation unit 15 performs calibration to correct the positions of the 3D points related to the captured data so that the position of the pixel corresponding to each of the 3D points matches the position of the respective 3D point. However, this is just one example, and the map generation unit 15 may weight multiple 3D points related to distance data according to distance and preferentially identify pixels corresponding to 3D points with larger weights. Specifically, the map generation unit 15 identifies pixels corresponding to the top G-th 3D points with larger weights, but does not identify pixels corresponding to 3D points from G+1 onwards. The map generation unit 15 performs calibration to correct the positions of the 3D points related to the captured data so that the positions of the pixels corresponding to the top G-th 3D points with the highest weights coincide with the positions of the top G-th 3D points with the highest weights.

[0050] In the above embodiment 3, the map generation device 3 is configured such that, before the map generation unit 15 adds the image data acquired by the data acquisition unit 11 to the distance data extracted by the data extraction unit 12, if there is a discrepancy between the positions of the 3D points related to the distance data and the positions of the 3D points related to the image data, the map generation device 3 performs calibration to correct the positions of the 3D points related to the image data so that the positions of both are aligned. Therefore, the map generation device 3 can suppress deterioration of map generation accuracy even if the 3D point cloud data acquired from the sensor 1 includes distance data indicating the distance to the 3D points of objects located far from the sensor, and can also suppress deterioration of map generation accuracy even if there is a discrepancy between the positions of the 3D points related to the distance data and the positions of the 3D points related to the image data.

[0051] The map generation unit 15 performs calibration to correct the positions of the 3D points related to the image data so that the positions of both are aligned. This allows for suppression of the degradation of map generation accuracy even if there is a discrepancy between the positions of the 3D points related to distance data and the positions of the 3D points related to image data. However, if the positions of the 3D points related to distance data are refracted by, for example, a window or the surface of water, the above calibration alone may not be sufficient to suppress the degradation of map generation accuracy. To prevent a deterioration in map generation accuracy when the position of a 3D point is refractional, the map generation unit 15 (or map generation unit 13) may choose not to use distance data related to the refractional position when generating a map of the observation area if the position of the 3D point related to the image data acquired by the data acquisition unit 11 is refractional.

[0052] Specifically, it is as follows: For example, a learning model is prepared to determine whether or not the position of a 3D point related to the captured data is refractional. The learning model is a model that, during training, is given captured data when the position of the 3D point is refractional and captured data when the position of the 3D point is not refractional, and is trained to determine whether or not the position of the 3D point is refractional. The learning model may be built into the map generation unit 15, etc., or it may be provided outside the map generation device 3. The map generation unit 15, etc., provides the captured data acquired by the data acquisition unit 11, or the captured data after calibration, to the learning model described above. If the position of a 3D point related to the captured data is refraction, the learning model outputs information indicating that it is refraction to the map generation unit 15, etc., and if the position of a 3D point related to the captured data is not refraction, it outputs information indicating that it is not refraction to the map generation unit 15, etc. If the map generation unit 15 receives information from the learning model indicating that the position of a 3D point is not refracted, it generates a map of the observation area based on the distance data related to that position. The method for generating the map of the observation area is as described above. If the map generation unit 15 outputs information from the learning model indicating that the position of a 3D point is refracted, it will not use distance data related to the refracted position when generating a map of the observation area.

[0053] Furthermore, this disclosure allows for free combination of each embodiment, modification of any component in each embodiment, or omission of any component in each embodiment. [Explanation of Symbols]

[0054] 1 Sensor, 2 Camera, 3 Map generation device, 11 Data acquisition unit, 12 Data extraction unit, 13 Map generation unit, 14 Data extraction unit, 15 Map generation unit, 21 Data acquisition circuit, 22 Data extraction circuit, 23 Map generation circuit, 24 Data extraction circuit, 25 Map generation circuit, 31 Memory, 32 Processor.

Claims

1. A data acquisition unit that obtains 3D point cloud data, including distance data indicating the distance to each 3D point, from a sensor that measures the distance to one or more 3D points on an object present within the observation area, A data extraction unit extracts distance data from the 3D point cloud data acquired by the data acquisition unit, wherein the distance to a 3D point is within a first threshold. A map generation unit generates a map of the observation area based on the distance data extracted by the data extraction unit. A map generation device equipped with this device.

2. The data extraction unit, The map generation apparatus according to claim 1, characterized in that it extracts distance data from the three-dimensional point cloud data acquired by the data acquisition unit in which the distance to a three-dimensional point is within the first threshold, and if the number of extracted distance data is greater than the second threshold, it performs a threshold reduction process to lower the first threshold, and extracts distance data from the three-dimensional point cloud data acquired by the data acquisition unit in which the distance to a three-dimensional point is within the first threshold after the threshold reduction process.

3. The data extraction unit, The map generation apparatus according to claim 1, characterized in that, from the three-dimensional point cloud data acquired by the data acquisition unit, distance data to three-dimensional points that are within the first threshold is extracted, and if the number of extracted distance data is less than a third threshold, a threshold increase process is performed to raise the first threshold, and distance data to three-dimensional points that are within the first threshold after the threshold increase process is extracted from the three-dimensional point cloud data acquired by the data acquisition unit.

4. The data acquisition unit, In addition to acquiring 3D point cloud data from the aforementioned sensor, the camera that photographs the observation area also acquires image data of the observation area. The map generation unit, The map generation device according to claim 1, characterized in that it adds photographic data acquired by the data acquisition unit to distance data extracted by the data extraction unit, and generates a map of the observation area based on the distance data with the photographic data added.

5. The map generation unit, The map generation device according to claim 4, characterized in that it extracts color information of pixels corresponding to three-dimensional points of an object from the photographic data acquired by the data acquisition unit, and adds the extracted color information to distance data indicating the distance to the three-dimensional points, thereby adding the photographic data to the distance data.

6. The map generation unit, The map generation apparatus according to claim 1, characterized in that, before adding the photographic data acquired by the data acquisition unit to the distance data extracted by the data extraction unit, if there is a discrepancy between the position of the three-dimensional point related to the distance data and the position of the three-dimensional point related to the photographic data, calibration is performed to correct the position of the three-dimensional point related to the photographic data so that the positions of both are aligned.

7. The data acquisition unit, In addition to acquiring 3D point cloud data from the aforementioned sensor, the camera that photographs the observation area also acquires image data of the observation area. The map generation unit, The map generation device according to claim 1, characterized in that if the position of a three-dimensional point related to the imaging data acquired by the data acquisition unit is refracted, distance data related to the refracted position is not used in generating the map of the observation area.

8. The data acquisition unit acquires 3D point cloud data, including distance data indicating the distance to each 3D point, from sensors that measure the distance to one or more 3D points on an object present within the observation area. The data extraction unit extracts distance data from the 3D point cloud data acquired by the data acquisition unit, where the distance to a 3D point is within a first threshold. The map generation unit generates a map of the observation area based on the distance data extracted by the data extraction unit. Map generation method.

9. A sensor that measures the distance to one or more three-dimensional points on an object within an observation area and outputs three-dimensional point cloud data that includes distance data indicating the distance to each three-dimensional point, A data acquisition unit that acquires 3D point cloud data output from the aforementioned sensor, A data extraction unit extracts distance data from the 3D point cloud data acquired by the data acquisition unit, wherein the distance to a 3D point is within a first threshold. A map generation unit generates a map of the observation area based on the distance data extracted by the data extraction unit. A map generation system equipped with this feature.

Images