Information processing apparatus, information processing method, and computer program
The information processing device enhances SLAM by integrating map data based on sensor reliability and absolute indexes, improving the accuracy of integrated maps and correcting for low-accuracy data, addressing the challenge of varying map accuracy in SLAM systems.
Patent Information
- Application Number
- JP2023074077
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing SLAM technologies face challenges in integrating maps with varying accuracy levels, leading to potential degradation of the integrated map's accuracy when low-accuracy map data is included.
An information processing device that acquires and integrates map data based on sensor reliability, using attribute information and absolute indexes to correct and weight map data according to their accuracy, thereby enhancing the overall map integration process.
Improves the accuracy of integrated maps by correcting and weighting map data based on sensor reliability and absolute indexes, ensuring higher precision in position and orientation estimation.
Smart Images

Figure 2024158682000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, an information processing method, a computer program, and the like. [Background technology]
[0002] SLAM (Simultaneous Localization and Mapping) technology is known as a method for estimating the position and orientation of a moving object using information acquired by sensors such as cameras and distance sensors mounted on the moving object. SLAM estimates the self-position and orientation using a three-dimensional environmental map to grasp information such as the positions of surrounding objects.
[0003] When the moving range of a moving object is wide, a large environmental map is required, and it takes a lot of time to generate the environmental map. As a method for efficiently generating a large environmental map, a map integration technique that integrates multiple maps is known.
[0004] When integrating maps, it is necessary to integrate the maps while ensuring consistency between areas that are measured overlappingly on multiple maps. Patent Document 1 discloses a method of correcting each map using absolute indicators and then integrating them. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent No. 5930346 Summary of the Invention [Problem to be solved by the invention]
[0006] However, in Patent Document 1, when overlapping measurement areas are integrated, if the accuracy of one of the map data is low, the accuracy of the integrated map may be low.
[0007] Therefore, an object of the present invention is to provide an information processing device capable of improving the accuracy of an integrated map. [Means for solving the problem]
[0008] An information processing device according to one aspect of the present invention includes: a map acquisition means for acquiring a plurality of map data in which measurement position and orientation information of the sensor is recorded for a plurality of measurement points along a movement path of the sensor; a map reliability determination means for calculating a reliability of each of the plurality of map data acquired by the map acquisition means; and a map integration unit that integrates the plurality of map data based on the reliability calculated by the map reliability determination unit. Effect of the Invention
[0009] According to the present invention, it is possible to realize an information processing device capable of improving the accuracy of an integrated map. [Brief description of the drawings]
[0010] [Figure 1] 1 is a functional block diagram illustrating a configuration example of an information processing system according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to a first embodiment. [Diagram 3] 4 is a flowchart showing a map integration process according to the first embodiment. [Figure 4] FIG. 11 is a functional block diagram illustrating a configuration example of an information processing system according to a third embodiment. [Diagram 5] 13 is a flowchart showing details of a map reliability determination step S500 according to the third embodiment. [Figure 6] 4A to 4C are diagrams illustrating examples of display on a display unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, the embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same reference numerals are used for the same members or elements, and duplicated descriptions are omitted or simplified. EXAMPLES
[0012] In this embodiment, a method for integrating multiple map data that can be used by a mobile body equipped with a sensor to perform position and orientation estimation, autonomous driving, and the like in an environment based on reliability will be described. Note that the position and orientation here means position and orientation. Note that the mobile body in this embodiment is not limited to an autonomous mobile body, but includes robots, drones, automobiles with driving assistance functions, and other mobile devices. Also, the information processing device in this embodiment may be mounted on the mobile body, or may be a terminal such as a PC provided outside the mobile body.
[0013] The map data to be integrated in this embodiment will be described. The map data is composed of data of one or more measurement points (key frames, etc.). Each measurement point data has information of a sensor for measuring position and orientation (hereinafter abbreviated as sensor) (image data captured by the sensor, etc.) and the position and orientation information of the sensor.
[0014] When a mobile body equipped with a sensor creates map data, each measurement point is placed at a specified interval along the moving path of the mobile body. As an example, we will use map data in which one measurement point is placed approximately every 1m of the moving path.
[0015] Here, an explanation will be given of errors (drift) in the position and orientation that occur when generating map data. Map data is generated by repeatedly generating measurement point data and estimating the current position and orientation using the measurement point data.
[0016] Therefore, the position and orientation error that occurs when generating measurement point data will become an error in subsequent position and orientation estimation using those measurement points, and the error accumulates each time measurement points are generated. This accumulated error is called drift. The effect of drift causes a scale error between the actual travel distance and the travel distance on the map data.
[0017] Fig. 1 is a functional block diagram showing a configuration example of an information processing system according to a first embodiment of the present invention. Some of the functional blocks shown in Fig. 1 are realized by causing a CPU or the like as a computer included in an information processing device 100 to execute a computer program stored in a memory as a storage medium.
[0018] However, a part or all of these may be realized by hardware, which may be a dedicated circuit (ASIC) or a processor (a reconfigurable processor, DSP), etc.
[0019] In addition, the functional blocks shown in Fig. 1 do not have to be built in the same housing, and may be configured by separate devices connected to each other via signal paths. The above explanation regarding Fig. 1 also applies to Fig. 4.
[0020] The storage unit 101 holds multiple pieces of map data. The map acquisition unit 102 acquires multiple pieces of specified map data from the storage unit 101. Here, the map acquisition unit 102 functions as a map acquisition means that acquires multiple pieces of map data in which measurement position and orientation information of the sensor is recorded for multiple measurement points along the movement path of the sensor.
[0021] The map reliability determination unit 103 calculates the reliability of the map data by using the map data acquired by the map acquisition unit 102. Here, the map reliability determination unit 103 functions as a map reliability determination means that calculates the reliability of each of the multiple map data acquired by the map acquisition unit 102. The map integration unit 104 performs integration by using the multiple map data acquired by the map acquisition unit 102 and the reliability calculated by the map reliability determination unit 103.
[0022] Here, the map integration unit 104 functions as a map integration means for integrating a plurality of map data based on the reliability calculated by the map reliability determination unit 103. The display unit 105 includes an image display such as an LCD or an organic EL, and functions as a display means for displaying the results of the map integration unit and the map reliability determination unit. The memory unit 101 may be provided inside the information processing device 100 or may be provided externally.
[0023] 2 is a diagram illustrating an example of a hardware configuration of an information processing device according to Example 1. The information processing device 100 has functions of a general PC device, and includes a CPU 211, a ROM 212, a RAM 213, a storage unit 214 such as an HDD or SSD, and a system bus 215 as a computer.
[0024] The CPU 211 uses the RAM 213 as a work memory, executes an operating system (OS) and various computer programs stored in the ROM 212, the storage unit 214, etc., and controls each unit via a system bus 215. For example, the computer programs executed by the CPU 211 include computer programs for executing the processes in Figs. 3 and 5, etc., which will be described later.
[0025] Next, Fig. 3 is a flowchart showing a map integration process according to the embodiment 1. Note that the CPU 211 as a computer in the information processing device 100 executes a computer program stored in a memory, thereby sequentially performing the operations of the steps in the flowchart in Fig. 3.
[0026] First, in step S300, a plurality of map data to be integrated are acquired from a storage device. Here, step S300 functions as a map acquisition step for acquiring a plurality of map data in which measurement position and orientation information of the sensor is recorded for a plurality of measurement points along the movement path of the sensor.
[0027] Next, in step S301, the reliability of each of the multiple map data acquired in step S300 is calculated. Details of the reliability calculation method will be described later. Note that step S301 functions as a map reliability determination step that calculates the reliability of each of the multiple map data acquired in step S300 as a map acquisition step.
[0028] Next, in step S302, map data is integrated based on the reliability calculated in step S301. The integration method will be described in detail later. Here, step S302 functions as a map integration step that integrates multiple map data based on the reliability calculated in step S301 as a map reliability determination step.
[0029] Next, the map reliability determination method in step S301 will be described. In this embodiment, the map data includes attribute information, and the accuracy of the map as a whole is calculated based on the attribute information added to the map, and the reliability of the map is determined based on the accuracy of the map. That is, the map reliability determination unit 103 calculates the reliability of each of the multiple map data based on the attribute information of the map data.
[0030] In this embodiment, the type of sensor used to create the map data is used as an example of attribute information. When using the sensor type as attribute information, a reliability table is prepared in advance for each sensor type, in which the reliability is set higher for sensor types with higher measurement performance according to the measurement method and measurement environment. In this embodiment, the reliability is set using at least one of the measurement accuracy and measurement range as an index of measurement performance.
[0031] When determining the reliability, the reliability is set by referring to the sensor used to create the map and the reliability table above. For example, a reliability table in which the reliability of LiDAR is set to 10 and the reliability of the stereo camera is set to 1 is used. In that case, when integrating map 1 created using LiDAR and map 2 created using the stereo camera, the reliability of map 1 is set to 10 and the reliability of map 2 is set to 1.
[0032] Next, the map integration method performed in step S302 will be described. In map integration, the positions and orientations of measurement points and the positions of feature points in the environment (map points, etc.) are corrected based on measurement information of the feature points in the environment. In this embodiment, the reliability of each map is compared for overlapping measurement areas of multiple maps to be integrated.
[0033] Then, using the positions and orientations of measurement points and the positions of characteristic points in the environment on a map with higher reliability as a reference, the positions of measurement points and characteristic points in the environment on a map with lower reliability are corrected. That is, the map integration unit 104 corrects map data whose reliability is lower than a predetermined threshold, using map data whose reliability is higher than a predetermined threshold as a reference.
[0034] As a correction method, for example, a transformation matrix including translation, rotation, and scale is calculated using the relative positions of measurement points on different maps, and is integrated with the positions and orientations of the measurement points and the positions of feature points in the environment. Also, in areas where measurements are only taken on a single map, correction is made using the transformation matrix calculated when correcting areas where measurements were taken in duplicate. In this embodiment, by doing this, the accuracy of the integrated map can be improved.
[0035] In this embodiment, one reliability is calculated for each map of the multiple maps to be integrated, but the reliability may be calculated by dividing the map into regions. For example, a region may be set for each group of measurement points of a certain number or for each measurement section. By dividing the regions and calculating the reliability for each region, the maps can be integrated using map information with higher accuracy for each region, and the accuracy of the integrated map will be higher.
[0036] In the above embodiment, the reliability of a map is determined using the sensor type as attribute information added to the map. However, the reliability may be determined based on the sensor characteristics, sensor installation positions, number of sensors, etc., of the sensors used when generating each map.
[0037] That is, the attribute information includes at least one of the following: sensor type, sensor characteristics, sensor installation position, and number of sensors. The sensor characteristics include at least one of the following: sensing field angle, sensing distance, and sensing accuracy.
[0038] Specifically, for example, the sensing field of view of the sensor used in map 1 may be R1, the sensing distance may be D1, and the standard deviation of the measurement error as the sensing accuracy may be σ1, and the reliability C1 of the map may be calculated using the following formula 1.
number
[0039] In addition, α, β, and γ are coefficients for scale correction of each characteristic value. As shown in formula 1, it is desirable to increase the reliability as the sensing field of view is wider, the sensing distance is longer, and the sensing accuracy is higher (the standard deviation is smaller). The reliability of map 2 is determined in the same manner.
[0040] When the sensor mounting position is used as attribute information, the reliability of the mounting position that makes it easy to sense the characteristic points in the environment is set high using the distribution of characteristic points in the measurement environment.
[0041] Specifically, in an environment where there are fewer features on the floor than on the top, it is difficult for a sensor attached facing downward to sense the features, so the reliability of a map created using a sensor attached facing downward is lower than that of a map created using a sensor attached facing upward.
[0042] For example, when a single moving object is equipped with multiple sensors, a reliability is determined for each sensor, and a single reliability is determined based on the multiple reliabilities. Specifically, the sum is set as the reliability. However, this method is not limited as long as a single reliability can be determined based on multiple reliabilities. For example, a weight may be set for each sensor and a weighted average may be taken as the overall reliability.
[0043] Furthermore, the attribute information of a map is not limited to sensor information. For example, when a map is created while moving or rotating a mobile object equipped with a sensor, each measurement point is placed along the moving path of the mobile object, so the moving path of the mobile object at the time of creating the map may be used as attribute information.
[0044] Attribute information of a movement path includes, for example, the shape of the movement path of a moving object, environmental data at the time of map creation, the relative positional relationship between specific areas (e.g., walls, pillars, plots, etc.) in the map creation environment and the movement path of the moving object, and changes in the placement of objects around the movement path.
[0045] That is, the attribute information includes at least one of the shape of the movement path of the mobile body on which the sensor is mounted, environmental data at the time of map data creation, the relative positional relationship between a specific area in the map data creation environment and the movement path of the mobile body, and changes in the placement of objects around the movement path.
[0046] When the shape of the moving path of the moving object at the time of map creation is used as attribute information, the presence or absence of loops in the path may be set as the reliability. In this case, the reliability may be calculated, for example, as the density of loops relative to the path length, so that the reliability is higher as the density of loops is higher.
[0047] When using the relative positional relationship between specific areas (e.g., walls, pillars, sections, etc.) in the map creation environment and the moving path of the moving object as attribute information, the number of specific areas whose distance from the path at the time of map creation is below a threshold can be used as the reliability. In other words, the reliability is increased for maps created with paths closer to more specific areas. However, without being limited to this, the reliability can also be calculated by setting a coefficient or weighting the number for each type of specific area and calculating the sum.
[0048] Also, the reliability may be calculated based on the magnitude of environmental changes in the corresponding areas of each map between the time the map was created and the time the map is used. That is, the attribute information of the map data may include environmental data at the time the map data was created, and the difference between the illuminance at the time the map was created and the illuminance at the time the map is used may be calculated as the environmental data, and a value inversely proportional to the difference may be set as the reliability. In this case, the smaller the illuminance difference, the higher the reliability of the map.
[0049] The environmental data at the time when the map data was created, which is attribute information of the map data, includes the date and time when the map data was created, etc. In other words, the reliability may be calculated by calculating a weighted sum of the reciprocals of the date difference and the time difference between the time when the map was created and the time when the map was used.
[0050] Specifically, the difference between the creation date of map 1 and the use date of the integrated map is D1, and the difference between the creation time of map 1 and the use time of the integrated map is T1, and the reliability C1 of map 1 may be calculated by the following formula 2. Note that α and β are weighting coefficients for the date and time differences, respectively. The reliability of map 2 is calculated in the same manner.
number
[0051] When the location of an object is used as attribute information, the inverse of the change in the location may be used as the reliability. The change may be calculated, for example, for each area as the sum of the number of objects, such as cardboard boxes, that have increased, moved, or disappeared, and the reliability may be increased as the change is smaller. Alternatively, for example, images captured from the same viewpoint when the map was created and when the map was used may be compared to calculate the similarity, and the reliability may be increased as the similarity is higher.
[0052] In this embodiment, an example in which one reliability determination method is used has been described, but a combination of multiple reliability determination methods may be used. Specifically, the reliability may be determined by each reliability determination method, and a weight may be set for the reliability determined by each method to obtain a weighted average. In addition, in this embodiment, an example in which the same transformation matrix is used to correct the position and orientation of the measurement points and the positions of the feature points in the environment during map integration has been described, but the correction method is not limited to this.
[0053] For example, the positions and orientations of the measurement points may be corrected by pose graph optimization. In this case, the scale of the map with low reliability is corrected to match the scale of the map with high reliability. To achieve this, the weight of the error in the relative positions and orientations between measurement points on different maps is set to be larger than the weight of the error in the relative positions and orientations between measurement points on the same map.
[0054] For example, let us consider a case where map 1 is more reliable than map 2, and the positions and orientations of measurement points on map 2 are corrected. If the error between the target values of the relative positions and orientations of measurement points on maps 1 and 2 is E12, and the error between the target values of the relative positions and orientations of measurement points on map 2 is E22, the residual R12 of the pose graph optimization is calculated using the following formula 3. Note that α and β are weighting coefficients for the respective errors.
number
[0055] In addition, bundle adjustment may be performed to correct the entire map. In this case, the weight of the reprojection error of the feature points in the environment measured on the map with high reliability is set large.
[0056] For example, let us consider a case where map 1 is more reliable than map 2, and the positions and orientations of the measurement points on map 2 and the positions of the feature points in the environment are corrected. Let the number of measurement points on map 2 be M2, the total number of feature points in the environment measured at measurement point i on map 2 that are also measured on map 1 be N2i1, and the total number of feature points in the environment that are measured only on map 2 be N2i2.
[0057] Furthermore, if the reprojection error of feature point j in the environment at measurement point i is eij, the residual error R of the bundle adjustment can be calculated by the following equation 4.
number
[0058] In this embodiment, the positions and orientations of the measurement points of the highly reliable map and the positions of the characteristic points in the environment are fixed as references when integrating maps, but this is not limiting, and the positions and orientations of the measurement points and the positions of the characteristic points in the environment may also be corrected for maps with high reliability. In other words, the map integration unit 104 may weight the integration of multiple map data based on the reliability of each map data.
[0059] For example, when correcting the relative position and orientation between measurement points by pose graph optimization, the reliability can be multiplied by the error of the relative position and orientation to be optimized. In this case, if the reliability is set for each measurement point, the weight of the relative position and orientation between two measurement points can be set to the higher of the two reliability. This setting method reduces the weight of the error of the relative position and orientation between measurement points with low reliability, and when optimized, the error of the relative position and orientation with a measurement point with high reliability is reduced.
[0060] In this embodiment, an example in which two maps are integrated has been described, but three or more maps may be integrated. When three or more maps are integrated, they are integrated in the same manner as described above. Specifically, maps may be integrated by successively adding maps, or three or more maps may be integrated at once. EXAMPLES
[0061] In the first embodiment, an example was described in which the accuracy of the map's potential is calculated based on the attribute information of the map in the map reliability determination step of S301 in Fig. 3, and the reliability is determined by calculating the accuracy of the map itself. In the second embodiment, an example is described in which the accuracy of the map itself is calculated, and the higher the accuracy, the higher the reliability.
[0062] In the second embodiment, the accuracy of the map is calculated using, for example, information on feature points in the environment included in the map data (for example, the number of feature points) as an evaluation index. That is, in the map reliability determination step of S301 in Fig. 3, the accuracy of the map itself is calculated based on the above evaluation index to determine the reliability.
[0063] In addition, in the second embodiment, for example, when the number of feature points in the environment is used as the reliability, the number of feature points in the environment being measured is calculated for each group of measurement points, and when integrating maps, the number of feature points for each measurement point is compared in the overlapping measurement sections of multiple maps.
[0064] That is, in the second embodiment, for example, when the number of feature points in the environment measured at measurement point i on map 1 is N1i and the number of measurement points in the overlapping measurement section is M1, the reliability C1 of the map is calculated using the following formula 5. In formula 5, the reliability increases as the number of feature points in the environment in the overlapping measurement section increases. Similarly, the reliability of map 2 is calculated using formula 5, and the reliability is compared when integrating maps.
number
[0065] The method of calculating the number of feature points in the environment as the reliability is not limited to the above method, and the reliability may be calculated by using the total number of feature points in the environment and the number of measurement points to calculate the average number of feature points for each measurement point. Also, the density of feature points in the environment may be calculated for each divided unit area (e.g., 1 cubic meter), and the number of areas where the density is equal to or greater than a threshold may be calculated as the reliability.
[0066] Since the accuracy of position and orientation measurement increases as the spatial distribution of feature points in the environment becomes wider, the accuracy of map data can be estimated with higher accuracy by using the density of feature points for each region. Note that in the second embodiment, the number of feature points in the environment is used as an evaluation index for the accuracy of the map.
[0067] However, the evaluation index is not limited to this. For example, the evenness of the distribution of feature points in the environment may be used as information about feature points included in the map data. The evenness of the distribution of feature points is calculated based on whether the feature points are present evenly in the captured image data.
[0068] The determination of whether or not there is a bias in the distribution of feature points is made by determining whether or not a predetermined number of feature points are present for each partition according to the division method specified by the user or for each partition according to the division method set uniquely to the system. Specifically, the sum of the number of partitions having a predetermined number of feature points or more is calculated.
[0069] The reliability of a map, which calculates the suitability of map information based on the distribution of feature points of an object, is the ratio of the distribution of feature points to the total number of sections. Therefore, the closer to 1 ((total number of sections with a predetermined number or more of feature points) / total number of sections) is, the higher the reliability is determined to be. Note that the aforementioned sections may be two-dimensional or three-dimensional.
[0070] In the second embodiment, the number of feature points in the environment is used as an evaluation index for the accuracy of the map, but the evaluation index is not limited to this. For example, the measurement error for each measurement point may be used as information about the feature points.
[0071] In the case of Visual SLAM, the average of the errors (reprojection errors) between the two-dimensional coordinates at which each feature point is observed in the image of each measurement point and the two-dimensional coordinates obtained by projecting the estimated relative positions of the feature points onto the image can be calculated, and the reciprocal of the average can be used as the reliability.
[0072] For example, let N1i be the number of feature points in the environment measured at measurement point i on map 1, M1 be the number of measurement points in the overlapping measurement section, and e1ij be the reprojection error of feature point j in the environment measured at measurement point i. In this case, the reliability C1 of the map may be calculated, for example, by the following formula 6. In formula 6, the smaller the reprojection error, the higher the reliability.
number
[0073] Note that the above formula is not limited to formula 6, so long as the smaller the reprojection error, the higher the reliability. For example, a threshold may be set for the reprojection error, and the reciprocal of the total number of feature points whose reprojection error exceeds the threshold may be used as the reliability. In the case of LiDAR SLAM, the error may be calculated by comparing the observed coordinates of each feature point for each measurement point with the coordinates of the feature point on the map.
[0074] Furthermore, drift errors accumulate during map creation, but the errors can be reduced by loop closing processing. Therefore, it is possible to determine for each measurement point whether loop closing processing has been performed, and to increase the reliability of measurement points for which loop closing processing has been performed. In other words, the reliability of map data may be calculated based on information regarding whether loop closing has been performed.
[0075] For example, the loop closure status of measurement point i on map 1 may be Li = 1 (loop closure implemented) or Li = 0 (loop closure not implemented), the number of measurement points in the overlapping measurement section may be M1, and the reliability C1 of the map may be calculated using the following formula 7. According to formula 7, the reliability increases as the number of measurement points where the loop is closed increases.
number
[0076] Note that the reliability may be calculated so long as the reliability reflects whether or not loop closing has been performed, and is not limited to the above formula 7. For example, the overlapping measurement section may be divided into unit distances (1 m), and the presence or absence of a measurement point where loop closing processing has been performed may be determined for each divided section, so that the distribution of whether or not loop closing processing has been performed on the measurement route may be used as the reliability.
[0077] Also, the reliability of the map data may be determined by determining whether or not a moving object exists when the map is created. That is, the reliability of the map data may be calculated based on information regarding the presence or absence of a moving object. The moving object may be estimated by pattern matching using SSD (Sum of Squared Difference) based on the image data and the moving object template data.
[0078] In addition, a moving body may be any object that can move, and includes a moving device equipped with a moving mechanism, an autonomous moving body, a cart, a belt conveyor, an elevator, a human being, and the like.
[0079] Furthermore, moving objects also include trees swaying due to external forces such as wind. If the similarity calculated by pattern matching using SSD is less than a predetermined threshold, it may be determined that no moving object is present, and if it is equal to or greater than the predetermined threshold, it may be determined that a moving object is present. In addition, the reliability of the map may be the inverse of the total number of moving objects estimated at each measurement point divided by the number of measurement points.
[0080] By doing so, the map becomes more reliable the less moving objects are present. In the above explanation, the presence or absence of information on a moving object, i.e., the presence or absence of a moving object in the image data of the sensor, is estimated based on pattern matching using SSD, but other methods may be used as long as they can calculate the similarity.
[0081] For example, an alternative index to SSD is Sum of Absolute Difference (SAD). In addition, an image recognition method using a trained model such as Convolutional Neural Network (CNN) may be used to determine whether a moving object exists.
[0082] Also, the reliability may be calculated using the sensor characteristics used to create the map and the relative positions of the characteristic points in the environment measured for each measurement point. Specifically, the relative positions of the measurement points and the characteristic points in the environment may be compared with the sensing field of view and the sensing distance, and if the measurement results are within the sensing field of view and the sensing distance, they may be determined as valid measurement results, and the total number of valid measurement results may be set as the reliability of the map.
[0083] Specifically, when the number of measurement points on map 1 is N1 and the total number of feature points in the environment being measured by measurement point i whose relative positions to the measurement point are within the sensing field of view and closer than the sensing distance is M1i, the reliability C1 may be calculated, for example, using the following equation 8.
number
[0084] In formula 8, the reliability increases as the number of feature points in the environment in the effective measurement range per measurement point increases. The reliability is also determined for map 2 using formula 8. In this way, the information on the feature points for calculating the reliability includes the number of feature points in the environment in the effective measurement range per measurement point, etc. EXAMPLES
[0085] In the first embodiment, an example was described in which the expected accuracy of the map is calculated based on the attribute information of the map in the map reliability determination step of S301 in Fig. 3, and the reliability is determined by calculating the accuracy of the map itself based on the absolute index and the evaluation index. In the third embodiment, an example will be described in which the accuracy of the map itself is calculated based on the absolute index and the evaluation index, and the calculated accuracy is determined by the reliability.
[0086] The absolute index in the third embodiment refers to an index that can be used as the true value of the position of a specific point in a map creation environment or the true value of the relative position between two points. Absolute indexes include design values using CAD or the like, and actual measurements such as the distance between walls using a sensor with higher accuracy than the sensor used to create the map. In this embodiment, an example will be described in which an actual measurement value from a sensor with higher accuracy than the sensor used to create the map is used as an absolute index.
[0087] Fig. 4 is a functional block diagram showing a configuration example of an information processing system according to a third embodiment. The information processing device in the third embodiment differs from the first embodiment in that an absolute index acquisition unit 406 is added to the configuration described in the first embodiment. Note that functional blocks 401 to 405 in Fig. 4 are similar to functional blocks 101 to 105 in Fig. 1, and therefore description thereof will be omitted.
[0088] The absolute index acquisition unit 406 functions as an absolute index acquisition means for acquiring absolute indexes related to map data from the storage unit 401. The map reliability determination unit 403 determines the reliability of the map data based on a comparison between the absolute indexes acquired by the absolute index acquisition unit 406 and the map data acquired by the map acquisition unit 402. In addition, the flow of the map integration process in this embodiment is such that step S301 shown in Fig. 3 is replaced with a map reliability determination step S500.
[0089] Fig. 5 is a flowchart showing details of the map reliability determination step S500 according to the embodiment 3. The CPU 211 as a computer in the information processing device 100 executes a computer program stored in a memory, thereby sequentially performing the operations of the steps in the flowchart in Fig. 3. The process described in Fig. 5 starts when an instruction to start the map reliability determination process is given following step S300.
[0090] In step S501, a loop is started to calculate an error for each absolute marker. In step S502, a group of measurement points corresponding to the absolute marker is determined. Specifically, for each point where an absolute marker is obtained, corresponding measurement points on a map are registered in advance, and the measurement points corresponding to these absolute markers are obtained when the reliability is determined.
[0091] For example, a matching process may be performed between an image of a point where an absolute index is obtained and an image of each measurement point, and the measurement point with the highest similarity may be determined as the measurement point corresponding to the absolute index.
[0092] In step S503, the error between the position and orientation of the acquired absolute marker and the position and orientation of the measurement point group corresponding to the absolute marker is calculated. For example, if the relative positions of two predetermined measurement points in the map data of map 1 are P1ij and the actual distance measurements of the two predetermined measurement points by a high-precision sensor are d1ij, the error between these measurement points can be calculated as an evaluation value E1ij by the following formula 9.
number
[0093] Step S504 is the end step of the loop started at step S501, and error calculation is repeated for all absolute indexes in steps S501 to S504. When error calculation for all absolute indexes is completed in step S504, the process proceeds to step S505.
[0094] In step S505, the reciprocal of the sum of the errors obtained in steps S501 to S504 is calculated as the map reliability. That is, if the total number of evaluation values of the measurement points calculated for map 1 is N1, the reliability C1 of map 1 is calculated using the following formula 10. The reliability of map 2 is also calculated using formula 10.
number
[0095] The method of calculating the evaluation value of the measurement point group is not limited to the calculation method of formula 10, depending on the absolute index that can be used. For example, when the relative position and orientation between two measurement points is obtained as an absolute index, the relative position and orientation in the map data may be calculated using the positions and orientations of the two measurement points on the map data, and the error from the relative position and orientation obtained as the absolute index may be calculated. In this case, the position error and orientation error may be calculated by setting coefficients respectively, integrating them, and calculating the sum.
[0096] In this way, the reliability of the map data is calculated by comparing it with the absolute index, and the calculated reliability can be used to improve the accuracy of the integrated map. For example, the accuracy of each map can be calculated by using the actual measurement values of a high-performance sensor, and multiple maps can be integrated with high accuracy based on the map with the higher accuracy.
[0097] In this embodiment, the actual measurement value is used as the absolute index, but the present invention is not limited to this, and the design value in the CAD model may be used as the absolute index. Specifically, for example, the design value of the relative position between columns of a building may be used as the absolute index. When determining the measurement point in step S502, the relative position information with respect to the column that is the target of the CAD model is confirmed.
[0098] The relative positions of the pillar and the measurement points may be measured using a laser distance sensor when creating the map. In other words, the position of the pillar on the map data is calculated using the position and orientation of the measurement points and their relative positions to the pillar. The position of pillar i, which is the modeling target measured using the measurement point cloud on map 1, is expressed as P1i, and the position of the pillar on the CAD model is expressed by the following equation 11.
number
[0099] When the total number of pillars is N1, the reliability C1 of map 1 is calculated using the following formula 12. The error in the relative position of each pillar with respect to pillar i is calculated, and the greater the sum of the errors, the lower the reliability.
number
[0100] The object to be modeled is not limited to a pillar. For example, it may be a wall, a staircase, or a window. Furthermore, the method of calculating the relative position between the object to be modeled and the measurement point is not limited to actual measurement. For example, a marker attached to a wall may be detected to calculate the relative position.
[0101] In addition, a specific object to be modeled may be detected using a deep learning model such as FCN or SegFormer, which assigns a label of an object type to each pixel of an image. The detected result may then be used to calculate the relative position of the specific object. This can improve the efficiency of map creation.
[0102] Note that FCN stands for Fully Convolutional Network, and SegFormer stands for Simple and Efficient Design for Semantic Segmentation with Transformers.
[0103] 6 is a diagram for explaining a display example on the display unit 105. On the display unit 105, 601 indicates attribute information, 602 indicates reliability, 603 indicates a map before integration, and 604 indicates a map after integration.
[0104] As shown in Fig. 6, the attribute information, reliability, and maps before integration for each of maps A and B are displayed, and then the integrated map is displayed. That is, the map data and the reliability of the map data are displayed in association with each other. By displaying in this manner, it becomes easy to verify the validity of the integrated map.
[0105] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to the above embodiments, and various modifications are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The above embodiments may be combined, and the present invention includes the following combinations.
[0106] (Configuration 1) An information processing device comprising: a map acquisition means for acquiring a plurality of map data in which measurement position and orientation information of a sensor is recorded for a plurality of measurement points along a movement path of the sensor; a map reliability determination means for calculating the reliability of each of the plurality of map data acquired by the map acquisition means; and a map integration means for integrating the plurality of map data based on the reliability calculated by the map reliability determination means.
[0107] (Configuration 2) The information processing device according to configuration 1, wherein the map data includes attribute information, and the map reliability determination means calculates the reliability of each of the plurality of map data based on the attribute information of the map data.
[0108] (Configuration 3) The information processing device according to configuration 2, wherein the attribute information includes at least one of the type of the sensor used in creating the map data, the sensor characteristics, the sensor mounting position, and the number of sensors.
[0109] (Configuration 4) The information processing device according to configuration 3, wherein the sensor characteristics include at least one of the sensing field of view, the sensing distance, and the sensing accuracy of the sensor.
[0110] (Configuration 5) An information processing device according to any one of configurations 2 to 4, characterized in that the attribute information includes at least one of the shape of the movement path of a moving body equipped with the sensor, environmental data at the time of map data creation, the relative positional relationship between a specific area in the map data creation environment and the movement path of the moving body, and changes in the arrangement of objects around the movement path.
[0111] (Configuration 6) The information processing device according to configuration 5, wherein the environmental data at the time of creating the map data includes a date and time at the time of creating the map data.
[0112] (Configuration 7) An information processing device described in any one of configurations 1 to 6, characterized in that the map reliability determination means calculates the reliability of the map data based on at least one of information regarding feature points contained in the map data, information regarding the presence or absence of moving objects, and information regarding whether or not loop closing has been performed.
[0113] (Configuration 8) Further comprising an absolute index acquisition means for acquiring an absolute index related to the map data, 8. The information processing device according to any one of configurations 1 to 7, wherein the map reliability determination means calculates the reliability of the map data based on a comparison between the absolute index and the map data.
[0114] (Configuration 9) The information processing device according to any one of configurations 1 to 8, wherein the map integration means corrects map data whose reliability is lower than a predetermined threshold value based on map data whose reliability is higher than a predetermined threshold value.
[0115] (Configuration 10) The information processing device according to any one of configurations 1 to 9, wherein the map integration means weights the plurality of map data when integrating the map data based on the reliability of each of the map data.
[0116] (Configuration 11) The information processing device according to any one of configurations 1 to 10, further comprising a display means for displaying the map data in association with the reliability of the map data.
[0117] (Method) An information processing method comprising: a map acquisition step of acquiring multiple map data in which measurement position and orientation information of the sensor is recorded for multiple measurement points along the sensor's movement path; a map reliability determination step of calculating the reliability of each of the map data acquired in the map acquisition step; and a map integration step of integrating the multiple map data based on the reliability calculated in the map reliability determination step.
[0118] (Program) A computer program for controlling each means of the information processing device according to any one of configurations 1 to 11 by a computer.
[0119] In order to realize a part or all of the control in the above-mentioned embodiment, a computer program for realizing the functions of the above-mentioned embodiment may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the information processing device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]
[0120] 100: Information processing device 101: Storage section 102: Map acquisition section 103: Map reliability determination unit 104: Map integration section 105: Display section
Claims
1. a map acquisition means for acquiring a plurality of map data in which measurement position and orientation information of the sensor is recorded for a plurality of measurement points along a movement path of the sensor; a map reliability determination means for calculating a reliability of each of the plurality of map data acquired by the map acquisition means; and a map integration unit that integrates the plurality of map data based on the reliability calculated by the map reliability determination unit.
2. The map data includes attribute information, 2. The information processing apparatus according to claim 1, wherein said map reliability determining means calculates the reliability of each of said plurality of map data based on said attribute information of said map data.
3. 3. The information processing apparatus according to claim 2, wherein the attribute information includes at least one of the type of the sensor used in creating the map data, the sensor characteristics, the sensor mounting position, and the number of sensors.
4. The information processing apparatus according to claim 3 , wherein the sensor characteristics include at least one of a sensing viewing angle, a sensing distance, and a sensing accuracy of the sensor.
5. The information processing device according to claim 2, characterized in that the attribute information includes at least one of the shape of the movement path of a moving body equipped with the sensor, environmental data at the time of map data creation, the relative positional relationship between a specific area in the map data creation environment and the movement path of the moving body, and changes in the arrangement of objects around the movement path.
6. 6. The information processing apparatus according to claim 5, wherein the environmental data at the time of creating the map data includes a date and time at the time of creating the map data.
7. The information processing device according to claim 1, characterized in that the map reliability determination means calculates the reliability of the map data based on at least one of information regarding characteristic points contained in the map data, information regarding the presence or absence of moving objects, and information regarding whether or not loop closing has been implemented.
8. further comprising an absolute index acquisition means for acquiring an absolute index relating to the map data; 2. The information processing apparatus according to claim 1, wherein the map reliability determining means calculates the reliability of the map data based on a comparison between the absolute index and the map data.
9. 2. The information processing apparatus according to claim 1, wherein the map integration means corrects map data whose reliability is lower than a predetermined threshold value, based on map data whose reliability is higher than a predetermined threshold value.
10. 2. The information processing apparatus according to claim 1, wherein said map integration means weights the plurality of map data based on the reliability of each of said map data when integrating said plurality of map data.
11. 2. The information processing apparatus according to claim 1, further comprising display means for displaying said map data in association with a reliability of said map data.
12. a map acquisition step of acquiring a plurality of map data in which measurement position and orientation information of the sensor is recorded for a plurality of measurement points along a movement path of the sensor; a map reliability determination step of calculating a reliability of each of the map data acquired in the map acquisition step; a map integration step of integrating the plurality of map data based on the reliability calculated in the map reliability determination step.
13. A computer program for controlling each unit of the information processing apparatus according to any one of claims 1 to 11 by a computer.
Citation Information
Patent Citations
Switching device of PSK modulator and demodulator
JP1984030346A
Cited By
Method and apparatus for providing a route for a vehicle
DE102024211035A1
