Information processing device, mobile object, information processing system, and computer program

Abstract

To provide an information processing device which can maintain high accuracy of a map for position estimation.SOLUTION: An information processing device capable of communicating with at least one mobile object comprises feature amount map management means which manages a feature amount map having, for each division area obtained by dividing a real space into lattice shapes, a feature amount calculated on the basis of previously acquired point group data included in the division area, the division area being associated with a position in the real space. The feature amount map management means comprises: reliability degree calculation means which calculates a reliability degree for each division area so as to make the reliability degree of the division area lower as a conformity degree is smaller on the basis of the conformity degree between the point group data in performing position estimation processing based on the feature amount map with the mobile object and the feature amount map; and feature amount map update means which updates the feature amount map on the basis of the reliability degree for each division area.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a mobile object, an information processing system, a computer program, and the like. [Background technology]

[0002] Conventionally, self-location estimation technology such as that disclosed in Patent Document 1 has been known as a means for determining the position of a moving object or the like in real space. This type of location estimation technology estimates the position of a moving object on a location estimation map by matching a pre-created map for location estimation with sensor data acquired in real time from a sensor mounted on the moving object while the moving object is traveling. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-54315 Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned location estimation technology requires that a map for location estimation be created in advance. Therefore, when location estimation is performed, differences may occur between the map for location estimation and the surrounding environment of the mobile unit at the time of location estimation due to changes over time in trees and buildings around the mobile unit, the presence or absence of other mobile units, measurement noise, etc.

[0005] If position estimation is performed when there is a difference between the map used for position estimation and the surrounding environment of the mobile body at the time of position estimation, there is a problem that the accuracy of the estimated position decreases.

[0006] Therefore, one object of the present invention is to provide an information processing device that can maintain high accuracy of a map for position estimation. [Means for solving the problem]

[0007] In order to achieve the above object, one aspect of the present invention is to provide: An information processing device capable of communicating with at least one mobile object, a feature amount map management means for managing a feature amount map for each divided area obtained by dividing a real space into a grid, the feature amount map having feature amounts calculated based on point cloud data included in the divided area and acquired in advance, the divided areas being associated with positions in the real space; The feature map management means a reliability calculation means for calculating the reliability of each divided region based on a degree of match between point cloud data obtained when the mobile object performs a position estimation process based on the feature amount map and the feature amount map, such that the smaller the degree of match, the lower the reliability of the divided region; and a feature amount map updating means for updating the feature amount map based on the reliability of each of the divided regions. [Effects of the Invention]

[0008] According to the present invention, it is possible to realize an information processing device that can maintain high accuracy of a map for position estimation. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of the internal configuration of a server 1 according to a first embodiment of the present invention. [Figure 2] 2 is a diagram illustrating an example of the internal configuration of a moving object 2 according to the first embodiment. FIG. [Figure 3] 10 is a flowchart illustrating an example of a processing flow of the server 1 according to the first embodiment. [Figure 4] 10(A) to 10(C) are diagrams illustrating an example of a method for generating a feature amount map in step S310. [Figure 5] 10 is a flowchart illustrating an example of a processing flow of a moving object 2 according to the first embodiment. [Figure 6]10A and 10B are diagrams illustrating a detailed example of the position estimation performed by the moving object 2 in step S540 according to the first embodiment. [Figure 7] 10(A) to 10(C) are diagrams illustrating a detailed example of a feature map updating method according to the first embodiment. [Figure 8] 10 is a flowchart illustrating an example of a processing flow of the server 1 according to the second embodiment. [Figure 9] 10A and 10B are diagrams illustrating an example of a feature map according to the second embodiment. [Figure 10] 10 is a flowchart illustrating an example of a processing flow of a moving object 2 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each drawing, the same members or elements are designated by the same reference numerals, and duplicate descriptions will be omitted or simplified.

[0011] <Embodiment 1> In this embodiment, a method for maintaining high accuracy of a map for position estimation even if the environment has changed since the map for position estimation was created will be described. Note that this embodiment will be described taking an information processing system including a server 1 as an information processing device and a mobile object 2 as an example.

[0012] The moving body 2 is, for example, a vehicle that can travel autonomously and move to a destination while appropriately communicating with the server 1. That is, the moving body in this embodiment is, for example, an autonomous moving body such as an AGV (Automated Guided Vehicle) or an AMR (Autonomous Mobile Robot).

[0013] In this embodiment, an example will be described in which the moving body 2 is a vehicle that moves on wheels, but it may also be a drone that can move using, for example, a propeller.

[0014] That is, the mobile object may be any type of mobile device, such as an automobile, train, ship, airplane, robot, drone, etc. At least a part of the information processing device of the embodiment may or may not be mounted on the mobile object. The present invention can also be applied to remote control of the mobile object or driving assistance of the mobile object.

[0015] 1 is a diagram illustrating an example of the internal configuration of a server 1 according to a first embodiment of the present invention. The server 1 comprises a CPU 110, a RAM 120, a ROM 130, a storage 140, a communication unit 150, etc., and each unit is connected via a bus 199 so as to be able to communicate with each other. The server 1 also functions as an information processing device that can communicate with at least one mobile object.

[0016] The CPU 110 controls the operation of each part of the server 1 and functions as a control means in the server 1. The ROM 130 stores a computer program that describes the processing procedures of the CPU 110 as a computer, and the RAM 120 serves as a work memory in which the control program and data are temporarily stored.

[0017] The storage 140 stores data necessary for execution by the CPU 110, execution results, and other data. The RAM 120, the ROM 130, and the storage 140 function as storage media. The communication unit 150 is configured to be able to communicate with multiple mobile objects 2 via the Internet or a wireless LAN.

[0018] 2 is a diagram illustrating an example of the internal configuration of the moving object 2 according to embodiment 1. The moving object 2 includes a CPU 210, a RAM 220, a ROM 230, a storage 240, a communication unit 250, a distance sensor 260, a driving unit 270, etc., and each unit is connected via a bus 299 so as to be able to communicate with each other.

[0019] The operations of the CPU 210, RAM 220, ROM 230, storage 240, and communication unit 250 are similar to those of the CPU 110, RAM 120, ROM 130, storage 140, and communication unit 150 of the server 1, and therefore will not be described here. The CPU 210 functions as a control unit in the mobile object.

[0020] The distance sensor 260 is a sensor that can measure the distance from the distance sensor 260 to objects around the moving body 2, and can use, for example, LiDAR (Light Detection and Ranging). The distance sensor 260 can compile distance data to a plurality of measurement points and output it as point cloud data.

[0021] The distance sensor 260 may be configured to generate point cloud data, and may be configured to acquire the surrounding distances and generate point cloud data using, for example, a stereo camera, a ToF (Time of Flight) sensor, or an image sensor with a phase difference detection function.

[0022] The driving unit 270 is configured to control the speed and direction by controlling the rotation speed, gears, steering angle, etc. of the motor and engine in response to instructions from the CPU 210, and to physically move the moving body 2 to any position.

[0023] Fig. 3 is a flowchart illustrating an example of a processing flow of the server 1 according to embodiment 1. Note that the CPU 110, which serves as a computer in the server 1, executes a computer program stored in memory, thereby sequentially performing the operations of the steps in the flowchart in Fig. 3.

[0024] In step S310, the CPU 110 reads the point cloud map from the storage 140, generates a feature amount map, and stores the feature amount map in the storage 140. The operation of step S310 may be performed periodically at a cycle different from that of the flow shown in FIG.

[0025] 4(A) to 4(C) are diagrams illustrating an example of a method for generating a feature amount map in step S310. Each point in Fig. 4(A) represents an object around the road, and point cloud data consisting of these multiple points is collected and associated with real space coordinates, which is called a point cloud map 41.

[0026] Next, the CPU 110 divides the point cloud map 41 into rectangular division areas 42 of the same predetermined size as shown in FIG. 4(B), and calculates the feature amount for each division area 42.

[0027] That is, the feature amount map in this embodiment is a feature amount map in which, for each divided area obtained by dividing the real space into a grid, feature amounts are calculated based on pre-acquired point cloud data included in the divided area, and the divided areas are associated with positions in the real space. Also, the CPU 110 functions as a feature amount map management means for managing the feature amount map.

[0028] When the position estimation algorithm implemented by the moving body 2 is NDT (Normal Distribution Transform) scan matching, the CPU 110 calculates the mean μ and variance-covariance matrix V of the point cloud data within the divided region as feature quantities using the following equations 1 and 2, respectively.

[0029]

number

number

[0030] Here, pi represents the coordinate vector of the point cloud data contained in the divided region, and n represents the number of point cloud data contained in the divided region. By calculating the mean μ and variance-covariance matrix V in this way and approximating the distribution of the point cloud data within the divided region to a normal distribution, it is possible to represent the characteristics of the distribution of the point cloud on the point cloud map with less information.

[0031] 4(C) is a feature amount map that schematically illustrates the distribution of feature amounts 43 calculated by the above formula. The CPU 110 performs the same process for all areas on the point cloud map to generate a feature amount map. The CPU 110 then stores the generated feature amount map in the storage 140.

[0032] In this embodiment, an example in which the divided regions are two-dimensional rectangles has been described, but they may also be three-dimensional rectangles (cuboids, voxels). It is desirable that the division method for the divided regions corresponds to real space. For example, it is desirable that the horizontal direction be divided based on latitude and longitude or a plane rectangular coordinate system.

[0033] In addition, it is desirable to divide the vertical direction based on the geoid surface. The coordinates of the point cloud data in the point cloud map are pi = [x, y] for two dimensions and pi = [x, y, z] for three dimensions.

[0034] Next, in step S320, CPU 110 determines whether feature map transmission request information has been received from moving object 2 via communication unit 150. If CPU 110 determines that feature map transmission request information has been received from moving object 2, it proceeds to step S330; if it determines that feature map transmission request information has not been received, it repeats step S320.

[0035] Next, in step S330, based on the feature map transmission request information received in step S320, the CPU 110 reads the feature map from the storage 140. Then, the CPU 110 transmits the feature map for a predetermined range via the communication unit 150 to the moving object 2 that requested the feature map.

[0036] If the feature map transmission request information received in step S320 includes requested coordinate information, a predetermined range of the feature map around the requested coordinates may be extracted and transmitted. Also, if the feature map transmission request information includes requested range information of the feature map, a feature map of the requested range may be extracted and transmitted.

[0037] Next, in step S340, CPU 110 determines whether or not it has received score information for position estimation from moving object 2 via communication unit 150 within a predetermined time after transmitting the feature amount map in step S330.

[0038] If CPU 110 determines that score information has been received from moving object 2 within the predetermined time, it proceeds to step S350, but if it determines that score information has not been received within the predetermined time, it ends the flow of Fig. 3. The flow of Fig. 3 is repeated at predetermined intervals. Details of the score information will be described later.

[0039] Next, in step S350, CPU 110 calculates and updates the reliability for each divided region in the feature map in step S340. Step S350 functions as a reliability calculation step (reliability calculation means). Details of the reliability calculation in step S350 will be described later.

[0040] Next, in step S360, CPU 110 updates the feature amount map based on the reliability information calculated in step S350. Here, step S360 functions as a feature amount map updating step (feature amount map updating means) that updates the feature amount map based on the reliability of each divided region. The feature amount map updating method will be described in detail later.

[0041] Fig. 5 is a flowchart illustrating an example of a processing flow of the moving body 2 according to the first embodiment, and shows an example of a processing flow executed by the CPU 210 of the moving body 2 when estimating a position. Note that the CPU 210, which serves as a computer within the moving body 2, executes a computer program stored in a memory, thereby sequentially performing the operations of the steps in the flowchart of Fig. 5, and the flow of Fig. 5 is repeatedly executed at a predetermined cycle.

[0042] First, in step S510, the CPU 210 transmits feature amount map transmission request information to the server 1 via the communication unit 250. At this time, the feature amount map transmission request information can include coordinate information and range information of the feature amount map to be acquired.

[0043] For example, a Global Navigation Satellite System (GNSS) sensor or the like may be mounted on the mobile object 2, and the position information of the mobile object acquired by the GNSS sensor may be used as the coordinate information included in the feature map transmission request information.

[0044] Alternatively, the coordinate information included in the feature map transmission request information may be the location information obtained the last time the mobile object 2 performed self-location estimation. If range information is to be included in the feature map transmission request information, it is desirable to determine the range information taking into consideration the performance of the mobile object 2, the communication status with the server 1, and the like.

[0045] For example, it is desirable to request a feature quantity map with a wider range as the speed of the moving body 2 increases. Also, it is desirable to request a feature quantity map with a wider range as the capacity of the storage 240 or the capacity of the RAM 220 of the moving body 2 increases. Furthermore, it is desirable to request a feature quantity map with a wider range as the communication speed between the moving body 2 and the server 1 decreases (communication delay increases).

[0046] Next, in step S520, the CPU 210 determines whether or not a feature amount map has been received from the server 1 via the communication unit 250 within a predetermined time after transmitting the feature amount map transmission request in step S510. Here, step S520 functions as an acquisition step (acquisition means) for acquiring a feature amount map from the information processing device.

[0047] In step S520, if the CPU 210 determines that the feature amount map has been received from the server 1 within the predetermined time, the CPU 210 stores the received feature amount map in the storage 240 or the RAM 220, and then proceeds to step S530.

[0048] On the other hand, if the CPU 210 determines in step S520 that the message has not been received from the server 1 within the predetermined time, the flow of Fig. 5 ends. The flow of Fig. 5 is repeated at a predetermined cycle. Depending on the speed of the moving object 2, the cycle of the flow of Fig. 5 may be shortened as the speed increases.

[0049] In step S530, the CPU 210 controls the distance sensor 260 to acquire real-time point cloud data around the moving object 2 and store it in the RAM 220. At this time, if the amount of real-time point cloud data is large, a configuration may be adopted in which downsampling processing is performed. For example, a configuration may be adopted in which a voxel grid filter or random sampling is used as the downsampling processing.

[0050] In step S540, the CPU 210 performs a matching calculation between the feature amount map determined to have been acquired in step S520 and the real-time point cloud data acquired in step S530, thereby estimating the position of the moving object 2. Here, step S540 functions as a position estimation step (position estimation means) that estimates the self-position based on the feature amount map.

[0051] 6A and 6B are diagrams illustrating a detailed example of the position estimation performed by the moving object 2 in step S540 according to the first embodiment.

[0052] For example, as shown in the following equation 3, a transformation matrix M is used to transform coordinates from the local coordinate system X of the moving object to the coordinate system X' in the feature map. Before position estimation is performed, the correct transformation matrix M is unknown, but the optimal transformation matrix M can be calculated by performing a matching calculation between the real-time point cloud data and the feature map. X′=MX (Formula 3)

[0053] FIG. 6(A) is a diagram showing a schematic diagram of the positional relationship between the real-time point cloud data and the feature map before the start of the matching calculation, and shows the moving object position 600 transformed into the coordinate system of the feature map using a hypothetical transformation matrix, and the real-time point cloud data 601.

[0054] A mobile object position 600 in FIG. 6A is an initial position for performing matching calculations, and as the initial position, for example, the result of the previous position estimation execution or coordinate information acquired by GNSS is used.

[0055] Here, for a divided area that includes the coordinate system X' of the real-time point cloud data 601 that has been coordinate-transformed using Equation 3, a score Si for each divided area is calculated based on the feature amounts μ and V of the divided area, for example, using Equation 4 below.

number

[0056] If there is no real-time point cloud data or no feature in a divided area, the score of that divided area is set to zero, or the divided area is excluded from the calculation as a non-matching target.

[0057] Then, based on the score Si of each divided region, the overall score S when coordinates are transformed using the transformation matrix M is calculated using the following formula 5. Here, q represents the number of divided regions to be matched.

number

[0058] The larger this total score S, the better the match between the real-time point cloud data and the feature map. In other words, the total score S corresponds to the degree of match between the point cloud data and the feature map when a position estimation process based on the feature map is performed by a moving object. Therefore, the result of position estimation can be obtained by performing a matching calculation using an arbitrary optimization algorithm to find the transformation matrix M that maximizes this total score S.

[0059] The optimization algorithm can be, for example, the Newton method. Note that, depending on the optimization algorithm, it may be difficult to perform the optimization calculation of Equation 5. In such cases, the score Si for each divided region and the total score S may be replaced with an arbitrary approximate formula.

[0060] 6(B) shows an example of a moving object position 610 transformed into a feature map coordinate system using the transformation matrix M calculated by the matching calculation, and real-time point cloud data 611. As described above, the feature map of this embodiment is associated with real space, such as latitude and longitude, so that the position of the moving object 2 in real space can be identified with high accuracy by using the transformation matrix M calculated by the matching calculation.

[0061] Finally, in step S550, CPU 210 transmits the score information of each divided area calculated in step S540 to server 1 via communication unit 250. The score information of each divided area is associated with the coordinate information and index information of the divided area. Here, step S550 functions as a transmission step (transmission means) that transmits, to the information processing device, point cloud data obtained when self-location estimation is performed and information on the degree of compatibility between the feature amount map.

[0062] As described above, the moving object 2 periodically repeats steps S510 to S550, thereby enabling the moving object 2 to estimate its latest position with high accuracy.

[0063] Next, the details and effects of the processes of steps S340 to S360 executed by the CPU 110 of the server 1 will be described.

[0064] As described above, in step S350 (reliability calculation step), CPU 110 calculates and updates the reliability of each divided region in the feature map based on the score information received in step S340. More specifically, the reliability of each divided region is calculated so that the smaller the score Si (degree of suitability), the lower the reliability of the divided region.

[0065] That is, the score Si corresponds to the reliability, and as described above, the score Si for each divided area calculated by Equation 4 represents the degree of matching (degree of compatibility) between the feature amount in each divided area and the real-time point cloud data.

[0066] 7A to 7C are diagrams illustrating a detailed example of a feature map updating method according to embodiment 1. For example, assume that the matching calculation is completed and the position of the moving object 2 estimated as a result is in the state shown in Fig. 7A. Comparing the divided area 70 with the divided area 71, the point cloud data 711 and 712 in the divided area 71 are located near the average compared to the variance of the feature 710, and therefore the score Si in the divided area 71 is calculated to be a relatively large value.

[0067] On the other hand, since the real-time point cloud data 701 in the divided area 70 is farther from the mean than the variance of the feature amount 700, a relatively small value is calculated as the score Si in the divided area 70. In this way, the reason why the score Si becomes a small value when the matching calculation is completed is thought to be that the environment in the divided area is likely to have changed between the time when the point cloud map is acquired and generated and the time when the moving object 2 acquires the real-time point cloud data.

[0068] For example, a possible cause of an environmental change is when pedestrians or other moving objects such as vehicles that were not present when the point cloud map was created are present when the real-time point cloud data is acquired. Alternatively, a possible cause of an environmental change is when pedestrians or other moving objects that were present when the point cloud map was created are no longer present when the real-time point cloud data is acquired.

[0069] Specifically, for example, when real-time point cloud data is acquired, the same divided area may contain both point cloud data for a building wall and point cloud data for other moving objects, and the divided area on the point cloud map may contain only the point cloud for the building wall. In this case, the difference between the real-time point cloud data and the feature map feature amount is large, so a small score Si is calculated. Other factors that can cause environmental changes include fallen leaves from trees and objects under construction.

[0070] Furthermore, by calculating the reliability based on score information from multiple different moving objects, the reliability of the reliability itself can be improved. Even if the accuracy of the distance sensor installed in the moving object is low, the possibility of incorrectly updating the map due to measurement errors of the distance sensor can be reduced.

[0071] Furthermore, when using a LiDAR or similar distance sensor, the accuracy of the distance sensor generally decreases as the distance from the moving object increases. Therefore, the score may be lowered depending on the distance from the position of the moving object to the point cloud data or the divided area at the time when the matching calculation is completed.

[0072] The reliability for each divided region may be calculated as follows: That is, the reliability ci of the divided region index i may be calculated by the following equation 6, for example.

number

[0073] In the above formula 6, j is the number of the moving object, J is the number of moving objects that received the score, Aj is the accuracy of the distance sensor of moving object j, and Di,j is the distance from moving object j to segmentation area index i. Also, 0≦Aj≦1, and the closer to 1, the higher the sensor accuracy. Also, f(Di,j) is 0≦f(Di,j)≦1, and represents a function that approaches 0 as Di,j increases.

[0074] It is also possible to obtain scores based on real-time point cloud data at multiple different times, and if the scores are low, calculate a low reliability. Specifically, when obtaining scores multiple times at multiple different times, it is desirable to configure the system so that the more times a score below a predetermined value is calculated, the lower the reliability is calculated. In this way, the reliability is calculated and updated in step S350.

[0075] Next, in step S360, based on the reliability information calculated in step S350, if the reliability is lower than a predetermined threshold, CPU 110 determines that updating of the feature amount map is necessary and updates the feature amount map.

[0076] Specifically, the CPU 110 of this embodiment updates the feature map by deleting the feature of the feature map in a divided area whose reliability is lower than a predetermined threshold and which is determined to need updating. Alternatively, the CPU 110 sets both the mean and the variance-covariance to zero. By doing so, the next time a matching calculation is performed for the moving object 2 or another moving object, the divided area in question can be excluded from the matching calculation and prevented from contributing to the position estimation result.

[0077] That is, in this embodiment, such divided areas are not used for position estimation because environmental changes occur between the point cloud map on which the feature map is based and the real-time point cloud data.

[0078] By using the above method, it is possible to maintain a high level of accuracy in the feature map, which in turn improves the accuracy of location estimation. In addition, by excluding unreliable divided regions from the calculation, it is possible to reduce the cost of matching calculations for divided regions that are not important for location estimation.

[0079] In step S360, the CPU 110 of the server 1 may regenerate the feature map based on the new reliability data updated in step S350.

[0080] In this case, the CPU 110 of the server 1 transmits a transmission request for real-time point cloud data to the mobile body in order to acquire real-time point cloud data for the divided area determined to have low reliability. The transmission request for real-time point cloud data includes information that can identify the divided area whose feature values ​​are to be updated, such as the index and coordinates of the divided area.

[0081] Then, when the CPU 210 of the moving object 2 determines that it has received the transmission request information for the real-time point cloud data, it transmits to the server 1 the real-time point cloud data of the requested divided area.

[0082] After transmitting the transmission request information for the real-time point cloud data, the CPU 110 of the server 1 receives the real-time point cloud data returned from the moving object 2, and then calculates a feature amount based on the received real-time point cloud data. Then, the CPU 110 updates the calculated feature amount as a new feature amount for the divided area and stores it in the storage 140.

[0083] For example, when the CPU 110 of the server 1 determines that the reliability of the divided area 70 in FIG. 7A is low, the CPU 110 transmits a transmission request for the real-time point cloud data of the divided area 70 to the moving object 2.

[0084] The CPU 210 of the moving object 2 transmits the real-time point cloud data 701 included in the divided area 70 to the server 1 in response to the request to transmit the real-time point cloud data.

[0085] 7(C), the CPU 110 of the server 1 calculates the feature amounts of the divided area 70 based on the received real-time point cloud data 701, and updates the feature amount map by setting the calculated feature amounts as new feature amounts 705 for the divided area 70. The updated feature amount map is then stored in the storage 140.

[0086] That is, point cloud data for a divided area where the reliability of the feature map is determined to be below a predetermined value is acquired from a moving body, and feature amounts are calculated based on the acquired point cloud data and updated as new feature amounts for that divided area.

[0087] 7(C), in a divided area determined to have low reliability, such as divided area 70, an environmental change has occurred in the point cloud map on which the feature map is based and in the real-time point cloud data. Therefore, as described above, in this embodiment, the feature map is updated by updating the feature amounts of a divided area determined to have low reliability, so that the feature map can be maintained with high accuracy.

[0088] Furthermore, the next time a matching calculation for self-position estimation is performed by the moving body 2 or another moving body, etc., the updated feature amount map can be referenced, thereby improving the accuracy of position estimation.

[0089] Although the example has been described in which NDT matching is used as the matching algorithm for position estimation, this is not limitative. For example, an ICP (Iterative Closest Point) algorithm may be used as the matching algorithm for position estimation.

[0090] In this case, a point cloud map for each divided area is used as a feature map, and the reliability for each divided area can be calculated based on the sum of the distances between the point cloud map and the real-time point cloud data of the moving object within the divided area, thereby achieving the same effect.

[0091] Furthermore, the processes executed by each unit of the server 1 in this embodiment may be divided and executed by multiple servers or information processing devices. For example, the process of generating the feature map in step S310 in Fig. 3 may be executed by a server other than the server 1, and the result may be sent to the server 1, which then acquires and manages the feature map.

[0092] Alternatively, the feature map generation process may be performed in advance by another information processing device or the like, and the generated feature map data may be stored in the storage 140 of the server 1, and the server 1 may acquire and manage the feature map.

[0093] As described above, by operating each part of the server 1 and the mobile body 2 in this embodiment, the accuracy of the location estimation map can be maintained at a high level even if changes have occurred in the actual environment since the feature map, which is the location estimation map, was generated.

[0094] Note that the calculation and update of the reliability in step S350 may be performed by the moving body 2. Also, the matching calculation and score calculation in step S540 may be performed by the server 1. In that case, the server 1 may receive real-time point cloud data from the moving body 2 instead of the score.

[0095] <Embodiment 2> The second embodiment differs from the first embodiment in that the map for location estimation is updated so that it can be used by various mobile bodies. Note that the server 1, the mobile body 2, and their internal configurations in this embodiment are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0096] Fig. 8 is a flowchart illustrating an example of a processing flow of the server 1 according to the second embodiment. The CPU 110, which serves as a computer in the server 1, executes a computer program stored in memory, thereby sequentially performing the operations of the steps in the flowchart in Fig. 8. The flow in Fig. 8 is repeated at a predetermined cycle. The operation of step S810 may also be performed periodically at a cycle different from that of the flow shown in Fig. 3.

[0097] 8, the CPU 110 of the server 1 reads the point cloud map from the storage 140, generates a feature amount map, and stores it in the storage 140. When generating the feature amount map, the area is divided, and multiple feature amount maps are generated for each size of each divided area, just like in the first embodiment.

[0098] However, in step S810 of the second embodiment, the size of the divided regions is set according to the processing performance of the moving object. Also, when multiple moving objects have different processing performances, the region is divided into multiple different sizes, and multiple feature amount maps are generated for each divided region size, which is different from the first embodiment.

[0099] That is, the feature map of embodiment 2 is a plurality of feature maps in which the real space is divided into divided areas in a grid of different sizes, each having a feature calculated based on the point cloud data contained in the divided area, and the divided areas are associated with positions in the real space.

[0100] In this way, in the second embodiment, the size of the divided regions is set according to the processing performance of the mobile body, so that a feature map with a divided region size appropriate for the processing performance of the mobile body can be used.

[0101] Furthermore, if multiple moving bodies each have different processing capabilities, by generating a feature map for each of multiple different divided area sizes, it is possible to use a feature map of an appropriate divided area size to match the processing capabilities of each of the multiple moving bodies.

[0102] In step S820, it is determined whether transmission request information for the feature map has been received, and if it is determined that it has been received, it is further determined whether the transmission request information received from the mobile body 2 includes a request for the division area size.

[0103] In step S830, the CPU 110 in the second embodiment transmits a feature amount map of the requested division area size to the moving object 2 based on the feature amount map transmission request information received from the moving object 2 in step S320. The processing in step S840 is similar to step S340 in the first embodiment, and therefore description thereof will be omitted.

[0104] In step S850, the CPU 110 in this embodiment calculates the reliability of each divided area in the feature map based on the score information received from the moving object 2. At this time, not only the reliability of the divided areas of the division area size used to calculate the score information, but also the reliability of divided areas of a size smaller than the division area size used to calculate the score information is updated.

[0105] 9(A) and 9(B) are diagrams illustrating an example of a feature map according to the second embodiment, and FIG. 9(A) is the same as FIG. 7(A). FIG. 9(B) is a schematic diagram of a feature map generated with a division region size that is half the size of the division region of the feature map of FIG. 9(A) (the same as FIG. 7(A)). Division region 70 in FIG. 9(A) corresponds to division regions 7001 to 7004 in FIG. 9(B).

[0106] In the second embodiment, in step S850, the CPU 110 updates the reliability of the divided region 70 based on information obtained by calculating the score in the feature map of the divided region size shown in Fig. 9(A), for example. In addition, the CPU 110 also updates the reliability of the divided regions 7001 to 7004 of fine size corresponding to the divided region 70 in the same manner.

[0107] In this case, it is desirable to use the reliability of the large-sized divided region 70 as the reliability of the small-sized divided region. In other words, it is desirable to also determine that the reliability of a feature map generated based on a divided region smaller than the divided region whose reliability is determined to be low, which overlaps with the divided region, is low.

[0108] Next, in step S860, CPU 110 of the second embodiment determines that the feature map needs to be updated if the reliability information for each divided region size calculated in step S850 is lower than a predetermined threshold. Then, the feature map is updated for each divided region size. Note that the feature map updating method in step S860 can be the same as that in the first embodiment, and therefore will not be described here.

[0109] By doing so, the calculation cost required for updating the feature map can be reduced compared to when a matching calculation is performed for each divided region size and the feature map is updated for each divided region size.

[0110] Fig. 10 is a flowchart illustrating an example of a processing flow of the mobile object 2 according to the second embodiment. In step S1010 in Fig. 10, the CPU 210 of the mobile object 2 according to the second embodiment transmits feature map transmission request information to the server 1 via the communication unit 250. At this time, the feature map transmission request information differs from step S510 according to the first embodiment in that it includes information that can identify the size of the divided regions of the feature map that is to be acquired.

[0111] The size of the divided regions of the feature map is determined taking into consideration the calculation power and storage capacity of the mobile object 2 and the performance of the ranging sensor. For example, if the calculation power and storage capacity are small, the calculation cost and data size can be reduced by using a feature map with a large divided region size. Furthermore, if the number of points that can be acquired by the ranging sensor is small, the accuracy of position estimation can be improved by using a feature map with a large divided region size.

[0112] Steps S1020 to S1050 are similar to the processes in steps S520 to S550, and therefore a description thereof will be omitted.

[0113] As described above, by operating each part of the server 1 and the mobile body 2 in embodiment 2, the accuracy of the location estimation map can be maintained at a high level even if changes have occurred in the actual environment since the feature map, which is the location estimation map, was generated.

[0114] Furthermore, as described above, not only the division area size used for matching during position estimation, but also the reliability of feature amount maps with finer division area sizes is updated. Therefore, not only can the accuracy of subsequent position estimations by a moving body for which matching for position estimation has been calculated be improved, but the accuracy of position estimations performed by other moving bodies, etc., that refer to feature amount maps with different division area sizes can also be improved. Therefore, feature amount maps can be updated more efficiently and generally, and accuracy can be maintained.

[0115] In the above embodiment, an example of an information processing system has been described in which the server 1 and the mobile object 2, which serve as information processing devices, operate while communicating with each other. However, for example, the mobile object 2 may have some or all of the functions of the server 1. That is, the information processing systems in the above-described first and second embodiments may be built into the mobile object. Also, as mentioned above, the server 1 may have some of the functions of the mobile object 2 in the above-described first and second embodiments.

[0116] The present invention has been described above in detail based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible based on the spirit of the present invention, and these are not excluded from the scope of the present invention.

[0117] The present invention also includes those that realize the functions of the above embodiments using, for example, at least one processor such as a CPU, memory, or circuit (for example, ASIC). Also, multiple processors may be used to perform distributed processing.

[0118] In order to realize part or all of the control in the above-described embodiments, a computer program that realizes the functions of the above-described embodiments may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the information processing device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. The present invention also includes the following combinations.

[0119] (Configuration 1) An information processing device capable of communicating with at least one mobile body, the information processing device having a feature map management means for managing a feature map in which, for each divided area obtained by dividing a real space into a grid, the feature map has feature amounts calculated based on previously acquired point cloud data included in the divided area, and the divided area is associated with a position in the real space, the feature map management means having: a reliability calculation means for calculating the reliability of each divided area based on a degree of match between the point cloud data when the mobile body performs a position estimation process based on the feature map and the feature map, such that the smaller the degree of match, the lower the reliability of the divided area; and a feature map update means for updating the feature map based on the reliability of each divided area.

[0120] (Configuration 2) The information processing device according to Configuration 1, wherein the feature map is a plurality of feature maps in which the real space is divided into divided regions in a grid of different sizes, the divided regions having feature values ​​calculated based on the point cloud data included in the divided regions, and the divided regions are associated with positions in the real space.

[0121] (Configuration 3) The information processing device according to Configuration 2, wherein the reliability calculation means determines that the reliability of a feature map generated based on a divided area that overlaps with the divided area whose reliability is determined to be low and is smaller than the divided area whose reliability is determined to be low is also low.

[0122] (Configuration 4) The information processing device according to Configuration 2 or 3, characterized in that the feature map update means updates the feature map by deleting features in the divided areas where the reliability of the feature map is lower than a predetermined threshold.

[0123] (Configuration 5) The information processing device described in any one of configurations 1 to 4, characterized in that the feature map update means acquires from the moving body the point cloud data of the divided area in which the reliability of the feature map is determined to be equal to or less than a predetermined value, calculates a feature based on the acquired point cloud data, and updates it as a new feature of the divided area.

[0124] (Configuration 6) A mobile body capable of communicating with an information processing device, characterized in that the mobile body has: an acquisition means for acquiring from the information processing device, for each divided area obtained by dividing real space into a grid, a feature map having features calculated based on previously acquired point cloud data contained in the divided area, the feature map corresponding to a position in the real space; a position estimation means for estimating a self-position based on the feature map; and a transmission means for transmitting to the information processing device the point cloud data at the time of performing the self-position estimation and information on the degree of compatibility between the feature map and the feature map.

[0125] (Configuration 7) A moving body characterized by having: a feature map management means for managing a feature map in which, for each divided area obtained by dividing real space into a grid, the feature map has feature amounts calculated based on previously acquired point cloud data included in the divided area, and the divided area is associated with a position in the real space; a position estimation means for estimating a self-position based on the feature map; a reliability calculation means for calculating the reliability of each divided area based on the degree of match between the point cloud data at the time of performing the self-position estimation and the feature map, such that the smaller the degree of match, the lower the reliability of the divided area; and a feature map update means for updating the feature map based on the reliability of each divided area.

[0126] (Configuration 8) A system comprising at least one mobile body and an information processing device, wherein the information processing device comprises a feature map management means for managing a feature map in which, for each divided area obtained by dividing real space into a grid, the feature map has feature amounts calculated based on previously acquired point cloud data included in the divided area, and the divided areas are associated with positions in the real space, and the feature map management means comprises: a reliability calculation means for calculating the reliability for each divided area based on point cloud data obtained when the mobile body performs a position estimation process based on the feature map and compatibility information with the feature map, so that a divided area having a low compatibility included in the compatibility information has a low reliability; and a feature map update means for updating the feature map based on the reliability for each divided area, and the mobile body comprises: a position estimation means for performing a self-position estimation based on the feature map, and a transmission means for transmitting the point cloud data obtained when the self-position estimation was performed and the compatibility information with the feature map to the information processing device.

[0127] (Program) A computer program for controlling the information processing device according to any one of configurations 1 to 5 or each means of the information processing system according to claim 8 by a computer. [Explanation of symbols]

[0128] 1: Server 2: Moving object 41: Point Cloud Map 42: Split area 43: Features

Claims

1. An information processing device capable of communicating with at least one mobile object, a feature amount map management means for managing a feature amount map for each divided area obtained by dividing a real space into a grid, the feature amount map having feature amounts calculated based on point cloud data included in the divided area and acquired in advance, the divided areas being associated with positions in the real space; The feature map management means a reliability calculation means for calculating the reliability of each divided region based on a degree of match between point cloud data obtained when the mobile object performs a position estimation process based on the feature amount map and the feature amount map, such that the smaller the degree of match, the lower the reliability of the divided region; and a feature amount map updating means for updating the feature amount map based on the reliability for each of the divided regions.

2. The feature map is 2. The information processing device according to claim 1, wherein the real space is divided into grids of different sizes, and each of the divided areas has a feature calculated based on the point cloud data included in the divided area, and the divided areas are a plurality of feature maps associated with positions in the real space.

3. The reliability calculation means The information processing device according to claim 2 , further comprising: determining that the reliability of a feature map generated based on a divided area that overlaps with the divided area whose reliability is determined to be low and that is smaller than the divided area whose reliability is determined to be low is also low.

4. The feature map update means The information processing apparatus according to claim 2 , wherein the feature map is updated by deleting feature amounts in the divided regions where the reliability of the feature map is lower than a predetermined threshold value.

5. The feature map update means The information processing device according to claim 1, characterized in that the point cloud data of the divided area in which the reliability of the feature map is determined to be equal to or less than a predetermined value is acquired from the moving body, and a feature is calculated based on the acquired point cloud data and updated as a new feature of the divided area.

6. A mobile object capable of communicating with an information processing device, an acquisition means for acquiring from the information processing device, for each divided area obtained by dividing a real space into a grid, a feature amount map having feature amounts calculated based on point cloud data included in the divided area and acquired in advance, wherein the divided areas are associated with positions in the real space; a position estimation means for estimating a self-position based on the feature amount map; a transmitting means for transmitting point cloud data obtained when the self-position estimation is performed and information on the degree of conformance between the point cloud data and the feature map to the information processing device.

7. a feature amount map management means for managing a feature amount map in which a real space is divided into divided areas in a grid pattern, the feature amount map having feature amounts calculated based on point cloud data included in the divided areas and acquired in advance, and in which the divided areas are associated with positions in the real space; a position estimation means for estimating a self-position based on the feature amount map; a reliability calculation means for calculating the reliability of each divided region based on a degree of matching between point cloud data obtained when the self-location estimation is performed and the feature amount map, such that the smaller the degree of matching, the lower the reliability of the divided region; and a feature amount map updating unit that updates the feature amount map based on the reliability of each of the divided regions.

8. A system including at least one mobile object and an information processing device, The information processing device includes: a feature amount map management means for managing a feature amount map for each divided area obtained by dividing a real space into a grid, the feature amount map having feature amounts calculated based on point cloud data included in the divided area and acquired in advance, the divided areas being associated with positions in the real space; The feature map management means a reliability calculation means for calculating the reliability of each divided region based on compatibility information between the feature map and point cloud data obtained when the mobile object performs a position estimation process based on the feature map, so that a reliability of the divided region having a low compatibility included in the compatibility information is low; a feature amount map updating means for updating the feature amount map based on the reliability of each of the divided regions, The moving body is a position estimation means for estimating a self-position based on the feature amount map; a transmitting unit that transmits the point cloud data obtained when the self-position estimation is performed and the degree of conformance information between the point cloud data and the feature map to the information processing device.

9. A computer program for controlling the information processing device according to any one of claims 1 to 5 or the means of the information processing system according to claim 8 by a computer.

Images