Information processing device
The information processing device addresses VSLAM accuracy issues by collecting and weighting point cloud data based on map data attributes, ensuring high reliability and accuracy in comprehensive point cloud generation.
Patent Information
- Application Number
- JP2024138934
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-06
- Estimated Expiration
- Not applicable · inactive patent
Smart Images

Figure 2024157029000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device. [Background technology]
[0002] Conventionally, a method (SLAM: Simultaneous Localization and Mapping) is known in which a moving object simultaneously performs self-location estimation and creates an environmental map. As one type of SLAM, a method (VSLAM: Visual SLAM) using a video camera instead of a laser device has been proposed (see, for example, Patent Document 1). In VSLAM as described in Patent Document 1, the use of a video camera aims to reduce costs. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2014-222550 A Summary of the Invention [Problem to be solved by the invention]
[0004] In VSLAM as described in Patent Document 1, feature points are extracted from each image (frame) captured by a video camera, and self-location estimation and environmental map creation are performed based on changes in feature points between images. When creating an environmental map in this way, a method can be considered in which the accuracy is improved by correcting the point cloud data included in the environmental map by referring to existing map data. Furthermore, a method can be considered in which multiple point cloud data are collected and integrated for a specified section to further improve the accuracy.
[0005] However, when generating each point cloud data, the map data referenced is not necessarily the same. Therefore, if the collected multiple point cloud data are mixed with point cloud data generated using old map data or map data with low accuracy, the accuracy of the integrated point cloud data may decrease.
[0006] Therefore, an example of an object of the present invention is to provide an information processing device that can generate highly accurate overall point cloud data based on individual point cloud data generated using map data. [Means for solving the problem]
[0007] In order to solve the above-mentioned problems and achieve the object, the information processing device of the present invention described in claim 1 comprises a collection unit that collects individual point cloud data generated based on imaging information and map data, and a generation unit that generates comprehensive point cloud data based on a plurality of the individual point cloud data, wherein the collection unit acquires attribute information regarding the map data, and the generation unit determines a contribution degree for each of the plurality of individual point cloud data based on the attribute information, and generates the comprehensive point cloud data. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram showing an outline of an information processing device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] An embodiment of the present invention will be described below. An information processing device according to an embodiment of the present invention includes a collection unit that collects individual point cloud data generated based on imaging information and map data, and a generation unit that generates integrated point cloud data based on the plurality of individual point cloud data. The collection unit acquires attribute information on the map data used to generate the individual point cloud data for each of the plurality of individual point cloud data. The generation unit determines a contribution level for each of the plurality of individual point cloud data based on the attribute information, and generates integrated point cloud data.
[0010] According to the information processing device of this embodiment, the collection unit acquires attribute information related to map data and determines the contribution degree of individual point cloud data based on the attribute information, thereby increasing the contribution degree of individual point cloud data based on highly reliable map data and decreasing the contribution degree of individual point cloud data based on less reliable map data. This makes it possible to generate highly accurate overall point cloud data based on the individual point cloud data generated using map data.
[0011] The attribute information preferably includes time information regarding the time when the map data was last updated or created. In this case, the generation unit preferably increases the contribution of individual point cloud data based on map data that was last updated or created more recently. Map data tends to be more reliable the more recently it was last updated or created. Therefore, by increasing the contribution of individual point cloud data based on map data that was last updated or created more recently, it is possible to generate comprehensive point cloud data with high accuracy.
[0012] The attribute information may include frequency information regarding the cumulative number of updates or the version number of the map data. In this case, it is preferable that the generating unit increases the contribution of individual point cloud data based on map data with a large cumulative number of updates or a large version number. When map data has a large cumulative number of updates or a large version number, the last update is more recent and the reliability tends to be high. Therefore, by increasing the contribution of individual point cloud data based on map data with a large cumulative number of updates or a large version number, it is possible to generate comprehensive point cloud data with high accuracy.
[0013] The attribute information may include resolution information regarding the resolution of the map data. In this case, it is preferable that the generating unit increases the contribution of individual point cloud data based on map data with high resolution. The resolution of map data may differ depending on the creator (provider) and the purpose of use. Therefore, by increasing the contribution of individual point cloud data based on map data with high resolution, it is possible to generate comprehensive point cloud data with high accuracy.
[0014] An information processing method according to an embodiment of the present invention includes a collection step of collecting individual point cloud data generated based on imaging information and map data, and a generation step of generating integrated point cloud data based on the plurality of individual point cloud data. In the collection step, attribute information on the map data used to generate the individual point cloud data is acquired for each of the plurality of individual point cloud data. In the generation step, a contribution degree is determined for each of the plurality of individual point cloud data based on the attribute information, and integrated point cloud data is generated. According to this information processing method, by acquiring attribute information on the map data in the collection step and determining a contribution degree of the individual point cloud data based on the attribute information in the generation step, highly accurate integrated point cloud data can be generated based on the individual point cloud data generated using the map data.
[0015] The above-mentioned information processing method may be implemented as an information processing program that causes a computer to execute the information processing method. In this way, it is possible to generate highly accurate overall point cloud data based on the individual point cloud data generated using map data by using the computer.
[0016] The information processing program may be stored in a computer-readable recording medium. In this way, the program can be distributed as a standalone program in addition to being incorporated into a device, and version upgrades can be easily performed. EXAMPLES
[0017] An embodiment of the present invention will be specifically described below. As shown in Fig. 1, an information processing device 1 of this embodiment is mounted on an external server 20 that communicates with a moving object 10, for example, and includes a collection unit 2 and a generation unit 3. The moving object 10 is, for example, a vehicle, and is mounted with an imaging unit 11, an individual data generation unit 12, a communication unit 13, and a map information storage unit 14. Although Fig. 1 shows only one moving object 10, the external server 20 may be capable of communicating with a plurality of moving objects.
[0018] The imaging unit 11 is, for example, a video camera, and is arranged to capture an image of the front side in the traveling direction of the moving object 10. The moving image captured by the imaging unit 11 is composed of a plurality of image information (frames), and is transmitted to the individual data generating unit 12. The imaging unit may be capable of capturing still images continuously at a predetermined time interval.
[0019] The individual data generating unit 12 acquires image information from the imaging unit 11 and generates individual point cloud data by VSLAM. That is, the individual data generating unit 12 extracts feature points in the image information and generates point cloud data based on changes in the feature points. The point cloud data may be, for example, three-dimensional point cloud data. Note that the extracted feature points may be distinguishable from other objects and trackable, such as corners of an object, and the feature points may be extracted using, for example, an ORB (Oriented fast and Rotated Brief) algorithm or other algorithms.
[0020] In addition, when the individual data generating unit 12 generates the individual point cloud data, it also uses the map data stored in the map information storage unit 14. Furthermore, when the individual data generating unit 12 generates the individual point cloud data, it also uses information on the imaging position and imaging direction of the imaging unit 11. That is, by using current position information obtained by a self-position estimating unit such as a GPS (Global Positioning System) receiving unit and displacement amount information obtained by an in-vehicle sensor such as a vehicle speed pulse obtaining unit, a gyro sensor, an acceleration sensor, etc., it is possible to identify the imaging position and imaging direction of the imaging unit 11.
[0021] By acquiring map data and specifying the imaging position and imaging direction of the imaging unit 11, it is possible to estimate what objects the feature points extracted in each frame and the constituent points constituting the point cloud data correspond to. The individual data generation unit 12 specifies and removes, for example, feature points corresponding to movable objects (other vehicles, pedestrians, etc.) from among the multiple feature points extracted in each frame, based on map information and travel direction information. Furthermore, when generating individual point cloud data based on multiple feature point groups, the individual data generation unit 12 specifies and removes, for example, feature points corresponding to features (vegetation, etc.) that may change over time from among the constituent points constituting the individual point cloud data, based on map information and travel direction information. Note that the individual data generation unit 12 may execute only one of the removal of feature points and the removal of constituent points.
[0022] When generating individual point cloud data, the individual data generation unit 12 assigns attribute information on the map data to be used as a flag to the individual point cloud data. The attribute information includes at least one of time information on the last update or creation time, number information on the cumulative number of updates or version number, and resolution information on the resolution. The more recent the last update or creation time of the map data, the higher the reliability tends to be. Also, when the cumulative number of updates is large or the version number is large, the last update time is more recent and the reliability tends to be high.
[0023] The communication unit 13 is composed of circuits, antennas, etc. for communicating with networks such as the Internet or public lines, and transmits information (individual point cloud data) generated by the individual data generation unit 12 by communicating with the communication unit 21 of the external server 20.
[0024] The map information storage unit 14 is composed of, for example, a hard disk or a non-volatile memory, and stores existing map data. The map data may be any data that includes at least information about road shapes, and may be a city map, a line map, or an aeronautical map. Note that each mobile object does not need to be equipped with a map information storage unit, and may acquire map data by communicating with a data center or the like.
[0025] The collection unit 2 of the information processing device 1 acquires the individual point cloud data generated by the individual data generation unit 12 via the communication unit 21. The collection unit 2 collects the individual point cloud data by acquiring the individual point cloud data from multiple moving objects or acquiring the individual point cloud data multiple times from one moving object.
[0026] The generating unit 3 generates integrated point cloud data based on the multiple individual point cloud data collected by the collecting unit 2. The integrated point cloud data may be a combination of multiple individual point cloud data, or may indicate the average position of the constituent points of the multiple individual point cloud data. When generating the integrated point cloud data, the generating unit 3 determines the contribution degree of each individual point cloud data based on the attribute information of the map data, and weights the data according to the contribution degree, for example. At this time, the higher the reliability, the higher the contribution degree is set.
[0027] Depending on the attribute information, for example, when a certain number of years have passed since the last update or creation, the contribution degree may be set to 0 and processing may be performed in which the data is not used (removed). The contribution degree determined as above may be applied to the entire individual point cloud data, or may be applied only to the constituent points that are affected by the use of map data among the constituent points that make up the individual point cloud data.
[0028] The thus generated comprehensive point cloud data is used, for example, to estimate the self-position of a moving object. That is, when a feature is detected by an optical sensor (so-called LIDAR; Laser Imaging Detection and Ranging) mounted on the moving object, the self-position of the moving object can be estimated by comparing it with the comprehensive point cloud data.
[0029] With the above configuration, the collection unit 2 acquires attribute information related to map data and determines the contribution degree of individual point cloud data based on the attribute information, thereby increasing the contribution degree of individual point cloud data based on highly reliable map data and decreasing the contribution degree of individual point cloud data based on less reliable map data. This makes it possible to generate highly accurate comprehensive point cloud data based on the individual point cloud data generated using map data.
[0030] In addition, by increasing the contribution of individual point cloud data based on map data that was last updated or created recently, by increasing the contribution of individual point cloud data based on map data that has been updated a large number of times or has a large version number, and by increasing the contribution of individual point cloud data based on map data with high resolution, it is possible to generate highly accurate overall point cloud data.
[0031] The present invention is not limited to the above-described embodiment, but includes other configurations that can achieve the object of the present invention, and the following modifications are also included in the present invention.
[0032] For example, in the above embodiment, the individual data generating unit 12 assigns attribute information related to the map data as a flag to the individual point cloud data, but the attribute information may not be assigned to the individual point cloud data collected by the collecting unit 2. For example, the collecting unit 2 may acquire attribute information of the map data stored in the mobile body on which the individual point cloud data was generated. In addition, when the map data is not stored on the mobile body side and is acquired from a data center or the like, the collecting unit 2 may acquire attribute information of the map data from the data center or the like. In addition, the collecting unit 2 may estimate attribute information of the used map data based on the generation date and time of the individual point cloud data.
[0033] In the above embodiment, the individual data generating unit 12 removes the feature points and constituent points based on the map data when generating the individual point cloud data, but the feature points and constituent points do not have to be removed. For example, a reliability may be assigned to each constituent point of the individual point cloud data based on the map data.
[0034] In addition, the collection unit and the generation unit constituting the information processing device may not be mounted on an external server, and may be mounted on different devices or equipment. That is, each component of the information processing device may be located across multiple physically separated devices.
[0035] In addition, the best configurations and methods for implementing the present invention are disclosed in the above description, but the present invention is not limited thereto. That is, the present invention has been mainly illustrated and described with reference to specific embodiments, but those skilled in the art can make various modifications to the above-described embodiments in terms of shape, material, quantity, and other detailed configurations without departing from the scope of the technical idea and purpose of the present invention. Therefore, the descriptions limiting the shapes, materials, etc. disclosed above are illustrative descriptions for the purpose of facilitating understanding of the present invention, and do not limit the present invention, and therefore descriptions of the names of components that are free of some or all of the limitations on the shapes, materials, etc. are included in the present invention. [Explanation of symbols]
[0036] 1. Information processing device 2 Collection Department 3 Generation part
Claims
[Claim 1] A collection unit that collects individual point cloud data generated based on imaging information and map data; A generating unit that generates integrated point cloud data based on a plurality of the individual point cloud data, The collection unit acquires attribute information related to the map data, The information processing apparatus is characterized in that the generation unit determines a contribution rate for each of the plurality of individual point cloud data based on the attribute information, and generates the integrated point cloud data.
Citation Information
Patent Citations
Method for vslam optimization
JP2014222550A
