Information Processing Apparatus, Control Method, Program, and Storage Medium

Abstract

To provide an information processing device capable of creating a map with high precision by suitably fitting measured data to a matching material.SOLUTION: An information processing device 4 includes acquisition means, matching means, and correction means. The acquisition means acquires measurement data which is point group data including meta information on at least a part of a measurement point. The matching means performs matching of the measurement data and the matching material including meta information on at least a part of the measurement space in which the measurement data is measured on the basis of the meta information. The correction means generates the correction point group data Dc by correcting the measurement data based on the matching result.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a technology for generating maps.

Background Art

[0002] Conventionally, a technology for acquiring information necessary for generating a map based on the output of sensors installed in vehicles has been known. For example, Patent Document 1 discloses a system in which when each vehicle detects a change point in map data by a sensor, the map data is updated by transmitting data regarding the change point to a map management server. Further, Patent Document 2 discloses a scan matching method using reflection intensity. Furthermore, Non-Patent Document 1 discloses a scan matching method using color information.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When creating a map by SLAM (Simultaneous Localization and Mapping), large distortions sometimes occurred in the generated point cloud map. Such distortions occurred regardless of whether the measurements were made during a single run or multiple runs.

[0006] The present invention has been made to solve the above problems, and the main object thereof is to provide an information processing apparatus capable of suitably generating a point cloud map.

Means for Solving the Problems

[0007] The invention according to the claims is at least and an acquisition means for acquiring measurement data including meta information for at least some of the measurement points, and among the measurement data, the measurement data is measured measurement data corresponding to the alignment points selected in the measurement space is extracted, the extracted measurement data is processed to obtain processed data, and an alignment material including meta information for at least a part of the measurement space is aligned by matching based on the meta information; and a correction means for correcting the measurement data based on the result of the alignment.

[0008] Further, the invention according to the claims is a control method in which a computer extracts measurement data corresponding to alignment points selected in the measurement space from measurement data including meta information for at least some of the measurement points, obtains processed data obtained by processing the extracted measurement data, aligns the measurement data and an alignment material including meta information for at least a part of the measurement space by matching based on the meta information, and corrects the measurement data based on the result of the alignment. and also among the measurement data is measured measurement data including meta information for at least some of the measurement points,

[0009] Further, the invention according to the claims is at least andAn acquisition means for acquiring measurement data including meta information for some measurement points, and among the measurement data, the measurement data is measured Measurement data corresponding to the alignment points selected in the measurement space is extracted, the processed data obtained by processing the extracted measurement data, and an alignment material including meta information for at least a part of the measurement space are aligned by matching based on the meta information. An alignment means, and a program for causing a computer to function as a correction means for correcting the measurement data based on the result of the alignment.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0011] According to a preferred embodiment of the present invention, an information processing apparatus includes: an acquisition unit that acquires point cloud data including meta information for at least some measurement points; an alignment unit that aligns the point cloud data and an alignment material including meta information for at least a part of the measurement space in which the point cloud data is measured, based on the meta information; and a correction unit that corrects the point cloud data based on the result of the alignment. With this aspect, the information processing apparatus can suitably correct the point cloud data so as to accurately align it with the alignment material.

[0012] In one aspect of the above information processing apparatus, the information processing apparatus further includes an alignment point selection unit that selects alignment points in the measurement space, and the alignment unit performs the alignment with reference to the alignment points. In this aspect, the information processing apparatus can accurately perform the alignment by selecting points that are easy to match as the alignment points.

[0013] In another aspect of the above information processing apparatus, the meta information is feature information that is an index indicating a feature of an object, and the alignment unit performs the alignment based on the feature information. In this aspect, the information processing apparatus can perform the alignment with higher accuracy using the feature information and can suitably correct the point cloud data. In a preferred example, the point cloud data is data measured by a camera or a range sensor, and the feature information is information indicating a color or luminance measured by the camera or information indicating a reflection intensity measured by the range sensor.

[0014] In another aspect of the above information processing apparatus, the meta information includes information for specifying a floor that is the measurement space when measurement is performed in a facility, and the alignment unit performs alignment between the point cloud data and the alignment material corresponding to the floor specified by the meta information. With this aspect, even when measuring inside a facility having a plurality of floors, the information processing apparatus can suitably specify the floor to be the target for aligning the point cloud data and can suitably correct the point cloud data.

[0015] In another aspect of the information processing apparatus, the fitting material further has additional information which is information regarding the measurement space, and the correction means generates a point cloud map in which the additional information is added to the corrected measurement data. According to this aspect, the information processing apparatus can suitably generate a point cloud map that follows the additional information included in the fitting material.

[0016] In another aspect of the information processing apparatus, the point cloud data includes point cloud data based on the position of the external sensor measured by the external sensor and trajectory information indicating the trajectory of the measurement positions in the time series of the point cloud data, and the correction means corrects the trajectory information of the point cloud data based on the result of the fitting. According to this aspect, the information processing apparatus can suitably correct the point cloud data so as to fit the fitting material.

[0017] According to another preferred embodiment of the present invention, a control method is provided in which a computer acquires point cloud data including meta information for at least some of the measurement points, performs fitting based on the meta information on the point cloud data and a fitting material including meta information for at least a part of the measurement space in which the point cloud data is measured, and corrects the point cloud data based on the result of the fitting. By executing this control method, the computer can suitably correct the point cloud data so as to accurately fit the fitting material.

[0018] According to another preferred embodiment of the present invention, a program causes a computer to function as an acquisition means for acquiring point cloud data including meta information for at least some of the measurement points, a fitting means for performing fitting based on the meta information on the point cloud data and a fitting material including meta information for at least a part of the measurement space in which the point cloud data is measured, and a correction means for correcting the point cloud data based on the result of the fitting. By executing this program, the computer can suitably correct the point cloud data so as to accurately fit the fitting material. Preferably, the above program is stored in a storage medium.

Example

[0019] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0020] (1) System overview FIG. 1 is a schematic configuration diagram of a point cloud map generation system according to this embodiment. The point cloud map generation system shown in FIG. 1 is a system that generates a map based on measured point cloud data, and mainly includes a measuring device 1 that moves while performing measurement, and an information processing device 4 that generates a map based on the data measured by the measuring device 1.

[0021] The measuring device 1 generates measurement data "Im" to be supplied to the information processing device 4 by performing measurement around the measuring device 1 while moving within a space to be measured (also referred to as a "measurement space"). The measuring device 1 has various external sensors and uses any SLAM technology such as Visual-SLAM to generate measurement data Im. The measurement data Im includes point cloud data that is a point cloud of measurement points indicating the three-dimensional positions of the surfaces of surrounding objects, and trajectory information indicating the trajectory of the measurement positions of the generated point cloud data. Here, the "point cloud data" refers to point cloud map (integrated point cloud) data obtained by integrating data output by an external sensor (also referred to as "sensor data") based on the above trajectory. The point cloud data and the trajectory information are associated with each other by including date and time information and the like. The measuring device 1 is capable of data communication with the information processing device 4 and transmits the generated measurement data Im to the information processing device 4 at a predetermined timing.

[0022] Note that the measurement space may be a predetermined outdoor area, or may mainly be an area within a facility that is indoors. Also, when the facility has multiple floors, the measurement space may be any area within the facility. When the measurement space is within a facility, the measuring device 1 is, for example, a self-propelled robot. When the measurement space is outdoors, the measuring device 1 is, for example, a measurement vehicle traveling on a road.

[0023] The information processing device 4 receives the measurement data Im from the measuring device 1 and accumulates the received measurement data Im. Then, based on the measurement data Im, the information processing device 4 generates a point cloud map in the measurement space.

[0024] Note that the configuration of the point cloud map generation system shown in FIG. 1 is an example, and various modifications may be made to the configuration shown in FIG. 1.

[0025] For example, instead of acquiring the measurement data Im through data communication with the measuring device 1, the information processing device 4 may acquire the measurement data Im by reading the measurement data Im stored in the storage medium by the measuring device 1 from the storage medium. In this case, the above storage medium is electrically connected to the measuring device 1 during the measurement of the measuring vehicle, and the measurement data Im is written by the measuring device 1. Also, after the measurement of the measuring vehicle, the above storage medium is electrically connected to the information processing device 4, and the measurement data Im is read by the information processing device 4. Also, a plurality of measuring devices 1 may exist. Also, the information processing device 4 may be composed of a plurality of devices. In this case, the plurality of devices execute the pre-assigned processing and exchange the necessary data with each other between the devices. Also, the measuring device 1 may have a function corresponding to the information processing device 4 and execute the point cloud map generation process executed by the information processing device 4.

[0026] (2) Device configuration FIG. 2(A) is a block diagram showing the functional configuration of the measuring device 1. The measuring device 1 mainly includes a sensor group 2, an interface 11, a memory 12, and a controller 15. These elements are interconnected via a bus line.

[0027] The sensor group 2 is a plurality of sensors necessary for generating the measurement data Im, and includes an external sensor that generates sensor data by performing measurement outside the measuring device 1.

[0028] The external sensor may be a ranging sensor such as a lidar, or may be a camera (stereo camera) capable of measuring three-dimensional positions based on parallax, or a three-dimensional camera based on TOF (Time of Flight) or the like that does not use parallax. The sensor data is, for example, data based on the position and orientation of the measuring device 1 (specifically, the position and installation angle of the external sensor), and is, for example, data indicating a combination of distance and azimuth from the measurement position. Further, the sensor data and the point cloud data generated based on the sensor data include, as meta-information, information (also referred to as "feature information") that is an index representing features of the measurement target such as color information, luminance information, and reflection intensity information. In this case, the measuring device 1 may generate the above-described feature information that becomes meta-information based on the outputs of a plurality of external sensors. For example, the measuring device 1 may generate measurement data including point cloud data in which each measurement point has color information or luminance information based on the outputs of both the lidar and the camera.

[0029] Note that the sensor group 2 may further include an internal sensor. In this case, the internal sensor includes, for example, a plurality of sensors such as a GPS receiver, an acceleration sensor, a gyro sensor, and an IMU (Inertial Measurement Unit). The above GPS receiver may generate high-precision position information indicating the absolute position (for example, three-dimensional position of latitude, longitude, and altitude) of the measurement vehicle based on the RTK positioning method (that is, the interference positioning method). In this case, the controller 15 may generate trajectory information indicating the time-series trajectory of the measurement positions of the point cloud data generated by the external sensor based on the output of the internal sensor. Note that the internal sensor may be provided on the external sensor so as to directly detect the position and orientation of the external sensor. Further, the measuring device 1 may generate trajectory information obtained by converting the data indicating the position of the measuring device 1 output by the internal sensor into data indicating the position of the external sensor based on the information indicating the installation position and installation angle of the external sensor on the measuring device 1.

[0030] Also, when the measuring device 1 measures within a facility having a plurality of floors, information for specifying the floor to be measured may be included in the measurement data as meta information. In this case, for example, the measurement data includes, as meta information, the air pressure information output by the air pressure sensor provided in the measuring device 1, or the indoor positioning information indicating the indoor positioning result. The indoor positioning information may be positioning information using a beacon, positioning information based on IMES (Indoor MEssaging System), positioning information based on the radio wave intensity of a wireless LAN (Local Area Network), or positioning information based on RFID (Radio Frequency Identification).

[0031] The interface 11 performs an interface operation regarding the data transfer between the measuring device 1 and an external device. In this embodiment, the interface 11 stores the data generated by the sensor group 2 in the memory 12 based on the control of the controller 15.

[0032] The memory 12 is composed of various memories such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a non-volatile memory (including a hard disk drive, a flash memory, etc.). The memory 12 stores a program for the controller 15 to execute a predetermined process. Also, the memory 12 is used as a working memory of the controller 15. Note that the program executed by the controller 15 may be stored in a storage medium other than the memory 12.

[0033] Also, the memory 12 functionally has a point cloud data storage unit 16 and a trajectory information storage unit 17. The point cloud data storage unit 16 stores the generated point cloud data based on the control of the controller 15. The trajectory information storage unit 17 stores the generated trajectory information based on the control of the controller 15.

[0034] Note that at least one of the point cloud data storage unit 16 and the trajectory information storage unit 17 may be stored in an external storage device of the measuring device 1, such as a hard disk, connected to the measuring device 1 via the interface 11.

[0035] The controller 15 includes a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and controls the entire measuring device 1. In this case, the controller 15 executes a program stored in the memory 12 or the like to perform accumulation processing of data stored in the point cloud data storage unit 16 and the trajectory information storage unit 17, and transmission processing of the measurement data Im to the information processing device 4.

[0036] FIG. 2(B) is a block diagram showing the functional configuration of the information processing device 4. The information processing device 4 includes an interface 41, a memory 42, and a controller 45. These elements are interconnected via a bus line.

[0037] The interface 41 performs an interface operation related to data transfer between the information processing device 4 and an external device. In the present embodiment, the interface 41 receives the measurement data Im generated by the measuring device 1. The interface 41 may be a wireless interface for performing wireless communication with the measuring device 1, or may be a hardware interface for reading the measurement data Im from a storage medium or the like storing the measurement data Im. Further, the interface 41 may perform an interface operation with an input device that receives user input.

[0038] The memory 42 is composed of various memories such as a RAM, a ROM, and other non-volatile memories (including a hard disk drive, a flash memory, etc.). The memory 42 stores a program for the controller 45 to execute a predetermined process. Further, the memory 42 is used as a working memory of the controller 45. Note that the program executed by the controller 45 may be stored in a storage medium other than the memory 42.

[0039] Further, the memory 42 functionally includes a measurement data storage unit 46, a fitting material storage unit 47, and a point cloud map storage unit 48. The measurement data storage unit 46 stores the measurement data Im received from the measurement device 1.

[0040] The fitting material storage unit 47 stores fitting materials that are targets for fitting the measurement data stored in the measurement data storage unit 46. The fitting materials are map data (drawing data) indicating the structure and arrangement of the measurement space, and include, for example, data including drawings such as road ledgers, design drawings, or floor maps. Further, when the measurement space is a space within a facility with a hierarchical structure, the fitting material may be information indicating the drawings (floor maps) of each floor of the target facility. The data structure of the fitting material will be described with reference to FIG. 3.

[0041] The point cloud map storage unit 48 stores the measurement data corrected to be fitted to the fitting material as a point cloud map. The point cloud map may be point cloud data of three-dimensional measurement points in the measurement space, or may be voxel data representing point cloud data for each voxel indicating a regular grid in three-dimensional space.

[0042] Note that at least one of the measurement data storage unit 46, the fitting material storage unit 47, and the point cloud map storage unit 48 may be stored in an external storage device of the information processing device 4, such as a hard disk, connected to the information processing device 4 via the interface 41. The above storage device may be a server device that communicates with the information processing device 4. Further, the above storage device may be composed of a plurality of devices.

[0043] The controller 45 includes processors such as a CPU and a GPU, and controls the entire information processing device 4. In this case, the controller 45 performs processing related to the generation of the point cloud map by executing a program stored in the memory 42 and the like. The controller 45 functions as an "acquisition means", "fitting point selection means", "fitting means", "correction means", and a computer that executes a program.

[0044] Note that the processing executed by the controller 45 is not limited to being realized by software according to a program, and may be realized by any combination of hardware, firmware, and software, etc. Further, the processing executed by the controller 45 may be realized using a user-programmable integrated circuit such as, for example, an FPGA (Field-Programmable Gate Array) or a microcomputer. In this case, a program that the controller 45 executes in this embodiment may be realized using this integrated circuit. Thus, the controller 45 may be realized by hardware other than a processor.

[0045] Here, a supplementary explanation will be given with reference to FIG. 3 regarding the fitting material stored in the fitting material storage unit 47. FIG. 3 is an example of the data structure of the fitting material stored in the fitting material storage unit 47. The fitting material has target space identification information, structure / arrangement information, and meta information.

[0046] The target space identification information is identification information of the target space in which the structure and arrangement are shown in the structure / arrangement information described later. The target space identification information may be information indicating the absolute position such as the latitude and longitude of the target space, or may be identification information (for example, identification information of an administrative division, etc.) that can be specified from the absolute position. Further, the target space identification information may be information indicating the position in a relative coordinate system that is not associated with the absolute position.

[0047] The structure and arrangement information is information indicating the structure within the target space and the arrangement of features and the like within the space. Features include, for outdoor areas, ground features such as traffic lights and signboards, and for indoor areas, walls, floors, stairs, etc. The structure and arrangement information may be drawings such as design drawings of the target space (specifically, images showing the drawings). Also, the structure and arrangement information may have a multi-layer structure including a layer indicating the position of the wall surface, a layer indicating dimension lines, etc. Further, information indicating a scale may be added to the structure and arrangement information. Also, in the structure and arrangement information, a coordinate system that is the same as or mutually convertible with the coordinate system adopted in the trajectory information included in the measurement data (also referred to as the "material coordinate system") is set, and the arrangement of at least some of the structures or features is indicated by the material coordinate system. Also, the structure and arrangement information may have three-dimensional information (information on three-dimensional structures and arrangements) so that a three-dimensional model of the target space can be generated.

[0048] The meta information is data used in the process of aligning the measurement data by the measuring device 1 with the alignment material, and mainly includes feature information, hierarchical attribute information, and additional information.

[0049] The feature information is information indicating an index of the features of objects (including flat objects such as floors) that can be measured in the space targeted by the corresponding structure and arrangement information. The feature information may be, for example, color information of the object, information indicating the luminance when the object is measured, or information indicating the reflection intensity of the object. Note that the feature information does not necessarily need to be added to all objects existing in the target space, and may be added to some objects (for example, a dedicated painted space, etc.).

[0050] Hierarchical attribute information is information indicating the attributes of a floor when the corresponding structure / arrangement information targets a space in a facility having multiple floors. For example, when the measured data includes the value of a barometric pressure sensor as meta information, the hierarchical attribute information indicates the range of barometric pressure measured on the target floor. Similarly, when the measured data includes indoor positioning information as meta information, the hierarchical attribute information indicates the range of positions measured on the target floor. This hierarchical attribute information is also preferably used when performing specific processing (such as self-position estimation, etc.) by referring to map data including the hierarchical attribute information as meta information.

[0051] Additional information is additional information regarding the space targeted by the corresponding structure / arrangement information. The additional information is, for example, information regarding facilities (stores) existing in the target space (such as information regarding the facility name, genre, location, etc.), information regarding dedicated areas, and other facility information included in general map data. The additional information is added to the corrected measured data and stored in the point cloud map storage unit 48.

[0052] (3) Functional block FIG. 4 is a block diagram showing the functional configuration of the controller 45 of the information processing apparatus 4 in the present embodiment. As shown in FIG. 4, the controller 45 functionally includes a matching point selection unit 21, a processing unit 22, a deviation amount calculation unit 23, and a correction unit 24.

[0053] The matching point selection unit 21 selects a point (also referred to as the "matching point Pm") for matching the measured data with the matching material. For example, the matching point selection unit 21 selects an arbitrary partial space from the measurement space as the matching point Pm based on user input. In another example, the matching point selection unit 21 automatically selects a partial space of a predetermined size where the feature objects registered in the matching material exist as the matching point Pm. Note that the matching material may include information indicating the points to be selected as the matching point Pm in advance.

[0054] Based on the alignment point Pm selected by the alignment point selection unit 21, the processing unit 22 generates the processed data "Dp" by processing the target measurement data. Here, when the deviation amount calculation unit 23 described later performs two-dimensional matching (i.e., matching only in the horizontal direction), the processing unit 22 extracts the measurement data corresponding to the alignment point Pm from the measurement data storage unit 46, and performs orthographic transformation on the extracted measurement data to generate an ortho image. Then, the processing unit 22 supplies the ortho image generated for each alignment point Pm to the deviation amount calculation unit 23 as the processed data Dp. On the other hand, when the deviation amount calculation unit 23 described later performs three-dimensional matching (i.e., matching including the height direction), the processing unit 22 extracts the measurement data corresponding to the alignment point Pm from the measurement data storage unit 46, and supplies the extracted measurement data to the deviation amount calculation unit 23 as the processed data Dp. In addition to the extraction process of the measurement data corresponding to the alignment point Pm, the processing unit 22 may perform optional filtering processes such as downsampling of the point cloud by VGF (Voxel Grid Filter) or voxelization processing. In this case, the processed data Dp becomes point cloud data indicating the three-dimensional positions of the respective measurement points within the alignment point Pm. Also, in either the case of two-dimensional matching or three-dimensional matching, the processed data Dp includes the meta information contained in the extracted measurement data.

[0055] The deviation amount calculation unit 23 calculates the deviation amount "dM" of the trajectory indicated by the trajectory information of the measurement data by aligning the processed data Dp supplied from the processing unit 22 with the alignment material extracted from the alignment material storage unit 47. In this case, when the measurement data includes meta information for specifying a hierarchy, the deviation amount calculation unit 23 refers to the meta information and the hierarchy attribute information of the alignment material to select the alignment material (specifically, the structure / arrangement information) corresponding to the measured hierarchy. Note that the deviation amount calculation unit 23 may select the alignment material to be used by receiving a user input for specifying the alignment material to be aligned. Then, the deviation amount calculation unit 23 calculates the deviation amount dM by performing the matching between the processed data Dp and the alignment material based on the feature information which is the meta information. The matching based on the feature information will be described later. Note that the deviation amount calculation unit 23 may further calculate the deviation amount between the divided measurement data by dividing the measurement data based on the alignment point Pm and matching the divided measurement data with each other at the alignment point Pm. By applying this deviation amount to the divided measurement data, the deviation between the measurement data can be suitably reduced. Note that the division of the measurement data is not essential. For example, when the trajectory measured at the alignment point Pm is separated, there is no need to perform the division.

[0056] Based on the deviation amount dM calculated by the deviation amount calculation unit 23, the correction unit 24 generates point cloud data (also referred to as "corrected point cloud data Dc") obtained by correcting the target measurement data. In this case, the correction unit 24 corrects the trajectory information included in the target measurement data based on the deviation amount dM, and generates the corrected point cloud data Dc by correcting the corresponding point cloud data based on the corrected trajectory information. In this case, the corrected point cloud data Dc is point cloud data obtained by correcting the corresponding point cloud data based on the corrected trajectory, and each measurement point constituting the corrected point cloud data Dc is data indicating a three-dimensional position in the material coordinate system. Note that when there is additional information in the alignment material corresponding to the measurement space, the correction unit 24 may add the additional information to the generated corrected point cloud data Dc and store it in the point cloud map storage unit 48. Thereby, the correction unit 24 can suitably add information such as a store in the measurement space to the generated point cloud map.

[0057] (4) Alignment process Next, the alignment process executed by the deviation amount calculation unit 23 will be specifically described.

[0058] FIG. 5(A) is an orthographic image showing measurement points in the measurement space, and FIG. 5(B) is a design drawing registered as an alignment material. In FIG. 5(B), the selected alignment points "Pm1" to "Pm5" are clearly shown by a broken line frame. Further, in FIG. 5(A), the trajectory indicated by the trajectory information is clearly shown by a white line, and the portions corresponding to the alignment points Pm1 to Pm5 are clearly shown by a white broken line frame. In the examples of FIGS. 5(A) and 5(B), it is assumed that the feature information regarding the feature 50 existing in the alignment point Pm3 is included in the alignment material as meta information.

[0059] As shown in FIG. 5(A), the processing unit 22 converts the measurement data to be processed into an orthoimage by performing an orthographic transformation so that it becomes an image taken from the same viewpoint as the bonding material. Further, the deviation amount calculation unit 23 identifies the layer in which the measurement was performed based on the atmospheric pressure information or indoor positioning information included as meta information in the measurement data and the hierarchical attribute information of the bonding material stored in the bonding material storage unit 47. Then, the deviation amount calculation unit 23 recognizes the design drawing shown in FIG. 5(B) by extracting the structure / arrangement information of the identified layer from the bonding material storage unit 47.

[0060] Also, as shown in FIGS. 5(A) and 5(B), the bonding point selection unit 21 sets rectangular regions of a predetermined size within the measurement space including features such as wall surfaces and stationary objects as bonding points Pm1 to Pm5 in both the orthoimage and the design drawing which is the bonding material, respectively. For example, when the bonding point selection unit 21 selects the bonding point Pm based on user input, the orthoimage shown in FIG. 5(A) and the design drawing shown in FIG. 5(B) are respectively displayed on a display device such as a display, and an input for designating the rectangular region that becomes the bonding point Pm may be received on the displayed orthoimage and design drawing.

[0061] Then, the deviation amount calculation unit 23 performs template matching between the orthoimage shown in FIG. 5(A) and the design drawing shown in FIG. 5(B) at the bonding points Pm1 to Pm5. Then, the deviation amount calculation unit 23 calculates, as the deviation amount dM, the amount of movement necessary to translate and rotate the orthoimage within the bonding points Pm1 to Pm5 shown in FIG. 5(A) by template matching. In this case, the deviation amount calculation unit 23 calculates, as the deviation amount dM, the deviation between the position in the material coordinate system at each pixel of the orthoimage identified based on the measurement data for generating the orthoimage and the position in the material coordinate system at each pixel of the orthoimage identified by bonding.

[0062] FIG. 6(A) is an enlarged view of the alignment point Pm5 in the ortho-image shown in FIG. 5(A), and FIG. 6(B) is an enlarged view of the alignment point Pm5 in the design drawing shown in FIG. 5(B). For example, in the template matching at the alignment point Pm5, the columns and wall surfaces are displayed in both the ortho-image shown in FIG. 6(A) and the design drawing shown in FIG. 6(B). Therefore, the displacement amount calculation unit 23 calculates, as the displacement amount dM, the amount of movement of each pixel of the ortho-image within the alignment point Pm5, which is necessary for the positions of these feature objects to match between the ortho-image and the design drawing. Then, after calculating the displacement amounts dM corresponding to the necessary translation and rotation amounts of the ortho-image at the alignment points Pm1 to Pm5 respectively, rigid body transformation or non-rigid body transformation of the measurement data is performed based on these displacement amounts dM. When performing rigid body transformation based on the displacement amount dM, the displacement amount calculation unit 23 may calculate, as the displacement amount dM, the average value or other representative value of the displacement amounts corresponding to the necessary translation and rotation amounts of the ortho-image at the alignment points Pm1 to Pm5 respectively. Further, the displacement amount calculation unit 23 may calculate the displacement amount dM by performing alignment of the measurement data and the alignment material after excluding the data in the regions other than the alignment points Pm1 to Pm5 by mask processing.

[0063] Note that in the examples of FIGS. 5 and 6, the deviation amount calculation unit 23 performs alignment in the two-dimensional direction. Instead, alignment in three dimensions including the height direction may be performed. In this case, the deviation amount calculation unit 23 extracts measurement data indicating the position within the alignment point Pm as the processing data Dp. Then, the deviation amount calculation unit 23 generates a three-dimensional model of the measurement space from the alignment material, and performs matching between the generated three-dimensional model and the processing data Dp which is three-dimensional point cloud data indicating the position within the alignment point Pm. This matching method may be realized by various three-dimensional matching techniques. Examples of the three-dimensional matching techniques include matching by NDT (Normal Distributions Transform), matching by ICP (Iterative Closest Point), matching using feature quantities such as normal vectors, and three-dimensional template matching.

[0064] Next, a matching method based on feature information will be described. The deviation amount calculation unit 23 calculates the deviation amount dM based on the feature information which is meta information included in both the processing data Dp and the alignment material.

[0065] As a first example, consider a case where the feature 50 within the alignment point Pm3 has a specific color and the alignment material includes the color information of the feature 50 in advance as meta-information. In this case, among the processing data Dp corresponding to the alignment point Pm3, each measurement point forming the surface of the feature 50 also includes the color information of the feature 50 acquired by the external sensor of the measuring device 1. Then, when performing alignment at the alignment point Pm3, the deviation amount calculation unit 23 executes matching considering the color information. In this case, for example, the deviation amount calculation unit 23 sets the weighting value between the points or voxels to be matched based on the color information. Specifically, the deviation amount calculation unit 23 defines an evaluation function for evaluating the degree of matching such that points or voxels having similar color information are weighted more highly. Note that the deviation amount calculation unit 23 may perform matching based only on the color information instead of using the color information for calculating the weighting value, or may combine the matching result based only on the color information and the matching result using the color information for calculating the weighting value. A method of scan matching using color information is disclosed in, for example, Non-Patent Document 1.

[0066] As a second example, consider the case where information on the reflection intensity of the feature 50 is included in advance as meta-information in the mating material. In this case, among the processing data Dp corresponding to the mating point Pm3, information on the reflection intensity of the feature 50 acquired by the external sensor of the measuring device 1 is included at each measurement point forming the surface of the feature 50. Then, when performing mating at the mating point Pm3, the deviation amount calculation unit 23 executes matching taking this reflection intensity into account. In this case, the deviation amount calculation unit 23 defines an evaluation function for evaluating the degree of matching such that points or voxels having similar reflection intensities are weighted more highly. Note that the deviation amount calculation unit 23 may perform matching based only on the reflection intensity instead of using the reflection intensity for calculating the weighting value, or may combine the matching result based only on the reflection intensity and the matching result using the reflection intensity for calculating the weighting value. Note that a method of scan matching using reflection intensity is disclosed, for example, in Patent Document 2. Even when information on the luminance of the feature 50 is included in advance as meta-information in the mating material, the deviation amount calculation unit 23 may execute the same processing as the scan matching taking the reflection intensity into account.

[0067] In this way, by performing the mating process considering the meta-information, the deviation amount calculation unit 23 can more accurately calculate the deviation amount dM for performing mating to the mating material. When performing three-dimensional matching, instead of using weights based on direct color or reflection intensity, the deviation amount calculation unit 23 may perform matching by combining meta-information and others, such as an application of CC-ICP, and calculate the deviation amount dM based on the matching result.

[0068] Preferably, the deviation amount calculation unit 23 may assign a higher weight to the matching result of the superimposing point Pm where meta information exists than to the matching result of the superimposing point Pm where meta information does not exist. For example, in the examples of FIGS. 5 and 6, consider the case where the deviation amount dM is set by the weighted average of the deviation amounts, which are the matching results at each of the superimposing points Pm1 to Pm5. In this case, the deviation amount calculation unit 23 sets the weight value for the deviation amount of the superimposing point Pm3 where meta information exists to be higher by a predetermined ratio than the weight value for the deviation amounts of the other superimposing points Pm. In another preferred example, the deviation amount calculation unit 23 may calculate the deviation amount dM based only on the matching results between the data where meta information exists. In this case, in the examples of FIGS. 5 and 6, the deviation amount calculation unit 23 determines the deviation amount dM based only on the matching result at the superimposing point Pm3. According to these examples, the deviation amount calculation unit 23 can preferably calculate the deviation amount dM with a higher weight for the matching result at the location where alignment can be more accurately performed using meta information.

[0069] (5) Processing flow FIG. 7 is an example of a flowchart showing the procedure of the point cloud map generation process executed by the controller 45 of the information processing apparatus 4.

[0070] First, the controller 45 acquires the target measurement data (step S10). In this case, the controller 45 may, for example, acquire all the measurement data stored in the measurement data storage unit 46 from the measurement data storage unit 46, or may acquire the measurement data in the measurement space specified by user input or the like from the measurement data storage unit 46. Further, when the controller 45 receives the measurement data Im from the measurement device 1 via the interface 41, the controller 45 may regard the measurement data Im as the target measurement data.

[0071] Next, the alignment point selection unit 21 of the controller 45 selects one or more alignment points Pm (step S11). In this case, the alignment point selection unit 21 may select the alignment point Pm based on user input, or may automatically select, as the alignment point Pm, a partial space within the measurement space in which there are feature objects suitable for matching for calculating the deviation amount dM.

[0072] Next, the processing unit 22 generates processed data Dp obtained by processing the measurement data at the one or more alignment points Pm selected in step S11 (step S12). In this case, when two-dimensional matching is performed in step S13 described later, the processing unit 22 generates an orthographic image generated from the measurement data at the alignment point Pm as the processed data Dp. Further, when three-dimensional matching is performed in step S13 described later, the processing unit 22 extracts, as the processed data Dp, a point group of measurement points indicating positions within the alignment point Pm from among the point group of measurement points indicated by the measurement data. In addition to the extraction process of the measurement data corresponding to the alignment point Pm, the processing unit 22 may perform an arbitrary filtering process such as downsampling of the point group by VGF or voxelization processing.

[0073] Next, the deviation amount calculation unit 23 calculates the deviation amount dM of the measurement data with respect to the alignment material using the meta information (step S13). In this case, the deviation amount calculation unit 23 performs matching between the alignment material and the processed data Dp at the alignment point Pm based on the feature information included as meta information in the alignment material and the processed data Dp generated in step S12, respectively. Thereby, the deviation amount calculation unit 23 can perform highly accurate matching considering the feature information and accurately calculate the deviation amount dM. Further, when the measurement data includes air pressure information or indoor positioning information as meta information, the deviation amount calculation unit 23 refers to the meta information and the hierarchical attribute information included in the alignment material to identify the layer in which the measurement was performed, and extracts the structure / arrangement information of the identified layer from the alignment material storage unit 47.

[0074] Then, based on the deviation amount dM calculated in step S13, the correction unit 24 generates corrected point cloud data Dc (step S14). In this case, the trajectory information of the measurement data is corrected by the deviation amount dM calculated by the deviation amount calculation unit 23, and the point cloud data is corrected based on the corrected trajectory information, thereby generating the corrected point cloud data Dc aligned with the alignment material. After that, the correction unit 24 adds the generated corrected point cloud data Dc to the additional information of the corresponding alignment material and stores it in the point cloud map storage unit 48. In this way, by generating the corrected point cloud data Dc aligned with the alignment material, the information processing device 4 can generate a point cloud map that preferably inherits the additional information included in the alignment material.

[0075] As described above, the information processing device 4 in this embodiment includes an acquisition unit, an alignment unit, and a correction unit. The acquisition unit acquires measurement data, which is point cloud data including meta information for at least some of the measurement points. The alignment unit aligns the measurement data and the alignment material including meta information for at least a part of the measurement space in which the measurement data is measured, based on the meta information. The correction unit generates corrected point cloud data Dc by correcting the measurement data based on the result of the alignment. Thereby, the information processing device 4 can preferably generate a point cloud map accurately aligned with the alignment material.

[0076] In the above-described embodiments, the program can be stored using various types of non-transitory computer readable media and supplied to a controller or the like that is a computer. The non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memories (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).

[0077] The present invention has been described with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes can be made to the configuration and details of the present invention within the scope of the present invention. That is, the present invention naturally includes various modifications and corrections that those skilled in the art could make in accordance with the entire disclosure including the claims and the technical idea. Also, each disclosure of the above-cited patent documents and the like is incorporated herein by reference.

Explanation of Reference Numerals

[0078] 1 Measuring device 2 Sensor group 4 Information processing device 16 Point group data storage unit 17 Trajectory information storage unit 46 Measurement data storage unit 47 Fitting material storage unit 48 Point group map storage unit

Claims

1. An acquisition means for acquiring measurement data including meta information for at least some of the measurement points; Extracting measurement data corresponding to the alignment points selected in the measurement space in which the measurement data is measured from the measurement data, the processed data obtained by processing the extracted measurement data, and an alignment material including meta information for at least a part of the measurement space, and performing alignment by matching based on the meta information; an alignment means; A correction means for correcting the measurement data based on the result of the alignment; An information processing apparatus having the above.

2. Further comprising an alignment point selection means for selecting alignment points in the measurement space, The alignment means performs the alignment with reference to the alignment points. The information processing apparatus according to claim 1.

3. The meta information is feature information that is an index indicating the characteristics of an object, The alignment means performs the alignment based on the feature information. The information processing apparatus according to claim 1 or 2.

4. The measurement data is data measured by a camera or a range sensor, The feature information is information indicating the color or luminance measured by the camera, or information indicating the reflection intensity measured by the range sensor. The information processing apparatus according to claim 3.

5. The meta information includes information for specifying the floor that becomes the measurement space when measurement is performed in the facility, The alignment means performs alignment between the measurement data and the alignment material corresponding to the floor specified by the meta information. The information processing apparatus according to any one of claims 1 to 4.

6. The alignment material further has additional information that is information regarding the measurement space, The correction means generates a point cloud map in which the additional information is added to the corrected measurement data. The information processing apparatus according to any one of claims 1 to 5.

7. The measurement data includes point cloud data based on the position of the external sensor measured by the external sensor and trajectory information indicating the trajectory of the measurement position in the time series of the point cloud data, The correction means corrects the trajectory information of the point cloud data based on the result of the alignment. The information processing apparatus according to any one of claims 1 to 6.

8. The alignment material includes drawing data of the measurement space, The alignment means generates an orthoimage obtained by orthorectifying the extracted measurement data as the processing data, and aligns the orthoimage and the drawing data by matching based on the meta information. The information processing apparatus according to any one of claims 1 to 7.

9. A computer acquires measurement data including meta information for at least some of the measurement points, extracts the measurement data corresponding to the alignment points selected in the measurement space in which the measurement data is measured from the measurement data, and aligns the processed data obtained by processing the extracted measurement data and the alignment material including meta information for at least a part of the measurement space by matching based on the meta information, corrects the measurement data based on the result of the alignment, control method.

10. an acquisition means for acquiring measurement data including meta information for at least some of the measurement points, an alignment means for extracting the measurement data corresponding to the alignment points selected in the measurement space in which the measurement data is measured from the measurement data, and aligning the processed data obtained by processing the extracted measurement data and the alignment material including meta information for at least a part of the measurement space by matching based on the meta information, a correction means for correcting the measurement data based on the result of the alignment A program that causes a computer to function as

11. A storage medium storing the program according to claim 10.

Images