Geospatial information processing device and geospatial information processing method

Abstract

Contributing to the compilation of digital maps at lower cost. [Solution] In the geospatial information processing system 10 according to the present invention, a geospatial information processing device 70 acquires data recorded by a drive recorder from a data management server 50, estimates the position of the camera equipped in the drive recorder based on the recorded data, reconstructs a 3D model of the geospace around the camera in association with the estimated position of the camera, and then aligns the reconstructed 3D reconstructed space together with the estimated position of the camera with the real-world coordinate space, and then performs matching to determine which roads on the OSM the estimated position of the camera matches after alignment. This is particularly useful when editing OSM.

Description

[Technical Field]

[0001] The present invention relates to a geospatial information processing device and a geospatial information processing method, and more particularly to a geospatial information processing device and a geospatial information processing method that are applicable to editing (updating) digital maps. [Background technology]

[0002] In recent years, with the development of autonomous driving technology and advanced navigation systems, there has been an increasing demand for three-dimensional maps, a type of digital map. As a method for creating such three-dimensional maps, there is a method that uses LiDAR (Light Detection and Ranging), as disclosed in Non-Patent Document 1, for example, and in particular a method that uses a dedicated vehicle equipped with LiDAR. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "Reiwa 4 Patent Application Technology Trends Survey Report - LiDAR -", Japan Patent Office, March 2023, pp. 1-7 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the LiDAR method is quite costly, and the larger the area of ​​the digital map, the higher the cost.

[0005] Furthermore, rather than creating a new digital map, existing digital maps may be edited. In such cases, it is desirable to be able to edit digital maps at lower cost, especially in the case of open-platform (user-generated) digital maps such as the well-known Open Street Map (hereinafter referred to as "OSM").

[0006] Therefore, an object of the present invention is to provide a new geospatial information processing device and a new geospatial information processing method that will greatly contribute to editing digital maps at lower cost. [Means for solving the problem]

[0007] To achieve this object, the present invention includes a first invention relating to a geospatial information processing device and a second invention relating to a geospatial information processing method.

[0008] A first aspect of the present invention, relating to a geospatial information processing device, includes an image data input accepting means, a positioning data input accepting means, a 3D reconstruction means, a spatial alignment means, and a matching means. The image data input accepting means accepts input of image data obtained by a camera moving on a road sequentially capturing images of a geographical space including the camera's surroundings in a first period. The positioning data input accepting means accepts input of positioning data obtained by a positioning satellite receiver moving on the road together with the camera sequentially performing positioning in a second period different from the first period. The 3D reconstruction means estimates the position of the camera for each first period based on the image data accepted by the image data input accepting means, and reconstructs (restores) a 3D model of the geographical space captured by the camera in association with the estimated position of the camera. The spatial alignment means aligns the 3D reconstructed space reconstructed by the 3D reconstruction means with the real world coordinate space, together with the estimated position of the camera, using the positioning position for each second period represented by the positioning data accepted by the positioning data input accepting means as an index. The matching means then matches the estimated position of the camera after the matching by the spatial matching means with road information included in the predetermined map information, and the matching result is applied to editing the digital map.

[0009] For example, if the estimated position of the camera after alignment by the spatial alignment means matches the road position represented by any of the road information, the matching means determines that the estimated position of the camera matches the road on the map related to the road information. On the other hand, if the estimated position of the camera after the alignment by the spatial alignment means does not match any road position, the matching means identifies a road on the map that matches the estimated position of the camera based on the relationship between the distance from the estimated position of the camera and the movement direction of the estimated position of the camera. Here, the matching means determines the relationship between the movement direction of the estimated position of the camera based on the angle formed between the movement direction of the estimated position of the camera and the direction of the road on the map, taking into account whether the road on the map is one-way or two-way.

[0010] Furthermore, for example, the matching means may estimate the position of the camera after the spatial matching means performs the matching. The roads on the map within a predetermined search radius centered on the road may be used to determine the relationship between the distance from the estimated camera position and the movement direction of the estimated camera position.

[0011] A second aspect of the present invention, relating to a geospatial information processing method, is a geospatial information processing method implemented by a computer, in which a processor of the computer executes an image data input receiving step, a positioning data input receiving step, a 3D reconstruction step, a spatial alignment step, and a matching step. In the image data input receiving step, the processor receives input of image data obtained by a camera moving on a road sequentially capturing images of a geospatial space including the camera's surroundings in a first period. In the positioning data input receiving step, the processor receives input of positioning data obtained by a positioning satellite receiver moving on the road together with the camera sequentially performing positioning in a second period different from the first period. In the subsequent 3D reconstruction step, the processor estimates the position of the camera for each first period based on the image data received in the image data input receiving step, and reconstructs a 3D model of the geospatial space captured by the camera in association with the estimated position of the camera. Furthermore, in the spatial alignment step, the processor aligns the 3D reconstructed space reconstructed in the 3D reconstruction step with the real world coordinate space, using the positioning position for each second period represented by the positioning data accepted in the positioning data input acceptance step as an index, together with the estimated position of the camera. Then, in the matching step, the processor matches the estimated position of the camera after alignment in the spatial alignment step with road information included in predetermined map information. This matching result is applied to editing the digital map. In the matching step, if the estimated position of the camera after the spatial matching step matches a road position represented by any road information, the processor determines that the estimated position of the camera matches the road on the map related to the road information. On the other hand, if the estimated position of the camera after the spatial matching step does not match any road position, the processor identifies a road on the map that matches the estimated position of the camera based on the relationship between the distance from the estimated position of the camera and the movement direction of the estimated position of the camera. Here, the processor determines the relationship between the movement direction of the estimated position of the camera and the direction of the road on the map, taking into account whether the road on the map is one-way or two-way. [Effects of the Invention]

[0012] The present invention will make a significant contribution to the compilation of digital maps at lower cost, and is particularly beneficial for open-platform digital maps such as OSM. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram showing the overall configuration of a geospatial information processing system according to an embodiment of the present invention. [Figure 2] 3 is a diagram showing an example of an image captured by a drive recorder in the embodiment; FIG. [Figure 3] FIG. 2 is a block diagram showing the electrical configuration of the geospatial information processing device according to the embodiment. [Figure 4] FIG. 2 is a flowchart showing the flow of a series of processes performed by the geospatial information processing device in the embodiment. [Figure 5] 10 is a diagram showing an example of visualization of a 3D point cloud reconstruction space and an estimated position of a camera in the same embodiment. FIG. [Figure 6] FIG. 10 is a diagram showing an example of a matching result by similarity transformation using Procrustes analysis in the same embodiment. [Figure 7] 10 is a diagram showing an example of a correction space used for correction based on a positioning position in the embodiment. FIG. [Figure 8] 10A and 10B are diagrams showing an example of a correction result based on the measured position in the embodiment; [Figure 9] FIG. 2 is a diagram showing the basic configuration of OSM roads in the embodiment. [Figure 10] 3A and 3B are diagrams for explaining the procedure for road matching in the embodiment; [Figure 11] 10A and 10B are diagrams for explaining how to increase the resolution of OSM road shapes in the embodiment. [Figure 12] FIG. 10 is another diagram for explaining how to increase the resolution of the OSM road shape in the same embodiment. [Figure 13] 10 is a diagram for explaining the procedure for additional correction based on OSM roads in the embodiment. FIG. [Figure 14] FIG. 10 is a diagram showing an example of a correction space used for correction based on an OSM road position in the embodiment. [Figure 15] FIG. 10 is a diagram showing an example of an additional correction result based on an OSM road in the embodiment. [Figure 16]10A and 10B are diagrams showing an example of a captured image after lens distortion of the camera has been corrected in the embodiment. [Figure 17] FIG. 2 is a diagram showing an example of a road detection result using the object detection model in the embodiment. [Figure 18] FIG. 10 is a diagram showing an example of a segmentation result obtained by the segmentation model in the same embodiment. [Figure 19] FIG. 10 is a diagram showing an example of a road surface condition determination result using an image language embedding model in the embodiment. [Figure 20] FIG. 5 is a flowchart showing details of S5 in FIG. 4. [Figure 21] FIG. 5 is a flowchart showing details of S7 in FIG. [Figure 22] FIG. 5 is a flowchart showing details of S9 in FIG. 4. [Figure 23] FIG. 5 is a flowchart showing details of S11 in FIG. [Figure 24] FIG. 5 is a flowchart showing the details of the remaining part of S13 in FIG. [Figure 25] FIG. 5 is a flowchart showing details of S15 in FIG. 4. DETAILED DESCRIPTION OF THE INVENTION

[0014] An embodiment of the present invention will be described using a geospatial information processing system 10 shown in FIG. 1 as an example.

[0015] 1, a geospatial information processing system 10 according to this embodiment includes a data management server 30, a map database 50, and a geospatial information processing device 70, which are interconnected for mutual communication via a network 90. ​​The network 90 is, for example, the Internet, but may also be a LAN or a WAN, or may include a LAN or a WAN.

[0016] The data management server 30 has a database (not shown), which stores data recorded by a large number of drive recorders. The data recorded by the drive recorders includes image data (video data) captured by a camera equipped in the drive recorder and positioning data captured by a positioning satellite receiver (GPS receiver) built into the drive recorder. The positioning data includes positioning position data indicating the position and positioning date and time data (timestamp) indicating the date and time of positioning. Note that the captured image data does not include data indicating the time of capture.

[0017] The map database 50 is a database of digital maps, such as an OSM database. OSM is well known, so a detailed description thereof will be omitted.

[0018] The geospatial information processing device 70 then acquires any data from the drive recorder's recorded data stored in the data management server 30 from the data management server 30. As described above, the drive recorder's recorded data includes image data captured by a camera provided in the drive recorder and positioning data captured by a positioning satellite receiver built into the drive recorder. In particular, the captured image data includes multiple frames. When a vehicle (moving object) equipped with a drive recorder is traveling on a road, a frame includes (an image of) the road, as shown in FIG. 2, for example.

[0019] The geospatial information processing device 70 derives three-dimensional coordinate values ​​in the real world coordinate space (WGS84) of appropriate locations on the road included in each frame of the captured image data, more specifically, two-dimensional feature points 132 (see FIG. 18 , described later), and associates position information representing the derived three-dimensional coordinate values ​​with the two-dimensional feature points 132. Additionally, the geospatial information processing device 70 uses the two-dimensional feature points 132 on the road in each frame to estimate the actual width Wt of the road (in the real world coordinate space), and associates the estimation result with a three-dimensional point 102 (see FIG. 5 , described later) corresponding to the two-dimensional feature points 132 related to the estimation result. Furthermore, the geospatial information processing device 70 determines the surface condition of the road in each frame, particularly the type of road surface (e.g., asphalt, concrete, etc.), and associates the determination result with the three-dimensional point 102 corresponding to the two-dimensional feature points 132 on the road related to the determination result.

[0020] Such a geospatial information processing device 70 is configured, for example, by a personal computer (hereinafter referred to as "PC") as shown in Fig. 3. That is, the PC as the geospatial information processing device 70 has a control unit 72, an input / output interface (I / F) unit 74, an input device 76, a display device 78, an auxiliary storage unit 80, and a communication unit 82. Note that the PC as the geospatial information processing device 70 also has various other elements, but elements not directly related to the present invention are omitted from Fig. 4.

[0021] The control unit 72 is a control means responsible for overall control of the PC serving as the geospatial information processing device 70, and this control unit 72 has a CPU 72a as a control execution means. The control unit 72 also has a main memory unit 72b that is directly accessible by the CPU 72a. The main memory unit 72b includes, for example, a ROM and a RAM. Firmware including the BIOS is stored in the ROM. Meanwhile, the RAM provides a working area and a buffer area for the CPU 72a when it executes processes based on various programs included in the firmware, the operating system, and various pieces of software, such as the geospatial information processing software 80a described below.

[0022] The input / output interface unit 74 is an element that acts as a bridge between the control unit 72, particularly the CPU 72a, and each element such as the input device 76, and includes, for example, a chipset. Therefore, the input / output interface unit 74 is connected to the control unit 72 as well as each element such as the input device 76.

[0023] The input device 76 is an operation receiving means for receiving operations by an operator (not shown), and includes, for example, a keyboard and a mouse.

[0024] The display device 78 is a display means for displaying various information. This display means is, for example, a flat panel display such as a liquid crystal display or an organic EL display, but is not limited to these.

[0025] The auxiliary storage unit 80 is an auxiliary storage means and includes, for example, a hard disk drive. The auxiliary storage unit 80 stores an operating system and various software, including application software called geospatial information processing software 80a that causes the PC to function as the geospatial information processing device 70. The auxiliary storage unit 80 may include a rewritable non-volatile memory such as a flash memory instead of or in addition to the hard disk drive.

[0026] The communication unit 82 is a communication connection means that is responsible for connection with the above-mentioned network 90. ​​The connection between the communication unit 82 and the network 90 may be wired or wireless.

[0027] As described above, the geospatial information processing device 70 derives three-dimensional coordinate values ​​in the real-world coordinate space of two-dimensional feature points 132 on a road included in each frame of the captured image data, and associates position information representing the derived three-dimensional coordinate values ​​with the two-dimensional feature points 132. Additionally, the geospatial information processing device 70 estimates the actual width Wt of the road using the two-dimensional feature points 132 on the road in each frame, and associates the estimation result with a three-dimensional point 102 (described below) corresponding to the two-dimensional feature points 132 related to the estimation result. Furthermore, the geospatial information processing device 70 determines the surface condition of the road in each frame, and associates the determination result with a three-dimensional point 102 corresponding to the two-dimensional feature points 132 on the road related to the determination result. A series of processing steps performed by the geospatial information processing device 70 is shown in FIG. 4. In FIG. 4, reference numerals beginning with "S" identify each processing step, and in the following description, each processing step will be represented by this reference numeral.

[0028] As shown in FIG. 4, the geospatial information processing device 70 (more precisely, the CPU 72a) first acquires, in S1, data recorded by an arbitrary drive recorder from the data management server 30. As described above, the data recorded by the drive recorder includes image data captured by a camera provided in the drive recorder and positioning data captured by a positioning satellite receiver built into the drive recorder. Here, the frame rate of the captured image data is, for example, 7 fps. In other words, the frame update period of the captured image data is 1 / 7 second. On the other hand, the positioning data is acquired at a predetermined cycle, for example, every 3 seconds. In other words, the frame update period of the captured image data (first cycle) is shorter than the update period of the positioning data (second cycle), i.e., different from the update period of the positioning data. Furthermore, these two update periods are asynchronous with each other.

[0029] After S1 is executed, in the subsequent S3, the geospatial information processing device 70 reconstructs a 3D model of the geospatial space represented by the captured image data, as shown in FIG. 6, using the captured image data included in the recording data acquired in S1. This reconstruction is performed, for example, using the well-known SfM (Structure from Motion) method, and more specifically, using the well-known COLMAP method. For this reason, the geospatial information processing software 40ab includes COLMAP. Furthermore, during this reconstruction, distortion due to the camera lens is corrected before the reconstruction is performed. Furthermore, during the reconstruction, the camera position when each frame is acquired is estimated, and the estimated camera position Pc[j] (j; frame number) is associated with the 3D point cloud reconstruction space, which is the space of the reconstructed 3D point cloud 100. Note that, in addition to estimating the camera position, the camera's pose (pan, tilt, roll) and parameters are also estimated, but the camera's pose and parameters are not particularly used in this embodiment.

[0030] FIG. 5 is an example of visualization of a 3D point cloud reconstruction space and a camera estimated position Pc[j] for the same geographic space as that shown in FIG. 2. As shown in FIG. 5, a 3D point cloud 100 is composed of a large number of 3D points 102. In addition, in FIG. 5, the camera estimated position Pc[j] is shown superimposed on the 3D point cloud 100, thereby expressing that the camera estimated position Pc[j] is associated with the 3D point cloud reconstruction space. Note that FIG. 5 is merely an example of a visualization of the 3D point cloud reconstruction space and the camera estimated position Pc[j]; in reality, the 3D coordinate values ​​of each 3D point 102 and the 3D coordinate values ​​of the camera estimated position Pc[j] are obtained as processing results by SfM.

[0031] Referring again to FIG. 4, after execution of S3, the geospatial information processing device 70 aligns the 3D point cloud reconstruction space reconstructed in S3 with the real world coordinate space in the following S5. At the same time, the camera estimated position Pc[j] is also aligned with the real world coordinate space. This alignment is performed, for example, by a similarity transformation using Procrustes analysis. The procedure will be described with reference to FIG. 6.

[0032] FIG. 6 shows an example of the matching result obtained by similarity transformation using Procrustes analysis. Specifically, the positioning positions Pg[k] (k: positioning order) represented by the positioning position data are plotted on the OSM with blue circles, and the camera estimated positions Pc[j] after matching are plotted on the OSM with red circles. The geographic space shown in FIG. 6 differs from the geographic space shown in FIG. 2 (and FIG. 5). Furthermore, FIG. 6 plots the positioning positions Pg[k] for one minute, i.e., a total of 21 positioning positions Pg[k]. Additionally, FIG. 6 plots the camera estimated positions Pc[j] for one minute (strictly speaking, just under one minute) based on the captured image data obtained in parallel with the positioning positions Pg[k], i.e., a total of 420 camera estimated positions Pc[j]. 6, the positioning position Pg[k] at the bottom right is the first positioning position Pg[1], and the positioning position Pg[k] at the top left is the last positioning position Pg[K] (=Pg

[21] ). That is, the trajectory of the positioning position Pg[k] shows that the vehicle equipped with the drive recorder moved from the first positioning position Pg[1] to the last positioning position Pg[K]. The camera estimated position Pc[j] at the bottom right in FIG. 6 is the first camera estimated position Pc[1], and the camera estimated position Pc[j] at the top left in FIG. 6 is the last camera estimated position Pc[J] (=Pc

[0420] ).

[0033] This similarity transformation using Procrustes analysis is performed using each positioning position Pg[k] as an index, and therefore, a corresponding camera estimated position Pcg[k] corresponding to each positioning position Pg[k] is identified from all camera estimated positions Pc[j].

[0034] Specifically, the first positioning position Pg[1] (first positioning position) and the first camera estimated position Pc[1] (first estimated position) correspond to each other, and the last positioning position Pg[K] (second positioning position) and the last camera estimated position Pc[J] (second estimated position) correspond to each other. That is, the first camera estimated position Pc[1] is identified as the corresponding camera estimated position Pcg[1] corresponding to the first positioning position Pg[1], and the last camera estimated position Pc[J] is identified as the corresponding camera estimated position Pcg[K] (=Pcg

[21] ) corresponding to the last positioning position Pg[K].

[0035] Then, the corresponding camera estimated position Pcg[k] (third estimated position) corresponding to each of the positioning positions Pg[k] (third positioning positions) other than the first positioning position Pg[1] and the last positioning position Pg[k] is identified based on the idea that the degree of movement progress of the positioning position Pg[k] and the degree of movement progress of the camera estimated position Pc[j] are the same. More specifically, the corresponding camera estimated position Pcg[k] corresponding to the arbitrary positioning position Pg[k] is identified based on the idea that the spatial relationship of the arbitrary positioning position Pg[k] to the first positioning position Pg[1] and the last positioning position Pg[K] is the same as the spatial relationship of the corresponding camera estimated position Pcg[k] corresponding to the arbitrary positioning position Pg[k] to the first camera estimated position Pc[1] and the last camera estimated position Pc[J].

[0036] To this end, the path length Lcg[k] from the initial camera estimated position Pc[1] to the corresponding camera estimated position Pcg[k] corresponding to any positioning position Pg[k] is calculated based on the following equation 1. Here, Lg[k] is the path length from the initial positioning position Pg[1] to any positioning position Pg[k]. Lg_ALL is the total path length from the initial positioning position Pg[1] to the final positioning position Pg[K]. Lc_ALL is the total path length from the initial camera estimated position Pc[1] to the final camera estimated position Pc[J].

[0037] 《Formula 1》 Lcg[k]=(Lg[k] / Lg_ALL)·Lc_ALL

[0038] Then, the camera estimated position Pc[j] corresponding to the position Pt from the initial camera estimated position Pc[1] via the path length Lcg[k] based on Equation 1 is identified as the corresponding camera estimated position Pcg[k] corresponding to the arbitrary positioning position Pg[k]. Note that if there is no camera estimated position Pc[j] corresponding to the position Pt, the camera estimated position Pc[j] closest to the position Pt is identified as the corresponding camera estimated position Pcg[k] corresponding to the arbitrary positioning position Pg[k].

[0039] Then, a similarity transformation is performed using Procrustes analysis so that each corresponding camera estimated position Pcg[k] coincides with the positioning position Pg[k] corresponding to the corresponding camera estimated position Pcg[k]. As a result, the 3D point cloud reconstruction space, together with the camera estimated position Pc[j], is translated, rotated, and scaled to be aligned with the real world coordinate space.

[0040] Note that, when matching by similarity transformation using Procrustes analysis, preprocessing may be performed to correct variations in the positioning position Pg[k] and the camera estimated position Pc[j]. This preprocessing can be performed using, for example, a known Kalman filter. Performing such preprocessing improves the matching accuracy.

[0041] Here, each corresponding camera estimated position Pcg[k] after matching by similarity transformation using Procrustes analysis, that is, each corresponding camera estimated position Pcg[k] in the real world coordinate space, is defined by latitude and longitude, just like the positioning position Pg[k] corresponding to each corresponding camera estimated position Pcg[k]. Note that in the real world coordinate space, the vertical (perpendicular) axis is the Y axis, and the two horizontal axes are the X axis and the Z axis. In other words, coordinate values ​​on the XZ plane of the real world coordinate space are defined by latitude and longitude.

[0042] On the other hand, coordinate values ​​in the Y-axis direction of the real-world coordinate space are defined by, for example, altitude, but the Y-values ​​that represent the height positions of the respective 3D points 102 that make up the aligned 3D point cloud 100, particularly the distance between two 3D points 102 and 102 in the Y-axis direction, cannot be calculated quantitatively (in meters) without some kind of standard. This is because the positioning data from a positioning satellite receiver does not include information about the height direction.

[0043] Therefore, the geospatial information processing device 70 derives a scale factor Sf as a criterion based on the mutual ratio between the distance between each corresponding camera estimated position Pcg[k] after alignment and the geodesic distance between each corresponding camera estimated position Pcg[k].

[0044] Specifically, the geospatial information processing device 70 calculates the shortest distance ΔLcg[k] between two consecutive corresponding camera estimated positions Pcg[k] and Pcg[k+1] after matching. The geospatial information processing device 70 also calculates the geodesic distance ΔLa[k] between the two corresponding camera estimated positions Pcg[k] and Pcg[k+1] in the real world coordinate space. The geospatial information processing device 70 then calculates the ratio R[k] (=ΔLcg[k] / ΔLa[k]) of the shortest distance ΔLcg[k] to the geodesic distance ΔLa[k]. The geospatial information processing device 70 then determines the average of the ratios R[k] over all sections as the scale factor Sf, i.e., derives the scale factor Sf based on the following equation 2:

[0045] 《Formula 2》 Sf=(ΣR[k]) / (K-1) where k=1~K-1

[0046] In the Y-axis direction of the real world coordinate space, the mutual distance between the two three-dimensional points 102 and 102 is multiplied by this scale factor Sf, and the mutual distance can be quantitatively determined.

[0047] As mentioned above, Fig. 6 shows an example of the matching result by similarity transformation using Procrustes analysis, in which the positioning position Pg[k] is marked on the OSM with a blue circle and the matching estimated camera position Pc[j] is marked on the OSM with a red circle. In Fig. 6, the corresponding estimated camera position Pcg[k] corresponding to each positioning position Pg[k] is marked with a green circle.

[0048] Here, ideally, the corresponding camera estimated position Pcg[k] (green circle in FIG. 6) and the positioning position Pg[k] (blue circle in FIG. 6) after matching should match each other. However, in the example shown in FIG. 6, the corresponding camera estimated position Pcg[k] and the positioning position Pg[k] after matching do not match locally, that is, there is a slight deviation between them. This deviation is due to the camera path, scene characteristics, etc. Furthermore, this deviation is considered to occur throughout the entire 3D point cloud reconstruction space after matching.

[0049] Therefore, referring again to FIG. 4, in S7 following S5, the geospatial information processing device 70 corrects the entire 3D point cloud reconstruction space with the positioned position Pg[k] as the reference.

[0050] Specifically, the geospatial information processing device 70 calculates an error vector (correction offset) Vg[k] for each of the corresponding camera estimated positions Pcg[k] after matching with respect to the positioning position Pg[k]. This error vector Vg[k] includes an error vector Vg_LAT[k] for latitude and an error vector Vg_LON[k] for longitude.

[0051] The geospatial information processing device 70 then generates a corrected space (first correction space) as shown in FIG. 7 by radial basis function (RBF) interpolation, using the corresponding camera estimated position Pcg[k] after matching as a control point and the error vector Vg[k] as an interpolated value. In FIG. 7, the red circle represents the corresponding camera estimated position Pcg[k] after matching, or in other words, the corresponding camera estimated position Pcg[k] before the correction by S7. In FIG. 7, the green circle represents the corresponding camera estimated position Pcg′[k] after the correction by S7, or in other words, the corresponding camera estimated position Pcg′[k] that is the target of the correction by S7. The horizontal axis of the correction space shown in FIG. 7 represents longitude, the vertical axis of the correction space represents longitude, and the color of the correction space represents the degree (strength) of correction.

[0052] 7 to the aligned 3D point cloud reconstructed space, thereby correcting the entire 3D point cloud reconstructed space. As a result, each pre-correction corresponding camera estimated position Pcg[k] is corrected to the target corresponding camera estimated position Pcg'[k], and accordingly, the entire 3D point cloud reconstructed space is corrected.

[0053] Fig. 8 shows an example of the correction result by S7, that is, an example of the correction result based on the positioning position Pg[k]. Note that Fig. 8 shows an extracted part of the road on the OSM in Fig. 6 (hereinafter referred to as "OSM road"), the camera estimated position Pc[j] (red circle) and the positioning position Pg[k] (blue circle) written on the OSM, and the camera estimated position Pc'[j] after correction based on the positioning position Pg[k] (i.e., based on the correction space shown in Fig. 7) is shown with an orange circle.

[0054] As shown in Fig. 8, by performing correction based on the positioning position Pg[k], the corrected camera estimated position Pc'[j] follows the trajectory of the positioning position Pg[k]. Also, although it is not clear from Fig. 8, each corresponding camera estimated position Pcg'[k] coincides with the positioning position Pg[k]. At the same time, the entire 3D point cloud reconstruction space is corrected.

[0055] However, depending on the circumstances, an error may occur in the positioning position Pg[k]. For example, in FIG. 8, it is recognized that an error has occurred in the positioning position Pg[k] enclosed by the dashed rectangular frame 110 (the positioning position Pg[k] is slightly toward the center of the OSM road). In such a case, it is considered appropriate to further correct the entire 3D point cloud reconstruction space including the corrected camera estimated position Pc'[j], specifically, to perform additional correction based on the OSM road.

[0056] Therefore, referring again to FIG. 4, in S9 following S7, the geospatial information processing device 70 performs additional correction of the entire 3D point cloud reconstructed space based on the OSM road.

[0057] To this end, the geospatial information processing device 70 acquires information about OSM roads required for additional correction, that is, OSM road information, from the map database 50. The OSM road information is acquired from the map database 50 using Overpass_API.

[0058] The OSM road information for the OSM roads required for the additional correction refers to the OSM road information for the OSM roads that include one of the estimated camera positions Pc'[j] that are the subject of the additional correction, or the OSM road information for the OSM roads that are in the vicinity of the estimated camera position Pc'[j], more specifically, within a predetermined search radius Rs centered on the estimated camera position Pc'[j]. The search radius Rc is, for example, 10 m, but is not limited to this.

[0059] The geospatial information processing device 70 then identifies OSM roads that match each estimated camera position Pc'[j] from among the OSM roads represented by the OSM road information acquired from the map database 50, and more specifically, identifies the OSM_ID of the OSM road, performing so-called road matching. For example, if any estimated camera position Pc'[j] is included in any OSM road, the estimated camera position Pc'[j] is determined to match the OSM road that includes the estimated camera position Pc'[j]. On the other hand, for an estimated camera position Pc'[j] that is not included in any OSM road, road matching is performed based on the relationship between the shortest distance DS from the estimated camera position Pc'[j] and the movement direction DRc of the estimated camera position Pc'[j].

[0060] 9, an OSM road is defined by two nodes 120 and 120 and a linear edge 122 connecting these two nodes 120 and 120. If any of the camera estimated positions Pc'[j] is on any of the OSM roads (on the edge 122), it is determined that the OSM road matches the camera estimated position Pc'[j].

[0061] On the other hand, for a camera estimated position Pc'[j] that is not located on any OSM road, an OSM road that matches the camera estimated position Pc'[j] is identified based on a distance score Sds corresponding to the shortest distance DS from the camera estimated position Pc'[j] to each OSM road (represented by OSM road information obtained from the map database 50), and a direction score Sdr corresponding to the relationship between the movement direction DRc of the camera estimated position Pc'[j] and the orientation DRs of each OSM road.

[0062] The shortest distance DS here is the distance (orthogonal distance) from the estimated camera position Pc'[j] to the OSM road in a direction perpendicular to the trajectory of the estimated camera position Pc'[j], as shown in Figure 10. The distance score Sds is calculated based on the following equation 3. That is, the distance score Sds is a value obtained by normalizing the shortest distance DS by the above-mentioned search radius Rs.

[0063] 《Formula 3》 Sds=DS / Rs

[0064] The direction score Sdr is calculated based on the following equation 4. θdr in this equation 4 is the angle between the movement direction DRc of the camera estimated position Pc'[j] and the direction DRs of each OSM road, which is the error angle. In other words, the direction score Sdr is a value obtained by normalizing the error angle θdr by a predetermined angle of 90°.

[0065] 《Formula 4》 Sdr=θdr / 90°

[0066] It is possible to normalize the error angle θdr by 180° for the direction score Sdr, but to prevent erroneous road matching, it is more appropriate to normalize the error angle θdr by 90°, in other words, to treat 90° as the maximum error.The direction score Sdr also takes into account whether the OSM road is a one-way road or a two-way road.

[0067] For example, if an OSM road is a one-way road, the OSM road information is tagged with "oneway." For OSM roads represented by OSM road information tagged with "oneway," i.e., for one-way OSM roads, the direction score Sdr is calculated based on the above-mentioned formula 4.

[0068] On the other hand, for OSM roads represented by OSM road information that is not tagged as "oneway", i.e., for OSM roads with two-way traffic, the direction score Sdr is calculated based on Equation 4, and also based on the following Equation 5.

[0069] 《Formula 5》 Sdr=(180°-θdr) / 90°

[0070] Then, the smaller value of the direction score Sdr based on Equation 4 and the direction score Sdr based on Equation 5 is used for road matching.

[0071] After calculating the distance score Sds and direction score Sdr in this manner, the geospatial information processing device 70 identifies the OSM road with the smallest total score Sa, which is the sum of the distance score Sds and direction score Sdr, as the one that matches the camera estimated position Pc'[j].

[0072] To give a specific example, suppose the movement direction of a certain camera's estimated position Pc'[j] is north, and this certain camera's estimated position Pc'[j] is 9 m east of the actual (to-be-matched) OSM road A. Here, OSM road A is a one-way road heading north. Suppose there is another OSM road B, which is a one-way road heading southeast and is 1 m west of the certain camera's estimated position. In this case, the distance score Sds for OSM road A is 0.9 (= 9 m / 10 m), the direction score Sdr is 0.0 (= 0° / 90°), and the total score Sa is 0.9 (= 0.9 + 0.0). On the other hand, the distance score Sds for OSM road B is 0.1 (=1 m / 10 m), the direction score Sdr is 1.5 (=135° / 90°), and the total score Sa is 1.6 (=0.1 + 1.5). Therefore, a certain camera estimated position Pc'[j] is identified as matching with OSM road A, which has a smaller total score Sa.

[0073] After identifying the OSM roads that match each of the camera estimated positions Pc'[j] in this way, the geospatial information processing device 70 performs processing to increase the resolution (smooth) of the shape of the OSM roads.

[0074] That is, as explained with reference to Fig. 9, an OSM road is defined by two nodes 120 and 120 and a straight edge 122 connecting these two nodes 120 and 120. Therefore, as shown in Fig. 11, the resolution of the OSM road is low, especially in curved sections, and therefore the shape of the OSM road (road spline) tends to deviate from the shape of the actual road.

[0075] Therefore, the geospatial information processing device 70 performs processing to increase the resolution of the OSM road shapes. This processing is performed using the well-known cubic Hermite spline interpolation. As a result, the OSM road shapes are increased in resolution as shown in FIG. 12.

[0076] Then, the geospatial information processing device 70 identifies a position Ps[j] on the high-resolution OSM road that corresponds to each estimated camera position Pc'[j], as shown in Fig. 13. Note that the position closest to each estimated camera position Pc'[j] on the high-resolution OSM road is identified as the corresponding position Ps[j].

[0077] The geospatial information processing device 70 then calculates an error vector (correction offset) Vs[j] for each camera's estimated position Pc'[j] relative to its corresponding position Ps[j]. The error vector Vs[j] includes an error vector Vs_LAT[j] for latitude and an error vector Vs_LON[j] for longitude.

[0078] Furthermore, the geospatial information processing device 70 generates a correction space (second correction space) as shown in FIG. 14 by radial basis function interpolation, using each camera estimated position Pc'[j] as a control point and the error vector Vs[j] as an interpolated value. In FIG. 14, the red circle represents the camera estimated position Pc'[j] before the additional correction by S9, in other words, the camera estimated position Pc'[j] after the correction by S7. In addition, the green circle in FIG. 14 represents the camera estimated position Pc''[j] after the additional correction by S9, in other words, the camera estimated position Pc''[j] that is the target of the additional correction by S9. In addition, the horizontal axis of the correction space shown in FIG. 14 represents longitude, the vertical axis of the correction space represents longitude, and the color of the correction space represents the degree of additional correction.

[0079] The geospatial information processing device 70 applies the corrected space shown in FIG. 14 to the 3D point cloud reconstructed space after correction in S7, thereby additionally correcting the entire 3D point cloud reconstructed space. As a result, each camera estimated position Pc'[j] before the additional correction is corrected to the target camera estimated position Pc''[j], and accordingly, the entire 3D point cloud reconstructed space is corrected.

[0080] FIG. 15 shows an example of the result of additional correction by S9, that is, an example of the result of additional correction based on OSM roads. In FIG. 15, as in FIGS. 6 and 8, the positioning position Pg[k] is indicated by a blue circle, the corrected estimated camera position Pc'[j] is indicated by an orange circle, and the additionally corrected estimated camera position Pc"[j] is indicated by a pink circle.

[0081] As shown in Figure 15, by performing additional correction based on the OSM road, the camera estimated position Pc''[j] after this additional correction accurately follows the OSM road. In particular, the camera estimated position Pc''[j] within the area surrounded by the dashed rectangular frame 110 also accurately follows the OSM road. And, although it is not clear from Figure 15, the entire 3D point cloud reconstruction space is additionally corrected.

[0082] Referring again to FIG. 4, in S11 following S9, the geospatial information processing device 70 performs processing to associate two-dimensional feature points 132 on the road included in each frame of the captured image data with position information representing the three-dimensional coordinate values ​​of the two-dimensional feature points 132 in the real world coordinate space.

[0083] In the process of S11, the captured image data after the lens distortion of the camera has been corrected by SfM is used. For example, Fig. 16 shows an example of captured image data, more precisely, a certain frame, after the lens distortion correction by SfM has been applied to the captured image data of the same geographical space as shown in Fig. 2 (and Fig. 5).

[0084] The geospatial information processing device 70 performs road detection using an object detection model for frames after SfM lens distortion has been applied, in other words, for each frame. The object detection model used here is, for example, the well-known GroundingDINO. Therefore, GroundingDINO is set up in the geospatial information processing software 40ab. The GroundingDINO prompt for road detection is, for example, "building." The box threshold, which is a parameter, is set to, for example, 0.3, and the text threshold is set to, for example, 0.25.

[0085] As a result of this road detection using GroundingDINO, a bounding box 125 is attached to the detected road, as shown in Fig. 17, for example. The geographic space shown in Fig. 17 is the same as the geographic space shown in Fig. 2 (and Figs. 5 and 16). In addition, in the frame shown in Fig. 17, a two-dimensional coordinate system is set with the upper left corner as the origin, the horizontal axis as the X axis, and the vertical axis as the Y axis. This is also true for frames shown in other drawings such as Fig. 16.

[0086] After road detection using GroundingDINO, the geospatial information processing device 70 performs segmentation for each frame to divide the road area using a segmentation model. The segmentation model used here is, for example, the well-known SAM2. Therefore, SAM2 is set up in the geospatial information processing software 40ab. Furthermore, a bounding box 125 representing the road detection result using GroundingDINO is directly input as a prompt to SAM2. As a result, the road area within the bounding box 125 is separated from other areas, and a mask 130 is applied to the road area, as shown in FIG. 18.

[0087] After the segmentation using SAM2, the geospatial information processing device 70 extracts two-dimensional feature points 132 located within the road area (mask 130) segmented by the segmentation. That is, in the reconstruction of the three-dimensional point cloud 100 using SfM described above, two-dimensional feature points are detected from each frame, and the three-dimensional point cloud 100 is reconstructed based on the two-dimensional coordinate values ​​of the two-dimensional feature points on the frame, the estimated camera position Pc[j], and the like. The two-dimensional feature points used to reconstruct the three-dimensional point cloud 100 are then associated with the three-dimensional points 102 corresponding to the two-dimensional feature points. The geospatial information processing device 70 extracts two-dimensional feature points 132 located within the road area segmented by SAM2 from the two-dimensional feature points of each frame. Note that FIG. 18 is also an example of a diagram showing two-dimensional feature points 132 located within a road area.

[0088] Furthermore, the geospatial information processing device 70 identifies three-dimensional points 102 that correspond to the extracted two-dimensional feature points 132, i.e., that correspond to the two-dimensional feature points 132 that are located within the road area. The geospatial information processing device 70 then excludes, from subsequent processing, outliers from the identified three-dimensional points 102 whose z-scores in the Y-axis direction of the real-world coordinate space exceed a predetermined value. This method is a standard filtering method (z-score filtering method) that uses SciPy, a well-known Python (registered trademark) library. Note that the predetermined value here is, for example, ±2, that is, a value corresponding to a confidence range of approximately 97%, but is not limited to this.

[0089] Then, the geospatial information processing device 70 associates the position information representing the three-dimensional coordinate values ​​of the three-dimensional points 102 after excluding the outliers with the two-dimensional feature points corresponding to the three-dimensional points 102. With this, the geospatial information processing device 70 ends S11 in FIG. 4.

[0090] In S13 following S11, the geospatial information processing device 70 estimates the actual width Wt of the road using two-dimensional feature points 132 within the road area in each frame, and performs processing to associate the estimation result with the three-dimensional point 102 corresponding to the two-dimensional feature point 132 related to the estimation result.

[0091] In the process of S13, the geospatial information processing device 70 first performs lane boundary detection for each frame using a lane boundary detection model. The lane boundary detection model used here is, for example, the well-known CLRerNET. Therefore, CLRerNET is set up in the geospatial information processing software 40ab.

[0092] 18, which is also used here, the geospatial information processing device 70 then groups the two-dimensional feature points 132a located near lane boundaries 134 detected by CLRerNET, by lane boundary 134. Whether each two-dimensional feature point 132 is located near a lane boundary 134 is determined based on the mutual distance between the two-dimensional feature point 132 and the lane boundary 134 on the frame.

[0093] Furthermore, the geospatial information processing device 70 pairs two-dimensional feature points 132a between each group that are on the same horizontal line (same scan line) 136, in other words, that have the same Y values ​​in the frame. Note that it is desirable to provide an appropriate margin for determining whether or not the two feature points are on the same horizontal line, that is, whether or not the Y values ​​in the frame are the same.

[0094] Then, the geospatial information processing device 70 calculates the distance between the three-dimensional points 102 corresponding to the paired two-dimensional feature points 132a, and estimates this calculated distance as the lane width Wt.

[0095] The geospatial information processing device 70 then associates the estimated lane width Wt with the 3D point 102 related to the estimation of the lane width Wt. With this, the geospatial information processing device 70 ends S13 in Fig. 4. Note that in S13, the average value of the lane width Wt estimated over a predetermined number of frames (for example, 5 to 10) may be used as the final estimation result in S13, or the final lane width Wt may be estimated by a so-called sliding window method.

[0096] In S15 following S13, the geospatial information processing device 70 determines the surface condition of the road in each frame and performs processing to associate the determination result with the 3D point 102 corresponding to the 2D feature point 132 on the road related to the determination result.

[0097] In the process of S15, the geospatial information processing device 70 first performs road detection for each frame using an object detection model, for example, GroundingDINO, as in the process of S11 described above. As a result, bounding boxes 140 are attached to the detected roads, as shown in FIG.

[0098] After the road detection by GroundingDINO is performed, the geospatial information processing device 70 determines the road surface condition using an image language embedding model for the road on the frame detected by GroundingDINO, that is, for the part with the bounding box 140. Note that as the image language embedding model, for example, the well-known CLIP is used. For example, the following prompt is input to this CLIP:

[0099] asphalt: “a paved road with an asphalt surface“ concrete: “a paved road with a concrete surface“ gravel: "an unpaved surface covered with gravel“ dirt: “an unpaved road with a dirt or mud surface“ grass: “an unpaved offroad surface covered with grass or foliage“ other: "an image with a foreground object or structure, or background“

[0100] As a result, the condition of the road surface in each frame, particularly the type of road surface, is next determined, and determination result display information 142 that visualizes the determination result is arranged in that frame, as shown in Fig. 19. Note that in Fig. 19, determination result display information 142 is arranged within bounding box 140, but the position at which this determination result display information 142 is arranged is not limited to this. In addition, determination accuracy display information 144 that visualizes the accuracy of the determination is arranged in a position near the upper left corner of the same frame, for example.

[0101] Then, the geospatial information processing device 70 associates the determination result information indicating the determination result of the road surface condition (road surface type) by CLIP with the 3D point 102 corresponding to the 2D feature point 132 on the road related to the determination result. With this, the geospatial information processing device 70 ends S15 in FIG. 4.

[0102] FIG. 20 is a flow diagram showing details of S5 in FIG. 4. As shown in FIG. 20, the geospatial information processing device 70 first identifies, in S101, corresponding camera estimated positions Pcg[k] corresponding to each positioning position Pg[k] among the camera estimated positions Pc[j]. Then, in S103, the geospatial information processing device 70 performs similarity transformation using Procrustes analysis. This aligns the 3D point cloud reconstruction space with the real world coordinate space. Furthermore, in S105, the geospatial information processing device 70 derives a scale factor Sf. With this, the geospatial information processing device 70 ends S5 in FIG. 4.

[0103] FIG. 21 is a flow diagram showing details of S7 in FIG. 4. As shown in FIG. 21, the geospatial information processing device 70 first calculates, in S201, an error vector Vg[k] of the corresponding camera estimated position Pcg[k] after alignment with respect to the positioned position Pg[k] by similarity transformation using Procrustes analysis. Then, in subsequent S203, the geospatial information processing device 70 generates a corrected space as shown in FIG. 7 by radial basis function interpolation, using the corresponding camera estimated position Pcg[k] after alignment as a control point and the error vector Vg[k] as an interpolated value. Furthermore, in subsequent S205, the geospatial information processing device 70 applies the corrected space generated in S203 to the 3D point cloud reconstructed space after alignment, thereby correcting the entire 3D point cloud reconstructed space. With this, the geospatial information processing device 70 ends S7 in FIG. 4.

[0104] FIG. 22 is a flow diagram showing details of S9 in FIG. 4. As shown in FIG. 22, the geospatial information processing device 70 first acquires OSM road information about OSM roads required for additional correction from the map database 50 in S301. Then, in S303, the geospatial information processing device 70 identifies OSM roads that match each estimated camera position Pc'[j], that is, identifies the OSM_ID of the OSM road. Furthermore, in S305, the geospatial information processing device 70 increases the resolution of the shape of the OSM road identified in S303 using known cubic Hermite spline interpolation. Then, in S307, the geospatial information processing device 70 identifies a position Ps on the OSM road that has been increased in resolution in S305, corresponding to each estimated camera position Pc'[j].

[0105] Then, in S309 following S307, the geospatial information processing device 70 calculates the error vector Vs[j] of each camera's estimated position Pc'[j] relative to the corresponding position Ps[j] identified in S307. Then, in S311, the geospatial information processing device 70 generates a corrected space as shown in FIG. 14 by radial basis function interpolation, using each camera's estimated position Pc'[j] as a control point and the error vector Vs[j] as an interpolated value. Then, in S313, the geospatial information processing device 70 applies the corrected space generated in S311 to the 3D point cloud reconstructed space corrected in S7 (S205) described above, thereby additionally correcting the entire 3D point cloud reconstructed space. With this, the geospatial information processing device 70 ends S9 in FIG. 4.

[0106] Furthermore, Fig. 23 is a flow diagram showing details of S11 in Fig. 4. As shown in Fig. 23, first, in S401, the geospatial information processing device 70 performs road detection for each frame of captured image data using an object detection model, for example, GroundingDINO.

[0107] Then, in the next step S403, the geospatial information processing device 70 performs segmentation on the detection result in S401 using a segmentation model, for example, SAM2, to divide the road area.

[0108] After S403, in the following S405, the geospatial information processing device 70 extracts two-dimensional feature points 132 within the road area (mask 130) divided by the segmentation in S403.

[0109] Then, in S407 following S405, the geospatial information processing device 70 identifies three-dimensional points 102 that correspond to each of the two-dimensional feature points 132 extracted in S405, i.e., correspond to the two-dimensional feature points 132 within the road area.

[0110] Furthermore, in S409 following S407, the geospatial information processing device 70 excludes from subsequent processing, among the 3D points 102 identified in S407, those whose z-score in the Y-axis direction of the real world coordinate space exceeds a predetermined value, as outliers.

[0111] Then, in S411 following S409, the geospatial information processing device 70 associates position information representing the three-dimensional coordinate values ​​of the three-dimensional points 102 after the outliers have been removed in S409 with the two-dimensional feature points corresponding to the three-dimensional points 102. This means that the three-dimensional coordinate values ​​in the real-world coordinate space of an appropriate point on the road in the frame are associated with an appropriate point on the road in the frame. With this, the geospatial information processing device 70 ends S11 in FIG. 4.

[0112] Moreover, Fig. 24 is a flow diagram showing details of S13 in Fig. 4. As shown in Fig. 24, first, in S501, the geospatial information processing device 70 performs lane boundary detection for each frame of captured image data using a lane boundary detection model, for example, CLRerNET.

[0113] Then, in the next step S503, the geospatial information processing device 70 groups the two-dimensional feature points 132a located close to the lane boundaries 134 detected in step S501 by the lane boundaries 134.

[0114] After S503, in the following S505, the geospatial information processing device 70 pairs two-dimensional feature points 132a that are on the same horizontal line, in other words, have the same Y value in the frame, between each of the groups grouped in S503.

[0115] Then, in S507 following S505, the geospatial information processing device 70 calculates the mutual distance between the three-dimensional points 102 corresponding to the two-dimensional feature points 132a paired in S505, and estimates this calculated mutual distance as the lane width Wt.

[0116] Then, in S509 following S507, the geospatial information processing device 70 associates the lane width Wt estimated in S507 with the 3D point 102 related to the estimation of the lane width Wt. This means that the estimation result of the actual lane width Wt of the road in the frame is associated with the road in the frame. With this, the geospatial information processing device 70 ends S13 in FIG. 4.

[0117] Furthermore, Fig. 25 is a flow diagram showing details of S15 in Fig. 4. As shown in Fig. 25, the geospatial information processing device 70 first performs road detection using an object detection model, for example, GroundingDINO, for each frame of captured image data in S601. The processing of S601 is similar to the processing of S401 in S11 described above.

[0118] Then, in the following S603, the geospatial information processing device 70 performs road surface condition determination using an image language embedding model, for example, CLIP, on the detection result in S601, that is, the portion with the bounding box 125 attached.

[0119] After S603, in the following S605, the geospatial information processing device 70 pairs two-dimensional feature points 132a that are on the same horizontal line, in other words, have the same Y value in the frame, between each of the groups grouped in S603.

[0120] Then, in S507 following S505, the geospatial information processing device 70 associates the determination result of S505 with the 3D point 102 related to the determination result. This means that the determination result of the surface condition of the road in the frame is associated with the road in the frame. With this, the geospatial information processing device 70 ends S15 in FIG. 4.

[0121] As described above, according to this embodiment, the position of the camera equipped in the drive recorder is estimated based on the data recorded by the drive recorder, and a 3D model of the geographic space around the camera is reconstructed in association with the estimated position of the camera. The reconstructed 3D reconstruction space is then aligned with the real-world coordinate space along with the estimated position of the camera. Furthermore, matching is performed to determine which road (OSM road) on the OSM the estimated position of the camera matches. This is particularly useful when editing OSM. Moreover, drive recorders are used, which are far cheaper than the LiDAR disclosed in the aforementioned Non-Patent Document 1 and are widely available on the market. Therefore, this embodiment significantly contributes to editing OSM at lower cost and is particularly beneficial for the open-platform digital map known as OSM.

[0122] In this embodiment, the geospatial information processing device 70 that executes S1 in Fig. 4, or more precisely, the CPU 72a, is an example of an image data input accepting means according to the present invention and also an example of a positioning data input accepting means according to the present invention. The CPU 72a that executes S3 in Fig. 4 is an example of a 3D reconstruction means according to the present invention, and the CPU 72a that executes S5 in Fig. 4 is an example of a spatial matching means according to the present invention. And the CPU 72a that executes S303 in Fig. 22 is an example of a matching means according to the present invention.

[0123] The present embodiment is merely a specific example of the present invention and does not limit the scope of the present invention. The present invention can be applied to various aspects other than the present embodiment.

[0124] For example, although the geospatial information processing device 70 is configured by a PC in the above embodiment, the present invention is not limited to this. The geospatial information processing device 70 may be configured by a device other than a PC, particularly a dedicated device.

[0125] Furthermore, although the data recorded by the drive recorder is assumed to be acquired from the data management server 30, this is not limitative. For example, the data recorded by the drive recorder may be acquired directly from the drive recorder.

[0126] Furthermore, a digital map database other than OSM, particularly an open platform digital map database, may be adopted as the map database 50.

[0127] The present invention is applicable not only to the creation and updating of digital maps, but also to fields other than digital maps, such as the metaverse and games.

[0128] Furthermore, the present invention is not limited to being provided in the form of a geospatial information processing device, but can also be provided in the form of a geospatial information processing method. [Explanation of symbols]

[0129] 10...Geospatial Information Processing System 30...Data management server 50 … Map database 70 ... Geospatial information processing device 72 ... Control section 72a...CPU 72b… Main memory section 80 … Auxiliary storage unit 80a ... Geospatial information processing software 100...3D point cloud 102 … 3D point 12 ... Control section 12a … APP 12b... Main memory section Pc[j] … Estimated camera position Pc'[j] ... estimated camera position after correction Pc”[j] … Estimated camera position after additional correction Pcg[k] … Estimated corresponding camera position Pcg'[k] … Estimated corresponding camera position after correction Pg[k] … Positioning position Ps[j] … Corresponding position

Claims

1. an image data input receiving means for receiving input of image data obtained by sequentially capturing images of a geographical space around a camera moving on a road in a first cycle; a positioning data input receiving means for receiving input of positioning data obtained by sequentially measuring positions at a second period different from the first period by a positioning satellite receiver that moves on the road together with the camera; a three-dimensional reconstruction means for estimating a position of the camera for each of the first periods based on the image data and reconstructing a three-dimensional model of the geographical space in association with the estimated position of the camera; a space alignment means for aligning the three-dimensional reconstruction space reconstructed by the three-dimensional reconstruction means with a real world coordinate space by using the positioning position for each second period represented by the positioning data as an index together with the estimated position of the camera; a matching means for matching the estimated position of the camera after the matching by the spatial matching means with road information included in predetermined map information, The matching means determines that the estimated position of the camera matches the road on the map related to the road information when the estimated position of the camera after alignment by the spatial alignment means matches the road position represented by any of the road information, and when the estimated position of the camera does not match any of the road positions, identifies the road on the map that matches the estimated position of the camera based on the relationship between the distance from the estimated position of the camera and the direction of movement of the estimated position of the camera, and here determines the relationship with the direction of movement of the estimated position of the camera based on the angle formed by the direction of movement of the estimated position of the camera and the orientation of the road on the map taking into account whether the road on the map is one-way or two-way.

2. The geospatial information processing device described in Claim 1, wherein the matching means determines the relationship between the distance from the estimated position of the camera and the direction of movement of the estimated position of the camera for roads on the map that are within a predetermined search radius centered on the estimated position of the camera after alignment by the spatial alignment means.

3. A geospatial information processing method by a computer, comprising: The processor of the computer an image data input receiving step of receiving input of image data obtained by sequentially capturing images of a geographical space around a camera moving on a road in a first cycle; a positioning data input receiving step of receiving input of positioning data obtained by sequentially measuring positions at a second period different from the first period by a positioning satellite receiver that moves on the road together with the camera; a three-dimensional reconstruction step of estimating a position of the camera for each of the first periods based on the image data and reconstructing a three-dimensional model of the geographical space in association with the estimated position of the camera; a space alignment step of aligning the three-dimensional reconstruction space reconstructed by the three-dimensional reconstruction step with a real world coordinate space using the positioning position for each second period represented by the positioning data as an index together with the estimated position of the camera; a matching step of matching the estimated position of the camera after the spatial matching step with road information included in predetermined map information; In the matching step, if the estimated position of the camera after matching in the spatial matching step matches the road position represented by any of the road information, the processor determines that the estimated position of the camera matches the road on the map related to the road information, and if the estimated position of the camera does not match any of the road positions, the processor identifies the road on the map that matches the estimated position of the camera based on the relationship between the distance from the estimated position of the camera and the direction of movement of the estimated position of the camera, and here determines the relationship between the direction of movement of the estimated position of the camera and the direction of the road on the map taking into account whether the road on the map is one-way or two-way. This is a geospatial information processing method.

Images