Environment map creation system, environment map creation method, and program
The environment map generation system improves map accuracy by incorporating imaging timing and distance information to estimate camera positions and orientations, addressing inaccuracies in existing methods like SfM, and reduces computation time through selective keyframe processing.
Patent Information
- Application Number
- PCT/JP2026/016983
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2026-04-28
- Publication Date
- 2026-11-19
AI Technical Summary
Existing methods for generating environment maps using images, such as SfM, often result in inaccurate position and orientation estimates due to the lack of consideration of imaging timing information, leading to potential mismatches in feature point matching.
An environment map generation system that acquires environmental images and distance information, generates co-visibility relationships based on imaging timing and distance information, and reacquires feature points to estimate the three-dimensional position and orientation of the camera, using techniques like SfM and VSLAM to improve accuracy.
The system enhances the accuracy of environment maps by reducing misestimations in position and orientation, while also reducing computation time and initial estimation errors, and provides failure signals for images with undetectable feature points.
Smart Images

Figure JP2026016983_19112026_PF_FP_ABST
Abstract
Description
Environment map generation system, environment map generation method, and program
[0001] The present disclosure relates to an environment map generation system, an environment map generation method, and a program.
[0002] Various techniques for generating an environment map using images have been proposed. For example, Patent Document 1 discloses a technique for calculating the position and orientation of an imaging apparatus using time-series images of a space captured by the imaging apparatus. The images are captured in accordance with the movement of a user wearing a head-mounted display device. The information processing apparatus according to Patent Document 1 evaluates the imaging state of the imaging apparatus based on the calculated position and orientation.
[0003] Japanese Unexamined Patent Application Publication No. 2021-192304
[0004] One of the methods for generating an environment map using images is SfM (Structure from Motion). SfM is a technique that inputs environment images, estimates the position and orientation of the camera that captured the environment images, and generates an environment map. When the input environment images are similar to each other, SfM may estimate an incorrect position and orientation. Therefore, the accuracy of the environment map generated by SfM may not be sufficient.
[0005] The present disclosure has been made to solve such a problem, and an object thereof is to provide an environment map generation system, an environment map generation method, and a program capable of improving the accuracy of an environment map generated using images.
[0006] An environment map generation system according to the present disclosure includes an acquisition unit configured to acquire environment images captured by a camera and distance information between feature points in the environment images and the camera, a co-visibility relationship generation unit configured to generate a co-visibility relationship between the environment images based on the environment images, imaging timing information of the environment images, and the distance information, and an estimation unit configured to reacquire feature points in the environment images, and estimate the three-dimensional position and orientation of the camera by comparing positions of common feature points in different environment images among the reacquired feature points based on the co-visibility relationship.
[0007] The environmental map creation method according to this disclosure involves a computer acquiring an environmental image captured by a camera and distance information between feature points in the environmental image and the camera, generating a conspicuous relationship between the environmental images based on the environmental image, the timing information of the environmental image capture, and the distance information, acquiring feature points in the environmental images again, and estimating the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the newly acquired feature points based on the conspicuous relationship.
[0008] The program relating to this disclosure causes a computer to perform the following steps: acquire an environmental image captured by a camera and distance information between feature points in the environmental image and the camera; generate a conspicuous relationship between the environmental images based on the environmental image, the imaging timing information of the environmental image, and the distance information; acquire feature points in the environmental image again and estimate the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the newly acquired feature points based on the conspicuous relationship.
[0009] This disclosure provides an environmental map creation system, an environmental map creation method, and a program that can improve the accuracy of environmental maps created using images.
[0010] Figure 1 is a block diagram showing the configuration of the environmental map creation system according to this disclosure. Figure 2 is a flowchart showing an example of the flow of the environmental map creation method by the environmental map creation system. Figure 3 is a block diagram showing the configuration of the environmental map creation system according to this disclosure. Figure 4 is a flowchart showing an example of the co-view relationship generation process when using VSLAM. Figure 5 is a flowchart showing an example of the keyframe selection process. Figure 6 is a flowchart showing an example of the co-view relationship generation process between environmental images. Figure 7 is a flowchart showing an example of the keyframe removal process. Figure 8 is a flowchart showing an example of the estimation process by the estimation unit. Figure 9A is a comparison diagram before and after similar keyframe removal. Figure 9B is a comparison diagram before and after similar keyframe removal. Figure 10 is a table comparing the time required to create the environmental map. Figure 11 is a diagram showing an example of the hardware configuration of the environmental map creation system.
[0011] (Embodiment 1) Hereinafter, Embodiment 1 of the present disclosure will be described with reference to the drawings. Figure 1 is a block diagram showing the configuration of the environmental mapping system 1 according to the present disclosure. The environmental mapping system 1 is typically one or more computer devices that operate by a processor executing a program stored in memory. In other words, the environmental mapping system 1 may be composed of one computer device or two or more computer devices. For example, the environmental mapping system 1 is one or more server devices.
[0012] The Environmental Mapping System 1 is a system that creates an environmental map based on environmental images captured by a camera. Here, environmental images refer to multiple images of an arbitrary environment. The environment is also referred to as space. In other words, the Environmental Mapping System 1 is a system that creates a map of a space based on images of that space captured by a camera. The map created by the Environmental Mapping System 1 is a three-dimensional map. Furthermore, an environment is a region having one or more objects from which feature points can be extracted from at least one environmental image. The environment may be an indoor environment such as a warehouse, office, or commercial facility, or an outdoor environment such as a park, street, or intersection.
[0013] The environmental map creation system 1 comprises an acquisition unit 11, a consensual relationship generation unit 12, and an estimation unit 13. The acquisition unit 11, the consensual relationship generation unit 12, and the estimation unit 13 are typically software or modules whose processing is performed by a processor executing a program stored in memory. The acquisition unit 11, the consensual relationship generation unit 12, and the estimation unit 13 may also be hardware such as circuits or chips. That is, the acquisition unit 11, the consensual relationship generation unit 12, and the estimation unit 13 may each be configured by different computer devices.
[0014] The acquisition unit 11 and the co-view relationship generation unit 12, and the co-view relationship generation unit 12 and the estimation unit 13 are connected in a way that enables data communication. Each data communication may be performed via a wired connection or a wireless connection. The data communication may be performed within the same computer device, via an internet line, or using short-range wireless communication technology. The type of communication protocol used for data communication is not limited.
[0015] The acquisition unit 11 acquires an environmental image captured by a camera and distance information between feature points in the environmental image and the camera. The acquisition unit 11 may also include a camera in its configuration. That is, the acquisition unit 11 may include an imaging means. The acquisition unit 11 may also receive environmental images from a camera. The camera is typically a visible light camera that captures two-dimensional images. That is, the environmental image is typically a two-dimensional image. The environmental image is also typically a series of images captured at a predetermined frame rate. The distance information acquired by the acquisition unit 11 is also called depth information. That is, the acquisition unit 11 acquires depth information between feature points in the environmental image and the camera. The acquisition unit 11 may acquire the environmental image and distance information from different devices or from the same device.
[0016] The constellation relationship generation unit 12 generates constellation relationships between environmental images based on the environmental images acquired by the acquisition unit 11, the imaging timing information of the environmental images, and the distance information acquired by the acquisition unit 11. The imaging timing information of the environmental images may be the time information when the environmental images were captured, or it may be information indicating the chronological relationship between the environmental images. A constellation relationship between environmental images refers to a relationship in which the same feature point or landmark is observed in multiple environmental images. Here, a feature point refers to a point in the environmental image where two-dimensional information is clear. On the other hand, a landmark refers to a point in the environment where three-dimensional information is clear. That is, a feature point refers to a point in the environmental image where two-dimensional coordinates can be estimated, and a landmark refers to a point in the environment where three-dimensional coordinates can be estimated. The constellation relationship generation unit 12 may generate constellation relationships for all environmental images acquired by the acquisition unit 11, or it may generate constellation relationships for some of the environmental images.
[0017] The co-viewing relationship generation unit 12 performs feature point matching, typically in the order of older environmental images to newer ones, based on the imaging timing information of the environmental images. Specifically, the co-viewing relationship generation unit 12 compares the newer environmental images with the older environmental images and matches the feature points in the environmental images. Then, the co-viewing relationship generation unit 12 estimates the three-dimensional information of landmarks using the distance information acquired by the acquisition unit 11. The co-viewing relationship generation unit 12 compares the newer environmental images with the older environmental images and generates a co-viewing relationship between the environmental images based on the shared landmark relationships between the environmental images.
[0018] The estimation unit 13 reacquires feature points from the environmental image acquired by the acquisition unit 11. Then, based on the constancy relationship between environmental images generated by the constancy relationship generation unit 12, the estimation unit 13 estimates the three-dimensional position and orientation of the camera that captured the environmental image by comparing the positions of common feature points in different environmental images among the reacquired feature points. The three-dimensional position and orientation refers to, for example, the camera's position and orientation in a world coordinate system. The three-dimensional position and orientation is also simply referred to as position and orientation. The camera's position and orientation are typically expressed by three parameters each. The three-dimensional position and orientation may be represented by a table, vector, or matrix. By estimating the camera's three-dimensional position and orientation, the estimation unit 13 creates an environmental map. The environmental map is, for example, point cloud data consisting of the three-dimensional coordinates of landmarks. The estimation unit 13 can estimate the camera's three-dimensional position and orientation using, for example, SfM (Structure from Motion). Here, SfM is a three-dimensional position and orientation estimation means that does not use environmental image capture timing information as an input element. In other words, the estimation unit 13 estimates the three-dimensional position and orientation of the camera without using the imaging timing information of the environmental image as an input element. Specifically, the estimation unit 13 can reacquire feature points in the environmental image acquired by the acquisition unit 11 using SfM. Then, the estimation unit 13 can estimate the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images using SfM.
[0019] Next, the flow of the environmental map creation method by the environmental map creation system 1 will be explained. Figure 2 is a flowchart showing an example of the flow of the environmental map creation method by the environmental map creation system 1. First, the acquisition unit 11 acquires an environmental image captured by the camera and distance information between feature points in the environmental image and the camera (S101). Next, the constellation relationship generation unit 12 generates a constellation relationship between environmental images based on the environmental image acquired by the acquisition unit 11, the imaging timing information of the environmental image, and the distance information acquired by the acquisition unit 11 (S102). After that, the estimation unit 13 acquires the feature points in the environmental image acquired by the acquisition unit 11 again. Then, the estimation unit 13 estimates the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the feature points acquired again, based on the constellation relationship between environmental images generated by the constellation relationship generation unit 12 (S103).
[0020] As described above, in this embodiment, the environmental map creation system 1 generates a conspicuous relationship between environmental images based on the environmental image captured by the camera, the imaging timing information of the environmental image, and the distance information between the feature points and the camera. Next, the environmental map creation system 1 acquires the feature points in the environmental image again. Then, based on the conspicuous relationship, the environmental map creation system 1 estimates the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the newly acquired feature points. In this way, the environmental map creation system 1 can create an environmental map. When creating an environmental map using a technique such as SfM, such a technique may estimate an incorrect position and orientation. This is because such a technique estimates the position and orientation without using the imaging timing information of the environmental image as an input element. The environmental map creation system 1 first generates a conspicuous relationship using the imaging timing information of the environmental image and the distance information between the feature points in the environmental image and the camera. Then, the environmental map creation system 1 acquires the feature points in the environmental image again. Subsequently, the environmental mapping system 1 can estimate the camera's three-dimensional position and orientation by comparing the positions of common feature points in different environmental images using the concentric relationships between pre-generated environmental images. Therefore, the environmental mapping system 1 can reduce misestimations of position and orientation. As a result, the environmental mapping system 1 can create highly accurate environmental maps. This allows the environmental mapping system 1 to improve the accuracy of environmental maps created using images.
[0021] (Embodiment 2) This second embodiment is a specific example of the first embodiment described above. Figure 3 is a block diagram showing the configuration of the environmental map creation system 2 according to this disclosure. The environmental map creation system 2 comprises an acquisition unit 21, a co-view relationship generation unit 22, a keyframe selection unit 23, a keyframe storage unit 24, a keyframe removal unit 25, and an estimation unit 26. The components of the environmental map creation system 2 are typically software or modules whose processing is performed by a processor executing a program stored in memory. The components of the environmental map creation system 2 may also be hardware such as circuits or chips. That is, the components of the environmental map creation system 2 may each be composed of different computer devices. Hereafter, for the sake of clarity, explanations that overlap with the first embodiment will be omitted as appropriate.
[0022] The acquisition unit 21 acquires an environmental image captured by the camera, and distance information between feature points in the environmental image and the camera. In other words, the acquisition unit 21 corresponds to the acquisition unit 11 in the environmental map creation system 1. The camera from which the acquisition unit 21 acquires the environmental image is typically a visible light camera that captures two-dimensional images. A visible light camera is also called an RGB camera. Here, the camera may be a visible light camera, or a camera that captures grayscale environmental images. The camera may also be an infrared camera or a stereo camera.
[0023] The acquisition unit 21 may acquire environmental images captured by multiple cameras. In this case, the focal length and optical center of these multiple cameras may be the same. That is, the intrinsic parameters of these multiple cameras may be the same. This can be achieved by using the same model of camera and the same image size. However, the intrinsic parameters of the multiple cameras do not have to be the same. That is, the intrinsic parameters of each camera may be different. Also, the image size may be different for each image captured.
[0024] The distance information between feature points in the environmental image and the camera acquired by the acquisition unit 21 is, for example, the depth information of the pixels in the environmental image. That is, the distance information may be the depth information for each pixel in the environmental image. This allows the acquisition unit 21 to acquire distance information between feature points in the environmental image and the camera. Depth information can be acquired, for example, by a depth camera. A depth camera is also called a depth camera. A depth camera can acquire depth information using stereo vision or ToF (Time of Flight).
[0025] Environmental images and distance information may be acquired simultaneously. For example, the acquisition unit 21 can acquire this information using an RGBD camera capable of acquiring environmental images and depth information for each pixel in the image. This allows the acquisition unit 21 to acquire environmental images and depth information for each pixel in the image in real time. The acquisition unit 21 may store this acquired information in a storage unit (not shown) or transmit it to the co-view relationship generation unit 22.
[0026] The co-view relationship generation unit 22 generates co-view relationships between environmental images based on the environmental images acquired by the acquisition unit 21, the timing information of the imaging of the environmental images, and the distance information acquired by the acquisition unit 21. In other words, the co-view relationship generation unit 22 corresponds to the co-view relationship generation unit 12 in the environmental map creation system 1.
[0027] The co-viewing relationship generation unit 22 generates co-viewing relationships between environmental images using VSLAM (Visual Simultaneous Localization and Mapping). VSLAM is also called Visual SLAM. That is, the co-viewing relationship generation unit 22 continuously inputs environmental images and distance information acquired by the acquisition unit 21 in a time series. Here, the co-viewing relationship generation unit 22 can grasp the timing information between environmental images, that is, the chronological relationship of the environmental images. The co-viewing relationship generation unit 22 may receive and input environmental images and distance information acquired by the acquisition unit 21 in real time, or it may receive and input environmental images etc. acquired by the acquisition unit 21 in advance retrospectively.
[0028] The co-viewing relationship generation unit 22 extracts feature points captured in environmental images that have a contextual relationship. The co-viewing relationship generation unit 22 extracts feature points using methods such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), or FAST (Features from Accelerated Segment Test). The co-viewing relationship generation unit 22 may also extract feature points using machine learning models such as CNN (Convolutional Neural Network).
[0029] Next, the co-view relationship generation unit 22 compares the newly input environmental image with the keyframe and performs feature point matching using distance information. A keyframe is an image selected from the environmental image acquired by the acquisition unit 21 that satisfies predetermined requirements. Specifically, a keyframe is selected when the camera position has changed sufficiently or when many new landmarks are detected. The keyframe is selected by the keyframe selection unit 23, which will be described later.
[0030] Subsequently, the co-viewing relationship generation unit 22 estimates the camera's three-dimensional position and orientation based on the imaging timing information of the newly input environmental image and the result of feature point matching with the keyframe. Then, by performing three-dimensional surveying based on the position and orientation of the landmark in the keyframe and the change in the camera's position and orientation relative to the keyframe, it estimates the three-dimensional information of the landmark in the newly input environmental image. Here, three-dimensional surveying refers to, for example, triangulation. Subsequently, by performing 2D (Dimension)-3D matching, the co-viewing relationship generation unit 22 creates an environmental map from the estimated camera position and orientation and the three-dimensional information of the landmark. In other words, by using VSLAM, the co-viewing relationship generation unit 22 can estimate the three-dimensional position based on the environmental image, imaging timing information, and distance information, before the estimation by the estimation unit 13.
[0031] The covisibility relationship generation unit 22 generates covisibility relationships between keyframes during the process of estimating the camera's position and orientation and creating an environmental map using VSLAM. Specifically, the covisibility relationship generation unit 22 generates a covisibility graph (CG) as the covisibility relationship between keyframes. A CG is an undirected graph structure that connects keyframes with common landmarks using edges. Nodes in a CG are keyframes. In other words, the covisibility relationship generation unit 22 can generate a CG in which edges are weighted based on the number of common landmarks. Typically, the covisibility relationship generation unit 22 connects keyframes with a certain number or more common landmarks using edges. The covisibility relationship generation unit 22 generates a CG based on keyframe information stored in the keyframe storage unit 24, which will be described later. The covisibility relationship generation unit 22 can transmit the generated CG to the keyframe storage unit 24.
[0032] When a newly input environmental image is selected as a keyframe by the keyframe selection unit 23, the co-view relationship generation unit 22 searches for common landmarks between the new keyframe and the previously selected keyframes. If there are a certain number or more common landmarks, the co-view relationship generation unit 22 can generate CG by connecting the newly selected keyframe and the corresponding keyframes with edges. The co-view relationship generation unit 22 estimates the camera's position and orientation and creates an environmental map by referring to the co-view relationships between the keyframes generated by the CG.
[0033] Here, the co-view relationship generation unit 22 does not have to estimate the camera's position and orientation and create an environment map based on keyframes. That is, the co-view relationship generation unit 22 may estimate the camera's position and orientation and create an environment map without selecting keyframes from the environment images acquired by the acquisition unit 21. Similarly, the co-view relationship generation unit 22 may generate co-view relationships between environment images without selecting keyframes from the environment images. That is, the co-view relationship generation unit 22 may generate CG between environment images rather than between keyframes.
[0034] If the co-viewing relationship generation unit 22 is unable to estimate the three-dimensional position and orientation of the camera using VSLAM, the co-viewing relationship generation unit 22 may output an estimation failure signal. The case in which the co-viewing relationship generation unit 22 cannot estimate the position and orientation is, for example, when it is unable to extract feature points from the environmental image acquired by the acquisition unit 21. In such a case, the co-viewing relationship generation unit 22 cannot perform feature point matching and the associated camera position and orientation estimation, and therefore outputs an estimation failure signal. The estimation failure signal is a signal indicating that the estimation process by the co-viewing relationship generation unit 22 is not possible. The co-viewing relationship generation unit 22 may output the estimation failure signal to the user of the environmental map creation system 2, or to another computer device (not shown). When outputting to the user, the co-viewing relationship generation unit 22 may display a warning on the computer device used by the user, or output a warning sound.
[0035] The constellation relationship generation unit 22 can output relevant environmental image information if three-dimensional position estimation is not possible. Relevant environmental image information refers to, for example, information relating to an environmental image from which the constellation relationship generation unit 22 was unable to extract feature points. Specifically, the constellation relationship generation unit 22 may output the environmental image itself, or it may output identification information or other metadata of the environmental image.
[0036] The co-view relationship generation unit 22 transmits the environmental image acquired by the acquisition unit 21, the three-dimensional information of landmarks captured in the environmental image, and the estimated camera position and orientation information to the keyframe selection unit 23.
[0037] The keyframe selection unit 23 selects multiple keyframes from the environmental image. The keyframe selection unit 23 selects keyframes when the camera position has changed sufficiently or when many new landmarks have been detected. The keyframe selection unit 23 receives the environmental image, three-dimensional information of landmarks captured in the environmental image, and estimated camera position and orientation information from the co-view relationship generation unit 22.
[0038] When selecting a keyframe after a significant change in camera position, the keyframe selection unit 23 compares the camera's position and orientation relative to the newly input environmental image to the co-view relationship generation unit 22 with the camera's position and orientation relative to the environmental image previously selected as a keyframe. If the distance or angle between the existing keyframe and the new environmental image is greater than or equal to a predetermined value, the keyframe selection unit 23 selects the new environmental image as a keyframe. Here, the keyframe selection unit 23 may also select the new environmental image as a keyframe if both the distance and angle between it and the existing keyframe are greater than or equal to a predetermined value.
[0039] When selecting a keyframe in the event that many new landmarks not present in existing keyframes are detected, the keyframe selection unit 23 compares the environmental image in which the co-view relationship generation unit 22 has newly detected landmarks with the landmarks in the existing keyframes. If a certain number of landmarks differ between the existing keyframes and the new environmental image, the keyframe selection unit 23 selects the new environmental image as a keyframe.
[0040] The keyframe selection unit 23 transmits the newly selected keyframe, the three-dimensional information of the landmark in the keyframe, and the camera's position and orientation information relative to the keyframe to the keyframe storage unit 24. If the co-viewing relationship generation unit 22 does not perform camera position and orientation estimation and environmental map creation based on the keyframe, the keyframe selection unit 23 does not select a keyframe from the environmental image. In this case, the keyframe selection unit 23 transmits the environmental image, the three-dimensional information of the landmark in the environmental image, and the camera's position and orientation information relative to the environmental image to the keyframe storage unit 24.
[0041] The keyframe storage unit 24 is composed of a non-volatile storage device such as a hard disk or flash memory. The keyframe storage unit 24 can store keyframes, three-dimensional information of landmarks in the keyframes, and camera position and orientation information for the keyframes transmitted from the keyframe selection unit 23. Here, "storage" may be read as memory, recording, and preservation. The keyframe storage unit 24 can also store CG transmitted from the co-viewing relationship generation unit 22. The keyframe storage unit 24 stores this information related to keyframes in association with each other. The keyframe storage unit 24 can output necessary information according to instructions from the co-viewing relationship generation unit 22, the keyframe selection unit 23, the keyframe removal unit 25, and the estimation unit 26. If the co-viewing relationship generation unit 22 does not perform camera position and orientation estimation and environmental map creation based on keyframes, the keyframe storage unit 24 stores information related to environmental images transmitted from the keyframe selection unit 23.
[0042] The keyframe removal unit 25 removes similar keyframes that have a similar relationship to a predetermined keyframe from a plurality of keyframes selected by the keyframe selection unit 23. That is, the keyframe removal unit 25 removes similar keyframes that have a similar relationship from the keyframe storage unit 24 by comparing the keyframes stored in the keyframe storage unit 24. Here, "remove" may be read as "delete".
[0043] The key frame removal unit 25 can, for example, define key frames with a distance difference and an angle difference between key frames less than a predetermined value as similar key frames. In other words, the key frame removal unit 25 can define key frames with an estimated inter-camera distance difference and angle difference relative to the key frame that are less than a predetermined value as similar key frames. The key frame removal unit 25 may define key frames with only the distance difference between key frames less than a predetermined value as similar key frames, or may define key frames with only the angle difference between key frames less than a predetermined value as similar key frames. The key frame removal unit 25 can remove other key frames that are determined to be similar key frames to a certain key frame from the key frame storage unit 24. When a certain key frame is defined as a reference, and there are a plurality of similar key frames for the reference key frame, the key frame removal unit 25 can remove all of the plurality of similar key frames from the key frame storage unit 24.
[0044] Furthermore, the key frame removal unit 25 can remove key frames from the key frame storage unit 24 based on the number of similar key frames that a key frame has. For example, the key frame removal unit 25 may calculate the number of similar key frames for each key frame, and remove the key frame with the largest number of similar key frames from the key frame storage unit 24. In addition, the key frame removal unit 25 may remove key frames for which the number of similar key frames is equal to or greater than a predetermined value.
[0045] For example, after removing the key frame with the largest number of similar key frames, the key frame removal unit 25 may then remove the key frame with the next largest number of similar key frames. In this manner, the key frame removal unit 25 may repeat the key frame removal processing until no key frames having similar key frames are detected.
[0046] The estimation unit 26 reacquires feature points from the environmental image acquired by the acquisition unit 21. Then, based on the co-viewing relationships between environmental images generated by the co-viewing relationship generation unit 22, the estimation unit 26 estimates the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the reacquired feature points. In other words, the estimation unit 26 corresponds to the estimation unit 13 in the environmental map creation system 1. In the environmental map creation system 2, the estimation unit 26 estimates the three-dimensional position and orientation of the camera based on the CG, which is the co-viewing relationship between keyframes, stored in the keyframe storage unit 24. The estimation unit 26 estimates the three-dimensional position and orientation based on the keyframes after similar keyframes have been removed by the keyframe removal unit 25. Specifically, the estimation unit 26 acquires feature points from the keyframes after similar keyframes have been removed by the keyframe removal unit 25. The estimation unit 26 then estimates the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different keyframes based on the conspicuous relationship generated by the conspicuous relationship generation unit 22. Note that "common feature points" in the environmental images (keyframes) refer to, for example, the same feature points in different environmental images (keyframes).
[0047] The estimation unit 26 estimates the three-dimensional position and orientation of the camera using SfM. That is, if the con-view relationship generation unit 22 has already estimated the three-dimensional position and orientation of the camera before the estimation unit 26 does so, the estimation unit 26 re-estimates the position and orientation using SfM. Specifically, the estimation unit 26 can estimate the position and orientation using incremental SfM.
[0048] First, the estimation unit 26 extracts feature points in the key frames accumulated in the key frame accumulation unit 24. In other words, the estimation unit 26 acquires the feature points in the key frames accumulated in the key frame accumulation unit 24. Here, the estimation unit 26 extracts the feature points using SIFT, SURF, FAST, or CNN. Next, the estimation unit 26 performs feature point matching between the key frames. Here, the estimation unit 26 inputs the co-visibility relationship between the environment images based on CG accumulated in the key frame accumulation unit 24 to perform the feature point matching. In other words, the estimation unit 26 performs the feature point matching by comparing the positions of common feature points in different key frames based on the CG accumulated in the key frame accumulation unit 24. That is, the estimation unit 26 performs the feature point matching between key frames connected in the CG, or between key frames connected by edges having a predetermined weight. Then, the estimation unit 26 estimates the three-dimensional position and orientation of the camera based on the feature point matching, and estimates three-dimensional information of landmarks through three-dimensional surveying. Accordingly, the estimation unit 26 can create an environmental map.
[0049] The estimation unit 26 can create a dense environmental map based on the created environmental map, that is, the point cloud data of landmarks and the three-dimensional position and orientation of the camera. Specifically, the estimation unit 26 calculates depth information for each pixel in a key frame by performing, for example, stereo matching, based on the key frame, the three-dimensional information of the landmarks imaged in the key frame, and the estimated position and orientation information of the camera. Then, by integrating the depth information of these key frames, the estimation unit 26 can create a dense environmental map.
[0050] The estimation unit 26 can input the created environmental map as initial data to a position and orientation estimation system (not shown) that estimates the camera's position and orientation using VSLAM. That is, the estimation unit 26 can output the environmental map in a format that can be input to the VSLAM used in the system. Specifically, the environmental map may consist of keyframes, three-dimensional information of landmarks captured in the keyframes, and constellation relationships between keyframes. By using the environmental map generated by the estimation unit 26 as initial data, the position and orientation estimation system using VSLAM can estimate the camera's three-dimensional position and orientation with higher accuracy compared to when such initial data is not available.
[0051] If the co-viewing relationship generation unit 22 does not estimate the camera's position and orientation and create an environmental map based on keyframes, the estimation unit 26 estimates the camera's three-dimensional position and orientation and creates an environmental map based on the environmental images and CG between environmental images stored in the keyframe storage unit 24.
[0052] Next, the process of creating an environmental map using the environmental map creation system 2 will be explained. Figure 4 is a flowchart showing an example of the co-view relationship generation process when using VSLAM. First, the co-view relationship generation unit 22 receives the environmental image acquired by the acquisition unit 21, the imaging timing information of the environmental image, and the distance information (S201). Next, the co-view relationship generation unit 22 extracts feature points captured in the input environmental image (S202). Then, the co-view relationship generation unit 22 acquires the keyframes accumulated from the keyframe accumulation unit 24 (S203). After that, the co-view relationship generation unit 22 performs feature point matching based on a comparison between the newly input environmental image and the keyframes and the distance information acquired by the acquisition unit 21 (S204).
[0053] Subsequently, the constellation relationship generation unit 22 estimates the camera's three-dimensional position and orientation based on the newly input environmental image acquisition timing information and the result of feature point matching with the keyframe (S205). Then, the constellation relationship generation unit 22 estimates the three-dimensional information of landmarks in the environmental image by performing three-dimensional surveying based on the position and orientation of landmarks in the keyframe and the change in the camera's position and orientation relative to the keyframe (S206). Then, the constellation relationship generation unit 22 creates an environmental map from the estimated camera position and orientation and the three-dimensional information of landmarks (S207).
[0054] Figure 5 is a flowchart showing an example of the keyframe selection process. First, the co-view relationship generation unit 22 estimates the three-dimensional position and orientation of the camera and the three-dimensional information of the landmark (S208). Here, step S208 corresponds to steps S205 and S206 in Figure 4. Next, the co-view relationship generation unit 22 transmits the environment image, the three-dimensional information of the landmark, and the camera's position and orientation information to the keyframe selection unit 23 (S209). Next, the keyframe selection unit 23 compares the environment image received from the co-view relationship generation unit 22 with the keyframes already stored in the keyframe storage unit 24 and determines whether the environment image satisfies the requirements for a keyframe (S210). If it is determined to be YES in step S210, the keyframe selection unit 23 selects the environment image as a keyframe and stores it in the keyframe storage unit 24 (S211). If it is determined to be NO in step S210, the process ends.
[0055] Figure 6 is a flowchart showing an example of the process for generating a co-view relationship between environmental images. First, the keyframe selection unit 23 stores a new keyframe in the keyframe storage unit 24 (S212). Here, step S212 corresponds to step S211 in Figure 5. Next, the co-view relationship generation unit 22 acquires the new keyframe from the keyframe storage unit 24 (S213). Next, the co-view relationship generation unit 22 updates the CG based on the new keyframe (S214). After that, the co-view relationship generation unit 22 stores the updated CG in the keyframe storage unit 24 (S215).
[0056] Figure 7 is a flowchart showing an example of the keyframe removal process. In Figure 7, keyframes whose distance difference and angle difference are both less than a predetermined value are identified as similar keyframes. First, the keyframe removal unit 25 selects keyframes stored in the keyframe storage unit 24 (S216). Next, the keyframe removal unit 25 calculates the distance difference and angle difference between the keyframe and other keyframes (S217). Next, the keyframe removal unit 25 determines whether the distance difference is less than a predetermined value (S218). If the result in step S218 is YES, the keyframe removal unit 25 then determines whether the angle difference is less than a predetermined value (S219). If the result in step S219 is YES, the keyframe removal unit 25 counts the other keyframe as a similar keyframe (S220).
[0057] If the result is NO after processing in step S220 or in step S218 or S219, the keyframe removal unit 25 determines whether a comparison has been made with all other keyframes (S221). If the result is NO in step S221, the process returns to step S217, and the keyframe removal unit 25 calculates the distance difference and angle difference between the keyframe and the other keyframes. If the result is YES in step S221, the keyframe removal unit 25 determines whether a comparison has been made with all other keyframes (S222). If the result is NO in step S222, the process returns to step S216, and the keyframe removal unit 25 selects a different keyframe.
[0058] If the result in step S222 is YES, the keyframe removal unit 25 removes the keyframe with the most similar frames from the keyframe storage unit 24 (S223). Next, the keyframe removal unit 25 determines whether there is a keyframe that has at least one similar keyframe (S224). If the result in step S224 is YES, the process returns to step S223, and the keyframe removal unit 25 removes the keyframe with the most similar frames. If the result in step S224 is NO, the process ends.
[0059] Figure 8 is a flowchart illustrating an example of the estimation process performed by the estimation unit 26. First, the estimation unit 26 receives input from the keyframes stored in the keyframe storage unit 24 and the concentric relationships between the keyframes (S225). Next, the estimation unit 26 extracts feature points from the input keyframes (S226). In other words, the estimation unit 26 obtains feature points from the input keyframes. Then, the estimation unit 26 matches the feature points between keyframes that have concentric relationships (S227). In other words, the estimation unit 26 performs feature point matching by comparing the positions of common feature points in different keyframes. After that, the estimation unit 26 estimates the three-dimensional position and orientation of the camera based on the feature point matching (S228). Subsequently, the estimation unit 26 estimates the three-dimensional information of landmarks in the keyframes by three-dimensional surveying (S229). Finally, the estimation unit 26 creates an environmental map from the estimated position and orientation of the camera and the three-dimensional information of landmarks (S230).
[0060] Thus, according to this embodiment, the environmental map creation system 2 can concretely realize the environmental map creation system 1. The effects of the environmental map creation system 2 are as follows. That is, when using a technique that estimates the three-dimensional position and orientation of the camera without using the imaging timing information of the environmental image as an input element, such as SfM, the accuracy of the environmental map may not be sufficient. This is because SfM performs feature point matching on the order of the environmental images and on all of the images, resulting in matching between images taken at different locations. In other words, because SfM does not use the imaging timing information of the environmental image as an input element, there is a risk of mismatching of feature points.
[0061] According to the environmental mapping system 2, the estimation unit 26, which uses SfM, receives not only environmental images but also the conspicuous relationships between environmental images as input. These conspicuous relationships are generated considering the imaging timing information. This allows the estimation unit 26 to receive the relative positional relationships between environmental images, thereby suppressing mismatching of feature points. As a result, the environmental mapping system 2 can improve the accuracy of position and orientation estimation and environmental mapping. Furthermore, the environmental mapping system 2 can reduce the time required for environmental mapping because it suppresses the need for the estimation unit 26 to re-estimate position and orientation. The environmental mapping system 2 can generate a covisibility graph (CG) as a conspicuous relationship. Since the CG is a conspicuous relationship represented by a graph structure, the estimation unit 26 can use the CG to perform feature point matching using SfM.
[0062] Furthermore, the environmental mapping system 2 can acquire environmental images captured by multiple cameras. In this case, the environmental mapping system 2 can acquire environmental images captured using the same camera model and with the same image size. When performing three-dimensional position and orientation estimation using SfM, SfM sets default intrinsic parameters for each acquired image to estimate the position and orientation. Therefore, estimation errors due to such incorrect intrinsic parameters can affect the final estimation result. By acquiring environmental images captured using the same camera model and with the same image size, the environmental mapping system 2 can uniquely determine the intrinsic parameters of the cameras, thereby suppressing the initial estimation errors described above. As a result, the need to re-estimate the position and orientation can be reduced, and therefore, the environmental mapping system 2 can reduce the time required to create the environmental map.
[0063] The environmental mapping system 2 can estimate the initial three-dimensional position and orientation using VSLAM based on the environmental image, imaging timing information, and distance information. Furthermore, the environmental mapping system 2 can output an estimation failure signal if it is unable to estimate the three-dimensional position and orientation. Techniques that estimate position and orientation based on environmental images, such as SfM and VSLAM, may fail to estimate the position and orientation if sufficient feature points cannot be detected in the image. Also, offline processing techniques like SfM require significantly more computation time than online processing techniques like VSLAM. Therefore, the environmental mapping system 2 performs VSLAM position and orientation estimation as a preliminary step to SfM position and orientation estimation. By outputting an estimation failure signal when VSLAM fails to estimate the position and orientation, the environmental mapping system 2 can detect images where feature points cannot be detected early. This reduces the likelihood of estimation failures in SfM, thereby reducing the environmental mapping time. Furthermore, the environmental mapping system 2 can output environmental image information whose three-dimensional position and orientation could not be estimated when an unestimable signal is output. This allows the environmental mapping system 2 to notify users of specific locations from which feature points could not be extracted.
[0064] Furthermore, the environmental mapping system 2 can select multiple keyframes from the environmental image and remove similar keyframes that have a similar relationship to a predetermined keyframe from the selected multiple keyframes. Then, the environmental mapping system 2 can estimate the three-dimensional position and orientation based on the keyframes after removing the similar keyframes. Offline processing techniques such as SfM require a great deal of computation time for position and orientation estimation, as described above. Therefore, the environmental mapping system 2 can reduce the number of images processed by SfM by creating keyframes using VSLAM. Furthermore, the environmental mapping system 2 can further reduce the number of images by removing similar keyframes from the keyframes. In this way, the environmental mapping system 2 can reduce computation time while maintaining a high level of accuracy in position and orientation estimation.
[0065] Here, the reduction in the number of images by removing keyframes is illustrated with diagrams. Figures 9A and 9B are comparison diagrams before and after the removal of similar keyframes. Figure 9A shows the keyframes at a point in the environment before the removal of similar keyframes. On the other hand, Figure 9B shows the keyframes at the same point after the removal of similar keyframes. In Figures 9A and 9B, the triangles represent the camera position, i.e., the keyframe position. In Figures 9A and 9B, keyframes with a distance difference of less than 0.3m and an angle difference of less than 2° are considered similar keyframes, according to the flowchart shown in Figure 7. In Figure 9A, the total number of keyframes for the environment was 2844, but by removing similar keyframes, the number of keyframes in Figure 9B has been reduced to 1683. In this way, the environment mapping system 2 can reduce the computation time using SfM by reducing the number of keyframes.
[0066] This section describes the effect of reducing computation time by using this embodiment. Figure 10 is a table comparing the time required to create an environmental map. The upper part of Figure 10 shows the case when the environmental map creation system 2 is used (with technology), and the lower part shows the case when the environmental map creation system 2 is not used (without technology). Specifically, the upper part of Figure 10 shows the case when the environmental map is created using SfM after running VSLAM, and the lower part shows the case when the environmental map is created using SfM only. By using the environmental map creation system 2, although the image acquisition time increases from 0.57 hours to 1.43 hours, the total time for creating the environmental map can be significantly reduced from 13.45 hours to 2.65 hours. In addition, when using SfM only, there are 5 retakes due to failures in environmental map creation, whereas by using the environmental map creation system 2, no SfM retakes occur. Therefore, the environmental map creation system 2 can significantly shorten the final creation period from 6 days (including 2 holidays) to 0.5 days.
[0067] The environmental map created by the environmental map creation system 2 can be input as initial data to the position and orientation estimation system that estimates the camera's position and orientation using VSLAM. That is, the position and orientation estimation system can estimate the camera's position and orientation by performing feature point matching based on the three-dimensional position information of keyframes and landmarks related to the environmental map created by the environmental map creation system 2. Since the environmental map creation system 2 can generate a highly accurate environmental map, the position and orientation estimation system can estimate the camera's position and orientation with high accuracy by using this environmental map.
[0068] (Hardware Configuration Example) Figure 11 shows an example of the hardware configuration of the environmental mapping system 3. In Figure 11, the environmental mapping system 3 includes a processor 31 and a memory 32. The processor 31 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor 31 may include multiple processors. The memory 32 is composed of a combination of volatile memory and non-volatile memory. The memory 32 may include storage located away from the processor 31. In this case, the processor 31 may access the memory 32 via an I / O (Input / Output) interface, which is not shown.
[0069] In the above example, the program can be stored and provided to the computer using various types of non-transitory computer-readable medium. Non-transitory computer-readable medium includes various types of tangible storage medium. Examples of non-transitory computer-readable medium include magnetic storage media (e.g., magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs). Alternatively, the program may be provided to the computer using various types of transient computer-readable medium. Examples of transient computer-readable medium include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable medium can supply the program to the computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels. Computers include various information processing devices such as PCs, servers, CPUs, MPUs, FPGAs (Field Programmable Gate Arrays), and ASICs (Application Specific Integrated Circuits).
[0070] In the embodiments described above, the disclosure was explained as a hardware configuration, but the disclosure is not limited thereto. The disclosure can also be implemented by having a CPU execute a computer program to perform the environmental map creation method by the environmental map creation system 1 shown in Figure 2 and the environmental map creation method by the environmental map creation system 2 shown in Figures 4 to 8.
[0071] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.
[0072] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments, rather than being associated with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.
[0073] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) An environmental map creation system comprising: an acquisition unit that acquires an environmental image captured by a camera and distance information between feature points in the environmental image and the camera; a co-visibility relationship generation unit that generates a co-visibility relationship between the environmental images based on the environmental image, the imaging timing information of the environmental image, and the distance information; and an estimation unit that acquires feature points in the environmental image again and estimates the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the reacquired feature points based on the co-visibility relationship. (Note 2) The environmental map creation system according to Note 1, wherein the co-visibility relationship generation unit generates a co-visibility graph as the co-visibility relationship, and the estimation unit estimates the three-dimensional position and orientation based on the co-visibility graph. (Note 3) The environmental map creation system according to Note 1 or 2, wherein the co-viewing relationship generation unit estimates the three-dimensional position and orientation based on the environmental image, the imaging timing information, and the distance information before estimation by the estimation unit, and outputs an estimation failure signal if the three-dimensional position and orientation could not be estimated. (Note 4) The environmental map creation system according to Note 3, wherein the co-viewing relationship generation unit outputs the environmental image information for which the three-dimensional position and orientation could not be estimated. (Note 5) The environmental map creation system according to any one of Notes 1 to 4, further comprising: a keyframe selection unit that selects a plurality of keyframes from the environmental image; and a keyframe removal unit that removes similar keyframes that have a similar relationship with a predetermined keyframe from the plurality of selected keyframes, wherein the estimation unit acquires feature points in the keyframes after the removal of the similar keyframes, and estimates the three-dimensional position and orientation by comparing the positions of common feature points in different keyframes among the acquired feature points based on the co-viewing relationship. (Note 6) The environment map creation system as described in Note 5, wherein the keyframe removal unit selects keyframes whose distance difference and angle difference between keyframes are less than a predetermined value as similar keyframes.(Note 7) The environment map creation system according to Note 5 or 6, wherein the keyframe removal unit removes the keyframe based on the number of similar keyframes the keyframe has. (Note 8) The environment map creation system according to any one of Notes 1 to 7, wherein the distance information is the depth information of pixels in the environment image. (Note 9) The environment map creation system according to any one of Notes 1 to 8, wherein the co-view relationship generation unit generates the co-view relationship using VSLAM (Visual Simultaneous Localization and Mapping). (Note 10) The environment map creation system according to any one of Notes 7 to 9, wherein the keyframe removal unit removes the keyframe with the largest number of similar keyframes. (Note 11) The environment map creation system according to Note 10, wherein the keyframe removal unit repeats the removal process until no more keyframes having similar keyframes are detected. (Note 12) An environmental map creation method comprising: a computer acquiring an environmental image captured by a camera and distance information between feature points in the environmental image and the camera; generating a constellation relationship between the environmental images based on the environmental image, the imaging timing information of the environmental image, and the distance information; acquiring feature points in the environmental image again and estimating the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the newly acquired feature points based on the constellation relationship. (Note 13) A program that causes a computer to perform the following steps: acquiring an environmental image captured by a camera and distance information between feature points in the environmental image and the camera; generating a constellation relationship between the environmental images based on the environmental image, the imaging timing information of the environmental image, and the distance information; acquiring feature points in the environmental image again and estimating the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the newly acquired feature points based on the constellation relationship.
[0074] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 11 that are subordinate to Appendice 1 may also be subordinate to Appendices 12 to 13 in the same manner as those described in Appendices 2 to 11. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.
[0075] This application claims priority based on Japanese Patent Application No. 2025-081807, filed on 15 May 2025, and incorporates all of its disclosures herein.
[0076] 1. Environmental map creation system 2. Environmental map creation system 3. Environmental map creation system 11. Acquisition unit 12. Co-view relationship generation unit 13. Estimation unit 21. Acquisition unit 22. Co-view relationship generation unit 23. Keyframe selection unit 24. Keyframe storage unit 25. Keyframe removal unit 26. Estimation unit 31. Processor 32. Memory
Claims
1. An environmental map creation system comprising: an acquisition unit that acquires an environmental image captured by a camera and distance information between feature points in the environmental image and the camera; a co-view relationship generation unit that generates a co-view relationship between the environmental images based on the environmental image, the imaging timing information of the environmental image, and the distance information; and an estimation unit that acquires feature points in the environmental image again and estimates the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the reacquired feature points based on the co-view relationship.
2. The environmental map creation system according to claim 1, wherein the co-visibility relationship generation unit generates a co-visibility graph as the co-visibility relationship, and the estimation unit estimates the three-dimensional position and orientation based on the co-visibility graph.
3. The environmental map creation system according to claim 1, wherein the co-view relationship generation unit estimates the three-dimensional position and orientation based on the environmental image, the imaging timing information, and the distance information, before estimation by the estimation unit, and outputs an estimation failure signal if the three-dimensional position and orientation could not be estimated.
4. The environmental map creation system according to claim 3, wherein the co-view relationship generation unit outputs the environmental image information for which the three-dimensional position and orientation could not be estimated.
5. The environmental map creation system according to any one of claims 1 to 4, further comprising: a keyframe selection unit that selects a plurality of keyframes from the environmental image; and a keyframe removal unit that removes similar keyframes that have a similar relationship with a predetermined keyframe from the plurality of selected keyframes, wherein the estimation unit acquires feature points in the keyframes after the removal of the similar keyframes, and estimates the three-dimensional position and orientation by comparing the positions of common feature points in different keyframes among the acquired feature points based on the concentric relationship.
6. The environmental map creation system according to claim 5, wherein the keyframe removal unit selects keyframes whose distance difference and angle difference between keyframes are less than a predetermined value as similar keyframes.
7. The environment map creation system according to claim 5, wherein the keyframe removal unit removes the keyframe based on the number of similar keyframes that the keyframe has.
8. The environmental map creation system according to claim 1, wherein the distance information is the depth information of pixels in the environmental image.
9. The environmental map creation system according to claim 1, wherein the co-viewing relationship generation unit generates the co-viewing relationship using VSLAM (Visual Simultaneous Localization and Mapping).
10. The environment map creation system according to claim 7, wherein the keyframe removal unit removes the keyframe with the largest number of similar keyframes.
11. The environmental map creation system according to claim 10, wherein the keyframe removal unit repeats the removal process until no more keyframes having similar keyframes are detected.
12. An environmental map creation method comprising: a computer acquiring an environmental image captured by a camera and distance information between feature points in the environmental image and the camera; generating a conspicuous relationship between the environmental images based on the environmental image, the imaging timing information of the environmental image, and the distance information; acquiring feature points in the environmental images again; and estimating the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the reacquired feature points based on the conspicuous relationship.
13. A program that causes a computer to perform the following steps: acquiring an environmental image captured by a camera and distance information between feature points in the environmental image and the camera; generating a concentric relationship between the environmental images based on the environmental image, the imaging timing information of the environmental image, and the distance information; and acquiring feature points in the environmental image again and estimating the three-dimensional position and orientation of the camera by comparing the positions of common feature points in different environmental images among the newly acquired feature points based on the concentric relationship.
