Facility recognition system and facility recognition method

Abstract

To provide a facility recognition system capable of precisely recognizing a facility by focusing on a specific facility.SOLUTION: The facility recognition system includes: an input unit that receives a series of panorama images picked up by a 360-degree camera as the input; a camera position and orientation estimation unit that estimates the camera position and orientation of the 360-degree camera based on the panorama image series; a facility image generation unit that generates a series of facility image of the facility images relevant to the facility based on the series of panorama images; and a facility model estimation unit that estimates a facility model of the facility by generating a 3D facility point cloud relevant to the facility based on the series of facility images and the camera position and orientation.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a facility recognition system and a facility recognition method. [Background technology]

[0002] Mobile mapping systems (MMS) are known that are equipped with measuring devices such as laser scanners, digital cameras, and GPS on vehicles and collect three-dimensional shapes of the road surroundings in the form of 3D point clouds while driving. For example, in Patent Document 1, a facility is recognized by matching a 3D point cloud acquired by a laser scanner with a 3D model (three-dimensional model) of the facility.

[0003] Meanwhile, there is known a technique for grasping the status of facilities around a road, such as utility poles, by using images captured by a camera. For example, in Patent Document 2, the position and orientation of a camera are estimated from the correspondence between feature points of an image captured by a camera and a wide-area image. The technology for estimating camera position and orientation from multi-view images is known as SfM (Structure from Motion), and the technology for generating a 3D point cloud from multi-view images is known as MVS (Multi-View Stereo). These technologies are explained in, for example, Non-Patent Document 1 and Non-Patent Document 2. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] WO2014-155715 publication [Patent Document 2] JP 2016-019053 A [Non-patent literature]

[0005] [Non-Patent Document 1] Onur Ozyesil, Vladislav Voroninski, Ronen Basri, Amit Singer, “A Survey of Structure from Motion”, Computer Vision and Pattern Recognition, 2017, arXiv:1701.08493v2[cs.CV]. [Non-Patent Document 2] SMSeitz, B. Curless, J. Diebel, D. Scharstein and R. Szeliski, “A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms,” 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), 2006, pp. 519-528, doi:10.1109 / CVPR.2006.19. Summary of the Invention [Problem to be solved by the invention]

[0006] In Patent Document 1, a highly accurate 3D point cloud obtained by a laser scanner is used. This makes it possible to match with a 3D model of the facility and recognize the facility. On the other hand, there is a need to recognize facilities using a camera without using a laser scanner.

[0007] In Patent Document 2, it is possible to visually confirm a specific facility on an image, but it is difficult to recognize the facility with high accuracy as three-dimensional information.

[0008] It is possible to generate a 3D point cloud from multi-view images using SfM and MVS as described in Non-Patent Document 1 and Non-Patent Document 2. However, unlike a highly accurate 3D point cloud acquired by a laser scanner, it is difficult to apply matching with a 3D model of the facility.

[0009] An object of the present invention is to enable a facility recognition system to recognize a facility with high accuracy by focusing on a specific facility. [Means for solving the problem]

[0010] An equipment recognition system of one embodiment of the present invention is characterized in having an input unit that accepts as input a series of panoramic images captured by a 360-degree camera, a camera position and attitude estimation unit that estimates a camera position and attitude of the 360-degree camera based on the series of panoramic images, an equipment image generation unit that generates an equipment image series of equipment images related to equipment based on the series of panoramic images, and an equipment model estimation unit that generates a three-dimensional equipment point cloud for the equipment based on the equipment image series and the camera position and attitude, and estimates an equipment model of the equipment.

[0011] An equipment recognition method of one embodiment of the present invention is characterized by comprising an input step of accepting as input a series of panoramic images captured by a 360-degree camera; a camera position and attitude estimation step of estimating a camera position and attitude of the 360-degree camera based on the series of panoramic images; an equipment image generation step of generating an equipment image series of equipment images relating to equipment based on the series of panoramic images; and an equipment model estimation step of generating a three-dimensional equipment point cloud for the equipment based on the equipment image series and the camera position and attitude, and estimating an equipment model of the equipment. Effect of the Invention

[0012] According to one aspect of the present invention, in an equipment recognition system, by focusing on specific equipment, it is possible to recognize the equipment with high accuracy. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a configuration diagram of a facility recognition system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a process flow of the facility recognition system in the first embodiment. [Diagram 3] FIG. 13 is a diagram showing a processing flow for estimating the position and orientation of a camera using a series of panoramic images or a series of equipment images as input. [Figure 4] FIG. 11 is a diagram illustrating an example of a determination flow for determining whether or not to employ individual features in camera position and orientation estimation. [Figure 5A] FIG. 4 is a diagram illustrating an example of an input image. [Figure 5B] FIG. 13 is a diagram showing a point cloud resulting from feature extraction corresponding to an input image. [Figure 5C] FIG. 13 is a diagram showing a point cloud of features determined to have low density. [Figure 5D] FIG. 13 is a diagram showing a point cloud of features determined to have high density. [Figure 6] FIG. 13 is a diagram illustrating an example of a facility selection screen. [Figure 7] FIG. 2 is a diagram showing an example of a data format of a panoramic image. [Figure 8A] FIG. 13 is a diagram showing an example of a screen configuration displaying a list of equipment image series. [Figure 8B] FIG. 13 is a diagram showing an example of a screen configuration on which a map of camera placement is displayed. [Figure 9] FIG. 13 is a diagram showing a process flow for generating 3D point clouds and estimating an equipment model. [Figure 10A] 11 is a diagram showing an example of the results of edge extraction performed on an equipment image. FIG. [Figure 10B] FIG. 13 is a diagram illustrating an example of utility pole coordinate estimation. [Figure 11A] FIG. 13 shows a 3D display of a panoramic image of a point cloud when the present invention is not applied. [Figure 11B] FIG. 13 is a diagram showing a 3D display of a point cloud equipment image when the present invention is applied. [Figure 12] FIG. 11 is a configuration diagram of a facility recognition system according to a second embodiment. [Figure 13] FIG. 11 is a configuration diagram of a facility recognition system according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] In conventional SfM, which simultaneously estimates the camera position and orientation and the 3D reconstruction of the target object from multi-viewpoint images, the generated 3D point cloud has a large position error, making it difficult to recognize the facility with high accuracy. Therefore, in this invention, by focusing on a specific facility, the position estimation accuracy of the 3D point cloud is locally improved, and the facility is recognized with high accuracy and a 3D model of the facility is estimated.

[0015] Hereinafter, the embodiments will be described with reference to the drawings. EXAMPLES

[0016] A configuration of a facility recognition system according to a first embodiment will be described with reference to FIG. The facility recognition system 100 of the first embodiment has a processing unit 110 and a memory 120. The processing unit 110 has an input unit 111, a camera position and orientation estimation unit 112, a facility selection unit 113, a facility image generation unit 114, and a facility model estimation unit 115. A display device 140 such as a display and an input device 141 such as a mouse and keyboard are connected to the processing unit 110.

[0017] The memory 120 stores a panoramic image sequence 121 , setting parameters 122 , camera position and orientation 123 , track information 124 , an equipment image sequence 125 , and equipment model parameters 126 .

[0018] A series of panoramic images 130 captured by a 360-degree camera and setting parameters 131 are input to the input unit 11. Here, the setting parameters 131 are not essential, but may be input when they are used in internal processing.

[0019] Here, a 360-degree camera is a camera that can capture images in all directions (up, down, left, right, etc.) at once. For example, a 360-degree camera can capture images with multiple lenses and stitch them together to create an image of the entire 360-degree view, making it possible to create a panoramic image in a single shot.

[0020] The camera position and orientation estimation unit 112 generates track information 124 from the panoramic image sequence 130 by feature matching, and estimates the relative relationship between the camera positions and orientations 123 of the 360-degree cameras that captured each panoramic image.

[0021] The facility selection unit 113 selects the target facility by some method, such as using a GUI (Graphical User Interface), etc. Here, the target facility refers to, for example, a utility pole.

[0022] The equipment image generation unit 114 generates an equipment image sequence 125 by extracting the target equipment from the panoramic image sequence 130. The equipment model estimation unit 115 generates a 3D point cloud around the target equipment by multi-viewpoint stereo matching between equipment images whose camera positions and orientations 123 are known, and estimates the equipment model (equipment model parameters 126). Here, stereo matching refers to a technique for obtaining three-dimensional information from two-dimensional image data.

[0023] A process flow of the facility recognition system 100 in the first embodiment will be described with reference to FIG. First, a panoramic image sequence 130 is input (step 201). When using the sequence in internal processing, setting parameters 131 may be input.

[0024] Next, the approximate positional relationship between the positions and orientations of the cameras that captured each panoramic image is estimated from the panoramic image sequence 130 (step 202). Next, the target facility is selected (step 203). Next, a series of equipment images with the target equipment at the center is generated from the panoramic image series 130 by planar projection transformation (step 204). Next, the camera position and orientation are re-estimated from the equipment image series (step 205). Here, the camera position of the equipment image and the camera position of the original panoramic image are the same, and there is a certain transformation relationship between the orientations. For this reason, the subsequent camera position and orientation estimation (S205) may be omitted. However, since the previous camera position and orientation estimation (S202) contains errors, it is expected that the accuracy will be improved by the subsequent camera position and orientation estimation (step 205).

[0025] Finally, a 3D point cloud around the target equipment is generated by stereo matching between equipment images with known camera positions and orientations, and an equipment model is estimated (step 206).

[0026] The above is processing for one piece of equipment, but by adding target equipment one by one, it is possible to estimate the positional relationship between equipment.

[0027] Referring to FIG. 3, a process flow for estimating the position and orientation of a camera using the panoramic image sequence 121 or the facility image sequence 125 as input is shown. This is the same process as for 3D reconstruction using conventional SfM.

[0028] First, for each image in the image sequence, local image features are detected in a general manner (step 301). Next, feature matching is performed for each image pair in the image sequence to find the correspondence of features (step 302). Although not specifically mentioned, general measures to reduce parallax error may be taken, such as excluding images with close camera positions from among the image pairs. Here, feature point matching refers to matching similar feature points between different images.

[0029] The feature correspondences for the image pairs are compiled to generate tracks (step 303). A track indicates the correspondences of features common to multiple images. Finally, the 3D coordinates of the features and the position and pose of each camera are estimated from the tracks (step 304).

[0030] FIG. 4 shows an example of a flow of determining whether or not to adopt each feature in camera position and orientation estimation. First, it is determined whether the object is not a target object (step 401). If the result of the determination is that the object is not a target object, the feature is deleted (step 402). If it is determined that the object is not a non-target object, it is determined whether the object is in the vicinity of the target facility (step 403). If it is determined that the area is in the vicinity of the target facility, the characteristics are registered (step 404). If it is determined that the area is not in the vicinity of the target facility, it is determined whether the feature density is smaller than a threshold value (step 401). If it is determined that the feature density is smaller than the threshold, the feature is registered (step 404). If it is determined that the feature density is not smaller than the threshold, the feature is deleted (step 402).

[0031] There are methods for estimating the camera position while eliminating features that are out of correspondence, such as RANSAC (Random Sample Consensus). However, it is desirable to remove unnecessary features that may cause erroneous correspondence in advance. Therefore, as described above, it is determined whether or not to adopt each feature.

[0032] For example, if the feature is a moving object such as a vehicle, it is deleted because there is a possibility that there will be no correspondence between camera images taken at different times. On the other hand, if the feature is a target facility, it is registered.

[0033] Here, vehicles and equipment can be detected from images using general object recognition such as YOLO (You Only Look Once) (see, for example, J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You Only Look Once: Unified,Real-Time Object Detection,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 779-788, doi:10.1109 / CVPR.2016.91.).

[0034] On the other hand, areas with large changes in shading, such as trees, are easily detected as local image features and can lead to erroneous correspondence. Therefore, features in areas with high feature density are excluded as noise, and features in areas with low feature density are used. Feature density can be calculated as the number of feature points in the vicinity of a feature of interest, and the distance considered to be in the vicinity and the density threshold are each specified by setting parameters 131 (see Figure 1). Note that features of the target equipment should not be limited to local image features. For example, the top and bottom of an area detected by general object recognition may be added as features.

[0035] FIG. 5 shows an example of classifying a point cloud based on feature density. Fig. 5A is a diagram showing an example of an input image. Fig. 5B is a diagram showing a point cloud of a feature extraction result corresponding to the input image. Fig. 5C is a diagram showing a point cloud of a feature determined to have a low density. Fig. 5D is a diagram showing a point cloud of a feature determined to have a high density.

[0036] FIG. 6 shows an example of a facility selection screen used by the facility selection unit 113. As shown in FIG. An example of the case where the user is prompted to specify a facility via a pointer such as a mouse, which is the input device 14, will be described with reference to FIG.

[0037] A coarse primary 3D point cloud is generated by processing similar to that of conventional SfM. Figure 6 shows a screen in which this primary 3D point cloud is displayed in 3D on a display device 140. As with normal 3D displays, it is possible to zoom in / out, scroll, rotate, etc. by operating the input device 141 such as a mouse and keyboard.

[0038] The user designates a point on the target facility with a pointer such as a mouse on the screen displaying this primary 3D point cloud on the display device 140. Inside, as with normal 3D display, it is possible to obtain the 3D coordinates of the corresponding point in the 3D point cloud by performing a neighborhood search from the point 600 designated on the screen. For example, a 3D shape with this point 600 as the center is set as the facility area 610. The 3D shape of the facility area 610 can be a rectangular parallelepiped, a cylinder, a sphere, etc., and can be selected depending on the shape of the target facility. FIG. 6 shows an example in which a cylinder is selected.

[0039] 7 shows an example of a data format of a panoramic image, in which equirectangular projection is used. In Fig. 7, (a) shows an orthogonal coordinate system, (b) shows a unit sphere showing all directions, and (c) shows a panoramic image (equirectangular projection). Points on the unit sphere in (b) are expressed by longitude and latitude, and can be converted to and from the orthogonal coordinate system in (a) using the conversion formula shown in the figure. The panoramic image in (c) is an image in which points on a sphere are projected onto a cylinder, and can be converted to and from the longitude and latitude in (b) using the conversion formula shown in the figure.

[0040] The equipment image sequence 125 generated from the panoramic image sequence 121 is a planar projection image when the equipment is viewed from each 360-degree camera. A planar projection image when an arbitrary direction is viewed from a 360-degree camera is obtained by projecting points on a sphere onto a plane perpendicular to the optical axis of the camera. This is similar to a normal perspective projection transformation, so a description is omitted here.

[0041] FIG. 8 shows an example of the screen configuration for generating a series of facility images. Fig. 8A is a diagram showing an example of a screen configuration that displays a list of equipment image series. Fig. 8B is a diagram showing an example of a screen configuration that displays a map of camera placement. Fig. 8B shows an overhead view of the target equipment area and the placement of multiple cameras 1 to 5, and like a normal 3D display, it is possible to zoom in and out, scroll, rotate, etc. by operating the input device 141 such as a mouse and keyboard.

[0042] FIG. 8A shows a list of equipment image series 125 taken by 360-degree cameras 1 to 4 that are close to the target equipment. The list display of FIG. 8A and the map display of FIG. 8B are displayed on a display device 140 such as a display. The list display of FIG. 8A and the map display of FIG. 8B may be arranged separately on one screen, either vertically or horizontally, so that they can be linked to each other, for example, by changing the display color of the corresponding camera in FIG. 8B when an equipment image in FIG. 8A is selected. Furthermore, parameters such as the angle of view and the distance from the equipment may be adjusted in real time using a slider or the like.

[0043] FIG. 9 shows the process flow for 3D point cloud generation and facility model estimation. First, an image pair list is created from the equipment image sequence 125 (step 901). In order to reduce occlusion, measures such as finding the angles between the equipment and the two cameras and excluding image pairs with large angles may be taken.

[0044] Next, a depth map is generated for each image pair by stereo matching (step 902). Here, a depth map is a representation of the distance (depth) to an object using grayscale shading to represent the depth of a three-dimensional space. Next, the depth map is converted to a point cloud, and the point clouds from each image pair are merged (step 903). Finally, the point cloud and the equipment model are aligned to estimate equipment model parameters such as position and size (step 904).

[0045] For example, in the case of a utility pole, the equipment model can be represented by a cylindrical model. The parameters of the cylindrical model are represented by, for example, the central coordinates of the bottom surface of the cylinder, the direction of the central axis, the radius, and the height. The parameters of the cylindrical model can be estimated by an algorithm such as RANSAC.

[0046] The equipment model estimation unit 115 estimates an equipment model represented by a catenary curve (or a quadratic curve approximation of the catenary curve) based on a 3D point cloud that is a candidate for equipment, and outputs parameters of the equipment model. The model estimation performed by the equipment model estimation unit 115 is based on a general model estimation method such as RANSAC.

[0047] Figure 10 shows the improvement measures for the example of utility pole model estimation. As explained above, the angles between the equipment and the two cameras are calculated, and image pairs with large angles are excluded. This is because stereo matching requires a high overlap rate. We will explain a method for directly estimating equipment model parameters using such image pairs, without generating a point cloud using stereo matching.

[0048] Fig. 10A is a diagram showing an example of a result of performing edge extraction on an equipment image, and Fig. 10B is a diagram showing an example of utility pole coordinate estimation.

[0049] As shown in Fig. 10A, the contour of the utility pole is extracted by extracting long vertical parallel lines near the center of the image from the edge extraction results. Furthermore, the center line of the parallel lines is obtained to estimate the center line of the utility pole. From the results of obtaining the utility pole center lines for multiple facility images, the utility pole coordinates (X, Y), which are the map coordinates of the utility pole center, are estimated as shown in Fig. 10B.

[0050] The map coordinates of the utility pole center obtained here are used in facility model estimation. If the diameter of the utility pole is known from facility information, it can be set as a setting parameter. Here, it can be used to extract parallel lines whose spacing is within the range of this setting parameter.

[0051] FIG. 11 shows the effect of the present invention. Fig. 11A is a diagram showing a 3D display of a panoramic image of a point cloud when the present invention is not applied, and Fig. 11B is a diagram showing a 3D display of an equipment image of a point cloud when the present invention is applied.

[0052] In this way, in the present invention, by focusing on a specific facility, the accuracy of estimating the position of the 3D point cloud can be locally increased, thereby enabling facility recognition with high accuracy and estimating a 3D model of the facility (e.g., a utility pole).

[0053] It is possible that the equipment positions in the estimated results may be shifted due to the subsequent camera position and orientation estimation (step 205 in FIG. 2). The present invention is capable of generating a point cloud with high accuracy locally, but when viewed as a whole, it is not expected to accurately determine the positional relationship between equipment. Therefore, to estimate the positional relationship between equipment, the results of the previous camera position and orientation estimation (step 202 in FIG. 2) are used, and the point clouds are arranged so that their center coordinates and spread are consistent. EXAMPLES

[0054] Second embodiment The configuration of a facility recognition system 200 according to a second embodiment will be described with reference to FIG. The configuration of the facility identification system 200 of the second embodiment differs from the configuration of the facility identification system 100 of the first embodiment shown in Fig. 1 in that GPS data 132 is input to the facility identification system 200 of the second embodiment. The other configurations are the same as those of the facility identification system 100 of the first embodiment shown in Fig. 1, so a description thereof will be omitted.

[0055] The facility recognition system 200 of the second embodiment is assumed to be equipped with a 360-degree camera and a GPS on a measurement vehicle (for example, an automobile) and to capture images at regular intervals while traveling. GPS data 132 is input to an input unit 111. Then, a camera position and orientation estimation unit 112 uses the GPS data 132 as the initial position of the camera. EXAMPLES

[0056] Second embodiment The configuration of a facility recognition system 300 according to a second embodiment will be described with reference to FIG. The configuration of the facility recognition system 300 of the third embodiment differs from the configuration of the facility recognition system 200 of the second embodiment shown in Fig. 12 in that facility data 133 is input to the facility recognition system 300 of the third embodiment. The other configurations are the same as those of the facility recognition system 200 of the second embodiment shown in Fig. 12, so a description thereof will be omitted.

[0057] The facility recognition system 300 of the third embodiment is assumed to use facility data 133 stored in a facility management DB. The facility selection unit 113 selects a facility on a screen that displays the location of the facility data 133 as a map on the display device 140, and the facility image generation unit 114 uses the location of the facility data 133.

[0058] The facility management DB stores pre-registered management data for utility poles, etc. The management data for utility poles includes information such as the management number, type (model number, diameter, height), and installation location (address, coordinates), for example.

[0059] According to the above embodiment, by focusing on a specific piece of equipment, the position estimation accuracy of the 3D point cloud can be locally improved, thereby enabling equipment recognition with high accuracy and estimating a 3D model of the equipment. [Explanation of symbols]

[0060] 100 Equipment Identification System 110 Processing section 111 Input section 112 Camera position and orientation estimation unit 113 Equipment Selection Department 114 Equipment Image Generation Unit 115 Equipment Model Estimation Unit 120 Memory 140 Display device 141 Input Devices

Claims

1. an input unit that receives as an input a sequence of panoramic images captured by a 360-degree camera; a camera position and orientation estimation unit that estimates a camera position and orientation of the 360-degree camera based on the sequence of panoramic images; an equipment image generation unit that generates an equipment image series of equipment images relating to the equipment based on the panoramic image series; an equipment model estimation unit that generates a three-dimensional equipment point cloud for the equipment based on the equipment image series and the camera position and orientation, and estimates an equipment model of the equipment; An equipment recognition system comprising:

2. Further comprising an input device, a display device, and a facility selection unit, The camera position and orientation estimation unit generating a three-dimensional panoramic point cloud from the sequence of panoramic images based on the camera position and orientation, and displaying the generated three-dimensional panoramic point cloud on the display device; The facility selection unit is setting an equipment area in the vicinity of a point designated by the input device based on the three-dimensional panoramic point cloud displayed on the display device; The equipment image generating unit includes: The equipment recognition system according to claim 1 , wherein the equipment image sequence is generated using the equipment area.

3. The facility selection unit is 3. The facility recognition system according to claim 2, wherein the facility area is set to a three-dimensional shape corresponding to a shape of the facility.

4. The camera position and orientation estimation unit Detecting features of the sequence of panoramic images; Generate track information through feature matching; calculating a feature density of the detected features; The facility recognition system according to claim 1 , wherein when the feature density is lower than a predetermined threshold, the feature density is used for the feature matching to estimate the camera position and orientation.

5. The camera position and orientation estimation unit Detecting the facility from the sequence of panoramic images; The facility recognition system according to claim 4, wherein the feature matching is performed using the features in the vicinity of the facility.

6. The camera position and orientation estimation unit Detecting outliers from the sequence of panoramic images; The facility recognition system according to claim 4, wherein the feature matching is performed by deleting features of the non-target objects.

7. The equipment image generating unit includes: selecting a plurality of the 360-degree cameras that are disposed at a predetermined distance from the three-dimensional coordinates of the facility based on the camera positions and orientations; The equipment recognition system according to claim 1, characterized in that the equipment image series is generated by creating an equipment image centered on the equipment from the panoramic image corresponding to each of the selected 360-degree cameras.

8. The equipment image generating unit includes: The facility recognition system according to claim 7, characterized in that the facility image is created by planar projection transformation using an angle of view as a parameter.

9. The equipment model estimation unit is generating a plurality of image pairs from the sequence of facility images; converting each of the plurality of image pairs into an image pair 3D point cloud; The equipment recognition system according to claim 1 , wherein the image pair 3D point cloud is integrated to generate the 3D equipment point cloud.

10. The equipment model estimation unit is determining an angle between the equipment and the image pair for each of the image pairs; The facility recognition system according to claim 9, characterized in that the image pairs whose angle is greater than a predetermined threshold are excluded, a depth map is generated by stereo matching, and the image pairs are converted into a three-dimensional point cloud.

11. The equipment model estimation unit is determining an angle between the equipment and the image pair for each of the image pairs; For each of the equipment images corresponding to the image pair in which the angle is greater than a predetermined threshold, edge extraction is performed to extract parallel lines and obtain a center line; Estimating corresponding fixture coordinates between the image pairs on a plane perpendicular to the centerline; The equipment model estimation unit is The facility recognition system according to claim 9, wherein the facility coordinates are used to generate the three-dimensional facility point cloud and estimate a model of the facility.

12. The input unit includes: inputting GPS data of the 360 ​​degree camera corresponding to the sequence of panoramic images; The camera position and orientation estimation unit 2. The facility recognition system according to claim 1, wherein the camera position and orientation are estimated using the GPS data.

13. The input unit includes: Further input the specified equipment data, The camera position and orientation estimation unit The facility recognition system according to claim 12, wherein the camera position and orientation are estimated using the GPS data and the facility data.

14. Further comprising an input device, a display device, and a facility selection unit, The display device includes: Displaying the location of the equipment data on a map on a screen; The facility selection unit is Setting a position of the equipment data displayed on the screen of the display device and instructed from the input device; The equipment image generating unit includes: The facility recognition system according to claim 13, characterized in that the facility image sequence is generated using the location of the facility data.

15. an input step of accepting as an input a sequence of panoramic images captured by a 360-degree camera; a camera position and orientation estimation step of estimating a camera position and orientation of the 360-degree camera based on the sequence of panoramic images; an equipment image generation step of generating an equipment image series of equipment images relating to the equipment based on the panoramic image series; an equipment model estimation step of generating a three-dimensional equipment point cloud for the equipment based on the equipment image series and the camera position and orientation, and estimating an equipment model of the equipment; 13. An equipment recognition method comprising:

Images