Point Cloud Data Management System

Abstract

To provide a system capable of efficiently managing three-dimensional point cloud data.SOLUTION: Three-dimensional point cloud data acquisition means 22 of a terminal device 20 acquires outline region data of a desired feature from an attribute data server device 60. The three-dimensional point cloud data acquisition means 22 accesses a first point cloud server device 40a on the basis of the acquired outline region data of the feature, and acquires three-dimensional point cloud data included in the region as first three-dimensional point cloud data. Further, the three-dimensional point cloud data acquisition means 22 accesses a second point cloud server device 40b on the basis of the acquired outline region data of the feature, and acquires three-dimensional point cloud data included in the region as second three-dimensional point cloud data. Three-dimensional mode construction means 24 constructs a three-dimensional shape of the feature on the basis of the acquired first three-dimensional point cloud data and the second three-dimensional point cloud data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system for managing point cloud data obtained by measuring features. [Background technology]

[0002] Laser surveying devices are mounted on unmanned aerial vehicles (UAVs) and automobiles to measure the shape of the earth's surface and features. For example, in an MMS (Mobile Mapping System) using an automobile, a GPS receiver, video camera, IMU, and laser scanner are mounted on the roof of the automobile, and measurements are taken while the automobile is traveling along roads, etc. The video camera can be either analog or digital.

[0003] These measurements allow for the acquisition of 3D point cloud data that shows the earth's surface and the outlines of features. Each 3D point cloud data is assigned a position using absolute coordinates (plane rectangular coordinates, latitude, longitude, altitude, etc.) based on the position obtained by GPS and the scanning direction and distance of the laser scanner. In addition, video data (still image data) is generated by capturing images of the earth's surface and features using a video capture camera. This video data is linked to and recorded with the constantly changing image capture position.

[0004] Obtaining three-dimensional point cloud data allows us to understand the current state of the earth's surface and features, which can be useful for maintenance, map creation, and other purposes.

[0005] The inventors propose that attribute data including data indicating the area in absolute coordinates and a name be assigned to features in three-dimensional point cloud data. Summary of the Invention [Problem to be solved by the invention]

[0006] However, while the above attribute data has led to progress in standardization and the accumulation of data, there is a demand for a system that can efficiently manage three-dimensional point cloud data when using the attribute data for maintenance, etc.

[0007] An object of the present invention is to solve the above problems and to provide a system that can efficiently manage three-dimensional point cloud data. [Means for solving the problem]

[0008] Independently applicable features of the present invention are listed below.

[0009] (1)(2)(3) A point cloud data management system according to the present invention is a point cloud data management system comprising first and second point cloud data server devices, an attribute data server device, and a plurality of terminal devices capable of communicating with these server devices, wherein the first point cloud server device records a first three-dimensional point cloud data file from a first direction using absolute coordinates including a feature, the second point cloud server device records a second three-dimensional point cloud data file from a second direction using absolute coordinates including the same feature as the feature, and the attribute data server device records a three-dimensional point cloud data file indicated by the absolute coordinates that surrounds the outline of the feature. The terminal device has three-dimensional point cloud data acquisition means for acquiring the outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device, and three-dimensional model construction means for constructing the three-dimensional shape of the feature based on the acquired first and second three-dimensional point cloud data from the first and second directions.

[0010] Therefore, it is possible to easily search and acquire three-dimensional point cloud data of the same feature from multiple different directions, and to construct a more complete three-dimensional model.

[0011] (4) (5) (6) A point cloud data management system according to the present invention is a point cloud data management system comprising first and second point cloud data server devices, an attribute data server device, and a plurality of terminal devices capable of communicating with these server devices, wherein the first point cloud server device records a first three-dimensional point cloud data file at a first date and time based on absolute coordinates including a feature, the second point cloud server device records a second three-dimensional point cloud data file at a second date and time based on absolute coordinates including the same feature as the feature, and the attribute data server device records a three-dimensional point cloud data file indicated by absolute coordinates that surrounds the outline of the feature. The terminal device has a three-dimensional point cloud data acquisition means for acquiring the outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device, and a time-lapse change acquisition means for acquiring a time-lapse change of the feature based on the acquired first and second three-dimensional point cloud data at the first and second dates and times.

[0012] Therefore, it is possible to easily obtain three-dimensional point cloud data for the same feature measured at different dates and times, and to obtain the secular changes of the feature.

[0013] (7) A point cloud data management system according to the present invention is characterized in that the first point cloud server device and the second point cloud server device are constructed as a single point cloud server device.

[0014] (8) In the point cloud data management system according to the present invention, the attribute data server device is a server device constructed integrally with the point cloud server device.

[0015] (9) A point cloud data management method according to the present invention includes the steps of: recording a first three-dimensional point cloud data file from a first direction using absolute coordinates that includes a feature; recording a second three-dimensional point cloud data file from a second direction using absolute coordinates that includes the same feature as the feature; recording outline area data that is indicated by absolute coordinates and that surrounds the outline of the feature; acquiring the outline area data of the feature using a computer; acquiring first three-dimensional point cloud data of the feature corresponding to the first three-dimensional point cloud data file based on the outline area data; acquiring second three-dimensional point cloud data of the feature corresponding to the second three-dimensional point cloud data file; and obtaining synthesized three-dimensional point cloud data based on the acquired first and second three-dimensional point cloud data from the first and second directions using a computer.

[0016] Therefore, it is possible to easily search and acquire three-dimensional point cloud data of the same feature from a plurality of different directions, thereby obtaining three-dimensional point cloud data with higher accuracy.

[0017] (10) A three-dimensional point cloud data management method according to the present invention is characterized in that it includes the steps of: recording a first three-dimensional point cloud data file at a first date and time using absolute coordinates that includes a feature; recording a second three-dimensional point cloud data file at a second date and time using absolute coordinates that includes the same feature as the feature; recording outline area data that surrounds the outline of the feature and is indicated by absolute coordinates; acquiring the outline area data of the feature using a computer; acquiring first three-dimensional point cloud data of the feature contained in the first three-dimensional point cloud data file based on the outline area data; acquiring second three-dimensional point cloud data of the feature contained in the second three-dimensional point cloud data file; and acquiring changes over time of the feature using the first and second three-dimensional point cloud data acquired at the first and second dates and times.

[0018] Therefore, it is possible to easily search and acquire three-dimensional point cloud data for the same feature at different dates and times, and to acquire changes over time in the feature.

[0019] (11)(12)(13)(14)(15) The feature search system according to the present invention is a feature search system comprising a point cloud data server device, an attribute data server device and a plurality of terminal devices capable of communicating with these server devices, wherein the server device comprises a recording unit that records attribute data including video data with image capture position information attached, in which the features are captured when three-dimensional point cloud data of the plurality of features is measured while moving, and outline area data surrounding the outlines of the features; an attribute data transmission unit that transmits the attribute data of the plurality of features to the terminal device for feature selection by a user; and a data transmission unit that receives feature selection information from the terminal device and transmits the attribute data of the plurality of features to the terminal device. The terminal device is equipped with a feature position acquisition means for acquiring the feature position corresponding to the outer area data included in the attribute data of the selected feature, a video data extraction means for extracting from the video data a portion attached with imaging position information near the feature position and transmitting the extracted portion to a terminal device, a feature attribute data display means for acquiring the feature attribute data from a server device and displaying the data on a display unit for the user to select a feature, a selection information transmission means for transmitting information on the feature selection made by the user to the server device, and a video data playback means for receiving the extracted video data transmitted from the server device and playing the data on a display unit.

[0020] Therefore, features captured in the video can be easily searched for and displayed.

[0021] (16) The feature search system of the present invention is characterized in that the position of the outer shape area data is indicated by absolute coordinates, and the video data extraction means extracts portions of the video data that are attached with imaging position information near the feature position, and transmits the extracted video data.

[0022] Therefore, if there is one piece of contour area data, it is possible to search for features in multiple pieces of video data.

[0023] (17) The feature search system of the present invention is characterized in that the video data extraction means acquires three-dimensional point cloud data of the selected feature, and includes a video feature identification means for identifying and marking the corresponding feature in the extracted video based on at least the contour of the three-dimensional point cloud data.

[0024] Therefore, it becomes easy to find features in the video.

[0025] (18) A feature search system according to the present invention is characterized in that the server device is configured to include a plurality of individual server devices, and the video data or the attribute data is recorded in different individual server devices.

[0026] Therefore, it is possible to search for features from video data scattered across different individual server devices.

[0027] (19) A feature search system according to the present invention is characterized in that it records video data accompanied by imaging position information of a plurality of features captured while measuring three-dimensional point cloud data of the features while moving, and attribute data including outline area data surrounding the outline of the features, and a computer displays the attribute data of the plurality of features for a user to select a feature, and upon receiving the selection, the computer obtains the feature position corresponding to the outline area data included in the attribute data of the selected feature, and the computer extracts and displays a portion of the video data accompanied by imaging position information near the feature position.

[0028] Therefore, features captured in the video can be easily searched for and displayed.

[0029] (20)(21) The contour area data generating device of this invention comprises a planar contour determining means for determining a planar contour based on a plan view of a feature having absolute coordinates, a means for determining the lower and upper ends in the elevation direction based on three-dimensional point cloud data of the feature, and a three-dimensional contour determining means for determining the three-dimensional contour when the planar contour is continued from the lower end to the upper end as contour area data.

[0030] Therefore, it is possible to generate outline area data of features easily and accurately.

[0031] In the embodiment, step S13 corresponds to the "three-dimensional point cloud data acquisition means."

[0032] In the embodiment, step S15 corresponds to the "three-dimensional model construction means."

[0033] In the embodiment, step S18 corresponds to the "means for acquiring change over time."

[0034] In the embodiment, step S51 corresponds to the "attribute data transmitting means."

[0035] In the embodiment, step S52 corresponds to the "fun feature position acquisition means."

[0036] In the embodiment, step S54 corresponds to the "moving image data extraction means."

[0037] In the embodiment, step S12 corresponds to the "facility attribute data display means."

[0038] In the embodiment, step S19 corresponds to the "selection information transmitting means."

[0039] In this embodiment, step S20 corresponds to the "video data playback means." The term "program" is a concept that includes not only programs that can be executed directly by a CPU, but also source-format programs, compressed programs, encrypted programs, and the like. [Brief explanation of the drawings]

[0040] [Figure 1] 1 is a functional block diagram of a point cloud data management system according to a first embodiment of the present invention. [Figure 2] This is the system configuration of a point cloud data management system. [Figure 3] This is the hardware configuration of the terminal device. [Figure 4] 10 is a flowchart of a terminal program. [Figure 5] 1 is an example of three-dimensional point cloud data. [Figure 6] FIG. 1 illustrates a process for detecting ground points. [Figure 7] This is an example of design data for a feature. [Figure 8] FIG. 10 is a diagram showing three-dimensional point cloud data superimposed on a plan view of a feature. [Figure 9] This is an example of 3D point cloud data of features. [Figure 10] 10 shows example items of attribute data. [Figure 11] 10A and 10B are diagrams for explaining a process of estimating a feature name based on three-dimensional point cloud data of the feature. [Figure 12] 1 is a flowchart of acquiring 3D point cloud data of features and generating a 3D model. [Figure 13] 1 is a flowchart of acquiring 3D point cloud data of features and generating a 3D model. [Figure 14] 10 is a display example of the outer shape of a feature. [Figure 15] FIG. 15A is first three-dimensional point cloud data of a feature from a first direction, and FIG. 15B is second three-dimensional point cloud data of a feature from a second direction. [Figure 16] This is a composite 3D point cloud data. [Figure 17] Figure 17A shows the generated three-dimensional model, and Figure 17B shows an example of the measurement direction. [Figure 18] FIG. 18A shows the first three-dimensional point cloud data, FIG. 18B shows the second three-dimensional point cloud data, and FIG. 18C shows the combined three-dimensional point cloud data. [Figure 19] FIG. 10 is a functional block diagram of a point cloud data management system according to a second embodiment. [Figure 20] 10 is a flowchart for acquiring a change over time. [Figure 21] 10 is a flowchart for acquiring a change over time. [Figure 22] This is an example of how features change over time. [Figure 23] 10 is an example of a screen prompting the user to select three-dimensional point cloud data from different dates and times. [Figure 24] FIG. 10 is a functional block diagram of a feature search system according to a third embodiment. [Figure 25] This is the system configuration of the feature search system. [Figure 26] 10 is a flowchart of a feature search. [Figure 27] FIG. 2 is a diagram showing the structure of video data. [Figure 28] FIG. 2 is a diagram showing an imaging area. [Figure 29] 10 is a flowchart of video extraction. [Figure 30] 1 is a diagram showing the relationship between an imaging trajectory 200 and a feature 106 at each time. [Figure 31] FIG. 10 is a diagram showing an image in which search target features are marked. DETAILED DESCRIPTION OF THE INVENTION

[0041] 1. First embodiment 1.1 Overall structure Figure 1 shows the overall configuration of a point cloud data management system according to one embodiment of the present invention. A first point cloud server device 40a stores a first three-dimensional point cloud data file from a first direction using absolute coordinates that include features. A second point cloud server device 40b stores a second three-dimensional point cloud data file from a second direction using absolute coordinates that include features. An attribute data server device 60 stores outline area data using absolute coordinates that surround the outlines of features.

[0042] Here, features include, for example, road design standard information (mileposts, road centerlines, measurement points, etc.), road surface features (roadway sections, sidewalk sections, trackbeds, etc.), features that share an area with the road surface (stop lines, dividing lines, pedestrian crossings, road markings, etc.), features other than the road surface (signposts, lighting poles, traffic lights, footbridges, etc.), and features that indicate the road surface (retaining walls, bridges, slopes, tunnels, sheds, etc.). Note that this is not limited to road features.

[0043] The terminal device 20 is capable of communicating with these server devices. The three-dimensional point cloud data acquisition means 22 of the terminal device 20 acquires the outline area data of a desired feature from the attribute data server device 60. The three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40a based on the acquired outline area data of the feature, and acquires the three-dimensional point cloud data included in the area as first three-dimensional point cloud data. Furthermore, the three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40b based on the acquired outline area data of the feature, and acquires the three-dimensional point cloud data included in the area as second three-dimensional point cloud data.

[0044] The three-dimensional model construction means 24 constructs the three-dimensional shape of the feature based on the acquired first three-dimensional point cloud data and second three-dimensional point cloud data. In this way, by taking advantage of the fact that the outline area data is assigned absolute coordinates, it is possible to collect three-dimensional point cloud data from different directions of the target feature that is recorded in a distributed manner on the Internet, etc., and therefore it is possible to form a more complete three-dimensional model of the feature.

[0045] 1.2 System Configuration Figure 2 shows the system configuration of a point cloud data management system according to one embodiment. Point cloud server devices 40a, 40b,..., 40n are provided on the Internet. An attribute data server device 60 is also provided. Terminal devices 20a, 20b,..., 20n are provided on these server devices 40a, 40b,..., 40n, 60 so as to be able to communicate via the Internet.

[0046] The hardware configuration of the terminal device 20 is shown in Figure 3. A CPU (or a GPU, etc., the same applies below) 80 is connected to a memory 82, a display 84, a hard disk 86, a DVD-ROM drive 88, a keyboard / mouse 90, and a communication circuit 92. The communication circuit 92 is for connecting to the Internet.

[0047] The hard disk 86 stores an operating system 94 and a terminal program 96. The terminal program 96 performs its functions in cooperation with the operating system 94. These programs were originally recorded on a DVD-ROM 98 and were installed onto the hard disk 86 via the DVD-ROM drive 88.

[0048] The point cloud server device 40 and the attribute data server device 60 have the same hardware configuration. However, in the point cloud server device 40, a point cloud server program is recorded instead of the terminal program 96. In addition, in the attribute data server device 60, an attribute data server program is recorded instead of the terminal program 96.

[0049] 1.3 Publishing point cloud data The surveying company generates three-dimensional point cloud data and videos of roads and features using an MMS or the like installed in a vehicle, etc. The generated three-dimensional point cloud data is recorded on a recording medium such as a DVD-ROM, and then imported into the hard disk 86 of the terminal device 20.

[0050] The surveying company generates attribute data based on the recorded three-dimensional point cloud data by starting the terminal program 96. A flowchart of the attribute data generation process in the terminal program 96 is shown in FIG.

[0051] The CPU 80 of the terminal device 20a first reads out the recorded three-dimensional point cloud data (step S1). An example of the three-dimensional point cloud data is shown in FIG. 5. In this embodiment, the three-dimensional point cloud data is obtained by measuring the color of the object to be measured and the laser reflection intensity. The three-dimensional point cloud data also includes information on the measurement method and the measurement equipment.

[0052] The CPU 80 groups points within a predetermined distance from each other in the three-dimensional point cloud data into clusters. This results in clusters of various sizes. The CPU 80 then removes clusters that are below a predetermined volume, a predetermined number of points, or a predetermined point density as noise (step S2).

[0053] Next, the CPU 80 extracts ground points (ground surface) using a method such as Cloth Simulation (step S2). In the ground point extraction, the CPU 80 first inverts the elevation values ​​of the three-dimensional point cloud data. For example, if there is three-dimensional point cloud data of a cross section as shown in Figure 6A (the point cloud is represented as a line in the figure), inverted three-dimensional point cloud data as shown in Figure 6B can be obtained.

[0054] Next, the CPU 80 simulates the inverted 3D point cloud data as if a cloth were draped over it from above. The simulated cloth is shown by a dashed line in Figure 6C. The CPU 80 then extracts the 3D point cloud data that the simulated cloth contacts as ground points. The CPU 80 then re-inverts the elevation values ​​to obtain ground points as shown in Figure 6D.

[0055] The ground points extracted in this way are generally accurate, but as shown in Figure 6D, they may include some features in the vicinity 100 where the features exist. Therefore, the normal direction of the line formed by each extracted ground point is calculated, and parts where the normal line forms an angle of a certain degree or more (for example, 30 degrees or more) with respect to the vertical direction are excluded from the ground points. This allows us to obtain ground points such as those shown in Figure 6E.

[0056] In this embodiment, Cloth Simulation is used to extract ground points, but ground points may be extracted by other methods such as the lowest point extraction method.

[0057] After extracting the ground points as described above, the CPU 80 removes the ground points from the three-dimensional point cloud data, thereby obtaining three-dimensional point cloud data of only features that exist on the ground (step S4).

[0058] Next, the CPU 80 reads out design data that indicates the planar arrangement of the features stored on the hard disk 86. This design data is a drawing of the features at the time of design, obtained by measuring the above-mentioned three-dimensional point cloud data. An example of a design drawing is shown in FIG. 7. The CPU 80 extracts the planar shape of each feature based on this design data (step S5). In this embodiment, the planar shape is extracted as a rectangle that contains the planar outer shape of the feature. Note that the planar outer shape of each feature may also be extracted as the planar shape as is.

[0059] Next, the CPU 80 overlays the three-dimensional point cloud data of only the feature on this planar shape. The overlay is performed by matching the absolute coordinates of the design data on which the planar shape is based with the absolute coordinates of the three-dimensional point cloud data. At this time, the height position of the planar shape is matched to the height (altitude) of the ground point (ground surface) in the three-dimensional point cloud data. Figure 8 shows the matched state. Figure 8 shows only the pedestrian bridge portion of the features in Figures 6 and 7. The frame line 300 indicates a rectangular planar shape that surrounds the planar outer shape of the pedestrian bridge (feature), and it can be seen that the position of this planar shape 300 matches the three-dimensional point cloud data of the pedestrian bridge (feature). Note that the planar outer shape is originally represented as a plane, but in Figure 8 it is shown with height for ease of understanding.

[0060] For each feature, the CPU 80 selects the highest elevation from the 3D point cloud data corresponding to the planar shape of the feature. The height is applied to the planar shape to determine the area of ​​the feature. This makes it possible to identify the three-dimensional shape (rectangular parallelepiped) surrounding the feature, as shown in FIG. 9. The CPU 80 records the three-dimensional shape generated in this way as outline area data. In this embodiment, the outline area data is recorded using the planar shape and the position and height based on absolute coordinates of horizontal azimuth and height.

[0061] Next, the CPU 80 overlays the area of ​​each feature on the three-dimensional point cloud data and displays it on the display 84. The operator looks at this screen and selects the area of ​​each feature with the mouse 90. This displays an input screen for attribute data such as the feature name, so the operator enters the data. Furthermore, based on the coordinates of the vertices of the three-dimensional shape of the outer shape area data, the coordinates of the center position (or any other position representative of the feature) are calculated and used as the coordinate position of the feature. In this way, attribute data can be generated and recorded. Figure 10 shows the attribute data items in this embodiment. The creation date and time is the creation date and time (measurement date and time) of the three-dimensional point cloud data. The feature name is the name of the feature. The feature area is outer shape data that indicates the outer shape of the feature. The feature position is the center position calculated above. In this embodiment, the three-dimensional point cloud data is recorded in XML format.

[0062] In the above, the operator inputs the feature name, but the feature name may be estimated based on three-dimensional point cloud data. For example, as shown in FIG. 11, three-dimensional point cloud data 6 of the feature is projected onto a plane at different angles, and the projected image and feature name are used as learning data to train a deep learning program (other machine learning programs may also be used). Learning is performed on various features to obtain a trained feature estimation program. The feature estimation program may be provided with three-dimensional point cloud data for which the feature is to be estimated, and the program may automatically estimate and assign the feature name. Alternatively, the estimated feature name may be displayed for the operator to confirm.

[0063] In addition, since there is a possibility that the three-dimensional point cloud data 6 and the outer shape area are misaligned, they may be displayed on the display 84, and the operator may align them using a mouse or the like so that their positions match.

[0064] In this way, the attribute data can be obtained.

[0065] In the above example, the outline area of ​​a feature is determined by assigning height to the planar data. However, if there is no planar data such as design data, the outline area may be determined from the 3D point cloud data itself.

[0066] For example, the outline area can be determined as follows: Divide the three-dimensional space into small cubic grids, and if there are points in adjacent grids above, below, left, right, or diagonally, combine them into one. For example, a spatial labeling technique using connected components can be used.

[0067] For each area organized as a grid containing points, areas with a predetermined volume, a predetermined number of points, or a predetermined point density are removed as noise. The remaining areas correspond to features. For each area, a rectangular parallelepiped is set to surround the area. This rectangular parallelepiped is the outline of the feature.

[0068] In this way, attribute data corresponding to three-dimensional point cloud data can be generated in the terminal device 20. The surveying business delivers this three-dimensional point cloud data and attribute data to the delivery destination.

[0069] Furthermore, at the discretion of the surveying company, only the attribute data can be uploaded to the attribute data server device 60. This allows companies that require three-dimensional point cloud data to access the attribute data server device 60 and refer to the attribute data. If the attribute data contains information about the company that provides the corresponding three-dimensional point cloud data, companies that require three-dimensional point cloud data can use this information to request the surveying company to obtain the three-dimensional point cloud data.

[0070] The three-dimensional point cloud data may also be uploaded to the point cloud server device 40. If the requester of the three-dimensional point cloud data is a local government or the like, they may be asked to upload it to a specific point cloud server device 40. This makes the three-dimensional point cloud data and attribute data publicly available on the Internet.

[0071] Furthermore, the 3D point cloud data and the outer area data of the attribute data are expressed in absolute coordinates. Therefore, if there are multiple 3D point cloud data for the same feature uploaded by different businesses to different point cloud servers, these multiple 3D point cloud data can be acquired as long as at least one attribute data is uploaded. For example, if there are 3D point cloud data for the same feature from different directions by different businesses, acquiring both data can provide more complete 3D point cloud data.

[0072] 1.4 3D model construction process Next, a process for constructing a three-dimensional model based on the plurality of three-dimensional point cloud data released as described above will be described.

[0073] The following describes the case where a three-dimensional model of a specific feature is constructed.

[0074] 12 and 13 show flowcharts for constructing a three-dimensional model. In the figures, the terminal device is shown as a flowchart of the terminal program, the attribute data server device is shown as a flowchart of the attribute data server program, and the point cloud server device is shown as a flowchart of the point cloud server program.

[0075] First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as terminal device 20b) accesses the attribute data server 60 and specifies the location where the desired feature is located (step S11). The location is specified by specifying a range, for example, by latitude and longitude. An address or place name may be entered (or searched), and this may be converted into a latitude and longitude range. Alternatively, an area clicked or selected on the map may be converted into a latitude and longitude range.

[0076] In response to this, the CPU of the attribute server device 60 (hereinafter sometimes abbreviated as the attribute server device) searches the attribute data and extracts the outline area data within the specified location (which may also include the vicinity outside the location) (step S21). The attribute server device 60 transmits the extracted outline area data (planar shape data and its position and height in absolute coordinates), feature names, etc. to the terminal device 20b.

[0077] The terminal device 20b displays the outline area and the feature name on the display 84 based on the received outline area data (step S12). An example of the display is shown in Fig. 14. The outline areas 102 to 112 of each feature are shown, and the feature name is shown nearby.

[0078] The user operates the mouse 90 of the terminal device 20b to select the outline region of a desired feature. Here, the description will proceed assuming that the outline region 110 of the building has been selected. In response to this selection, the terminal device 20b specifies the outline region data of the outline region 110 of the building and transmits a request for three-dimensional point cloud data to each of the point cloud server devices 40a to 40n (step S13).

[0079] Each point cloud server device 40a-40n that receives this request determines whether it holds the three-dimensional point cloud data for the outer region 110, and if so, returns it. Here, the explanation will be given assuming that two of the point cloud server devices 40a-40n, namely, 40a and 40b, hold the three-dimensional point cloud data for the outer region 110 of the building.

[0080] The CPU of the point cloud server device 40a (hereinafter, sometimes abbreviated as the point cloud server device 40a) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S31).

[0081] Similarly, the CPU of the point cloud server device 40b (hereinafter, may be abbreviated as point cloud server device 40b) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S41).

[0082] The point cloud server devices 40a and 40b transmit three-dimensional point cloud data of an area that has been widened to take into account measurement errors relative to the specified outer area 110. Measurement errors vary depending on the measurement method and equipment, and therefore also depend on the measurement method and equipment used to measure the three-dimensional point cloud data. Therefore, the widened area varies depending on the measurement method and equipment.

[0083] The terminal device 20b receives three-dimensional point cloud data (first three-dimensional point cloud data) from the point cloud server device 40a and three-dimensional point cloud data (second three-dimensional point cloud data) from the point cloud server device 40b. FIG. 15A shows an example of the first three-dimensional point cloud data, and FIG. 15B shows an example of the second three-dimensional point cloud data. As shown in FIGS. 15A and 15B, the first three-dimensional point cloud data was measured from the front side of the rectangular parallelepiped as viewed from the paper surface. The second three-dimensional point cloud data was measured from the rear side as viewed from the paper surface of the rectangular parallelepiped.

[0084] The terminal device 20b aligns and synthesizes the first three-dimensional point cloud data and the second three-dimensional point cloud data to generate synthesized three-dimensional point cloud data as shown in Fig. 16 (step S14). As is clear from this figure, it is possible to obtain three-dimensional point cloud data with a larger amount of information (having measurement data from both the front and rear sides) than when the first three-dimensional point cloud data and the second three-dimensional point cloud data are used alone.

[0085] The composite 3D point cloud data thus obtained can be used for various purposes. In this embodiment, a 3D model of a feature is generated based on the composite 3D point cloud data.

[0086] The terminal device 20b generates an outline surface of the feature based on the composite 3D point cloud data (step S15). This process can be performed, for example, using a 3D convex hull algorithm (http: / / www.qhull.org / download / ). FIG. 17A shows the generated 3D model of the feature. This 3D model can be more complete than a 3D model generated using only the first 3D point cloud data or a 3D model generated using only the second 3D point cloud data. The terminal device 20b displays this on the display 84 and records it on the hard disk 86 (step S15).

[0087] According to this embodiment, it is possible to easily search for multiple 3D point cloud data for the same feature scattered across the Internet and combine them to obtain composite 3D point cloud data. Furthermore, the above search and combination can be performed as long as one piece of attribute data is generated based on any one of the multiple 3D point cloud data for the same feature. This is because the attribute data is independent of the 3D point cloud data and the outer area data of the attribute data is defined in absolute coordinates.

[0088] 1.5 Other (1) In the above embodiment, two pieces of three-dimensional point cloud data are acquired to generate the composite three-dimensional point cloud data. However, three or more pieces of three-dimensional point cloud data may be acquired to generate the composite three-dimensional point cloud data.

[0089] (2) In the above embodiment, composite 3D point cloud data is obtained for one feature. However, composite 3D point cloud data may be obtained for multiple features. Also, composite 3D data may be obtained for all features within a predetermined range. Examples of such data are shown in Figures 18A, 18B, and 18C. Figure 18A shows the first 3D point cloud data, Figure 18B shows the second 3D point cloud data, and Figure 18C shows the composite 3D point cloud data.

[0090] (3) In the above embodiment, 3D point cloud data measured in different directions is synthesized. However, 3D point cloud data measured at different distances to features or measured by different measuring instruments (for example, MMS and UAV) may also be synthesized.

[0091] (4) In the above embodiment, the case where only one attribute data item is found for a location in step S21 has been described. However, there are cases where different measurement companies create 3D point cloud data for the same location and each uploads attribute data, or where the same company creates 3D point cloud data for the same location at different times and each uploads attribute data. In such cases, multiple attribute data items are found for the specified location. In this case, the attribute data server device 60 transmits a message to the terminal device 20b indicating that multiple attribute data items exist. The terminal device 20b displays the creation date and time, creation device, etc. of the multiple attribute data items on the display 84, allowing the operator to select one. Based on the selected attribute data item, the outline area is transmitted (step S21).

[0092] (5) In the above embodiment, the terminal device 20b makes the three-dimensional point cloud data request (step S13). However, the attribute data server device 60 may make this request and transmit the result to the terminal device 20b.

[0093] (6) In the above embodiment, the terminal device 20b synthesizes the 3D point cloud data and generates the 3D model. However, either or both of these may be performed by the attribute data server device 60, and the results may be transmitted to the terminal device 20b.

[0094] (7) In the above embodiment, when the terminal device 20b acquires the three-dimensional point cloud data in step S14, it does not display the direction from which the data was measured. However, to clarify this, as shown in FIG. 17B, an arrow may be used to display the measurement direction and distance relative to the feature based on the data attached to the acquired three-dimensional point cloud data. The base of the arrow indicates the measurement position, and the direction of the arrow indicates the measurement direction. In the case of FIG. 17B, it is clear that there is three-dimensional point cloud data measured from two locations.

[0095] Therefore, based on this screen display, the next time you make a measurement, it is preferable to measure from a direction other than the arrow if you want to improve the quality of the composite 3D point cloud data. Also, if you want to know the changes in features over time, it is preferable to measure from the same position and direction as the arrow.

[0096] (8) In the above embodiment, the first and second three-dimensional point cloud data are simply combined to generate the combined three-dimensional point cloud data. However, if the first three-dimensional point cloud data and the second three-dimensional point cloud data exist for the same location in the three-dimensional point cloud data, only the three-dimensional point cloud data with the highest measurement accuracy may be used to generate the combined three-dimensional point cloud data for that location.

[0097] (9) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are configured as separate servers, but they may be configured as a single server device.

[0098] (10) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b, and the attribute data server device 60 were constructed as separate server devices, but they may also be constructed as a single server device.

[0099] (11) The above-described embodiments and modifications can be implemented in combination with other embodiments as long as it does not contradict the essence of the embodiments.

[0100] 2. Second embodiment 2.1 Overall structure 19 shows the overall configuration of a point cloud data management system according to a second embodiment of the present invention. A first three-dimensional point cloud data file at a first date and time based on absolute coordinates including features is recorded in a first point cloud server device 40a. A second three-dimensional point cloud data file at a second date and time based on absolute coordinates including features is recorded in a second point cloud server device 40b. An attribute data server device 60 records outline area data based on absolute coordinates that surrounds the outlines of features.

[0101] The terminal device 20 is capable of communicating with these server devices. The three-dimensional point cloud data acquisition means 22 of the terminal device 20 acquires the outline area data of a desired feature from the attribute data server device 60. The three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40a based on the acquired outline area data of the feature, and acquires the three-dimensional point cloud data included in the area as first three-dimensional point cloud data. Furthermore, the three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40b based on the acquired outline area data of the feature, and acquires the three-dimensional point cloud data included in the area as second three-dimensional point cloud data.

[0102] The time-dependent change acquisition means 25 calculates the change in shape of the feature between the first date and time and the second date and time based on the acquired first three-dimensional point cloud data at the first date and time and the acquired second three-dimensional point cloud data at the second date and time.

[0103] 2.2 System Configuration The system configuration is the same as in the first embodiment.

[0104] 2.3 Calculation of changes over time Next, a process for acquiring changes over time of features based on a plurality of published three-dimensional point cloud data will be described.

[0105] 20 and 21 show flowcharts for acquiring changes over time in features. In the figures, the terminal device is shown as a flowchart of the terminal program, the attribute data server device is shown as a flowchart of the attribute data server program, and the point cloud server device is shown as a flowchart of the point cloud server program.

[0106] First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as terminal device 20b) accesses the attribute data server 60 and specifies the location where the desired feature is located (step S11). The location is specified by specifying a range, for example, by latitude and longitude. An address or place name may be entered (or searched), and this may be converted into a latitude and longitude range. Alternatively, an area clicked or selected on the map may be converted into a latitude and longitude range.

[0107] In response to this, the CPU of the attribute server device 60 (hereinafter sometimes referred to as the attribute server device) searches the attribute data and extracts the outline area data within the specified location (step S21). The attribute server device 60 transmits the extracted outline area data (planar shape data, its position and height in absolute coordinates), the feature name, etc. to the terminal device 20b.

[0108] The terminal device 20b displays the outline area and the feature name on the display 84 based on the received outline area data (step S12). An example of the display is shown in Fig. 14. The outline areas 102 to 112 of each feature are shown, and the feature name is shown nearby.

[0109] The user operates the mouse 90 of the terminal device 20b to select the outline region of a desired feature. Here, the description will proceed assuming that the outline region 110 of the building has been selected. In response to this selection, the terminal device 20b specifies the outline region data of the outline region 110 of the building and transmits a request for three-dimensional point cloud data to each of the point cloud server devices 40a to 40n (step S13).

[0110] Each point cloud server device 40a-40n that receives this request determines whether it holds the three-dimensional point cloud data for the outer region 110, and if so, returns it. Here, the explanation will be given assuming that two of the point cloud server devices 40a-40n, namely, 40a and 40b, hold the three-dimensional point cloud data for the outer region 110 of the building.

[0111] The CPU of the point cloud server device 40a (hereinafter, sometimes abbreviated as the point cloud server device 40a) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S31).

[0112] Similarly, the CPU of the point cloud server device 40b (hereinafter, may be abbreviated as point cloud server device 40b) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S41).

[0113] The terminal device 20b receives three-dimensional point cloud data (first three-dimensional point cloud data) from the point cloud server device 40a and three-dimensional point cloud data (second three-dimensional point cloud data) from the point cloud server device 40b. Here, the first three-dimensional point cloud data and the second three-dimensional point cloud data are obtained by measuring features from substantially the same direction, but the measurement dates and times are different.

[0114] The terminal device 20b generates a three-dimensional model of the feature at the first date and time based on the first three-dimensional point cloud data (step S17), and also generates a three-dimensional model of the feature at the second date and time based on the second three-dimensional point cloud data (step S17).

[0115] The terminal device 20b displays the generated three-dimensional model at the first date and time and the three-dimensional model at the second date and time on the display 84 so that they can be compared (step S18). For example, as shown in FIG. 22, a three-dimensional model 120 at the first date and time and a three-dimensional model 122 at the second date and time are displayed. Below them, the name of the feature and the date of measurement are displayed. Therefore, in the example of FIG. 22, it can be seen by comparing the external shape of the building on February 24, 2010 with the external shape of the building on March 5, 2019, that has changed.

[0116] To facilitate comparison, the two may be displayed in different colors.

[0117] Furthermore, for example, if there is a situation where a feature does not exist at the first date and time but does exist at the second date and time (for example, a new road sign is installed), 3D point cloud data of the feature can be obtained if there is measurement data of the location at the second date and time. However, even if there is measurement data of the location at the first date and time, 3D point cloud data of the feature cannot be obtained. Therefore, by displaying a 3D model based on this, it becomes clear that the feature did not exist at the first date and time but did exist at the second date and time.

[0118] For the purpose of the above-mentioned comparison over time, it is preferable to use three-dimensional point cloud data in which the measurement direction relative to the feature is the same (or similar).

[0119] 2.4 Other (1) In the above embodiment, a three-dimensional model is generated using three-dimensional point cloud data at two dates and times, and then a comparison is performed. However, three-dimensional point cloud data at three or more dates and times may be acquired, and a three-dimensional model may be generated for each of the three-dimensional point cloud data, and then a comparison may be performed.

[0120] (2) In the above embodiment, a time-series comparison is performed for one feature. However, a time-series comparison may be performed for multiple features. Also, a time-series comparison may be performed for all features within a predetermined range.

[0121] (3) In the above embodiment, the terminal device 20b makes the request for three-dimensional point cloud data (step S13). However, the attribute data server device 60 may make this request and transmit the result to the terminal device 20b.

[0122] (6) In the above embodiment, the three-dimensional model is generated in the terminal device 20b. However, the three-dimensional model may be generated in the attribute data server device 60, and the result may be transmitted to the terminal device 20b.

[0123] (7) In the above embodiment, when the terminal device 20b acquires the three-dimensional point cloud data in step S17, the date and time when each piece of three-dimensional point cloud data was measured is not displayed. However, to clarify this, the measurement date and time of each piece of three-dimensional point cloud data may be displayed based on the data attached to the acquired three-dimensional point cloud data, as shown in FIG. 23. When there is three-dimensional point cloud data with many dates and times, it is preferable to display such a display. The user can view this, select a predetermined number of dates to compare, and display the three-dimensional model.

[0124] (8) The above-described embodiment and modified examples can be implemented in combination with other embodiments as long as the combination does not contradict the essence of the embodiment. For example, if there are multiple 3D point cloud data from different directions at the same time (or in the same month or year), composite 3D point cloud data may be generated to generate a 3D model, which may then be compared with the 3D model at another time.

[0125] (9) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are configured as separate servers, but they may be configured as a single server device.

[0126] (10) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b, and the attribute data server device 60 were constructed as separate server devices, but they may also be constructed as a single server device.

[0127] (11) In the above embodiment, the searched video data is played back on the terminal device. At this time, a map of the area where the video was captured, three-dimensional point cloud data, etc. may be displayed together. Furthermore, three-dimensional point cloud data viewed from the image capture position that changes over time may be displayed to match the playback of the video.

[0128] (12) In the above embodiment, three-dimensional models of features at two different points in time are displayed for comparison. However, the normal vectors of the outlines of the three-dimensional models may be displayed for both points in time to make the changes in the surface angles easier to understand.

[0129] It is also possible to divide the data into small areas (25cm cubes) and compare and display the number of points within each area, which allows you to compare the roughness of the surface of features.

[0130] Furthermore, the reflection intensity of each point or area may be compared, thereby allowing the smoothness of the surface to be compared.

[0131] The volumes of the three-dimensional models may also be compared, allowing the growth of plants, the condition of leaves, and so on to be compared.

[0132] (13) The above-described embodiments and modifications can be implemented in combination with other embodiments as long as it does not contradict the essence of the embodiments.

[0133] 3. Third embodiment 3.1 Overall structure 24 shows a functional block diagram of a feature search system according to the third embodiment of the present invention. Attribute data such as feature names, outline areas, and positions of three-dimensional point cloud data, as well as video data, are recorded in the recording unit 158 ​​of the feature search server device 150.

[0134] The attribute data transmission means 150 of the feature search server device 150 reads out the feature name or outer area corresponding to the three-dimensional point cloud data recorded in the recording unit 158 ​​and transmits it to the terminal device 40. The feature attribute data display means 182 of the terminal device 40 displays the received feature name or outer area of ​​the feature on the display unit 188.

[0135] The operator of the terminal device 40 uses the mouse 90 to view the feature name or outline area of ​​the displayed feature and select the feature for which they wish to search for video. The selection information sending means 184 sends to the feature search server device 150 which feature has been selected.

[0136] The feature position acquisition means 154 of the feature search server device 150 acquires the position of the selected feature. The video data transmission means 156 searches the video data in chronological order and extracts a portion that has imaging position information attached near the feature position. The extracted video data is transmitted to the terminal device 40. The video playback means 186 of the terminal device 40 plays the received video data on the display unit 188.

[0137] This makes it possible to easily search for and display locations where desired features are captured from a huge amount of video data.

[0138] 3.2 System Configuration Fig. 25 shows the system configuration of a point cloud data management system according to the third embodiment. Point cloud server devices 40a, 40b...40n are provided on the Internet. A feature search server device 62 is also provided. The point cloud server devices 40a, 40b...40n store three-dimensional point cloud data of measured features and video data (with image capture location information) captured during measurement. The feature search server device 62 stores attribute data such as that shown in Fig. 10. Terminal devices 20a, 20b...20n are provided to these server devices 40a, 40b...40n, 60 so as to be able to communicate with them via the Internet.

[0139] The hardware configuration of the terminal device 20 is the same as that shown in Fig. 3. The point cloud server device 40 and the feature search server device 62 also have similar hardware configurations. However, in the point cloud server device 40, a point cloud server program is recorded instead of the terminal program 96. Also, in the feature search server device 62, a feature search server program is recorded instead of the terminal program 96.

[0140] 3.3 Feature search process A flowchart for feature search is shown in Figure 26. In the figure, what is shown as a terminal device is a flowchart for a terminal program, and what is shown as a feature search server device is a flowchart for a feature search server program.

[0141] First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as terminal device 20b) accesses the feature search server device 62 and specifies the location where the desired feature is located (step S11). The location is specified by specifying a range, for example, by latitude and longitude. An address or place name may be entered (or searched), and this may be converted into a latitude and longitude range. Alternatively, an area clicked or selected on the map may be converted into a latitude and longitude range.

[0142] In response to this, the CPU of the feature search server device 62 (hereinafter sometimes abbreviated as feature search server device 62) searches the attribute data and extracts the outline area data within the specified location (step S51). The feature search server device 62 transmits the extracted outline area data (planar shape data, its position and height in absolute coordinates), the feature name, etc. to the terminal device 20b.

[0143] The terminal device 20b displays the outline area and the feature name on the display 84 based on the received outline area data (step S12). An example of the display is shown in Fig. 14. The outline areas 102 to 112 of each feature are shown, and the feature name is shown nearby.

[0144] The user operates the mouse 90 of the terminal device 20b to select the outline region of the desired feature. Here, the explanation will proceed assuming that the outline region 106 of the sign has been selected. In response to this selection, the terminal device 20b specifies the outline region data of the sign's outline region 106 and transmits an instruction to the feature search server device 62 to search for the portion of the video in which this sign is captured (step S19).

[0145] The feature search server device 62 reads the attribute data of the specified feature and acquires the position of the feature (position in absolute coordinates) (step S52). Next, the feature search server device 62 extracts a portion of the captured video data 162 in which the feature is captured, based on the feature position (step S53).

[0146] 27 shows the data structure of captured video data 162. Time stamps TS1, TS2, etc. indicating the time of capture are recorded in association with video main body data 170. The position of the camera (or a car equipped with a camera, etc.) that captured the video is also recorded in absolute coordinates. Although not shown, captured video data 162 also has associated and recorded imaging area information for the entire video.

[0147] Fig. 28 shows the concept of imaging area information. The solid line indicates the imaging path R. It shows that imaging started from the start point S and continued to the end point E. The dashed line that encloses this imaging path R indicates the imaging area AR. The imaging area AR is specified by, for example, the absolute coordinates of the diagonal (imaging area information).

[0148] 29 shows the details of the process of extracting video portions in step S53. Based on the position of the feature to be searched, the feature search server device 62 selects video data 162 whose imaging area includes the measurement position (step S531). The video data 162 selected in this way may include a portion in which the feature is captured. The feature search server device 62 searches each of the selected video data 162 to determine whether the captured portion is present.

[0149] In step S533, the feature search server device 62 reads out the imaging positions recorded in the video data 162 in chronological order (step S533). In the example of Fig. 27, imaging positions PS1, PS2, ..., PSn are read out.

[0150] The feature search server device 62 identifies the imaging position PSf that is closest to the location of the feature from among the read imaging positions PS1, PS2, ..., PSn (step S534). The feature search server device 62 determines whether the distance between the closest imaging position PSf and the feature position is equal to or less than a predetermined value (step S535). If it is equal to or less than the predetermined value, it means that the feature has been imaged at that location. The feature search server device 62 extracts video data for a predetermined time before and after the identified imaging position PSf (step S536). If the distance between the imaging position PSf and the feature position exceeds the predetermined value, the video portion is not extracted.

[0151] The above process is performed for all the moving images selected in step S531 (steps S532, S537).

[0152] After extracting the video data portion in this manner, the feature search server device 62 transmits it to the terminal device 20b (step S54).

[0153] In response to this, the terminal device 20b displays the extracted portion of the video data on the display 84 (step S20). If there are multiple video data, it is preferable to display a list of the video data together with the shooting date and time before playback, and let the user select one to play.

[0154] By watching this video, the user can grasp the status of the feature that was the search target as an image. For example, it can be used when wanting to know the status of the feature that can be identified on the screen display of Fig. 14 (checking the looseness detection marks (for example, match marks) on bolts and nuts, checking the state of appearance deterioration, etc.).

[0155] The above detection mark and the deterioration of the appearance may be checked by the terminal device using image processing or AI.

[0156] 3.4 Other (1) In the above embodiment, the feature positions in the attribute data and the capture positions in the video data are expressed in absolute coordinates. Therefore, even if attribute data corresponding to each video data is not prepared, it is possible to search for features from multiple video data with only one attribute data.

[0157] If attribute data is provided for each video data, the feature position and the image capture position do not need to be absolute coordinates as long as they are consistent between the two data. Also, if attribute data is provided for the video data, searches can be performed based on the date and time when the 3D point cloud data of the feature was created and the date and time when it was captured, rather than based on the feature position and the image capture position.

[0158] (2) In the above embodiment, the feature to be searched for is specified by the outline area and the feature name, as shown in Fig. 14. However, it is also possible to display only the feature name without displaying the outline area and allow the user to select it.

[0159] (3) In the above embodiment, the portion of the video in which the feature is captured is extracted and played back. However, in addition to this, the feature may be marked in the video and played back.

[0160] The feature search server device 62 can achieve this by performing the following processing. For example, suppose there is an outline area of ​​a feature as shown in Fig. 30, and a sign 106 is to be searched for. In the space where this outline area is located, a trajectory 200 is drawn when a video is captured. Since time is added to this trajectory 200, it is possible to know the image capture time at each point. At each image capture time t1, t2, etc., it is determined in which direction and at what distance the target feature 106 exists.

[0161] Based on this direction and distance, it calculates how the feature 106 is imaged at the corresponding time in the captured video. The appearance of the contour area of ​​the feature 106 generated in this way at each time is superimposed on the extracted video and transmitted to the terminal device 20b.

[0162] When this is played back on the terminal device 20b, an outline frame is displayed to surround the target feature in the image, as shown in Figure 31. Therefore, the user can easily find the target feature in the video.

[0163] (4) In the above embodiment, the point cloud server device 40 and the search server device 62 are configured as separate servers, but they may be configured as a single server device.

[0164] (5) In the above embodiment, a case where a video is searched for is described. However, a still image with an image capture location and image capture time can also be searched for using the same process.

[0165] (6) The above-described embodiments and modifications can be implemented in combination with other embodiments as long as it does not contradict the essence of the embodiments.

Claims

1. A feature search system including a server device and a plurality of terminal devices capable of communicating with the server device, The server device a recording unit that records attribute data including three-dimensional point cloud data of a plurality of features having position information, video data with information on the image capturing positions at which the features were captured, and outline area data surrounding the outlines of the features; an attribute data transmission means for transmitting attribute data of the plurality of features to a terminal device for a user to select a feature; a feature position acquisition means for receiving feature selection information from a terminal device and acquiring a feature position corresponding to the contour area data included in the attribute data of the selected feature; a video data extraction means for extracting from the video data a portion to which imaging position information in the vicinity of the feature position is added and transmitting the extracted portion to a terminal device; The terminal device a feature attribute data display means for acquiring attribute data of the features from the server device and displaying the data on a display unit for a user to select a feature; a selection information transmitting means for transmitting information on features selected by a user to a server device; a video data playback means for receiving the extracted video data transmitted from the server device and playing it on a display unit; A feature search system with

2. a recording unit that records attribute data including three-dimensional point cloud data of a plurality of features having position information, video data with information on the image capturing positions at which the features were captured, and outline area data surrounding the outlines of the features; an attribute data transmission means for transmitting attribute data of the plurality of features to a terminal device for a user to select a feature; a feature position acquisition means for receiving feature selection information from a terminal device and acquiring a feature position corresponding to the contour area data included in the attribute data of the selected feature; a video data extraction means for extracting from the video data a portion to which imaging position information in the vicinity of the feature position is added and transmitting the extracted portion to a terminal device; A server device comprising:

3. A feature search server program for realizing a server device by a computer, the program comprising: access means for accessing a recording unit that records attribute data including three-dimensional point cloud data of a plurality of features having position information, video data with information on the image capturing positions at which the features were captured, and outline area data surrounding the outlines of the features; an attribute data transmission means for transmitting attribute data of the plurality of features to a terminal device for a user to select a feature; a feature position acquisition means for receiving feature selection information from a terminal device and acquiring a feature position corresponding to the contour area data included in the attribute data of the selected feature; a server program for causing the server to function as a video data extraction means for extracting from the video data a portion to which imaging position information in the vicinity of the feature position is added and transmitting the extracted portion to a terminal device;

4. a feature attribute data display means for acquiring feature attribute data from a server device having an access means for accessing a recording unit that records attribute data including three-dimensional point cloud data of a plurality of feature objects having position information, video data with attached image capture position information of the feature objects, and outer shape area data surrounding the outer shapes of the feature objects, and for displaying the feature attribute data on a display unit for a user to select a feature; a selection information transmitting means for transmitting information on features selected by a user to a server device; a video data playback means for receiving the extracted video data transmitted from the server device and playing it on a display unit; A terminal device comprising:

5. A terminal program for realizing a terminal device by a computer, the program comprising: a feature attribute data display means for acquiring feature attribute data from a server device having an access means for accessing a recording unit that records attribute data including three-dimensional point cloud data of a plurality of feature objects having position information, video data with attached image capture position information of the feature objects, and outer shape area data surrounding the outer shapes of the feature objects, and for displaying the feature attribute data on a display unit for a user to select a feature; a selection information transmitting means for transmitting information on features selected by a user to a server device; A terminal program for causing the terminal to function as a video data playback means for receiving extracted video data transmitted from the server device and playing the data on the display unit.

6. In the program of claim 3, The position of the outer shape area data is indicated by absolute coordinates, The video data extraction means extracts portions of the video data that are assigned with imaging position information near the feature position, and transmits the extracted video data.

7. In the program of claim 3 or 6, The video data extraction means A program characterized by comprising a video feature identification means for acquiring three-dimensional point cloud data of a selected feature, and identifying and marking the corresponding feature in the extracted video based on at least the contour of the three-dimensional point cloud data.

8. In the program of any one of claims 3, 6 or 7, The server device is configured to include a plurality of individual server devices, The program is characterized in that the video data or the attribute data is recorded in different individual server devices.

9. In the program of any one of claims 3, 6 to 8, The server device includes a point cloud data server device that records the three-dimensional point cloud data, and a search server device that records the attribute data.

10. Recording attribute data including video data with image capture position information of the features captured when measuring three-dimensional point cloud data of a plurality of features while moving and outline area data surrounding the outlines of the features; a computer displaying attribute data of the plurality of features for feature selection by a user; The computer receives the selection, acquires a feature position corresponding to the outline area data included in the attribute data of the selected feature, and A feature search method, characterized in that a computer extracts and displays a portion of the video data near the feature location that is provided with imaging position information.

Images