Point cloud data management system
The point cloud data management system addresses the challenge of integrating attribute data with three-dimensional point cloud data by using multiple server devices to facilitate the construction of comprehensive three-dimensional models and detect temporal changes in features.
Patent Information
- Application Number
- JP2024145443
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-03-29
AI Technical Summary
Existing systems face challenges in efficiently managing and integrating attribute data with three-dimensional point cloud data obtained from terrestrial features, making it difficult to construct comprehensive and accurate three-dimensional models.
A point cloud data management system comprising first and second point cloud server devices, an attribute data server device, and terminal devices that facilitate the storage, retrieval, and synthesis of three-dimensional point cloud data from multiple directions and times, enabling the construction of complete three-dimensional models and the detection of temporal changes in features.
Enables efficient management and integration of three-dimensional point cloud data from various directions and times, allowing for the creation of more accurate three-dimensional models and the detection of temporal changes in features.
Smart Images

Figure 2024164212000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a system for managing point cloud data obtained by measuring features. [Background technology]
[0002] Laser surveying devices are mounted on unmanned aerial vehicles (UAVs) and automobiles to measure the shape of the earth's surface and features. For example, in an MMS (Mobile Mapping System) using an automobile, a GPS receiver, video camera, IMU, and laser scanner are mounted on the roof of the automobile, and measurements are taken while the automobile is traveling along roads. The video camera can be either analog or digital.
[0003] Through such measurements, it is possible to obtain 3D point cloud data that shows the earth's surface and the outlines of features. Each 3D point cloud data is given a position based on absolute coordinates (plane rectangular coordinates, latitude, longitude, altitude, etc.) according to the position obtained by GPS and the scanning direction and distance of the laser scanner. In addition, the earth's surface and features are captured by a video camera to generate video data (still image data). This video data is linked to and recorded with the image capture position, which changes from moment to moment.
[0004] Obtaining three-dimensional point cloud data makes it possible to know the current state of the earth's surface and features, which can be useful for maintenance, map creation, and other purposes.
[0005] The inventors propose assigning attribute data, including data indicating the area of each feature in the three-dimensional point cloud data using absolute coordinates and its name, to each feature in the three-dimensional point cloud data. Summary of the Invention [Problem to be solved by the invention]
[0006] However, while the above-mentioned attribute data is leading to progress in standardization and the accumulation of abundant data, at the same time, there is a demand for a system that can efficiently manage three-dimensional point cloud data when using the attribute data for maintenance, etc.
[0007] An object of the present invention is to provide a system that can solve the above problems and efficiently manage three-dimensional point cloud data. [Means for solving the problem]
[0008] Independently applicable features of the present invention are listed below.
[0009] (1)(2)(3) A point cloud data management system according to the present invention is a point cloud data management system comprising first and second point cloud data server devices, an attribute data server device and a plurality of terminal devices capable of communicating with these server devices, wherein the first point cloud server device records a first three-dimensional point cloud data file from a first direction in absolute coordinates including a feature, the second point cloud server device records a second three-dimensional point cloud data file from a second direction in absolute coordinates including the same feature as the feature, and the attribute data server device records a three-dimensional point cloud data file indicated by the absolute coordinates that surrounds the outline of the feature. The terminal device is equipped with a three-dimensional point cloud data acquisition means for acquiring the outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device, and a three-dimensional model construction means for constructing a three-dimensional shape of the feature based on the acquired first and second three-dimensional point cloud data from the first and second directions.
[0010] Therefore, it is possible to easily search and acquire three-dimensional point cloud data for the same feature from multiple different directions, and to construct a more complete three-dimensional model.
[0011] (4)(5)(6) A point cloud data management system according to the present invention is a point cloud data management system comprising a first and a second point cloud data server device, an attribute data server device and a plurality of terminal devices capable of communicating with these server devices, wherein the first point cloud server device records a first three-dimensional point cloud data file at a first date and time based on absolute coordinates including a feature, the second point cloud server device records a second three-dimensional point cloud data file at a second date and time based on absolute coordinates including the same feature as the feature, and the attribute data server device records a three-dimensional point cloud data file indicated by the absolute coordinates that surrounds the outline of the feature. The terminal device has a three-dimensional point cloud data acquisition means for acquiring the outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device, and a time-over-time change acquisition means for acquiring a change in the feature over time based on the acquired first and second three-dimensional point cloud data at the first and second dates and times.
[0012] Therefore, it is possible to easily obtain three-dimensional point cloud data for the same feature measured at different dates and times, and to obtain the secular changes of the feature.
[0013] (7) A point cloud data management system according to the present invention is characterized in that the first point cloud server device and the second point cloud server device are constructed as a single point cloud server device.
[0014] (8) A point cloud data management system according to the present invention is characterized in that an attribute data server device is a server device constructed integrally with the point cloud server device.
[0015] (9) A point cloud data management method according to the present invention includes recording a first three-dimensional point cloud data file from a first direction using absolute coordinates including a feature, recording a second three-dimensional point cloud data file from a second direction using absolute coordinates including the same feature as the feature, recording outline area data indicated by absolute coordinates that surrounds the outline of the feature, acquiring the outline area data of the feature by a computer, acquiring first three-dimensional point cloud data of the feature corresponding to the first three-dimensional point cloud data file based on the outline area data, acquiring second three-dimensional point cloud data of the feature corresponding to the second three-dimensional point cloud data file, and obtaining a synthesized three-dimensional point cloud data based on the acquired first and second three-dimensional point cloud data from the first and second directions by the computer.
[0016] Therefore, it is possible to easily search and acquire three-dimensional point cloud data for the same feature from a plurality of different directions, thereby obtaining more accurate three-dimensional point cloud data.
[0017] (10) A three-dimensional point cloud data management method according to the present invention includes: recording a first three-dimensional point cloud data file at a first date and time using absolute coordinates including a feature; recording a second three-dimensional point cloud data file at a second date and time using absolute coordinates including the same feature as the feature; recording outline area data indicating the absolute coordinates that surrounds the outline of the feature; acquiring, by a computer, the outline area data of the feature; acquiring, based on the outline area data, first three-dimensional point cloud data of the feature contained in the first three-dimensional point cloud data file; acquiring, based on the first and second three-dimensional point cloud data of the feature contained in the second three-dimensional point cloud data file; and acquiring, by the computer, changes over time of the feature based on the acquired first and second three-dimensional point cloud data at the first and second dates and times.
[0018] Therefore, it is possible to easily search for and obtain three-dimensional point cloud data for the same feature at different dates and times, and to obtain changes in the feature over time.
[0019] (11)(12)(13)(14)(15) A feature search system according to the present invention is a feature search system comprising a point cloud data server device, an attribute data server device and a plurality of terminal devices capable of communicating with these server devices, wherein the server device comprises a recording unit which records attribute data including video data with imaging position information attached, in which the features are imaged when three-dimensional point cloud data of the plurality of features is measured while moving, and outline area data surrounding the outlines of the features, an attribute data transmission unit which transmits the attribute data of the plurality of features to the terminal device for feature selection by a user, and a The terminal device is equipped with a feature position acquisition means for acquiring a feature position corresponding to the outer area data included in the attribute data of the selected feature, a video data extraction means for extracting from the video data a portion attached with imaging position information near the feature position and transmitting the extracted portion to a terminal device, and a feature attribute data display means for acquiring the feature attribute data from the server device and displaying the data on a display unit for the user to select a feature, a selection information transmission means for transmitting information of the feature selection made by the user to the server device, and a video data playback means for receiving the extracted video data transmitted from the server device and playing the data on the display unit.
[0020] Therefore, features captured in a video can be easily searched for and displayed.
[0021] (16) The feature search system of the present invention is characterized in that the position of the outer shape area data is indicated in absolute coordinates, and the video data extraction means extracts portions of the multiple video data that are assigned imaging position information near the feature position, and transmits the multiple extracted video data.
[0022] Therefore, if there is one piece of external region data, it is possible to search for features in multiple pieces of video data.
[0023] (17) The feature search system of the present invention is characterized in that the video data extraction means acquires three-dimensional point cloud data of a selected feature, and includes a video feature identification means for identifying and marking a corresponding feature in the extracted video based on at least a contour of the three-dimensional point cloud data.
[0024] This makes it easier to find features within a video.
[0025] (18) A feature search system according to the present invention is characterized in that the server device is configured to include a plurality of individual server devices, and the video data or the attribute data is recorded in the different individual server devices.
[0026] Therefore, it is possible to search for features from video data scattered across different individual server devices.
[0027] (19) A feature search system according to the present invention is characterized in that it records video data accompanied by imaging position information of a plurality of features captured while measuring three-dimensional point cloud data of the plurality of features while moving, and attribute data including outline area data enclosing the outlines of the features, and a computer displays the attribute data of the plurality of features for a user to select a feature, and upon receiving the selection, the computer obtains a feature position corresponding to the outline area data included in the attribute data of the selected feature, and the computer extracts and displays a portion of the video data accompanied by imaging position information near the feature position.
[0028] Therefore, features captured in a video can be easily searched for and displayed.
[0029] (20)(21) The outline area data generating device of the present invention comprises a plan outline determination means for determining a plan outline based on a plan view of an object having absolute coordinates, a means for determining the lower and upper ends in the elevation direction based on three-dimensional point cloud data of the object, and a three-dimensional outline determination means for treating the three-dimensional outline when the plan outline is continued from the lower end to the upper end as outline area data.
[0030] Therefore, the contour area data of the feature can be generated easily and accurately.
[0031] In this embodiment, step S13 corresponds to the "three-dimensional point cloud data acquisition means."
[0032] In this embodiment, step S15 corresponds to the "three-dimensional model construction means."
[0033] In this embodiment, step S18 corresponds to the "time-dependent change acquisition means."
[0034] In this embodiment, step S51 corresponds to the "attribute data transmitting means."
[0035] In this embodiment, step S52 corresponds to the "facility position acquisition means."
[0036] In this embodiment, step S54 corresponds to the "moving image data extraction means."
[0037] In this embodiment, step S12 corresponds to the "facility attribute data display means."
[0038] In this embodiment, step S19 corresponds to the "selection information transmitting means."
[0039] In this embodiment, step S20 corresponds to the "moving image data reproducing means." The term "program" is a concept that includes not only programs that can be executed directly by a CPU, but also programs in source format, compressed programs, encrypted programs, and the like. [Brief description of the drawings]
[0040] [Figure 1] 1 is a functional block diagram of a point cloud data management system according to a first embodiment of the present invention. [Diagram 2] This shows the system configuration of a point cloud data management system. [Diagram 3] This shows the hardware configuration of the terminal device. [Figure 4] 13 is a flowchart of a terminal program. [Diagram 5] 1 is an example of three-dimensional point cloud data. [Figure 6] FIG. 1 illustrates a process for detecting ground points. [Figure 7] This is an example of design data for a feature. [Figure 8] FIG. 1 is a diagram showing three-dimensional point cloud data superimposed on a plan view of a feature. [Figure 9] This is an example of 3D point cloud data of geographical features. [Figure 10] 13 is an example of attribute data items. [Figure 11] 11 is a diagram for explaining a process of estimating a feature name based on three-dimensional point cloud data of the feature. FIG. [Figure 12] 1 is a flowchart for acquiring 3D point cloud data of features and generating a 3D model. [Figure 13] 1 is a flowchart for acquiring 3D point cloud data of features and generating a 3D model. [Figure 14] 13 is a display example of the outline shape of a feature. [Figure 15] FIG. 15A is a first 3D point cloud data of a feature from a first direction, and FIG. 15B is a second 3D point cloud data of the feature from a second direction. [Figure 16] This is a composite 3D point cloud data. [Figure 17] Fig. 17A shows the generated three-dimensional model, and Fig. 17B shows an example of the measurement direction. [Figure 18] FIG. 18A shows the first three-dimensional point cloud data, FIG. 18B shows the second three-dimensional point cloud data, and FIG. 18C shows the composite three-dimensional point cloud data. [Figure 19] FIG. 11 is a functional block diagram of a point cloud data management system according to a second embodiment. [Figure 20] 13 is a flowchart for acquiring a change over time. [Figure 21] 13 is a flowchart for acquiring a change over time. [Figure 22] This is an example of how features change over time. [Diagram 23] 13 is an example of a screen prompting the user to select three-dimensional point cloud data with different dates and times. [Figure 24] FIG. 11 is a functional block diagram of a feature search system according to a third embodiment. [Diagram 25] This is the system configuration of the feature search system. [Figure 26] 1 is a flowchart of a feature search. [Figure 27] FIG. 2 is a diagram showing a configuration of video data. [Figure 28] FIG. [Figure 29] 13 is a flowchart of video extraction. [Diagram 30] FIG. 2 is a diagram showing the relationship between an imaging trajectory 200 and a feature 106 at each time. [Diagram 31] FIG. 13 is a diagram showing an image in which search target features are marked. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0041] 1. First embodiment 1.1 Overall structure Fig. 1 shows the overall configuration of a point cloud data management system according to an embodiment of the present invention. A first point cloud server device 40a stores a first three-dimensional point cloud data file from a first direction based on absolute coordinates including features. A second point cloud server device 40b stores a second three-dimensional point cloud data file from a second direction based on absolute coordinates including features. An attribute data server device 60 stores outline area data based on absolute coordinates that surrounds the outline of the features.
[0042] Here, features include, for example, road design standard information (mileposts, road centerlines, measurement points, etc.), road surface features (roadway sections, sidewalks, trackbeds, etc.), features that share an area with the road surface (stop lines, dividing lines, pedestrian crossings, road markings, etc.), features other than the road surface (signposts, lighting poles, traffic lights, pedestrian bridges, etc.), and features that indicate the road surface (retaining walls, bridges, slopes, tunnels, sheds, etc.). Note that this is not limited to road features.
[0043] The terminal device 20 can communicate with these server devices. The three-dimensional point cloud data acquisition means 22 of the terminal device 20 acquires the contour area data of a desired feature from the attribute data server device 60. The three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40a based on the acquired contour area data of the feature, and acquires the three-dimensional point cloud data included in the area as the first three-dimensional point cloud data. Furthermore, the three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40b based on the acquired contour area data of the feature, and acquires the three-dimensional point cloud data included in the area as the second three-dimensional point cloud data.
[0044] The three-dimensional model construction means 24 constructs the three-dimensional shape of the feature based on the first three-dimensional point cloud data and the second three-dimensional point cloud data acquired above. In this way, by utilizing the fact that the data is outer area data with absolute coordinates, it is possible to collect three-dimensional point cloud data from different directions of the target feature that is recorded in a dispersed manner on the Internet, etc. Therefore, it is possible to form a more complete three-dimensional model of the feature.
[0045] 1.2 System Configuration 2 shows a system configuration of a point cloud data management system according to an embodiment. Point cloud server devices 40a, 40b, . . . 40n are provided on the Internet. Also provided is an attribute data server device 60. These server devices 40a, 40b, . . . 40n, 60 are provided with terminal devices 20a, 20b, . . . 20n that are capable of communicating with each other via the Internet.
[0046] The hardware configuration of the terminal device 20 is shown in Fig. 3. A memory 82, a display 84, a hard disk 86, a DVD-ROM drive 88, a keyboard / mouse 90, and a communication circuit 92 are connected to a CPU (or a GPU, etc.; the same applies below) 80. The communication circuit 92 is for connecting to the Internet.
[0047] An operating system 94 and a terminal program 96 are recorded on the hard disk 86. The terminal program 96 performs its functions in cooperation with the operating system 94. These programs were recorded on a DVD-ROM 98 and installed on the hard disk 86 via the DVD-ROM drive 88.
[0048] The point cloud server device 40 and the attribute data server device 60 have the same hardware configuration. However, in the point cloud server device 40, a point cloud server program is recorded instead of the terminal program 96. In addition, in the attribute data server device 60, an attribute data server program is recorded instead of the terminal program 96.
[0049] 1.3 Publishing process of point cloud data The surveying company generates 3D point cloud data and videos of roads and features using an MMS or the like mounted on a vehicle, etc. The generated 3D point cloud data is recorded on a recording medium such as a DVD-ROM, and is imported into the hard disk 86 of the terminal device 20.
[0050] The surveying company starts the terminal program 96 and generates attribute data based on the recorded three-dimensional point cloud data. A flowchart of the attribute data generation process in the terminal program 96 is shown in FIG.
[0051] First, the CPU 80 of the terminal device 20a reads out the recorded three-dimensional point cloud data (step S1). An example of the three-dimensional point cloud data is shown in FIG. 5. In this embodiment, the three-dimensional point cloud data is obtained by measuring the color of the object to be measured and the reflection intensity of the laser. Furthermore, the three-dimensional point cloud data includes information on the measurement method and the measurement equipment.
[0052] The CPU 80 groups points within a predetermined distance from each other in the three-dimensional point cloud data into clusters. This results in clusters of various sizes. The CPU 80 then removes clusters that are less than a predetermined volume, a predetermined number of points, or a predetermined density of points as noise (step S2).
[0053] Next, the CPU 80 extracts ground points (ground surface) using a Cloth Simulation method or the like (step S2). In the ground point extraction, the CPU 80 first inverts the elevation values of the three-dimensional point cloud data. For example, if there is three-dimensional point cloud data of a cross section as shown in Fig. 6A (point clouds are shown as lines in the figure), inverted three-dimensional point cloud data as shown in Fig. 6B can be obtained.
[0054] Next, the CPU 80 performs a simulation on the inverted 3D point cloud data as if a cloth was draped over it from above. The simulated cloth is shown in dashed lines in Fig. 6C. The CPU 80 then extracts the 3D point cloud data that the simulated cloth contacts as ground points. The CPU 80 then re-inverts the elevation values to obtain ground points as shown in Fig. 6D.
[0055] The ground points extracted in this way are generally accurate, but as shown in Figure 6D, some features may be included in the vicinity 100 where the features exist. Therefore, the normal direction of the line formed by each extracted ground point is calculated, and the part where the normal line forms a predetermined angle or more (for example, 30 degrees or more) with respect to the vertical direction is excluded from the ground points. This allows the ground points shown in Figure 6E to be obtained.
[0056] In this embodiment, Cloth Simulation is used for extracting ground points, but ground points may be extracted by other methods such as a lowest point extraction method.
[0057] After extracting the ground points as described above, the CPU 80 removes the ground points from the three-dimensional point cloud data, thereby obtaining three-dimensional point cloud data of only the features existing on the ground (step S4).
[0058] Next, the CPU 80 reads out design data that indicates the planar arrangement of the features recorded on the hard disk 86. This design data is a drawing of the features at the time of design, obtained by measuring the above-mentioned three-dimensional point cloud data. An example of the design drawing is shown in FIG. 7. The CPU 80 extracts the planar shape of each feature based on this design data (step S5). In this embodiment, the planar shape is extracted as a rectangle that contains the planar outer shape of the feature. Note that the planar outer shape of each feature may also be extracted as the planar shape as it is.
[0059] Next, the CPU 80 overlays the three-dimensional point cloud data of only the feature on this planar shape. The absolute coordinates of the design data on which the planar shape is based are matched with the absolute coordinates of the three-dimensional point cloud data to perform the overlay. At this time, the height position of the planar shape is matched with the height (altitude) of the ground point (ground surface) of the three-dimensional point cloud data. FIG. 8 shows the matched state. FIG. 8 shows only the footbridge portion of the features in FIG. 6 and FIG. 7. A frame 300 indicates a rectangular planar shape that surrounds the planar outer shape of the footbridge (feature), and it can be seen that the planar shape 300 and the position of the three-dimensional point cloud data of the footbridge (feature) match. Note that the planar outer shape is originally represented by a plane, but in FIG. 8, it is shown with a height for ease of understanding.
[0060] For each feature, the CPU 80 selects the one with the highest elevation from the three-dimensional point cloud data corresponding to the planar shape of the feature. The height is applied to the planar shape to determine the area of the feature. This makes it possible to identify a three-dimensional shape (rectangular parallelepiped) that surrounds the feature, as shown in FIG. 9. The CPU 80 records the three-dimensional shape thus generated as outline area data. In this embodiment, the outline area data is recorded based on the planar shape and the position and height based on absolute coordinates of the horizontal azimuth and height.
[0061] Next, the CPU 80 displays the area of each feature on the display 84, overlaying it on the three-dimensional point cloud data. The operator looks at this screen and selects the area of each feature with the mouse 90. This displays an input screen for attribute data such as the feature name, and the operator inputs the data. Furthermore, based on the coordinates of the vertices of the three-dimensional shape of the outer shape area data, the coordinates of the center position (or a position representative of the feature may be used) are calculated and set as the coordinate position of the feature. In this manner, attribute data can be generated and recorded. FIG. 10 shows the items of attribute data in this embodiment. The creation date and time is the creation date and time (measurement date and time) of the three-dimensional point cloud data. The feature name is the name of the feature. The feature area is the outer shape data showing the outer shape of the feature. The feature position is the center position calculated above. In this embodiment, the three-dimensional point cloud data is recorded in XML format.
[0062] In the above, the operator inputs the feature name, but the feature name may be estimated based on the three-dimensional point cloud data. For example, as shown in FIG. 11, the three-dimensional point cloud data 6 of the feature is projected onto a plane at a different angle, and the projected image and the feature name are used as learning data to train a deep learning program (other machine learning programs may be used). Learning is performed on various features to obtain a trained feature estimation program. The feature estimation program may be provided with three-dimensional point cloud data of the target feature to be estimated, and the feature name may be automatically estimated and assigned. Alternatively, the estimated feature name may be displayed to allow the operator to confirm it.
[0063] In addition, since there is a possibility that the three-dimensional point cloud data 6 and the outer region are misaligned, they may be displayed on the display 84, and the operator may align them using a mouse or the like so that their positions match.
[0064] In this manner, the attribute data can be obtained.
[0065] In the above, the outline area of the feature is determined by assigning height to the plane data. However, if there is no plane data such as design data, the outline area may be determined from the 3D point cloud data itself.
[0066] For example, the outline region is determined as follows: Divide the three-dimensional space into small cubic grids, and if there are points in the grids adjacent to each other above, below, left, right, or diagonally, combine them into one. For example, a spatial labeling technique using connected components can be used.
[0067] For each area organized as a grid containing points, areas with a certain volume, a certain number of points, or a certain density of points are removed as noise. The remaining areas correspond to features. For each area, a rectangular parallelepiped is set to surround the area. This rectangular parallelepiped is the outline of the feature.
[0068] In this manner, attribute data corresponding to the three-dimensional point cloud data can be generated in the terminal device 20. The surveying business entity delivers the three-dimensional point cloud data and the attribute data to the delivery destination.
[0069] Furthermore, at the discretion of the surveying business operator, only the attribute data can be uploaded to the attribute data server device 60. This allows a company that requires three-dimensional point cloud data to access the attribute data server device 60 and refer to the attribute data. If information on a company that provides the corresponding three-dimensional point cloud data is recorded in the attribute data, the company that requires the three-dimensional point cloud data can request the surveying business operator to obtain the three-dimensional point cloud data based on this information.
[0070] Also, the three-dimensional point cloud data may be uploaded to the point cloud server device 40. When the requester of the three-dimensional point cloud data is a local government or the like, they may be asked to upload the data to a specific point cloud server device 40. This makes the three-dimensional point cloud data and attribute data publicly available on the Internet.
[0071] Furthermore, the three-dimensional point cloud data and the outer area data of the attribute data are shown in absolute coordinates. Therefore, when there are multiple three-dimensional point cloud data for the same feature uploaded to different point cloud server devices by different businesses, these multiple three-dimensional point cloud data can be acquired as long as at least one attribute data is uploaded. For example, when there are three-dimensional point cloud data for the same feature from different directions by different businesses, more complete three-dimensional point cloud data can be obtained by acquiring both.
[0072] 1.4 3D model construction process Next, a process for constructing a three-dimensional model based on a plurality of three-dimensional point cloud data released as described above will be described.
[0073] The following describes the case where a three-dimensional model of a specific feature is constructed.
[0074] 12 and 13 show flowcharts for constructing a three-dimensional model. In the drawings, the terminal device is shown as a flowchart of the terminal program, the attribute data server device is shown as a flowchart of the attribute data server program, and the point cloud server device is shown as a flowchart of the point cloud server program.
[0075] First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as terminal device 20b) accesses the attribute data server 60 and specifies the location where the desired feature is located (step S11). The location is specified by specifying a range, for example, by latitude and longitude. An address or place name may be entered (or searched), and this may be converted into a latitude and longitude range. Also, an area clicked or selected on the map may be converted into a latitude and longitude range.
[0076] In response to this, the CPU of the attribute server device 60 (hereinafter sometimes abbreviated as the attribute server device) searches the attribute data and extracts the contour area data within the specified location (which may include the vicinity outside the location) (step S21). The attribute server device 60 transmits the extracted contour area data (planar shape data and its position and height in absolute coordinates), feature name, etc. to the terminal device 20b.
[0077] The terminal device 20b displays the outline region and the feature name on the display 84 based on the received outline region data (step S12). An example of the display is shown in Fig. 14. The outline regions 102 to 112 of each feature are shown, and the feature name is shown in the vicinity thereof.
[0078] The user operates the mouse 90 of the terminal device 20b to select the outline region of a desired feature. Here, the explanation will proceed assuming that the outline region 110 of the building has been selected. In response to this selection, the terminal device 20b specifies the outline region data of the outline region 110 of the building and transmits a request for three-dimensional point cloud data to each of the point cloud server devices 40a to 40n (step S13).
[0079] Each of the point cloud server devices 40a to 40n that receives this request determines whether it holds the three-dimensional point cloud data of the outer region 110, and returns it if it does. Here, the explanation will be given assuming that two of the point cloud server devices 40a and 40b among the point cloud server devices 40a to 40n hold the three-dimensional point cloud data of the outer region 110 of the building.
[0080] The CPU of the point cloud server device 40a (hereinafter, may be abbreviated as the point cloud server device 40a) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S31).
[0081] Similarly, the CPU of the point cloud server device 40b (hereinafter, may be abbreviated as the point cloud server device 40b) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S41).
[0082] The point cloud server devices 40a and 40b transmit three-dimensional point cloud data of an area that is expanded to take into account measurement errors relative to the specified external area 110. Measurement errors vary depending on the measurement method and equipment, and therefore also vary depending on the measurement method and equipment used to measure the three-dimensional point cloud data. Therefore, the expanded area varies depending on the measurement method and equipment.
[0083] The terminal device 20b receives three-dimensional point cloud data (first three-dimensional point cloud data) from the point cloud server device 40a and three-dimensional point cloud data (second three-dimensional point cloud data) from the point cloud server device 40b. An example of the first three-dimensional point cloud data is shown in FIG. 15A, and an example of the second three-dimensional point cloud data is shown in FIG. 15B. As shown in FIG. 15A and FIG. 15B, the first three-dimensional point cloud data is measured from the front side facing the paper surface of the rectangular parallelepiped. The second three-dimensional point cloud data is measured from the rear side facing the paper surface of the rectangular parallelepiped.
[0084] The terminal device 20b aligns and synthesizes the first three-dimensional point cloud data and the second three-dimensional point cloud data to generate synthesized three-dimensional point cloud data as shown in Fig. 16 (step S14). As is clear from this figure, it is possible to obtain three-dimensional point cloud data with a larger amount of information (having both measurement data from the front and rear sides) than when the first three-dimensional point cloud data and the second three-dimensional point cloud data are used alone.
[0085] The composite 3D point cloud data thus obtained can be used for various purposes. In this embodiment, a 3D model of a feature is generated based on the composite 3D point cloud data.
[0086] The terminal device 20b generates an outer surface of the feature based on the composite 3D point cloud data (step S15). This process can be performed, for example, by a 3D convex hull algorithm (http: / / www.qhull.org / download / ). FIG. 17A shows the generated 3D model of the feature. It is possible to generate a 3D model that is more complete than a 3D model generated only from the first 3D point cloud data and a 3D model generated only from the second 3D point cloud data. The terminal device 20b displays this on the display 84 and records it on the hard disk 86 (step S15).
[0087] According to this embodiment, it is possible to easily search for multiple 3D point cloud data scattered on the Internet for the same feature, and to obtain composite 3D point cloud data by synthesizing the data. Also, if one attribute data is generated based on any one of the multiple 3D point cloud data for the same feature, the above search and synthesis can be performed. This is because the attribute data is independent of the 3D point cloud data, and the outer region data of the attribute data is defined in absolute coordinates.
[0088] 1.5 Other (1) In the above embodiment, two pieces of three-dimensional point cloud data are acquired to generate the composite three-dimensional point cloud data. However, three or more pieces of three-dimensional point cloud data may be acquired to generate the composite three-dimensional point cloud data.
[0089] (2) In the above embodiment, composite 3D point cloud data is obtained for one feature. However, composite 3D point cloud data may be obtained for multiple features. Also, composite 3D data may be obtained for all features within a predetermined range. Such examples are shown in Figures 18A, 18B, and 18C. Figure 18A shows the first 3D point cloud data, Figure 18B shows the second 3D point cloud data, and Figure 18C shows the composite 3D point cloud data.
[0090] (3) In the above embodiment, 3D point cloud data with different measurement directions is synthesized. However, 3D point cloud data with different measurement distances to features or 3D point cloud data with different measurement instruments (for example, measurements by MMS and UAV) may be synthesized.
[0091] (4) In the above embodiment, the case where only one attribute data is found for the location in step S21 has been described. However, there are cases where different measurement companies create three-dimensional point cloud data for the same location and each uploads attribute data, or even the same company creates three-dimensional point cloud data for the same location at different times and each uploads attribute data. In such cases, multiple attribute data are found for the specified location. In this case, the attribute data server device 60 transmits a message to the terminal device 20b that multiple attribute data are found. The terminal device 20b displays the creation date and time, the creation device, etc. for the multiple attribute data on the display 84 and allows the operator to select one. Based on the selected attribute data, the outer shape area is transmitted (step S21).
[0092] (5) In the above embodiment, the terminal device 20b makes the three-dimensional point cloud data request (step S13). However, the attribute data server device 60 may make this request and transmit the result to the terminal device 20b.
[0093] (6) In the above embodiment, the synthesis of the 3D point cloud data and the generation of the 3D model are performed in the terminal device 20b. However, either one or both of these may be performed in the attribute data server device 60, and the results may be transmitted to the terminal device 20b.
[0094] (7) In the above embodiment, when the terminal device 20b acquires the three-dimensional point cloud data in step S14, it does not display from which direction the data was measured. However, to clarify this, as shown in FIG. 17B, an arrow may be used to display the measurement direction and distance to the feature based on the data attached to the acquired three-dimensional point cloud data. The base of the arrow indicates the measurement position, and the direction of the arrow indicates the measurement direction. In the case of FIG. 17B, it is clear that there is three-dimensional point cloud data measured from two locations.
[0095] Therefore, based on this screen display, it is clear that the next time a measurement is made, it is preferable to measure from a direction other than the arrow if the quality of the composite 3D point cloud data is to be improved. Also, if the user wishes to know the temporal changes of features, it is preferable to make a measurement from the same position and direction as the arrow.
[0096] (8) In the above embodiment, the first and second three-dimensional point cloud data are simply combined to generate the combined three-dimensional point cloud data. However, when the first three-dimensional point cloud data and the second three-dimensional point cloud data exist for the same location among the three-dimensional point cloud data, only the three-dimensional point cloud data with high measurement accuracy for that location may be used to generate the combined three-dimensional point cloud data.
[0097] (9) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are configured as separate server devices. However, they may be configured as a single server device.
[0098] (10) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b and the attribute data server device 60 were constructed as separate server devices, but they may also be constructed as a single server device.
[0099] (11) The above-described embodiments and variations can be implemented in combination with other embodiments as long as this does not go against the essence of the invention.
[0100] 2. Second embodiment 2.1 Overall configuration Fig. 19 shows the overall configuration of a point cloud data management system according to a second embodiment of the present invention. A first three-dimensional point cloud data file at a first date and time based on absolute coordinates including features is recorded in a first point cloud server device 40a. A second three-dimensional point cloud data file at a second date and time based on absolute coordinates including features is recorded in a second point cloud server device 40b. An attribute data server device 60 records outer shape area data based on absolute coordinates that surrounds the outer shape of features.
[0101] The terminal device 20 can communicate with these server devices. The three-dimensional point cloud data acquisition means 22 of the terminal device 20 acquires the contour area data of a desired feature from the attribute data server device 60. The three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40a based on the acquired contour area data of the feature, and acquires the three-dimensional point cloud data included in the area as the first three-dimensional point cloud data. Furthermore, the three-dimensional point cloud data acquisition means 22 accesses the first point cloud server device 40b based on the acquired contour area data of the feature, and acquires the three-dimensional point cloud data included in the area as the second three-dimensional point cloud data.
[0102] The time-dependent change acquisition means 25 calculates the change in shape of the feature between the first date and time and the second date and time based on the acquired first three-dimensional point cloud data at the first date and time and the acquired second three-dimensional point cloud data at the second date and time.
[0103] 2.2 System Configuration The system configuration is the same as in the first embodiment.
[0104] 2.3 Calculation of changes over time Next, a process for acquiring changes over time of features based on a plurality of published three-dimensional point cloud data will be described.
[0105] 20 and 21 show flowcharts for acquiring changes over time of features. In the figures, the terminal device is shown as a flowchart of the terminal program, the attribute data server device is shown as a flowchart of the attribute data server program, and the point cloud server device is shown as a flowchart of the point cloud server program.
[0106] First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as terminal device 20b) accesses the attribute data server 60 and specifies the location where the desired feature is located (step S11). The location is specified by specifying a range, for example, by latitude and longitude. An address or place name may be entered (or searched), and this may be converted into a latitude and longitude range. Also, an area clicked or selected on the map may be converted into a latitude and longitude range.
[0107] In response to this, the CPU of the attribute server device 60 (hereinafter sometimes abbreviated as the attribute server device) searches the attribute data and extracts the outline area data within the specified location (step S21). The attribute server device 60 transmits the extracted outline area data (planar shape data and its position and height in absolute coordinates), feature name, etc. to the terminal device 20b.
[0108] The terminal device 20b displays the outline region and the feature name on the display 84 based on the received outline region data (step S12). An example of the display is shown in Fig. 14. The outline regions 102 to 112 of each feature are shown, and the feature name is shown in the vicinity thereof.
[0109] The user operates the mouse 90 of the terminal device 20b to select the outline region of a desired feature. Here, the explanation will proceed assuming that the outline region 110 of the building has been selected. In response to this selection, the terminal device 20b specifies the outline region data of the outline region 110 of the building and transmits a request for three-dimensional point cloud data to each of the point cloud server devices 40a to 40n (step S13).
[0110] Each of the point cloud server devices 40a to 40n that receives this request determines whether it holds the three-dimensional point cloud data of the outer region 110, and returns it if it does. Here, the explanation will be given assuming that two of the point cloud server devices 40a and 40b among the point cloud server devices 40a to 40n hold the three-dimensional point cloud data of the outer region 110 of the building.
[0111] The CPU of the point cloud server device 40a (hereinafter, may be abbreviated as the point cloud server device 40a) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S31).
[0112] Similarly, the CPU of the point cloud server device 40b (hereinafter, may be abbreviated as the point cloud server device 40b) returns the three-dimensional point cloud data of the outer region 110 of the building to the terminal device 20b (step S41).
[0113] The terminal device 20b receives three-dimensional point cloud data (first three-dimensional point cloud data) from the point cloud server device 40a and three-dimensional point cloud data (second three-dimensional point cloud data) from the point cloud server device 40b. Here, the first three-dimensional point cloud data and the second three-dimensional point cloud data are obtained by measuring features from substantially the same direction. However, the measurement dates and times are different.
[0114] The terminal device 20b generates a three-dimensional model of the feature at the first date and time based on the first three-dimensional point cloud data (step S17), and also generates a three-dimensional model of the feature at the second date and time based on the second three-dimensional point cloud data (step S17).
[0115] The terminal device 20b displays the generated three-dimensional model at the first date and time and the three-dimensional model at the second date and time on the display 84 so that they can be compared (step S18). For example, as shown in FIG. 22, a three-dimensional model 120 at the first date and time and a three-dimensional model 122 at the second date and time are displayed. Below that, the name of the feature and the date of measurement are displayed. Therefore, in the example of FIG. 22, it can be seen by comparing the changes in the external shape of the building on February 24, 2010 and the external shape of the building on March 5, 2019.
[0116] For ease of comparison, the two may be displayed in different colors.
[0117] Also, for example, if there is a situation where a feature does not exist at the first date and time but does exist at the second date and time (for example, a new road sign has been installed), 3D point cloud data of the feature can be obtained if there is measurement data of the location at the second date and time. However, even if there is measurement data of the location at the first date and time, 3D point cloud data of the feature cannot be obtained. Therefore, by displaying a 3D model based on this, it becomes clear that the feature does not exist at the first date and time but exists at the second date and time.
[0118] For the purpose of the above-mentioned time-series comparison, it is preferable to use three-dimensional point cloud data in which the measurement direction relative to the feature is the same (or similar).
[0119] 2.4 Other (1) In the above embodiment, a 3D model is generated using 3D point cloud data at two dates and times, and then a comparison is performed. However, 3D point cloud data at three or more dates and times may be acquired, 3D models may be generated for each, and then a comparison may be performed.
[0120] (2) In the above embodiment, a comparison over time is performed for one feature. However, a comparison over time may be performed for multiple features. Also, a comparison over time may be performed for all features within a predetermined range.
[0121] (3) In the above embodiment, the terminal device 20b makes the three-dimensional point cloud data request (step S13). However, the attribute data server device 60 may make this request and transmit the result to the terminal device 20b.
[0122] (6) In the above embodiment, the three-dimensional model is generated in the terminal device 20b. However, the generation may be performed in the attribute data server device 60, and the result may be transmitted to the terminal device 20b.
[0123] (7) In the above embodiment, when the terminal device 20b acquires the three-dimensional point cloud data in step S17, the date and time when each piece of three-dimensional point cloud data was measured is not displayed. However, in order to clarify this, the measurement date and time of each piece of three-dimensional point cloud data may be displayed based on the data attached to the acquired three-dimensional point cloud data, as shown in FIG. 23. When there is three-dimensional point cloud data with many dates and times, it is preferable to display such a display. The user can view this, select a predetermined number of dates to be compared, and display the three-dimensional model.
[0124] (8) The above embodiment and modified examples can be combined with other embodiments as long as they are not contrary to the essence of the embodiment and modified examples. For example, when there are multiple 3D point cloud data from different directions at the same date and time (or the same month or year), a composite 3D point cloud data may be generated to generate a 3D model, which may be compared with the 3D model at another date and time.
[0125] (9) In the above embodiment, the first point cloud server device 40a and the second point cloud server device 40b are configured as separate server devices. However, they may be configured as a single server device.
[0126] (10) In the above embodiment, the first point cloud server device 40a, the second point cloud server device 40b and the attribute data server device 60 were constructed as separate server devices, but they may also be constructed as a single server device.
[0127] (11) In the above embodiment, the searched video data is played back on the terminal device. At this time, a map of the vicinity where the video was captured, three-dimensional point cloud data, etc. may be displayed together. Also, three-dimensional point cloud data viewed from an imaging position that changes with time may be displayed so as to match the playback of the video.
[0128] (12) In the above embodiment, the 3D models of the features at two points in time are displayed for comparison. However, the normal vectors of the outlines of the 3D models may be displayed for both models to make it easier to see the angle changes of the surface.
[0129] It is also possible to divide the area into small areas (25 cm cubic areas) and compare and display the number of points within each area. This allows the roughness of the surface of features to be compared.
[0130] Furthermore, the reflection intensity of each point or area may be compared, allowing the smoothness of the surface to be compared.
[0131] In addition, the volumes of the three-dimensional models may be compared, which allows the growth of plants, the condition of leaves, and the like to be compared.
[0132] (13) The above-described embodiments and variations can be implemented in combination with other embodiments as long as this does not go against the essence of the invention.
[0133] 3. Third embodiment 3.1 Overall configuration 24 shows a functional block diagram of a feature search system according to the third embodiment of the present invention. Attribute data such as feature names, outer area, and positions of three-dimensional point cloud data, and video data are recorded in a recording unit 158 of the feature search server device 150.
[0134] The attribute data transmission means 150 of the feature search server device 150 reads out the feature name or outer area corresponding to the three-dimensional point cloud data recorded in the recording unit 158, and transmits it to the terminal device 40. The feature attribute data display means 182 of the terminal device 40 displays the received feature name or outer area of the feature on the display unit 188.
[0135] The operator of the terminal device 40 uses the mouse 90 to look at the feature names or outline areas of the displayed features and selects the feature for which he or she wishes to search for video. The selection information transmitting means 184 transmits to the feature search server device 150 which feature has been selected.
[0136] The feature position acquisition means 154 of the feature search server device 150 acquires the position of the selected feature. The video data transmission means 156 searches the video data in chronological order and extracts a portion with imaging position information attached near the feature position. The extracted video data is transmitted to the terminal device 40. The video playback means 186 of the terminal device 40 plays the received video data on the display unit 188.
[0137] This makes it possible to easily search for and display locations where desired features are captured from a huge amount of video data.
[0138] 3.2 System Configuration Fig. 25 shows the system configuration of a point cloud data management system according to the third embodiment. Point cloud server devices 40a, 40b,..., 40n are provided on the Internet. In addition, a feature search server device 62 is also provided. In the point cloud server devices 40a, 40b,..., 40n, three-dimensional point cloud data of measured features and video data (with imaging position information) captured during measurement are recorded. In the feature search server device 62, attribute data as shown in Fig. 10 is recorded. In these server devices 40a, 40b,..., 40n, 60, terminal devices 20a, 20b,..., 20n are provided so as to be able to communicate with each other via the Internet.
[0139] The hardware configuration of the terminal device 20 is the same as that shown in Fig. 3. The point cloud server device 40 and the feature search server device 62 also have the same hardware configuration. However, in the point cloud server device 40, a point cloud server program is recorded instead of the terminal program 96. Also, in the feature search server device 62, a feature search server program is recorded instead of the terminal program 96.
[0140] 3.3 Feature search process A flowchart of feature search is shown in Fig. 26. In the figure, what is shown as a terminal device is a flowchart of a terminal program, and what is shown as a feature search server device is a flowchart of a feature search server program.
[0141] First, the CPU 80 of the terminal device 20b (hereinafter sometimes abbreviated as terminal device 20b) accesses the feature search server device 62 and specifies the location where the desired feature is located (step S11). The location is specified by specifying a range, for example, using latitude and longitude. An address or place name may be entered (or searched), and this may be converted into a latitude and longitude range. Also, an area clicked or selected on the map may be converted into a latitude and longitude range.
[0142] In response to this, the CPU of the feature search server device 62 (hereinafter sometimes abbreviated as feature search server device 62) searches the attribute data and extracts the outline region data within the specified location (step S51). The feature search server device 62 transmits the extracted outline region data (planar shape data and its position and height in absolute coordinates), feature name, etc. to the terminal device 20b.
[0143] The terminal device 20b displays the outline region and the feature name on the display 84 based on the received outline region data (step S12). An example of the display is shown in Fig. 14. The outline regions 102 to 112 of each feature are shown, and the feature name is shown in the vicinity thereof.
[0144] The user operates the mouse 90 of the terminal device 20b to select the outline region of the desired feature. Here, the explanation will proceed assuming that the outline region 106 of the sign has been selected. In response to this selection, the terminal device 20b transmits an instruction to the feature search server device 62 to specify the outline region data of the outline region 106 of the sign and to search for the part of the video in which this sign is captured (step S19).
[0145] The feature search server device 62 reads attribute data of the specified feature and acquires the position of the feature (position in absolute coordinates) (step S52). Next, the feature search server device 62 extracts a portion of the captured video data 162 in which the feature is captured, based on the feature position (step S53).
[0146] 27 shows the data structure of the captured video data 162. Time stamps TS1, TS2, ... indicating the time of capture are recorded in association with the video main body data 170. The position of the camera (or the car equipped with the camera) that captured the video is recorded in absolute coordinates. Although not shown, the captured video data 162 is recorded in association with imaging area information for the entire video.
[0147] Fig. 28 shows the concept of imaging area information. The imaging path R is shown by a solid line. It shows that imaging started from a starting point S and was performed up to an end point E. The imaging area AR is shown by a dashed line that circumscribes this imaging path R. The imaging area AR is specified by, for example, the absolute coordinates of the diagonal (imaging area information).
[0148] Fig. 29 shows details of the process of extracting video portions in step S53. Based on the position of the feature to be searched, the feature search server device 62 selects video data 162 whose imaging area includes the measurement position (step S531). The video data 162 selected in this way may include a portion in which the feature is imaged. The feature search server device 62 searches each of the selected video data 162 to determine whether the imaged portion is present.
[0149] In step S533, the feature search server device 62 reads out the imaging positions recorded in the video data 162 in chronological order (step S533). In the example of Fig. 27, the imaging positions PS1, PS2, ..., PSn are read out.
[0150] The feature search server device 62 identifies the imaging position PSf that is closest to the location of the feature among the read imaging positions PS1, PS2, ..., PSn (step S534). The feature search server device 62 judges whether the distance between the closest imaging position PSf and the feature position is equal to or less than a predetermined value (step S535). If it is equal to or less than the predetermined value, it means that the feature is imaged at that location. The feature search server device 62 extracts video data for a predetermined time before and after the identified imaging position PSf (step S536). If the distance between the imaging position PSf and the feature position exceeds the predetermined value, the video portion is not extracted.
[0151] The above process is performed for all the moving images selected in step S531 (steps S532, S537).
[0152] When the video data portion is extracted in this manner, the feature search server device 62 transmits it to the terminal device 20b (step S54).
[0153] In response to this, the terminal device 20b displays the extracted portion of the video data on the display 84 (step S20). When there are multiple video data, it is preferable to display a list of the video data together with the shooting date and time before playback, and allow the user to select one to play.
[0154] The user can view this video and understand the condition of the feature that is the subject of the search as an image. For example, it can be used when wanting to know the condition of the feature that can be specified on the screen display of Fig. 14 (confirmation of looseness detection marks (e.g., match marks) on bolts and nuts, confirmation of the deterioration of the appearance, etc.).
[0155] In addition, the above detection mark and the appearance deterioration status may be checked by the terminal device using image processing or AI.
[0156] 3.4 Other (1) In the above embodiment, the feature positions in the attribute data and the capture positions in the video data are expressed in absolute coordinates. Therefore, even if attribute data corresponding to each piece of video data is not prepared, one piece of attribute data is enough to search for features in multiple pieces of video data.
[0157] In addition, if attribute data is provided for each video data, the feature position and the image capture position do not need to be absolute coordinates as long as they are consistent between the two data. In addition, if there is attribute data corresponding to the video data, it is also possible to perform a search based on the date and time when the 3D point cloud data of the feature was created and the date and time when it was photographed, rather than a search based on the feature position and the image capture position.
[0158] (2) In the above embodiment, the feature to be searched for is specified by the outline area and the feature name as shown in Fig. 14. However, it is also possible to display only the feature name without displaying the outline area and to allow the user to select it.
[0159] (3) In the above embodiment, the portion of the video in which the feature is captured is extracted and played back. However, in addition to this, the feature may be marked in the video and played back.
[0160] The feature search server device 62 can achieve this by performing the following process. For example, assume that there is an outline area of a feature as shown in Fig. 30, and a sign 106 is to be searched for. In the space in which this outline area is located, a trajectory 200 is drawn when a video is captured. Since time is added to this trajectory 200, the image capturing time at each point can be known. At each image capturing time t1, t2, ..., it is determined in which direction and at what distance the target feature 106 exists.
[0161] Based on this direction and distance, it calculates how the feature 106 is imaged at the corresponding time in the captured video. The appearance of the outer region of the feature 106 generated in this way at each time is superimposed on the extracted video and transmitted to the terminal device 20b.
[0162] When this is played back on the terminal device 20b, a frame of an outline region is displayed so as to surround the target feature in the image, as shown in Fig. 31. Therefore, the user can easily find the target feature in the video.
[0163] (4) In the above embodiment, the point cloud server device 40 and the search server device 62 are configured as separate servers, but they may be configured as a single server device.
[0164] (5) In the above embodiment, a case where a video is searched for is described. However, a still image with a shooting location and shooting time can also be searched for by using the same process.
[0165] (6) The above-described embodiments and variations can be implemented in combination with other embodiments as long as this does not go against the essence of the invention.
Claims
1. A point cloud data management system including first and second point cloud data server devices, an attribute data server device, and a plurality of terminal devices capable of communicating with these server devices, the first point cloud server device records a first three-dimensional point cloud data file from a first direction based on absolute coordinates including features; the second point cloud server device records a second three-dimensional point cloud data file from a second direction based on absolute coordinates including the same feature as the feature; the attribute data server device records outer shape area data that encloses the outer shape of the feature and is indicated by absolute coordinates; The terminal device a three-dimensional point cloud data acquisition means for acquiring outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device; a three-dimensional model constructing means for constructing a three-dimensional shape of the feature based on the first and second three-dimensional point cloud data from the first and second directions; A point cloud data management system equipped with
2. a terminal device capable of communicating with a first point cloud server device which records a first three-dimensional point cloud data file from a first direction using absolute coordinates including a feature; a second point cloud server device which records a second three-dimensional point cloud data file from a second direction using absolute coordinates including the same feature as the first point cloud data file; and an attribute data server device which records outer shape area data indicated by absolute coordinates that surrounds the outer shape of the feature, a three-dimensional point cloud data acquisition means for acquiring outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device; a three-dimensional model constructing means for constructing a three-dimensional shape of the feature based on the first and second three-dimensional point cloud data from the first and second directions; A terminal device comprising:
3. A terminal program for implementing, by a computer, a terminal device capable of communicating with a first point cloud server device that records a first three-dimensional point cloud data file from a first direction using absolute coordinates including a feature, a second point cloud server device that records a second three-dimensional point cloud data file from a second direction using absolute coordinates including the same feature as the first point cloud data file, and an attribute data server device that records outer shape area data indicated by absolute coordinates that surrounds the outer shape of the feature, the terminal device comprising: a three-dimensional point cloud data acquisition means for acquiring outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device; A terminal program for causing the device to function as a three-dimensional model construction means for constructing a three-dimensional shape of the feature based on the first and second three-dimensional point cloud data acquired from the first and second directions.
4. A point cloud data management system including first and second point cloud data server devices, an attribute data server device, and a plurality of terminal devices capable of communicating with these server devices, The first point cloud server device records a first three-dimensional point cloud data file at a first date and time based on absolute coordinates including features, the second point cloud server device records a second three-dimensional point cloud data file at a second date and time based on absolute coordinates including the same feature as the feature; the attribute data server device records outer shape area data that encloses the outer shape of the feature and is indicated by absolute coordinates; The terminal device a three-dimensional point cloud data acquisition means for acquiring outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device; a time-dependent change acquisition means for acquiring a time-dependent change of the feature based on the acquired first and second three-dimensional point cloud data at the first and second dates and times; A point cloud data management system equipped with
5. a terminal device capable of communicating with a first point cloud server device that records a first three-dimensional point cloud data file at a first date and time based on absolute coordinates including a feature, a second point cloud server device that records a second three-dimensional point cloud data file at a second date and time based on absolute coordinates including the same feature as the first point cloud server device, and an attribute data server device that records outer shape area data indicated by absolute coordinates that surrounds the outer shape of the feature, a three-dimensional point cloud data acquisition means for acquiring outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device; a time-dependent change acquisition means for acquiring a time-dependent change of the feature based on the acquired first and second three-dimensional point cloud data at the first and second dates and times; A terminal device comprising:
6. A terminal program for implementing, by a computer, a terminal device capable of communicating with a first point cloud server device that records a first three-dimensional point cloud data file at a first date and time based on absolute coordinates including a feature, a second point cloud server device that records a second three-dimensional point cloud data file at a second date and time based on absolute coordinates including the same feature as the first point cloud server device, and an attribute data server device that records outer shape area data indicated by absolute coordinates that surrounds the outer shape of the feature, the terminal device comprising: a three-dimensional point cloud data acquisition means for acquiring outer region data of the feature from the attribute data server device, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file from the first point cloud server device based on the outer region data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file from the second point cloud server device; A terminal program for causing the terminal to function as a time-course change acquisition means for acquiring the change in the feature over time based on the first and second three-dimensional point cloud data acquired at the first and second dates and times.
7. In any one of the systems, devices, or programs according to claims 1 to 6, A system, device, or program, characterized in that the first point cloud server device and the second point cloud server device are constructed as a single point cloud server device.
8. The system, device or program of claim 7, The attribute data server device is a server device constructed integrally with the point cloud server device.
9. A first three-dimensional point cloud data file from a first direction based on absolute coordinates including a feature is recorded, and a second three-dimensional point cloud data file from a second direction based on absolute coordinates including the same feature as the first three-dimensional point cloud data file is recorded; Recording outer region data that encloses the outer shape of the feature and is indicated by absolute coordinates; by a computer, acquiring outline area data of the feature, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file based on the outline area data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file; A point cloud data management method for obtaining synthesized three-dimensional point cloud data based on first and second three-dimensional point cloud data acquired from first and second directions by a computer.
10. A first three-dimensional point cloud data file at a first date and time based on absolute coordinates including a feature is recorded, and a second three-dimensional point cloud data file at a second date and time based on absolute coordinates including the same feature as the first three-dimensional point cloud data file is recorded; Recording outer region data that encloses the outer shape of the feature and is indicated by absolute coordinates; by a computer, acquiring outline area data of the feature, acquiring first three-dimensional point cloud data of the feature corresponding to a first three-dimensional point cloud data file based on the outline area data, and acquiring second three-dimensional point cloud data of the feature corresponding to a second three-dimensional point cloud data file; A point cloud data management method in which a computer obtains changes in a feature over time based on first and second three-dimensional point cloud data obtained at first and second dates and times.
11. A feature search system including a point cloud data server device, an attribute data server device, and a plurality of terminal devices capable of communicating with these server devices, The server device includes: a recording unit that records attribute data including video data with image capture position information of the features captured when the three-dimensional point cloud data of the multiple features is measured while moving, and outer shape area data surrounding the outer shapes of the features; an attribute data transmission means for transmitting attribute data of the plurality of features to a terminal device for a user to select the feature; a feature position acquisition means for receiving feature selection information from a terminal device and acquiring a feature position corresponding to the contour area data included in the attribute data of the selected feature; a video data extraction means for extracting a portion of the video data that is associated with imaging position information in the vicinity of the feature position and transmitting the portion to a terminal device; The terminal device a feature attribute data display means for acquiring attribute data of the features from the server device and displaying the data on a display unit for a user to select a feature; A selection information transmission means for transmitting information on the features selected by a user to a server device; a video data reproducing means for receiving the extracted video data transmitted from the server device and reproducing the video data on a display unit; A feature search system with
12. a recording unit that records attribute data including video data with image capture position information of the features captured when the three-dimensional point cloud data of the multiple features is measured while moving, and outer shape area data surrounding the outer shapes of the features; an attribute data transmission means for transmitting attribute data of the plurality of features to a terminal device for a user to select the feature; a feature position acquisition means for receiving feature selection information from a terminal device and acquiring a feature position corresponding to the contour area data included in the attribute data of the selected feature; a video data extraction means for extracting a portion of the video data that is associated with imaging position information in the vicinity of the feature position and transmitting the portion to a terminal device; A feature search server device comprising:
13. A feature search server program for implementing a feature search server device by a computer, the program comprising: an access means for accessing a recording unit that records attribute data including video data with image capture position information of the features captured when the three-dimensional point cloud data of the multiple features is measured while moving, and outer shape area data surrounding the outer shapes of the features; an attribute data transmission means for transmitting attribute data of the plurality of features to a terminal device for a user to select the feature; a feature position acquisition means for receiving feature selection information from a terminal device and acquiring a feature position corresponding to the contour area data included in the attribute data of the selected feature; a feature search server program for causing the server to function as a video data extraction means for extracting a portion of the video data to which imaging position information in the vicinity of the feature position has been added and transmitting the portion to a terminal device;
14. a feature attribute data display means for acquiring feature attribute data from a server device having an access means for accessing a recording unit that records attribute data including video data with image capture position information of the features captured when measuring three-dimensional point cloud data of a plurality of features while moving and outer shape area data surrounding the outer shapes of the features, and for displaying the feature attribute data on a display unit for a user to select the feature; A selection information transmission means for transmitting information on the features selected by a user to a server device; a video data reproducing means for receiving the extracted video data transmitted from the server device and reproducing the video data on a display unit; A terminal device comprising:
15. A terminal program for implementing a terminal device by a computer, the program comprising: a feature attribute data display means for acquiring feature attribute data from a server device having an access means for accessing a recording unit that records attribute data including video data with image capture position information of the features captured when measuring three-dimensional point cloud data of a plurality of features while moving and outer shape area data surrounding the outer shapes of the features, and for displaying the feature attribute data on a display unit for a user to select the feature; A selection information transmission means for transmitting information on the features selected by a user to a server device; A terminal program for causing the terminal to function as a video data playback means for receiving the extracted video data transmitted from the server device and playing the video data on the display unit.
16. In any one of the systems, devices, or programs according to claims 11 to 15, The position of the outer shape area data is indicated by absolute coordinates, The video data extraction means extracts from a plurality of video data portions a portion of the video data that is associated with imaging position information in the vicinity of the feature position, and transmits the plurality of extracted video data portions.
17. In any one of the systems, devices, or programs according to claims 11 to 16, The video data extraction means includes: A system, device, or program characterized by comprising an in-video feature identification means for acquiring three-dimensional point cloud data of a selected feature, and identifying and marking the corresponding feature in the extracted video based on at least the contour of the three-dimensional point cloud data.
18. In any one of the systems, devices, or programs according to claims 11 to 17, The server device is configured to include a plurality of individual server devices, The system, device or program, wherein the video data or the attribute data is recorded in different individual server devices.
19. Recording attribute data including video data with image capture position information of the features captured while moving and outline area data surrounding the outlines of the features, when measuring three-dimensional point cloud data of a plurality of features while moving; a computer displays attribute data of the plurality of features for a user to select a feature; The computer receives the selection, obtains a feature position corresponding to the outline area data included in the attribute data of the selected feature, and A feature search method, comprising: a computer extracting from said video data a portion near said feature position to which imaging position information has been added, and displaying the portion.
20. A plane outline determining means for determining a plane outline based on a plan view of a feature having absolute coordinates; A means for determining the lower end and the upper end in the elevation direction based on the three-dimensional point cloud data of the feature; a three-dimensional outline determining means for determining an outline region data based on a three-dimensional outline obtained by connecting the planar outline from the lower end to the upper end; An outer shape area data generating device comprising:
21. An outline area data generating program for implementing an outline area data generating device by a computer, the program comprising: A plane outline determining means for determining a plane outline based on a plan view of a feature having absolute coordinates; A means for determining the lower end and the upper end in the elevation direction based on the three-dimensional point cloud data of the feature; an outline area data generating program for causing the program to function as a three-dimensional outline determining means for determining an outline area data to be a three-dimensional outline obtained by continuing the planar outline from the lower end to the upper end.
Citation Information
Patent Citations
Video processing device and video processing method
JP2017005299A
Image management apparatus, image management method and program
JP2017130146A
Facility management data extracting device, and facility management data extracting method
JP2017133838A
Video display method and video display device
US20180343442A1
Cited By
Fastener system with stabilizer ribs
US12384007B2
