Structure survey support system, structure survey support method and program

Abstract

[Problem] To make it easier to investigate structures around roads. [Solution] A structure investigation assistance system comprises: a first storage means that stores road image data captured continuously by a camera; a second storage means that stores detailed information on a structure around the road in association with position information of the structure on a map; a first selection means that selects image data for an investigation request from the first storage means when an investigation request for the structure is received; a second selection means that selects detailed information on the structure targeted for the investigation request from the second storage means; and an information presentation means that presents a map indicating the location of the structure, image data selected by the first selection means, and detailed information on the structure. The detailed information on the structure is configured using point cloud data obtained by 3D scanning the structure.

Description

[Technical field]

[0001] The present invention relates to a structure inspection support system, a structure inspection support method, and a recording medium. [Background technology]

[0002] Patent Document 1 discloses a road structure management system that can easily manage road structures. According to this document, a vehicle-side information processing device is disclosed that detects road structures from image data, identifies the type of the road structure, and then records image data in which the road structure is captured in association with the position and time at which the image data was captured. The document also discloses a second information processing device that maps the detected road structures on map data based on information on the road structures received from the vehicle-side information processing device, and makes it possible to display the map data together with information indicating the road structures. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-151967 A Summary of the Invention [Problem to be solved by the invention]

[0004] The road structure management system in Patent Document 1 discloses a configuration in which detailed information can be obtained by selecting information about a road structure displayed on a map. However, depending on the type of road structure, there may be multiple identical structures of the same type, which can make it difficult to identify the structure to be checked.

[0005] In addition, the road structure management system in Patent Document 1 identifies the type of road structure using image data captured by the vehicle's forward surveillance camera, so if the road structure is not clearly visible in the image data, it is expected that it will be difficult to identify it.

[0006] An object of the present invention is to provide a structure inspection support system, a structure inspection support method, and a recording medium that can contribute to facilitating the inspection of structures around roads. [Means for solving the problem]

[0007] According to a first aspect, there is provided a structure inspection support system including a first storage means for storing image data of a road continuously photographed by a camera, a second storage means for storing detailed information of structures around the road in association with position information of the structures on a map, a first selection means for selecting image data of the object of the inspection request from the first storage means when an inspection request for the structure is received, a second selection means for selecting detailed information of the object of the inspection request from the second storage means, and an information presentation means for presenting a map showing the location of the structure, the image data selected by the first selection means, and the detailed information of the structure. The structure inspection support system also includes a point cloud data obtained by 3D scanning the structure to configure the detailed information of the structure. 。

[0008] According to a second aspect, a structure inspection support method is provided which receives a request to inspect a structure, selects image data of the object of the inspection request from a first storage means which stores image data of a road continuously photographed by a camera, selects detailed information of the object of the inspection request from a second storage means which stores detailed information composed of point cloud data of the structure obtained by 3D scanning the structure linked to position information of the structure on a map, and presents a map showing the location of the structure, the image data selected by the first selection means, and the detailed information of the structure.

[0009] According to a third aspect, there is provided a computer-readable recording medium that stores a program for causing a computer to execute the following processes: receiving a request to inspect a structure, selecting image data of the object of the inspection request from a first storage means that stores image data of a road continuously photographed by a camera; selecting detailed information of the structure that is the object of the inspection request from a second storage means that stores detailed information composed of point cloud data of the structure linked to position information of the structure on a map and 3D scanned; and presenting a map showing the location of the structure, the image data selected by the first selection means, and the detailed information of the structure. Effect of the Invention

[0010] According to the present invention, a structure inspection support system, a structure inspection support method, and a recording medium are provided that can contribute to facilitating the inspection of structures around roads. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing a configuration of an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram for explaining an information presentation function of the structure inspection support system according to one embodiment of the present invention. [Diagram 3] 1 is a diagram showing a configuration of a structure inspection support system according to a first embodiment of the present invention. [Figure 4] 1 is a diagram showing an example of image data acquired by the structure inspection support system of the first embodiment of the present invention; [Diagram 5] 1 is a diagram for explaining point cloud data acquired by the structure inspection support system of the first embodiment of the present invention; [Figure 6] FIG. 13 is a diagram showing a state in which point cloud data is classified by object and labeled. [Figure 7] FIG. 13 is a diagram showing a state in which an object to which a label "roadside tree" has been added is highlighted. [Figure 8] FIG. 13 is a diagram showing a state in which an object to which a label "roadside tree" has been added is not displayed. [Figure 9]FIG. 2 is a diagram for explaining the difference between point cloud data and image data (visible image). [Figure 10] FIG. 1 is a diagram showing an example of a method for understanding the state of roadside trees using point cloud data. [Figure 11] 3 is a flowchart showing the operation (processing of image data) of the structure inspection support system according to the first embodiment of the present invention. [Figure 12] 4 is a flowchart showing the operation (processing of point cloud data) of the structure inspection support system according to the first embodiment of the present invention. [Figure 13] FIG. 2 is a sequence diagram showing an information presentation operation of the structure inspection support system according to the first embodiment of the present invention. [Figure 14] 3 is a diagram showing an example of an integrated screen presented on an in-vehicle terminal of the structure inspection support system of the first embodiment of the present invention; FIG. [Figure 15] FIG. 11 is a diagram showing another example of the integrated display screen presented on the vehicle-mounted terminal of the structure inspection support system of the first embodiment of the present invention. [Figure 16] FIG. 11 is a diagram showing the configuration of a structure inspection support system according to a second embodiment of the present invention. [Figure 17] FIG. 11 is a diagram showing an example of design drawing data held by the structure inspection support system of the second embodiment of the present invention. [Figure 18] FIG. 11 is a diagram showing an example of an integrated screen presented on an in-vehicle terminal of the structure inspection support system according to the second embodiment of the present invention. [Figure 19] FIG. 2 is a diagram showing the configuration of a computer that constitutes the structure inspection support system of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] First, an overview of one embodiment of the present invention will be described with reference to the drawings. Note that the reference numerals in the drawings are added to each element for convenience as an example to aid in understanding, and are not intended to limit the present invention to the illustrated embodiment. Also, the connection lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional. A unidirectional arrow is a schematic representation of the flow of a main signal (data), and does not exclude bidirectionality. A program is executed via a computer device, which includes, for example, a processor, a storage device, an input device, a communication interface, and a display device as necessary. Also, this computer device is configured to be able to communicate with devices (including computers) inside or outside the device via the communication interface, regardless of whether it is wired or wireless. Also, although there are ports or interfaces at the connection points of input and output of each block in the drawing, they are not shown.

[0013] In one embodiment, the present invention can be realized in a structure inspection support system 10 comprising a first memory means 11, a second memory means 12, a first selection means 13, a second selection means 14, and an information presentation means 15, as shown in FIG. 1.

[0014] More specifically, the first storage means 11 stores image data of a road continuously photographed by a camera. As the image data, data photographed by an on-board camera of a vehicle traveling on the road can be used.

[0015] Second storage means 12 stores point cloud data obtained by 3D scanning the structures as detailed information of the structures around the road, linked to the position information of the structures on the map. In this specification, the structures include not only civil engineering structures or architectural structures such as roads, buildings, dams, and levees, but also road accessories such as signs, guardrails, and convex mirrors.

[0016] When a request to inspect the structure is received, the first selection means 13 selects image data of the object of the inspection request from the first storage means.

[0017] A second selection means 14 selects detailed information about the structure that is the object of the survey request from the second storage means.

[0018] An information presentation means 15 presents a map showing the location of the structure, the image data selected by the first selection means, and detailed information about the structure.

[0019] For example, when a request is received from vehicle V on map M in the upper part of FIG. 2 to inspect a slope of a road as a structure at destination G, the structure inspection support system 10 operates as follows.

[0020] The structure inspection support system 10 selects, from the first storage means 11, image data of the process of the surveyor traveling from his / her current location to the structure.

[0021] The structure inspection support system 10 selects the point cloud data of the road slope from the second storage means 12 as detailed information of the structure that is the subject of the inspection request.

[0022] Then, the structure inspection support system 10 presents a map showing the location of the structure, the image data selected by the first selection means 13, and detailed information about the structure.

[0023] Fig. 2 is a diagram showing an example of a presentation form of a map, image data, and detailed information of a structure by the structure inspection support system 10. A vehicle V and a destination G where the structure is located are shown on a map M in the upper part of Fig. 2.

[0024] In the area D1 on the lower left side of Fig. 2, image data of the process of the vehicle V moving from the current location to the location G of the structure is displayed. In the example of Fig. 2, a collapsed part of the slope indicated by the reference numeral 20 is displayed in the area D1.

[0025] The display form of the image data in the area D1 is not limited to the form shown in FIG. 2. For example, a plurality of image data of the process of moving from the current location of the vehicle V to the location G of the structure may be displayed as thumbnail images, and when the user selects any thumbnail image, an enlarged image may be displayed. Also, a plurality of image data of the process of moving from the current location of the vehicle V to the location G of the structure may be continuously switched and displayed in an animated manner. In these cases, it is preferable that the image data displayed in the area D1 is image data actually taken during the process of moving from the current location of the vehicle V to the direction of the location G of the structure. In this way, by matching what is visible in the line of sight of the surveyor with the direction of the camera when the image data was taken, on-site confirmation is facilitated.

[0026] In the area D2 on the lower right side of Fig. 2, 3D scanned point cloud data is displayed as detailed information of the structure. For example, in the example of Fig. 2, point cloud data obtained by 3D scanning the slope on the side of the road is displayed.

[0027] While aboard vehicle V, a user who is about to board accesses structure inspection support system 10 from an in-vehicle terminal, a personal computer, his / her own mobile terminal, etc., and views the integrated and displayed information shown in Fig. 2. By referring to the screen shown in Fig. 2, the user can identify the structure to be checked, and further by referring to the point cloud data, the user can easily grasp the degree of damage to the structure, etc.

[0028] Furthermore, in the example of Fig. 2, point cloud data of the structure viewed from the same direction as the image data is presented. This makes it possible to facilitate on-site inspection work by users who are boarding or planning to board vehicle V. In this case, the direction of the point cloud data of the structure may be aligned with the direction of the image data displayed in area D1. This makes it easier for surveyors to grasp the structure.

[0029] [First embodiment] Next, a first embodiment of the present invention in which the present invention is applied to the inspection of roadside trees and signs in urban areas will be described in detail with reference to the drawings. FIG. 3 is a diagram showing the configuration of a structure inspection support system 100 according to the first embodiment of the present invention. Referring to FIG. 3, similar to FIG. 1, a configuration including a first storage means 111, a first selection means 113, an information presentation means 115, a second storage means 112, and a second selection means 114 is shown. Furthermore, the structure inspection support system 100 of this embodiment includes an image acquisition means 101, a point cloud data analysis means 116, a map information storage means 103, and a point cloud data acquisition means 102.

[0030] The image acquisition means 101 acquires image data of a road from a vehicle Va equipped with a camera 210, and stores the image data together with information on the image capture position and image capture direction in the first storage means 111. The vehicle Va may be a vehicle of a road administrator such as a local government, or may be a general vehicle. As the image capture position information of the image data, for example, position information measured by a GPS (Global Positioning System) of the vehicle Va or the camera 210 can be used. The image capture direction may be acquired from the vehicle, or may be determined from the image.

[0031] Fig. 4 is a diagram showing an example of image data acquired by the structure inspection support system 100 of this embodiment. Referring to Fig. 4, there is a roadside tree 803 in front of a traffic signal 801 and a street light 802, and the overhanging branches make it difficult to see. In the following explanation, an example of inspecting this roadside tree and a structure hidden in its shadow will be given.

[0032] The point cloud data acquisition means 102 acquires point cloud data obtained by scanning structures around the road from a vehicle Vb equipped with a 3D laser scanner 220 or the like, and stores the data in the second storage means 112.

[0033] The point cloud data described above is composed of, for example, three-dimensional coordinate values ​​(X, Y, Z) and color information (R, G, B). Such point cloud data can be obtained by, for example, emitting a line laser beam in the vertical and horizontal directions to the structure to be measured, measuring the time it takes for the laser pulse to travel between the measurement point on the measurement object and the sensor, and calculating the distance to the measurement point.

[0034] Such point cloud data can be obtained, for example, from a UAV (Unmanned Aerial Vehicle) equipped with a 3D laser scanner, etc. Furthermore, the vehicle Va and the vehicle Vb in FIG.

[0035] Fig. 5 is a diagram showing an example of point cloud data acquired by the structure inspection support system of this embodiment. The point cloud data shown in Fig. 5 is a set of three-dimensional coordinate values ​​(X, Y, Z), so roadside trees, street lights, signs, etc. cannot be distinguished from the data itself.

[0036] The point cloud data analysis means 116 performs object recognition and labels on the point cloud data using AI (Artificial Intelligence) and deletes unnecessary data. This process is also called (3D data) cleaning process. When the point cloud data analysis means 116 performs this cleaning process, it may refer to the image data stored in the first storage means 111. In this way, it is possible to improve the accuracy of object recognition from the point cloud data. By using the above-mentioned AI, the point cloud data analysis means 116 is able to estimate the entire image of a structure hidden in the image data by overlapping subjects, using the point cloud data. Therefore, the point cloud data analysis means 116 corresponds to an estimation means that estimates the entire image of a structure hidden in the image data by overlapping subjects, using the point cloud data.

[0037] Fig. 6 is a diagram showing a state in which the point cloud data of Fig. 5 is classified by object and labeled. In the example of Fig. 6, three roadside trees 803-1 to 803-3, a street light 802, a sign 804, and a traffic light 801 are respectively identified. By labeling in this way, it becomes possible to highlight only the roadside trees 803-1 to 803-3, as shown in Fig. 7. Also, by labeling in this way, it becomes possible to remove the information of the roadside trees 803-1 to 803-3 and display only the other structures, as shown in Fig. 8.

[0038] Fig. 9 is a diagram for explaining the difference between point cloud data and image data (visible image). In the case of image data (visible image), as shown by the dashed line in Fig. 9, the pole part of the street light 802 and the sign 804 are in the shadow of roadside trees 803-1 and 803-3, making it difficult to grasp. In the case of point cloud data after the above cleaning process, the data of the parts hidden in the shadow of roadside trees 803-1 and 803-3 is also complemented, making it possible to grasp the accurate positional relationship.

[0039] Fig. 10 is a diagram showing an example of a method for grasping the state of roadside trees using point cloud data. As described above, since point cloud data is expressed as three-dimensional coordinate values ​​(X, Y, Z), it is possible to quantitatively grasp the amount of projection of roadside tree 803-1 onto the road side, for example, as shown in d1 of Fig. 10. Similarly, by comparing the respective coordinates, it is possible to accurately grasp the width of roadside tree 803-3 that is hidden by roadside tree 803-3, as shown in d2 of Fig. 10.

[0040] Referring again to FIG. 3, the first storage means 111 stores image data acquired by the image acquisition means 101, the image data including information on the image capturing position and the image capturing direction.

[0041] The map information storage means 103 stores map information of an area in which a structure to be surveyed exists.

[0042] The second storage means 112 stores the point cloud data acquired by the point cloud data acquisition means 102.

[0043] When a request to survey the structure is received, the first selection means 113 selects from the first storage means 111 image data of the process from the surveyor's current location to the structure.

[0044] A second selection means 114 selects point cloud data from the second storage means 112 as detailed information of the structure that is the object of the survey request.

[0045] The information presentation means 115 presents this information by transmitting a map showing the location of the structure, the image data selected by the first selection means 113, and the point cloud data selected by the second selection means 114 to a specified output destination.

[0046] Next, the operation of this embodiment will be described in detail with reference to the drawings. Fig. 11 is a flow chart showing the operation (processing of image data) of the structure inspection support system of the first embodiment of the present invention. Referring to Fig. 11, the structure inspection support system 100 acquires image data from a vehicle Va equipped with a camera 210 (step S001).

[0047] Next, the structure inspection support system 100 identifies the photographing position and photographing direction information of the acquired image data (step S002). The photographing position of the image data is, for example, image It can be identified from the GPS location information added to the data. The shooting direction of the image data can be identified from changes in the GPS location information or from the image data itself.

[0048] Next, the structure inspection support system 100 stores the image data together with the identified photographing position and photographing direction information of the image data in the first storage means 111 (step S003).

[0049] By repeating the above process, image data for each point on the road is accumulated in the first storage means 111.

[0050] Fig. 12 is a flow chart showing the operation (processing of point cloud data) of the structure inspection support system according to the first embodiment of the present invention. Referring to Fig. 12, the structure inspection support system 100 acquires point cloud data obtained by scanning structures around a road from a vehicle Vb equipped with a 3D laser scanner 220 or the like (step S101).

[0051] Next, the structure inspection support system 100 performs a cleaning process on the acquired point cloud data, and recognizes and labels objects represented by the point cloud data (step S102).

[0052] Next, the structure inspection support system 100 stores the point cloud data that has been subjected to the object recognition and labeling in the second storage means 112 (step S103).

[0053] By repeating the above process, point cloud data of structures around the road is accumulated in the second storage means 112.

[0054] Next, the operation of the structure inspection support system 100 when a structure inspection request is received from a user will be described. Fig. 13 is a sequence diagram showing the information presentation operation of the structure inspection support system of the first embodiment of the present invention. Referring to Fig. 13, the structure inspection support system 100 displays a map of an area in which structures that can be inspected exist, on an in-vehicle terminal of a vehicle Vc or the like (step S201).

[0055] When the user of the vehicle-mounted terminal of the vehicle Vc who has referred to the map selects a structure, a request to inspect the selected structure is sent to the structure inspection support system 100 (step S202).

[0056] Upon receiving the request to inspect the structure, the structure inspection support system 100 selects image data and point cloud data from the first and second storage means 111 and 112, respectively, creates an integrated image by integrating these, and displays it on the display device of the vehicle-mounted terminal (step S203).

[0057] Fig. 14 is a diagram showing an example of an integrated screen presented on the in-vehicle terminal of the structure inspection support system 100 of the first embodiment of the present invention. In the example of Fig. 14, a marker V indicating the position of the vehicle Vc and a map showing the positions of the structures with markers G are displayed in the upper part of the screen. The area D1 in the lower left of Fig. 14 is a display area for image data, and image data of the process from the current location V of the surveyor to the position G of the structure is displayed. Moreover, the area D2 in the lower right of Fig. 14 is a display area for point cloud data, and a 3D wheel 3DW is displayed in the lower right of the area D2.

[0058] Thereafter, the structure inspection support system 100 accepts various operations from the vehicle-mounted terminal and changes the integrated screen according to the contents of the operations (step S204). For example, when the user of the vehicle-mounted terminal rotates the 3D wheel 3DW left, the area D2 of the integrated screen changes as shown in FIG. 15. Referring to FIG. 15, the point cloud data of the area D2 changes in response to the left rotation operation so that three roadside trees are visible from the side. The user of the vehicle-mounted terminal refers to such a display to understand the local situation and determine whether or not the roadside trees need to be pruned. Of course, the structure inspection support system 100 may also present the amount of overhang of roadside trees and the degree to which signs are hidden in response to a request from the user, as shown in FIG. 10.

[0059] As described above, according to this embodiment, it is possible to facilitate the investigation work of workers heading to investigate structures around roads. The reason for this is that a configuration is adopted that makes it easy to grasp the route to the structure to be investigated from the image data displayed in area D1 of the integrated screen, and to grasp the difference from the current situation, etc., from the point cloud data displayed in area D2 of the integrated screen.

[0060] [Second embodiment] Next, a second embodiment in which a function for presenting design drawing data of a structure is added to the above-mentioned structure inspection support system 100 will be described in detail with reference to the drawings. FIG. 16 is a diagram showing the configuration of a structure inspection support system 100a of the second embodiment of the present invention. The difference from the structure inspection support system 100 of the first embodiment shown in FIG. 3 is that a third storage means 117 is added, and the information presenting means 215 is configured so as to be able to display an integrated screen using design drawing data. Since the other configurations are the same as those of the first embodiment, the following description will focus on the differences.

[0061] The third storage means 117 stores design drawing data of the structure to be surveyed. Note that as this design drawing data, data provided by the manager of the structure or the like can be used.

[0062] Fig. 17 is a diagram showing an example of design drawing data held in the third storage means 117. In the example of Fig. 17, design drawing data in which dimensions dimA to dimD of a sign are written is stored.

[0063] When the information presentation means 215 receives a request for design plan data of a structure from a user of the in-vehicle terminal, it presents the design plan data on the integrated screen. Fig. 18 is a diagram showing an example of an integrated screen displayed on the in-vehicle terminal by the structure inspection support system 100a. In the example of Fig. 18, the design plan data is superimposed on the display area D1 of the image data. Workers heading to inspect structures around the road can refer to this design plan data to understand the damage status of signs and the consistency status with the point cloud data.

[0064] Although each embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiments, and further modifications, substitutions, and adjustments can be made without departing from the basic technical concept of the present invention. For example, the network configurations, element configurations, and data expression forms shown in the drawings are examples to aid in understanding the present invention, and the present invention is not limited to the configurations shown in these drawings.

[0065] For example, in the above-described first and second embodiments, the display destination of the integrated screen is the display device of the in-vehicle terminal of the vehicle Vc, but the display destination of the integrated screen is not limited to this. For example, the integrated screen may be displayed on a mobile terminal of a worker or a terminal of a manager of a structure of a local government or the like.

[0066] In the above embodiment, the image data is acquired from the vehicle Va, but the source of the image data is not limited to the vehicle. For example, the image data may be acquired from a camera installed on a moving object such as a drone, or may be acquired from a fixed camera. Of course, a configuration in which the first selection means appropriately selects and presents appropriate image data from these image data may also be adopted.

[0067] Similarly, in the above embodiment, the point cloud data is acquired from the vehicle Vb, but the source of the image data is not limited to the vehicle. For example, the point cloud data may be acquired from a 3D laser scanner installed on a moving body such as a drone, or may be acquired from a fixed camera. Of course, a configuration in which the second selection means appropriately selects and presents appropriate point cloud data from these point cloud data may also be adopted.

[0068] (Hardware configuration) In each embodiment of the present disclosure, each component of each device represents a functional block. A part or all of each component of each device is realized by an arbitrary combination of an information processing device 900 and a program as shown in Fig. 19. Fig. 19 is a block diagram showing an example of a hardware configuration of the information processing device 900 that realizes each component of each device. The information processing device 900 includes, as an example, the following configuration. ·CPU(Central Processing Unit)901 ROM (Read Only Memory) 902 ·RAM(Random Access Memory)903 Program 904 loaded into RAM 903 A storage device 905 for storing a program 904 A drive device 907 for reading and writing data from the recording medium 906 A communication interface 908 for connecting to a communication network 909 An input / output interface 910 for inputting and outputting data Bus 911 connecting each component

[0069] Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. That is, the CPU 901 in FIG. 19 executes a data acquisition program and a signal schedule creation program, and performs an update process for each calculation parameter stored in the RAM 903, the storage device 905, etc. The program 904 that realizes the function of each component of each device is stored in the storage device 905 or the ROM 902 in advance, for example, and is read by the CPU 901 as necessary. The program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in the recording medium 906 in advance, and the drive device 907 may read the program and supply it to the CPU 901.

[0070] Furthermore, this program 904 can display the processing results, including intermediate states, at each stage as necessary on a display device, or can communicate with the outside via a communication interface. Furthermore, this program 904 can be recorded in a computer-readable (non-transitive) recording medium.

[0071] There are various variations in the method of realizing each device. For example, each device may be realized by any combination of a separate information processing device 900 and a program for each component. Also, a plurality of components included in each device may be realized by any combination of one information processing device 900 and a program. That is, each part (processing means, function) of the signal control system shown in the first to fourth embodiments can be realized by a computer program that causes a processor mounted on the device to execute each of the above-mentioned processes using its hardware.

[0072] In addition, some or all of the components of each device are realized by other general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or may be configured by multiple chips connected via a bus.

[0073] Some or all of the components of each device may be realized by a combination of the above-mentioned circuits and programs.

[0074] When some or all of the components of each device are realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or distributed. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, etc., in a form in which each is connected via a communication network.

[0075] It should be noted that the above-described embodiments are preferred embodiments of the present disclosure, and the scope of the present disclosure is not limited to the above-described embodiments. In other words, a person skilled in the art can modify or substitute the above-described embodiments to construct various modified forms without departing from the gist of the present disclosure.

[0076] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0077] [Appendix 1] A first storage means for storing image data of a road continuously photographed by a camera; a second storage means for storing detailed information of structures around the road in association with position information of the structures on a map; a first selection means for selecting image data of the object of the request for inspection from the first storage means when the request for inspection of the structure is received; a second selection means for selecting detailed information of the structure that is the object of the survey request from the second storage means; a map showing the location of the structure, the image data selected by the first selection means, and information presentation means for presenting detailed information about the structure; A structure survey support system in which detailed information about the structure is constructed using point cloud data obtained by 3D scanning the structure. [Appendix 2] The first selection means of the above-mentioned structure inspection support system can be configured to select image data from the first storage means of the survey request target, the image data of the process of the survey person traveling from the current location to the structure. [Appendix 3] The above-mentioned structural inspection support system is The method further includes estimating means for estimating an entire image of a structure hidden by overlapping objects in the image data, using the point cloud data; The information presenting means may be configured to present detailed information of the structure estimated by the estimating means. [Appendix 4] The information presentation means of the above-mentioned structure inspection support system may be configured to present point cloud data of the structure viewed from the same direction as the image data selected by the first selection means. [Appendix 5] The structure inspection support system further includes a third storage means for storing design drawing data of the structure, The information presentation means may be configured to present design drawing data of the structure in response to a request from a user. [Appendix 6] receiving a request to inspect a structure, and selecting image data of a route from a first storage means for storing image data of a road continuously photographed by a camera mounted on a vehicle, the image data being taken along a route from a current location of the inspector to the structure; Selecting detailed information of the structure that is the subject of the survey request from a second storage means that stores detailed information composed of point cloud data obtained by 3D scanning the structure and linking it to position information of the structure on a map; presenting a map showing the location of the structure, the selected image data, and detailed information about the structure; Methods for supporting structural investigations. [Appendix 7] A process of accepting a request to inspect a structure, and selecting image data of a route from a first storage means that stores image data of a road continuously photographed by a camera mounted on a vehicle, the image data being taken from the current location of the inspector toward the structure; A process of selecting detailed information of the structure that is the subject of the survey request from a second storage means that stores detailed information composed of point cloud data obtained by 3D scanning the structure and linking it to position information of the structure on a map; A process of presenting a map showing the location of the structure, the selected image data, and detailed information about the structure; A computer-readable recording medium that stores a program for causing a computer to execute the above.

[0078] Note that, like Supplementary Note 1, the embodiments of Supplementary Note 6 to Supplementary Note 7 can be developed into the embodiments of Supplementary Note 2 to Supplementary Note 5.

[0079] The disclosures of the above patent documents are incorporated herein by reference and may be used as the basis or part of the present invention as necessary. Within the framework of the entire disclosure of the present invention (including the scope of claims), modifications and adjustments of the embodiments and examples are possible based on the basic technical ideas. Furthermore, within the framework of the disclosure of the present invention, various combinations or selections (including partial deletions) of various disclosed elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. In other words, the present invention naturally includes various modifications and corrections that a person skilled in the art would be able to make in accordance with the entire disclosure including the scope of claims and the technical ideas. In particular, with regard to the numerical ranges described in this document, any numerical value or small range included within the range should be interpreted as being specifically described even if not otherwise specified. Furthermore, the disclosures of the above cited documents may be used in part or in whole in combination with the disclosures of this document as part of the disclosure of the present invention in accordance with the spirit of the present invention as necessary, and are deemed to be included in the disclosures of this application. [Explanation of symbols]

[0080] 10, 100, 100a Structural Inspection Support System 11, 111 First storage means 12, 112 Second storage means 13, 113 First selection means 14, 114 Second selection means 15, 115, 215 Information presentation means 101 Image acquisition means 102 Point cloud data acquisition method 103 Map information storage means 116 Point Cloud Data Analysis Methods 117 Third storage means 801 Traffic lights 802 Street Light 803, 803-1~803-3 Street trees 804 signs 210 Camera 220 3D Laser Scanner Va, Vb, Vc Vehicles V, G markers D1, D2 area 3DW 3D Wheel 900 Information processing device 901 CPU(Central Processing Unit) 902 ROM (Read Only Memory) 903 RAM (Random Access Memory) 904 Program 905 Storage device 906 Recording media 907 Drive unit 908 Communication Interface 909 Communication Network 910 Input / Output Interface 911 Bus

Claims

1. a first storage means for storing image data of a road continuously photographed by a camera; a second storage means for storing detailed information of structures around the road in association with position information of the structures on a map; a first selection means for selecting image data of the object of the request for survey from the first storage means when the request for survey of the structure is received; a second selection means for selecting detailed information of the structure that is the object of the survey request from the second storage means; a map showing the location of the structure, the image data selected by the first selection means, and information presentation means for presenting detailed information about the structure; The detailed information of the structure is configured using point cloud data obtained by 3D scanning the structure, the information presentation means presents an amount of projection of the object in the point cloud data onto a road side. Structural survey support system.

2. 2. A structure inspection support system according to claim 1, wherein said first selection means selects from said first storage means image data of a route from the surveyor's current location to said structure as image data of the survey request target.

3. The method further includes estimating means for estimating an entire image of a structure hidden by overlapping objects in the image data, using the point cloud data; 3. The structure inspection support system according to claim 1, wherein the information presentation means presents detailed information about the structure estimated by the estimation means.

4. 4. The structure inspection support system according to claim 1, wherein the information presenting means presents point cloud data of the structure viewed from the same direction as the image data selected by the first selecting means.

5. Further, a third storage means for storing design drawing data of the structure is provided, 5. A structure inspection support system according to claim 1, wherein said information presentation means presents design drawing data of said structure in response to a request from a user.

6. A request for inspecting a structure is received, and image data of the object of the inspection request is selected from a first storage means for storing image data of a road continuously photographed by a camera; Selecting detailed information of the structure that is the subject of the survey request from a second storage means that stores detailed information composed of point cloud data obtained by 3D scanning the structure and linking it to position information of the structure on a map; presenting a map showing the location of the structure, the selected image data, and detailed information about the structure; In the presenting step, an amount of projection of the object in the point cloud data onto a road side is presented. Methods for supporting structural investigations.

7. A process of accepting a request to inspect a structure and selecting image data of the object of the inspection request from a first storage means that stores image data of a road continuously photographed by a camera; A process of selecting detailed information of the structure that is the subject of the survey request from a second storage means that stores detailed information composed of point cloud data obtained by 3D scanning the structure and linking it to position information of the structure on a map; A process of presenting a map showing the location of the structure, the selected image data, and detailed information about the structure; Run the following on your computer: In the process of presenting, an amount of projection of the object in the point cloud data onto a road side is presented. program.

Images