Facility inspection display device, information processing device, facility inspection display method, and facility inspection display program

Abstract

A facility inspection display device (100) comprises a locus display unit (110) that displays a locus (11) of coordinates serving as an origin when point group data on a facility (60) using LiDAR is acquired by moving the LiDAR in the facility (60), a point-group data display unit (120) that displays point group data acquired with the origin located at user-specified coordinates at a user-specified position (12) on the locus (11) specified by a user on the locus display unit (110), and a deterioration phenomenon display unit (140) that displays, as a deterioration phenomenon position (13) on the locus display unit (110), deterioration phenomenon coordinates serving as the origin when the point group data including a deterioration phenomenon of the facility (60) is acquired.

Description

[Technical Field]

[0001] The present invention relates to a facility inspection display device, an information processing device, a facility inspection display method, and a non-transitory computer-readable medium. [Background technology]

[0002] Technologies using mobile devices such as robots and drones are known for the purpose of inspecting the headrace channels at hydroelectric power plants.In addition, a deterioration phenomenon inspection solution is being considered that detects cracks and other damage that have occurred in the headrace channels from point cloud data acquired by LiDAR (Light Detection and Ranging).

[0003] When the granularity of point cloud data acquired by LiDAR is coarse, it is difficult to determine the size and type of deterioration, so it is necessary to also check image data such as video or still images taken with a camera.However, when taking video, large-capacity batteries and storage are required in water channels where long-term inspections are expected, which increases the weight and size of the equipment and makes it difficult to transport inside the water channels.

[0004] Patent Document 1 describes inspecting facilities using point cloud data and image data. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-015572 Summary of the Invention [Problem to be solved by the invention]

[0006] In the display method of Patent Document 1, it is difficult to synchronize the positions of point cloud data and image data (still images), and it is particularly difficult to link the coordinates of deterioration events included in the point cloud data with the image data.

[0007] There are LiDARs equipped with a device called an IMU (Inertial Measurement Unit) that detects three-dimensional inertial movement, and in this case, it is possible to record position information (trajectory data) in chronological order when point cloud data is acquired. By using an IMU, it is hoped that the position of point cloud data and image data (still images) can be synchronized, and degradation events can be linked to the image data corresponding to the degradation events, thereby improving the visibility of the degradation events.

[0008] Furthermore, when identifying the location of a deterioration event based on visual inspection or past inspection results, there is a possibility that image data corresponding to the deterioration event may be missing. It is therefore desirable to display the coordinates of a deterioration event where image data has been missing, thereby improving the visibility of the deterioration event.

[0009] The object of the present disclosure has been made to solve such problems, and is to provide a facility inspection display device, an information processing device, a facility inspection display method, and a non-transitory computer-readable medium that can improve visibility in displaying the locations of deterioration events. [Means for solving the problem]

[0010] The facility inspection display device according to the present disclosure includes a trajectory display means for displaying a trajectory of coordinates that serve as a base point when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility, a point cloud data display means for displaying the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user as the base point, and a deterioration event display means for displaying, on the trajectory display means, the deterioration event coordinates that serve as the base point when the point cloud data including deterioration events of the facility is acquired, as deterioration event positions.

[0011] The information processing device according to the present disclosure also includes a trajectory information storage means for storing trajectory information including a trajectory of coordinates that serve as base points when point cloud data of a facility is acquired using the LiDAR by moving the LiDAR within the facility, the coordinates of each of the multiple base points that constitute the trajectory, and each time that the LiDAR passed through each of the coordinates; a point cloud data information storage means for storing point cloud data information including each of the point cloud data acquired using each of the coordinates as the base point and each time that the LiDAR passed through each of the coordinates that served as the base points when each of the point cloud data was acquired; and a deterioration event coordinate extraction means for extracting, from the trajectory information and the point cloud data information, deterioration event coordinates that serve as the base points when the point cloud data including deterioration events of the facility is acquired.

[0012] Furthermore, the facility inspection display method according to the present disclosure includes the steps of: displaying, on a trajectory display means, a trajectory of coordinates that serve as a base point when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility; displaying, on a point cloud data display means, the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; and displaying, on the trajectory display means, the deterioration event coordinates that serve as the base point when the point cloud data including deterioration events of the facility is acquired, as deterioration event positions.

[0013] In addition, the facility inspection display program according to the present disclosure causes a computer to execute the following steps: displaying, on a trajectory display means, a trajectory of coordinates that serve as a base point when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility; displaying, on a point cloud data display means, the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; and displaying, on the trajectory display means, the deterioration event coordinates that serve as the base point when the point cloud data including deterioration events of the facility is acquired, as deterioration event positions. [Effects of the Invention]

[0014] According to the present disclosure, it is possible to provide a facility inspection display device, an information processing device, a facility inspection display method, and a non-transitory computer-readable medium that can improve visibility in displaying the locations of deterioration events. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a configuration diagram illustrating a facility inspection system according to a first embodiment. [Figure 2] 1 is a block diagram illustrating an information processing device according to a first embodiment. [Figure 3] 1 is a plan view illustrating an example of a display screen of a facility inspection display device according to a first embodiment. [Figure 4] FIG. 1 is a flowchart illustrating a facility inspection method according to the first embodiment. [Figure 5] 1 is a flowchart illustrating a facility inspection display method according to the first embodiment. [Figure 6] FIG. 10 is a plan view illustrating an example of a display screen of a facility inspection display device according to a second embodiment. [Figure 7] FIG. 10 is a flowchart illustrating a facility inspection display method according to a second embodiment. [Figure 8] FIG. 10 is a block diagram illustrating an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, the same elements in each drawing are designated by the same reference numerals, and duplicate explanations have been omitted as necessary.

[0017] (Embodiment 1) A facility inspection system according to a first embodiment will be described. The facility inspection system of this embodiment inspects a facility using point cloud data and image data of the facility. The facility is, for example, a headrace at a hydroelectric power plant, but is not limited to this. As long as point cloud data and image data can be acquired, the facility is not limited to a substation at a power plant, a road tunnel, or the like.

[0018] Fig. 1 is a configuration diagram illustrating a facility inspection system according to a first embodiment. As shown in Fig. 1, the facility inspection system 1 includes an inertial motion detection device 10, a point cloud data acquisition device 20, an image data acquisition device 30, a mobile body 40, a mobile body 41, an information processing device 50, and a facility inspection display device 100. The inertial motion detection device 10, the point cloud data acquisition device 20, the image data acquisition device 30, the mobile body 40, the mobile body 41, the information processing device 50, and the facility inspection display device 100 function as an inertial motion detection means, a point cloud data acquisition means, an image data acquisition means, a moving means, a moving means, an information processing means, and a facility inspection display means, respectively. Each configuration will be described below.

[0019] <Inertial motion detection device> The inertial motion detection device 10 detects inertial motion. The inertial motion detection device 10 is, for example, an IMU. Note that the inertial motion detection device 10 is not limited to an IMU, and may be any device that detects inertial motion as conceived by those skilled in the art, as long as it is a device that detects inertial motion. In the following, an IMU will be used as an example of the inertial motion detection device 10.

[0020] The IMU is mounted on the mobile object 40 together with the point cloud data acquisition device 20. The IMU acquires a trajectory 11 of the mobile object 40 moving through the facility 60. Since the mobile object 40 is also equipped with the point cloud data acquisition device 20, the IMU acquires the trajectory 11 of coordinates that serve as base points when acquiring point cloud data of the facility 60. The IMU also acquires the coordinates of each of the multiple base points that make up the trajectory 11. Furthermore, the IMU uses a timer to acquire the time at which the IMU passes through each of the coordinates of the multiple base points that make up the trajectory 11.

[0021] The coordinates that make up the trajectory 11 are continuous along the trajectory 11, but for example, coordinates P1 and P2 will be used as an example, as shown in Fig. 1. The coordinates P1 and P2 that make up the trajectory 11 are linked to the times t1 and t2 at which the IMU passed through the coordinates P1 and P2, respectively.

[0022] The trajectory 11 acquired by the IMU, the coordinates that make up the trajectory 11, and the times at which the IMU passed through each coordinate are referred to as trajectory information. That is, the trajectory information includes the trajectory 11 of the coordinates that serve as the base point when the point cloud data of the facility 60 was acquired, the coordinates of the multiple base points that make up the trajectory 11, and the times at which the IMU passed through each coordinate.

[0023] The IMU outputs the acquired trajectory information to the information processing device 50. The IMU may output the trajectory information to the information processing device 50 via a wireless or wired communication line, or may output the trajectory information to the information processing device 50 via a storage medium.

[0024] <Point cloud data acquisition device> The point cloud data acquisition device 20 acquires point cloud data. The point cloud data acquisition device 20 is, for example, a LiDAR. Note that the point cloud data acquisition device 20 is not limited to a LiDAR, and may be any device that acquires point cloud data as conceived by a person skilled in the art, as long as it is a device that acquires point cloud data. In the following, a LiDAR will be used as an example of the point cloud data acquisition device 20.

[0025] The LiDAR is mounted on the mobile object 40 together with the IMU. The LiDAR acquires point cloud data of the facility 60. Specifically, the LiDAR acquires point cloud data of the facility 60 from coordinates that serve as a base point, while moving the coordinates that serve as the base point using the mobile object 40. Therefore, the coordinates that serve as the base point when the point cloud data of the facility 60 is acquired form a trajectory 11. This trajectory 11 is acquired by the IMU. The LiDAR acquires each piece of point cloud data using each coordinate included in the trajectory 11 as a base point. In addition, the LiDAR acquires each time when the LiDAR passes each coordinate. In addition, each coordinate that constitutes the trajectory 11 is linked to each time when the LiDAR passed each coordinate. In addition, each piece of point cloud data acquired using each coordinate that constitutes the trajectory 11 as a base point is linked to each time when the LiDAR passed each coordinate.

[0026] For example, coordinates P1 and P2 will be used as an example, as shown in Fig. 1. Each of the coordinates P1 and P2 constituting the trajectory 11 is linked to the time t1 and t2 at which the LiDAR passed through each of the coordinates P1 and P2. Furthermore, each of the point cloud data G1 and G2 acquired using each of the coordinates P1 and P2 constituting the trajectory 11 as a base point is linked to the time t1 and t2 at which the LiDAR passed through each of the coordinates P1 and P2.

[0027] Each piece of point cloud data acquired by the LiDAR and the times at which the LiDAR passed through each coordinate that serves as the base point for each piece of point cloud data are called point cloud data information. That is, the point cloud data information includes each piece of point cloud data acquired using each coordinate as the base point and each time at which the LiDAR passed through each coordinate that serves as the base point when each piece of point cloud data was acquired.

[0028] The LiDAR outputs the acquired point cloud data information to the information processing device 50. The LiDAR may output the point cloud data information to the information processing device 50 via a wireless or wired communication line, or may output the point cloud data information to the information processing device 50 via a storage medium.

[0029] <Image data acquisition device> The image data acquisition device 30 acquires image data. The image data acquisition device 30 is, for example, a camera. Note that the image data acquisition device 30 is not limited to a camera, and may be any device that acquires image data as conceived by a person skilled in the art, as long as it is a device that acquires image data. In the following, a camera will be used as an example of the image data acquisition device 30.

[0030] It is desirable that the image data be still images. For example, if it is expected that the inspection of the facility 60 will take a long time, such as a waterway, moving images will require a large-capacity battery and storage device. Therefore, the weight and size of the equipment will make it difficult to inspect the facility 60. mosquito However, the image data may include, for example, fragmented video of a few seconds, as long as it does not make it difficult to inspect the facility 60. In the following, a still image will be used as an example of image data.

[0031] The camera is moved by a mobile object 41. The mobile object 41 is, for example, a person such as a worker. Note that the mobile object 41 is not limited to a person such as a worker, but may be a device similar to the mobile object 40 and separate from the mobile object 40. It is not excluded that the mobile object 41 and the mobile object 40 move together. The camera moves through the facility 60 together with the IMU and LiDAR.

[0032] The camera photographs deterioration phenomena in the facility 60 based on visual inspection by workers and past inspection information, etc. Deterioration phenomena include, for example, cracks on wall surfaces, peeling of wall surfaces, water leaks from wall surfaces, etc. Note that deterioration phenomena are not limited to wall surface deterioration, and may be any deterioration of facilities that would occur to a person skilled in the art. The camera may acquire the direction from the coordinates that serve as the base point when photographing the deterioration phenomenon to the deterioration phenomenon.

[0033] The camera has a timer. The timer is linked to the start of point cloud data acquisition by LiDAR. The camera acquires the time on the timer when acquiring image data. This means that the image data is linked to the time when the camera captured the image data.

[0034] For example, coordinates P1 and P2 will be described as an example, as shown in FIG. 1. Each of the coordinates P1 and P2 constituting the trajectory 11 is linked to the time t1 and t2 at which the LiDAR passed through each of the coordinates P1 and P2. The camera captures image data H1 at time t1. The image data H1 is then linked to time t1. As a result, the information processing device 50, which will be described later, links the image data H1 to the coordinate P1 on the trajectory 11, which is linked to time t1.

[0035] The image data acquired by the camera and the time when the image data was captured are called image data information. That is, the image data information includes the image data of the facility 60 and the time when the image data was captured.

[0036] The camera outputs the acquired image data information to the information processing device 50. The camera may output the image data information to the information processing device 50 via a wireless or wired communication line, or may output the image data information to the information processing device 50 via a storage medium.

[0037] <Mobile> The mobile bodies 40 and 41 move within the facility 60. The mobile bodies 40 and 41 are, for example, mobile robots, drones, vehicles, workers, etc. Note that the mobile bodies 40 and 41 are not limited to these, and may be anything conceivable to a person skilled in the art as long as they are equipped with an IMU, LiDAR, and a camera and can move within the facility 60. The mobile body 40 may be equipped with an IMU and LiDAR, and the mobile body 41 may be equipped with a camera. The mobile body 41 is, for example, a worker. Note that the mobile bodies 40 and 41 may be integrated together and move the IMU, LiDAR, and camera.

[0038] <Information processing device> Fig. 2 is a block diagram illustrating an information processing device according to embodiment 1. As shown in Fig. 2, an information processing device 50 includes a trajectory information storage unit 51, a point cloud data information storage unit 52, an image data information storage unit 53, a deterioration event coordinate extraction unit 54, and a distinction extraction unit 55. The trajectory information storage unit 51, the point cloud data information storage unit 52, the image data information storage unit 53, the deterioration event coordinate extraction unit 54, and the distinction extraction unit 55 function as trajectory information storage means, point cloud data information storage means, image data information storage means, deterioration event coordinate extraction means, and distinction extraction means, respectively.

[0039] The information processing device 50 may be configured with hardware including a microcomputer consisting of, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and an interface unit (I / F). The CPU performs storage processing, extraction processing, control processing, etc. The ROM stores storage programs, extraction programs, control programs, etc. executed by the CPU. The RAM stores various data such as trajectory information, point cloud data information, and image data information. The interface unit (I / F) inputs and outputs signals to and from the outside. The interface unit may include input devices such as a keyboard, touch panel, or mouse, and may also include output devices such as a display or speaker. The interface unit accepts data input operations by a user and outputs information to the user. The CPU, ROM, RAM, and interface unit are connected to each other via a data bus or the like.

[0040] The trajectory information storage unit 51 stores trajectory information output from the IMU. The trajectory information includes a trajectory 11 of coordinates that serve as base points when point cloud data of the facility 60 using the LiDAR is acquired by moving the LiDAR within the facility 60, the coordinates of multiple base points that make up the trajectory 11, and the times at which the LiDAR passed through each of the coordinates. The trajectory information storage unit 51 may store each of the coordinates that make up the trajectory 11 in association with the times at which the LiDAR passed through each of the coordinates.

[0041] The point cloud data information storage unit 52 stores point cloud data information output from the LiDAR. The point cloud data information includes each piece of point cloud data acquired using each coordinate constituting the trajectory 11 as a base point, and each time when the LiDAR passed each coordinate that served as the base point when each piece of point cloud data was acquired. The point cloud data information storage unit 52 may store each piece of point cloud data acquired using each coordinate constituting the trajectory 11 as a base point, in association with each time when the LiDAR passed each coordinate.

[0042] The image data information storage unit 53 stores image data information output from the camera. The image data information includes image data of the facility 60 and the time when the image data was acquired. The image data information storage unit 53 may store the image data in association with coordinates on the trajectory 11 that are associated with the same time as the time when the image was captured.

[0043] The deterioration event coordinate extraction unit 54 analyzes deterioration events of the facility 60 from the trajectory information and point cloud data information. The deterioration event coordinate extraction unit 54 may analyze deterioration events using past inspection results. For example, the deterioration event coordinate extraction unit 54 may compare past point cloud data of the facility 60 with the current point cloud data and analyze deterioration events from changed parts. Alternatively, the deterioration event coordinate extraction unit 54 may compare design data of the facility 60 with the current point cloud data and analyze deterioration events from changed parts.

[0044] Then, the deterioration event coordinate extraction unit 54 extracts, from the trajectory information and the point cloud data information, deterioration event coordinates that serve as a base point when point cloud data including deterioration events of the facility 60 is acquired. The deterioration event coordinate extraction unit 54 may store the extracted deterioration event coordinates in the trajectory information storage unit 51.

[0045] The distinction extraction unit 55 distinguishes and extracts deterioration event coordinates depending on whether image data is linked to the extracted deterioration event coordinates. Specifically, the distinction extraction unit 55 determines whether image data is linked to each deterioration event coordinate. Then, the distinction extraction unit 55 distinguishes and extracts deterioration event coordinates that are linked to image data from deterioration event coordinates that are not linked to image data.

[0046] The information processing device 50 outputs the acquired trajectory information, point cloud data information, image data information, and deterioration event coordinates to the facility inspection display device 100. The information processing device 50 may output this information to the facility inspection display device 100 via a wireless or wired communication line, or may output this information to the facility inspection display device 100 via a storage medium.

[0047] <Facility inspection display device> Fig. 3 is a plan view illustrating an example of a display screen of the facility inspection display device according to embodiment 1. As shown in Fig. 3, the facility inspection display device 100 includes a trajectory display unit 110, a point cloud data display unit 120, an image data display unit 130, a deterioration event display unit 140, a coordinate display unit 150, and a distinction display unit 160. The trajectory display unit 110, the point cloud data display unit 120, the image data display unit 130, the deterioration event display unit 140, the coordinate display unit 150, and the distinction display unit 160 function as trajectory display means, point cloud data display means, image data display means, deterioration event display means, coordinate display means, and distinction display means, respectively.

[0048] The facility inspection display device 100 is a user interface and may be configured with hardware including a microcomputer consisting of, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and an interface unit (I / F). The CPU performs display processing and control processing. The ROM stores display programs and control programs executed by the CPU. The RAM stores various data such as trajectory information, point cloud data information, and image data information. The interface unit (I / F) inputs and outputs signals to and from the outside. The interface unit may include input devices such as a keyboard, touch panel, or mouse, and may also include output devices such as a display or speaker. The interface unit accepts data input operations by the user and outputs information to the user. The CPU, ROM, RAM, and interface unit are connected to each other via a data bus or the like.

[0049] The trajectory display unit 110 displays a trajectory 11 of coordinates that serve as base points when point cloud data of the facility 60 using the LiDAR is acquired by moving the LiDAR within the facility 60. The trajectory display unit 110 displays, for example, the trajectory 11 viewed from one direction using a line. For example, the trajectory display unit 110 may display the trajectory 11 viewed from the side of the water conduit using a line, or may display the trajectory 11 viewed from above the water conduit using a line. Each coordinate that constitutes the trajectory 11 is linked to each time that the LiDAR passed through that coordinate.

[0050] The point cloud data display unit 120 displays the point cloud data G1 acquired using user-specified coordinates at a user-specified position 12 on the trajectory 11 specified by the user in the trajectory display unit 110 as a base point. In the trajectory display unit 110, a position may be displayed, for example, as a vertical line intersecting with coordinates on the trajectory 11. Specifically, the user-specified position 12 may be displayed, for example, as a vertical line intersecting with user-specified coordinates on the trajectory 11 in the trajectory display unit 110. For example, the user may select coordinates on the trajectory 11 by using a touch panel function on the trajectory display unit 110, or may select coordinates on the trajectory 11 using a pointer function of a mouse. Each piece of point cloud data acquired using each coordinate constituting the trajectory 11 as a base point is linked to each time when the LiDAR passed through each coordinate.

[0051] The image data display unit 130 displays image data of the facility 60. The image data is linked to coordinates on the trajectory 11 that are linked to the same time as the time at which the image data was acquired. When the user designates a user-designated position 12, if there is image data linked to the user-designated coordinates at the user-designated position 12, the image data display unit 130 displays the image data linked to the user-designated coordinates.

[0052] The deterioration event display unit 140 causes the trajectory display unit 110 to display, as the deterioration event position 13, the deterioration event coordinates that serve as the base point when point cloud data including the deterioration event of the facility 60 is acquired. Specifically, the deterioration event position 13 may be displayed on the trajectory display unit 110 as a vertical line that intersects with the deterioration event coordinates on the trajectory 11.

[0053] The coordinate display unit 150 displays coordinates that serve as a base point when the point cloud data is acquired. For example, the coordinate display unit 150 displays deterioration event coordinates. When a user designates a deterioration event position 13 as the user-designated position 12, the coordinate display unit 150 displays the deterioration event coordinates. When image data is linked to the deterioration event coordinates of the user-designated position 12, the image data display unit 130 displays the image data linked to the deterioration event coordinates. Therefore, the coordinates displayed by the coordinate display unit 150 can also be said to be coordinates that serve as a base point when the image data is acquired.

[0054] The coordinate display unit 150 may display at least one of the direction from the deterioration event coordinates to the deterioration event and the time when the LiDAR passed through the deterioration event coordinates.

[0055] The distinctive display unit 160 distinguishes between and displays the deterioration event positions 13 depending on whether image data is linked to the deterioration event coordinates. Specifically, for example, the distinctive display unit 160 displays the deterioration event positions 13 including deterioration event coordinates linked to image data with a solid line, and displays the deterioration event positions 14 including deterioration event coordinates not linked to image data with a dotted line. In this way, the distinctive display unit 160 distinguishes between and displays the deterioration event positions 13 and 14.

[0056] <Facility inspection method> Next, the facility inspection method of this embodiment will be described. Fig. 4 is a flowchart illustrating the facility inspection method according to the first embodiment. As shown in Fig. 4, steps S11 to S14 are operations in a facility 60 such as a water channel. Steps S15 to S19 are operations in an office or the like where the information processing device 50 and the facility inspection display device 100 are located.

[0057] As shown in step S11, measurements are started using the LiDAR and IMU in the facility 60. As a result, an inspection of the facility 60 is carried out using the LiDAR and IMU, and point cloud data and trajectory data of the facility 60 are acquired.

[0058] Meanwhile, as shown in step S12, in the facility 60, a timer for the camera is started in synchronization with the start of measurement by the LiDAR and IMU.

[0059] Next, as shown in step S13, the deterioration phenomenon is confirmed in the facility 60. Specifically, the location of the deterioration phenomenon in the facility 60 is confirmed based on the visual inspection of the worker and the results of past inspections.

[0060] Next, as shown in step S14, photographs of the deterioration phenomenon of the facility 60 are taken, and the timer at the time of photographing is recorded. Specifically, a worker or a mobile body 41 photographs the deterioration phenomenon of the facility 60 with a camera and acquires image data. Then, the time information of the timer is recorded when the image data is acquired. In this way, point cloud data information, trajectory information, and image data information of the facility 60 are acquired.

[0061] Next, as shown in steps S15 and S16, data is output. Specifically, after the inspection of the facility 60 is completed, the point cloud data and trajectory data acquired by the LiDAR and IMU are output to the information processing device 50. In addition, image data acquired by the camera is output to the information processing device 50.

[0062] Next, as shown in step S17, data is stored. Specifically, in the information processing device 50, the trajectory information storage unit 51 stores trajectory information output from the IMU. The trajectory information storage unit 51 may store each coordinate constituting the trajectory 11 in association with each time when the LiDAR passed through each coordinate. The point cloud data information storage unit 52 stores point cloud data information output from the LiDAR. The point cloud data information storage unit 52 may store each piece of point cloud data acquired using each coordinate constituting the trajectory 11 as a base point in association with each time when the LiDAR passed through each coordinate. The image data information storage unit 53 stores image data information output from the camera. The image data information storage unit 53 may store image data in association with coordinates on the trajectory 11 that are associated with the same time as the time when the image was captured.

[0063] Next, as shown in step S18, data analysis is performed. Specifically, in the information processing device 50, the deterioration event coordinate extraction unit 54 analyzes deterioration events of the facility 60 from the trajectory information and point cloud data information. Then, the deterioration event coordinate extraction unit 54 extracts deterioration event coordinates that serve as base points when point cloud data including deterioration events of the facility 60 is acquired from the trajectory information and point cloud data information. The deterioration event coordinate extraction unit 54 may store the extracted deterioration event coordinates in the trajectory information storage unit 51.

[0064] The distinction extraction unit 55 also distinguishes and extracts deterioration event coordinates based on the presence or absence of image data linked to the extracted deterioration event coordinates. Specifically, the distinction extraction unit 55 determines whether image data is linked to each deterioration event coordinate. Then, the distinction extraction unit 55 distinguishes and extracts deterioration event coordinates linked to image data from deterioration event coordinates not linked to image data.

[0065] Next, as shown in step S19, the retained data and analyzed data are output to the facility inspection display device 100. Specifically, the information processing device 50 outputs trajectory information, point cloud data information, image data information, deterioration event coordinates, etc. to the facility inspection display device 100.

[0066] <Facility inspection display method> Next, a facility inspection display method using the facility inspection display device 100 will be described. Fig. 5 is a flowchart illustrating the facility inspection display method according to the first embodiment. As shown in step S110 of Fig. 5, a trajectory 11 is displayed on the trajectory display unit 110. Specifically, the trajectory 11 of coordinates that serve as base points when point cloud data of the facility 60 using the LiDAR is acquired by moving the LiDAR in the facility 60 is displayed on the trajectory display unit 110.

[0067] Next, as shown in step S120, the point cloud data is displayed on the point cloud data display unit 120. Specifically, the point cloud data acquired using the user-specified coordinates at the user-specified position 12 on the trajectory 11 specified by the user in the trajectory display unit 110 as the base point is displayed on the point cloud data display unit 120.

[0068] Next, as shown in step S130, the image data is displayed on the image data display unit 130. Specifically, image data of the facility 60 is displayed on the image data display unit 130. At this time, each coordinate constituting the trajectory 11 is linked to each time when the LiDAR passed through each coordinate. Furthermore, each piece of point cloud data acquired using each coordinate constituting the trajectory 11 as a base point is linked to each time when the LiDAR passed through each coordinate. The image data is linked to the coordinate on the trajectory 11 that is linked to the same time as the time when it was acquired. Therefore, when there is image data linked to the user-specified coordinate, the image data is displayed on the image data display unit 130.

[0069] Next, as shown in step S140, the deterioration event position 13 is displayed on the trajectory display unit 110. Specifically, the deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility 60 is acquired are displayed on the trajectory display unit 110 as the deterioration event position.

[0070] Next, as shown in step S150, coordinates on the trajectory 11 are displayed on the coordinate display unit 150. Specifically, coordinates that serve as a base point when point cloud data is acquired are displayed on the coordinate display unit 150. For example, deterioration event coordinates that serve as a base point when point cloud data including deterioration events of the facility 60 is acquired may be displayed on the coordinate display unit 150. When displaying the deterioration event coordinates, the coordinate display unit 150 may display at least one of the direction from the deterioration event coordinates to the deterioration event and the time when the LiDAR passed the deterioration event coordinates.

[0071] Next, as shown in step S160, it is determined whether image data is linked to each deterioration event coordinate. If, in step S160, image data is linked to each deterioration event coordinate (YES), the process ends. On the other hand, if, in step S160, image data is not linked to each deterioration event coordinate (NO), the trajectory display unit 110 displays the deterioration event positions 13 and 14, distinguishing them based on whether image data is available, as shown in step S170. Specifically, if image data is linked to the deterioration event coordinate, the distinguishing display unit 160 displays the deterioration event position 13 with a solid vertical line, and if image data is not linked to the deterioration event coordinate, the distinguishing display unit 160 displays the deterioration event position 14 with a dotted vertical line. In this way, the distinguishing display unit 160 distinguishes between the deterioration event positions 13 and 14 based on whether image data is available. Then, the process ends.

[0072] Next, the effects of this embodiment will be described. In this embodiment, the LiDAR is moved together with the IMU in the facility 60 to acquire a trajectory of coordinates that serve as a base point when acquiring point cloud data of the facility 60. Therefore, it is possible to associate coordinates and time with the point cloud data, and it is possible to improve the accuracy of the position of the facility 60 in the point cloud data.

[0073] Furthermore, image data of the facility 60 is acquired using a timer synchronized with detection by the LiDAR and IMU. This allows the image data to be linked to time and coordinates, improving the accuracy of the position in the image data of the facility 60. In this way, this embodiment uses trajectory data acquired by the IMU to synchronize the point cloud data and image data. This allows the point cloud data and image data to be linked, and the deterioration event coordinates and image data to be linked.

[0074] Furthermore, in this embodiment, the deterioration event coordinates that serve as the base point when point cloud data including the deterioration events of the facility 60 is acquired are displayed as the deterioration event position on the trajectory display unit 110. The deterioration event coordinates may also be displayed on the coordinate display unit 150. The coordinate display unit 150 may also display at least one of the direction from the deterioration event coordinates to the deterioration event and the time when the LiDAR passed the deterioration event coordinates. This can improve the visibility of the display of the deterioration event position.

[0075] Furthermore, in this embodiment, the deterioration event positions 13 and 14 are distinguished and displayed on the trajectory display unit 110 depending on whether image data is linked to the deterioration event coordinates. Therefore, it is possible to display deterioration event coordinates that do not have corresponding image data.

[0076] (Embodiment 2) Next, a facility inspection display device and an information processing device according to a second embodiment will be described. In this embodiment, main features of the facility inspection display device and the information processing device will be extracted. FIG. 6 is a plan view illustrating an example of a display screen of the facility inspection display device according to the second embodiment. As shown in FIG. 6, the facility inspection display device 200 includes a trajectory display unit 110, a point cloud data display unit 120, and a deterioration event display unit 140.

[0077] The trajectory display unit 110 displays a trajectory of coordinates that serve as base points when point cloud data of the facility 60 using the LiDAR is acquired by moving the LiDAR within the facility 60. The point cloud data display unit 120 displays point cloud data G1 acquired using user-specified coordinates at a user-specified position 12 on a trajectory 11 specified by the user as a base point in the trajectory display unit 110. The deterioration event display unit 140 displays, on the trajectory display unit 110, deterioration event coordinates that serve as the base point when point cloud data including deterioration events of the facility 60 was acquired, as deterioration event positions 13.

[0078] 7 is a flowchart illustrating a facility inspection display method according to embodiment 2. As shown in step S210 of FIG. 7, a trajectory 11 is displayed on the trajectory display unit 110. Specifically, the trajectory 11 of coordinates that serve as base points when point cloud data of the facility 60 is acquired using the LiDAR by moving the LiDAR within the facility 60 is displayed on the trajectory display unit 110.

[0079] Next, as shown in step S220, the point cloud data is displayed on the point cloud data display unit 120. Specifically, the point cloud data acquired using the user-specified coordinates at the user-specified position 12 on the trajectory 11 specified by the user in the trajectory display unit 110 as the base point is displayed on the point cloud data display unit 120.

[0080] Next, as shown in step S230, the deterioration event position 13 is displayed on the trajectory display unit 110. Specifically, the deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility 60 is acquired are displayed on the trajectory display unit 110 as the deterioration event position.

[0081] Fig. 8 is a block diagram illustrating an information processing device according to embodiment 2. As shown in Fig. 8, the information processing device 70 includes a trajectory information storage unit 51, a point cloud data information storage unit 52, and a deterioration event coordinate extraction unit 54.

[0082] The trajectory information storage unit 51 stores trajectory information including a trajectory 11 of coordinates that serve as base points when point cloud data of the facility 60 is acquired using the LiDAR by moving the LiDAR within the facility 60, the coordinates of each of the multiple base points that make up the trajectory 11, and the times at which the LiDAR passed through each coordinate.

[0083] The point cloud data information storage unit 52 stores point cloud data information including each point cloud data acquired using each coordinate as a base point, and each time when the LiDAR passed through each coordinate that served as the base point when each point cloud data was acquired.

[0084] The deterioration event coordinate extraction unit 54 extracts deterioration event coordinates that serve as a base point when point cloud data including deterioration events of the facility 60 is acquired from the trajectory information and point cloud data information.

[0085] According to this embodiment, it is possible to improve the visibility of the display of the deterioration event position. Other configurations and effects of the second embodiment are included in the description of the first embodiment.

[0086] The present invention has been described above with reference to Embodiments 1 and 2, but the present invention is not limited to the above-described Embodiments 1 and 2. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. For example, an embodiment that combines the configurations of Embodiments 1 and 2 also falls within the scope of the technical idea. Furthermore, a facility inspection display program that causes a computer to execute the facility inspection display method of Embodiments 1 and 2 also falls within the technical scope of Embodiments 1 and 2.

[0087] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0088] (Appendix 1) a trajectory display means for displaying a trajectory of coordinates serving as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR in the facility; a point cloud data display means for displaying the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; a deterioration event display means for displaying, on the locus display means, a deterioration event coordinate that serves as the base point when the point cloud data including the deterioration events of the facility is acquired, as a deterioration event position; A facility inspection display device equipped with (Appendix 2) a coordinate display unit that displays the coordinates of the base point when the point cloud data is acquired; the coordinate display means displays the deterioration event coordinates. 1. A facility inspection display device as described in Appendix 1. (Appendix 3) the coordinate display means displays at least one of a direction from the deterioration event coordinates to the deterioration event and a time when the LiDAR passed through the deterioration event coordinates. 1. A facility inspection display device as described in Appendix 2. (Appendix 4) further comprising image data display means for displaying image data obtained by photographing the facility; Each coordinate constituting the trajectory is associated with each time when the LiDAR passed through each coordinate, Each point cloud data acquired using each coordinate constituting the trajectory as the base point is linked to each time when the LiDAR passed each of the coordinates, the image data is linked to the coordinates on the trajectory that are linked to the same time as the time at which the image data was acquired; When the image data associated with the user-specified coordinates exists, the image data display means displays the image data. 4. The facility inspection display device according to any one of appendices 1 to 3. (Appendix 5) and a distinguishing display means for distinguishing the deterioration event position depending on the presence or absence of the image data linked to the deterioration event coordinates and displaying the same on the locus display means. 10. A facility inspection display device as described in Appendix 4. (Appendix 6) a trajectory information storage means for storing trajectory information including a trajectory of coordinates serving as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility, the coordinates of each of the plurality of base points constituting the trajectory, and the times at which the LiDAR passed each of the coordinates; a point cloud data information storage means for storing point cloud data information including each point cloud data acquired using each of the coordinates as the base point and each time when the LiDAR passed through each of the coordinates that serve as the base point when each of the point cloud data was acquired; a deterioration event coordinate extraction means for extracting deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility is acquired from the trajectory information and the point cloud data information; An information processing device comprising: (Appendix 7) and an image data information storage means for storing image data information including image data of the facility and the time when the image data was acquired. The trajectory information storage means stores the coordinates constituting the trajectory in association with the times at which the LiDAR passed through the coordinates, the point cloud data information storage means stores the point cloud data acquired using the coordinates constituting the trajectory as the base point, in association with the time at which the LiDAR passed through each of the coordinates; the image data information storage means stores the image data in association with the coordinates on the trajectory that are associated with the same time as the time at which the image data was acquired; 7. The information processing device according to claim 6. (Appendix 8) The apparatus further includes a distinguishing and extracting means for distinguishing and extracting the deterioration event coordinates based on the presence or absence of the image data linked to the extracted deterioration event coordinates. 8. The information processing device according to claim 7. (Appendix 9) a step of displaying on a trajectory display means a trajectory of coordinates serving as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR in the facility; a step of displaying, on a point cloud data display means, the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; a step of displaying, on the trajectory display means, the deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility is acquired, as the deterioration event position; A facility inspection display method comprising: (Appendix 10) a step of displaying the coordinates of the base point when the point cloud data is acquired on a coordinate display means; In the step of displaying on the coordinate display means, the deterioration event coordinates are displayed on the coordinate display means. Facility inspection display method described in Appendix 9. (Appendix 11) In the step of displaying on the coordinate display means, at least one of a direction from the deterioration event coordinates to the deterioration event and a time when the LiDAR passed through the deterioration event coordinates is displayed on the coordinate display means. Facility inspection display method described in Appendix 10. (Appendix 12) further comprising a step of displaying image data of the facility on an image data display means; In the step of displaying the image data on an image data display means, Each coordinate constituting the trajectory is associated with each time when the LiDAR passed through each coordinate, Each point cloud data acquired using each coordinate constituting the trajectory as the base point is linked to each time when the LiDAR passed each of the coordinates, the image data is linked to the coordinates on the trajectory that are linked to the same time as the time at which the image data was acquired; If there is image data associated with the user-specified coordinates, the image data display means displays the image data. A facility inspection display method according to any one of appendices 9 to 11. (Appendix 13) The method further includes a step of distinguishing the deterioration event positions depending on whether or not the image data linked to the deterioration event coordinates is present, and displaying the same on the trajectory display means. Facility inspection display method described in Appendix 12. (Appendix 14) a step of displaying on a trajectory display means a trajectory of coordinates serving as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR in the facility; a step of displaying, on a point cloud data display means, the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; a step of displaying, on the trajectory display means, the deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility is acquired, as the deterioration event position; A non-transitory computer-readable medium storing a facility inspection display program that causes a computer to execute the above. (Appendix 15) a step of displaying the coordinates of the base point when the point cloud data is acquired on a coordinate display means; In the step of displaying on the coordinate display means, the deterioration event coordinates are displayed on the coordinate display means. A non-transitory computer-readable medium storing a facility inspection display program according to claim 14, which causes a computer to execute the program. (Appendix 16) In the step of displaying on the coordinate display means, at least one of a direction from the deterioration event coordinates to the deterioration event and a time when the LiDAR passed through the deterioration event coordinates is displayed on the coordinate display means. A non-transitory computer-readable medium storing a facility inspection display program according to claim 15, which causes a computer to execute the program. (Appendix 17) further comprising a step of displaying image data of the facility on an image data display means; In the step of displaying the image data on an image data display means, Each coordinate constituting the trajectory is associated with each time when the LiDAR passed through each coordinate, Each point cloud data acquired using each coordinate constituting the trajectory as the base point is linked to each time when the LiDAR passed each of the coordinates, the image data is linked to the coordinates on the trajectory that are linked to the same time as the time at which the image data was acquired; If there is image data associated with the user-specified coordinates, the image data display means displays the image data. A non-transitory computer-readable medium storing a facility inspection display program according to any one of appendices 14 to 16, which causes a computer to execute the program. (Appendix 18) A non-transitory computer-readable medium storing a facility inspection display program described in Appendix 17, which further causes a computer to execute a step of distinguishing the location of the deterioration event depending on whether or not the image data linked to the deterioration event coordinates is present and displaying the location on the trajectory display means.

[0089] In the above example, the program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path. [Explanation of symbols]

[0090] 1. Facility Inspection System 10 Inertial motion detection device 11 Trajectory 12 User-specified position 13, 14 Degradation event location 20 Point cloud data acquisition device 30 Image data acquisition device 40, 41 Mobile 50 Information processing equipment 51 Trajectory information holding unit 52 Point cloud data information storage unit 53 Image data information storage unit 54 Deterioration event coordinate extraction unit 55 Distinction extraction unit 60 facilities 70 Information processing equipment 100, 200 Facility inspection display device 110 Trajectory display section 120 Point cloud data display 130 Image data display unit 140 Deterioration event display unit 150 Coordinate display section 160 Distinction display unit

Claims

1. a trajectory display means for displaying a trajectory of coordinates serving as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility; a point cloud data display means for displaying the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; a deterioration event display means for displaying, on the locus display means, a deterioration event coordinate that serves as the base point when the point cloud data including the deterioration events of the facility is acquired, as a deterioration event position; image data display means for displaying image data obtained by photographing the facility; Equipped with Each coordinate constituting the trajectory is linked to each time when the LiDAR passed through each coordinate, Each point cloud data acquired using each coordinate constituting the trajectory as the base point is linked to each time when the LiDAR passed through each coordinate, the image data is linked to the coordinates on the trajectory that are linked to the same time as the time at which the image data was acquired; When the image data associated with the user-specified coordinates exists, the image data display means displays the image data; and a distinguishing display means for distinguishing the deterioration event position depending on the presence or absence of the image data linked to the deterioration event coordinates and displaying the same on the locus display means. Facility inspection display device.

2. a coordinate display unit that displays the coordinates of the base point when the point cloud data is acquired; the coordinate display means displays the deterioration event coordinates. The facility inspection display device according to claim 1 .

3. The coordinate display means displays at least one of a direction from the deterioration event coordinates to the deterioration event and a time when the LiDAR passed through the deterioration event coordinates. The facility inspection display device according to claim 2 .

4. a trajectory information storage means for storing trajectory information including a trajectory of coordinates serving as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility, the coordinates of each of the multiple base points that make up the trajectory, and the times at which the LiDAR passed through each of the coordinates; a point cloud data information storage means for storing point cloud data information including each point cloud data acquired using each of the coordinates as the base point and each time when the LiDAR passed through each of the coordinates that served as the base point when each of the point cloud data was acquired; a deterioration event coordinate extraction means for extracting deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility is acquired from the trajectory information and the point cloud data information; an image data information storage means for storing image data information including image data of the facility and the time when the image data was acquired; Equipped with The trajectory information storage means stores each of the coordinates constituting the trajectory in association with each of the times when the LiDAR passed through each of the coordinates, The point cloud data information storage means stores the point cloud data acquired using the coordinates constituting the trajectory as the base point, in association with the time when the LiDAR passed through each of the coordinates; the image data information holding means holds the image data in association with the coordinates on the trajectory that are linked to the same time as the time at which the image data was acquired; The apparatus further includes a distinguishing and extracting means for distinguishing and extracting the deterioration event coordinates based on the presence or absence of the image data linked to the extracted deterioration event coordinates. Information processing device.

5. a step of displaying on a trajectory display means a trajectory of coordinates that serve as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility; a step of displaying, on a point cloud data display means, the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; a step of displaying, on the trajectory display means, the deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility is acquired, as the deterioration event position; a step of displaying image data of the facility on an image data display means; Equipped with In the step of displaying the image data on an image data display means, Each coordinate constituting the trajectory is linked to each time when the LiDAR passed through each coordinate, Each point cloud data acquired using each coordinate constituting the trajectory as the base point is linked to each time when the LiDAR passed through each coordinate, the image data is linked to the coordinates on the trajectory that are linked to the same time as the time at which the image data was acquired; If there is image data associated with the user-specified coordinates, the image data display means displays the image data; The method further includes a step of distinguishing the deterioration event positions depending on whether or not the image data linked to the deterioration event coordinates is present, and displaying the same on the trajectory display means. Facility inspection display method.

6. The method further comprises the step of displaying, on a coordinate display means, the coordinates that serve as the base point when the point cloud data is acquired; In the step of displaying on the coordinate display means, the deterioration event coordinates are displayed on the coordinate display means. The facility inspection display method according to claim 5.

7. In the step of displaying on the coordinate display means, the coordinate display means displays at least one of the direction from the deterioration event coordinates to the deterioration event and the time when the LiDAR passed through the deterioration event coordinates. The facility inspection display method according to claim 6.

8. a step of displaying on a trajectory display means a trajectory of coordinates that serve as base points when point cloud data of the facility is acquired using the LiDAR by moving the LiDAR within the facility; a step of displaying, on a point cloud data display means, the point cloud data acquired using user-specified coordinates at a user-specified position on the trajectory specified by the user on the trajectory display means as the base point; a step of displaying, on the trajectory display means, the deterioration event coordinates that serve as the base point when the point cloud data including the deterioration events of the facility is acquired, as the deterioration event position; a step of displaying image data of the facility on an image data display means; Equipped with In the step of displaying the image data on an image data display means, Each coordinate constituting the trajectory is linked to each time when the LiDAR passed through each coordinate, Each point cloud data acquired using each coordinate constituting the trajectory as the base point is linked to each time when the LiDAR passed through each coordinate, the image data is linked to the coordinates on the trajectory that are linked to the same time as the time at which the image data was acquired; If there is image data associated with the user-specified coordinates, the image data display means displays the image data; and further causing the computer to execute a step of distinguishing the deterioration event positions depending on whether or not the image data linked to the deterioration event coordinates is present and displaying the same on the trajectory display means. Facility inspection display program.

9. The method further comprises the step of displaying, on a coordinate display means, the coordinates that serve as the base point when the point cloud data is acquired; In the step of displaying on the coordinate display means, the deterioration event coordinates are displayed on the coordinate display means.

9. The facility inspection display program according to claim 8, which causes a computer to execute the following.

10. In the step of displaying on the coordinate display means, the coordinate display means displays at least one of the direction from the deterioration event coordinates to the deterioration event and the time when the LiDAR passed through the deterioration event coordinates.

10. The facility inspection display program according to claim 9, which causes a computer to execute the following.

Images