Methods and systems for visualizing changes in monitored targets

Abstract

This invention provides a method and system that allows monitors and other personnel to easily visualize changes in monitored targets. [Solution] A method for visualizing changes in a monitored object involves acquiring first three-dimensional point cloud data of the monitored object at a first time (S1), acquiring second three-dimensional point cloud data of the monitored object at a second time different from the first time (S2), generating first projection data of the first three-dimensional point cloud data onto a predetermined plane (S3), generating second projection data of the second three-dimensional point cloud data onto the predetermined plane (S4), overlaying the first projection data and the second projection data to extract differences, generating difference data showing the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the first projection data and the second projection data (S5), and displaying the difference data as an image (S6, S7).

Description

Technical Field

[0001] The present invention relates to a method and a system for visualizing changes in a monitoring target.

Background Art

[0002] Patent Document 1 describes a technique for photographing a target installed on a slope such as a cliff with a plurality of displacement detection devices installed at different positions, and detecting displacement information of the slope where the target is installed using the obtained images.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above prior art, there is a possibility that changes occurring in a part of the slope, particularly changes in a part away from the target, cannot be detected. Since such changes can also be precursors to serious accidents such as collapses, it is desirable that they can be easily grasped by monitors and the like. Note that such a demand is not limited to slopes such as cliffs, but is common to natural and artificial objects that require monitoring or for which it is desirable to monitor.

[0005] Therefore, an object of the present invention is to provide a method and a system that can visualize changes in a monitoring target so that they can be easily grasped by monitors and the like.

Means for Solving the Problems

[0006] According to one aspect of the present invention, a method for visualizing changes in a monitored object is provided. The provided method includes: a first step of acquiring first three-dimensional point cloud data of the monitored object at a first time; a second step of acquiring second three-dimensional point cloud data of the monitored object at a second time different from the first time; a third step of generating first projection data of the first three-dimensional point cloud data onto a predetermined plane, wherein each point of the first projection data has coordinate values ​​in a direction perpendicular to the predetermined plane; a fourth step of generating second projection data of the second three-dimensional point cloud data onto the predetermined plane, wherein each point of the second projection data has coordinate values ​​in a direction perpendicular to the predetermined plane; a fifth step of overlaying the first projection data and the second projection data and extracting the difference to generate difference data showing the difference in coordinate values ​​in a direction perpendicular to the predetermined plane between the first projection data and the second projection data; and a sixth step of displaying the difference data as an image.

[0007] According to another aspect of the present invention, a system for visualizing changes in a monitored object is provided. The provided system includes a point cloud data acquisition device for acquiring three-dimensional point cloud data, a point cloud data processing device for processing the three-dimensional point cloud data, and a display device. The point cloud data processing device includes a projection data generation unit that generates first projection data onto a predetermined plane of first three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device at a first time, and second projection data onto the predetermined plane of second three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device at a second time different from the first time, a difference data generation unit that performs difference extraction by superimposing the first projection data and the second projection data and generates difference data showing the difference in coordinate values ​​in a direction perpendicular to the predetermined plane between the first projection data and the second projection data, and an image data generation unit that generates difference image data based on the difference data. The display device displays the difference image data. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a method and system that can visualize changes in the monitored object so that monitors and others can easily grasp them. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the schematic configuration of the visualization system according to the embodiment. [Figure 2] This is a functional block diagram of the point cloud data processing unit that constitutes the visualization system. [Figure 3] This diagram illustrates an example of extracting (calculating) the difference in coordinate values ​​perpendicular to a given surface between two projection data sets onto a given surface. [Figure 4] This flowchart shows an example of the processes performed by the visualization system. [Figure 5] This figure shows an example of three-dimensional point cloud data of the surface of a monitored object (retaining wall) after coordinate transformation to the local coordinate system. [Figure 6] Figure 5 shows an overview of the XY plane projection data of the three-dimensional point cloud data of the surface of the monitored object (retaining wall) onto the XY plane. [Figure 7] Figure 5 shows an overview of the XZ plane projection data of the three-dimensional point cloud data of the surface of the monitored object (retaining wall) onto the XZ plane. [Figure 8] This figure shows an example of displaying the first difference image data based on Z-coordinate difference data, which shows the difference in Z-coordinate values. [Figure 9] This figure shows an example of displaying a second difference image data based on Y-coordinate difference data, which shows the difference in Y-coordinate values. [Figure 10] This figure shows an example of displaying the first difference-enlarged image data based on the enlarged Z-coordinate difference data. [Figure 11] This figure shows an example of displaying a second difference-enlarged image data based on the enlarged Y-coordinate value difference data. [Figure 12] This figure shows an example of planar unfolded data from three-dimensional point cloud data of the inner surface of a tunnel. [Figure 13]This figure shows an example of displaying differential image data based on Z-coordinate difference data, which shows the difference in Z-coordinate values. [Figure 14] This figure shows an example of displaying magnified differential image data based on magnified Z-coordinate difference data. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings.

[0011] Figure 1 is a block diagram illustrating the schematic configuration of a visualization system 1 according to one embodiment of the present invention. The visualization system 1 according to this embodiment is configured to acquire three-dimensional point cloud data of a monitored object and to visualize changes (movements) of the monitored object based on the acquired three-dimensional point cloud data, thereby enabling monitors and others to easily grasp changes (movements) of the monitored object. The monitored object is a natural or artificial object that needs to be monitored or is desirable to monitor. Although not particularly limited, the monitored object may include natural slopes, artificial slopes, retaining walls, tunnels, and / or bridges. As shown in Figure 1, the visualization system 1 includes a point cloud data acquisition device 3, a point cloud data processing device 5, and a display device 7.

[0012] The point cloud data acquisition device 3 is, for example, a 3D laser scanner, which scans the target being monitored and acquires three-dimensional point cloud data of the target being monitored. The three-dimensional point cloud data of the target being monitored includes position data (three-dimensional coordinate values) of each reflection point of the laser scan light on the target being monitored (hereinafter simply referred to as "each point"). In this embodiment, each point in the three-dimensional point cloud data of the target being monitored acquired by the point cloud data acquisition device 3 has three-dimensional coordinate values ​​relative to a predetermined reference point. The point cloud data acquisition device 3 is mainly fixedly installed on the ground. However, it is not limited to this, and the point cloud data acquisition device 3 may be mounted on an unmanned aerial vehicle (UAV) or a ground mobile vehicle.

[0013] The point cloud data processing device 5 processes the three-dimensional point cloud data of the monitoring target acquired by the point cloud data acquisition device 3. The point cloud data processing device 5 is composed of, for example, a computer having at least one processor, at least one memory, a user interface, and a communication interface, etc. Each functional unit is realized by the processor reading and executing a program in the memory. FIG. 2 is a functional block diagram of the point cloud data processing device 5. As shown in FIG. 2, the point cloud data processing device 5 includes, as functional units, a point cloud data input unit 51, a coordinate conversion unit 52, a projection data generation unit 53, a difference data generation unit 54, and an image data generation unit 55.

[0014] The point cloud data input unit 51 inputs the three-dimensional point cloud data of the monitoring target acquired by the point cloud data acquisition device 3. In the present embodiment, the point cloud data input unit 51 inputs the three-dimensional point cloud data of the monitoring target acquired by the point cloud data acquisition device 3 at at least two different times.

[0015] The coordinate conversion unit 52 performs coordinate conversion of the three-dimensional point cloud data of the monitoring target input to the point cloud data input unit 51 as necessary. Specifically, the coordinate conversion unit 52 converts the three-dimensional point cloud data of the monitoring target input to the point cloud data input unit 51 into a local coordinate system set for the monitoring target. By this coordinate conversion, each point of the three-dimensional point cloud data of the monitoring target comes to have coordinate values (X coordinate value, Y coordinate value, Z coordinate value) of the local coordinate system. The local coordinate system can be arbitrarily set according to the monitoring target. Although not particularly limited, the local coordinate system may be such that the X-axis represents the "width (left and right) direction" of the monitoring target, the Y-axis represents the "depth (front and back) direction" of the monitoring target, and the Z-axis represents the "height (up and down) direction" of the monitoring target. Note that the coordinate conversion unit 52 may be omitted when it is not necessary to convert the three-dimensional point cloud data of the monitoring target into a local coordinate system.

[0016] The projection data generation unit 53 generates projection data onto a predetermined plane (projection plane) of the three-dimensional point cloud data of the monitored object input to the point cloud data input unit 51 or the three-dimensional point cloud data of the monitored object that has been coordinate-transformed by the coordinate transformation unit 52. The generated projection data consists of each point of the three-dimensional point cloud data of the monitored object arranged two-dimensionally on the predetermined plane, but each point in the generated projection data has coordinate values ​​in a direction perpendicular to the predetermined plane.

[0017] In this embodiment, the projection data generation unit 53 orthorectifies the three-dimensional point cloud data of the monitored object, which has been coordinate-transformed by the coordinate transformation unit 52, onto a predetermined plane. When multiple points overlap on the predetermined plane, the unit deletes all but the point with the largest coordinate value in the direction perpendicular to the predetermined plane (in other words, the point closest in the projection direction), thereby eliminating data duplication caused by orthorectification and generating projection data. Specifically, the predetermined plane is the XY plane (corresponding to the vertical plane), the XZ plane (corresponding to the horizontal plane), and / or the YX plane (corresponding to the sub-projection plane) of the local coordinate system.

[0018] Alternatively, the projection data generation unit 53 generates projection data by unfolding and projecting the three-dimensional point cloud data of the monitored object, which has been input to the point cloud data input unit 51, onto a plane (for example, a horizontal plane) (i.e., unfolding it into a plane).

[0019] The difference data generation unit 54 overlays two corresponding projection data generated by the projection data generation unit 53, extracts the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the two projection data, and generates difference data indicating that difference.

[0020] In this embodiment, the difference data generation unit 54 performs difference extraction by superimposing a first projection data generated based on the first three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device 3 at a first time (e.g., initial time) and a second projection data generated based on the three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device 3 at a second time different from the first time (e.g., after a predetermined time has elapsed), thereby generating difference data that shows the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the first projection data and the second projection data. The interval between the first time and the second time can be arbitrarily set according to the monitored object, etc.

[0021] Here, with reference to Figure 3, an example of extraction (calculation) of the difference in coordinate values ​​perpendicular to the predetermined plane between two corresponding projection data (first projection data, second projection data) by the difference data generation unit 54 will be explained.

[0022] The difference data generation unit 54 first represents the first projection data as a collection of multiple triangular meshes by forming a triangular mesh at three adjacent points in the first projection data, and takes the average of the coordinate values ​​of the three points forming each triangular mesh in the direction perpendicular to the predetermined plane as the coordinate value of each triangular mesh in the direction perpendicular to the predetermined plane.

[0023] Next, the difference data generation unit 54 represents the second projection data as a collection of multiple triangular meshes by forming triangular meshes at three adjacent points in the second projection data, and takes the average of the coordinate values ​​of the three points forming each triangular mesh in the direction perpendicular to the predetermined plane as the coordinate value of each triangular mesh in the direction perpendicular to the predetermined plane.

[0024] Then, the difference data generation unit 54 overlays the first projection data and the second projection data and extracts (calculates) the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the corresponding triangular meshes.

[0025] However, if the monitored object changes between the first and second periods, for example, the triangular mesh of the first projection data and the triangular mesh of the second projection data will not be exactly the same. In other words, as shown in Figure 3, when the first and second projection data are superimposed, the triangular mesh of the second projection data (solid line) may be shifted relative to the triangular mesh of the first projection data (dashed line). Therefore, the difference data generation unit 54 calculates the difference in the coordinate values ​​perpendicular to the predetermined plane based on the coordinate values ​​perpendicular to the predetermined plane of the triangular mesh of the second projection data perpendicular to the predetermined plane of each of the at least one triangular meshes of the first projection data that overlaps with at least a part of the triangular mesh of the second projection data, and the degree of overlap between the triangular mesh of the second projection data and each of the at least one triangular meshes of the first projection data.

[0026] For example, if there are three triangular meshes in the first projection data that partially overlap with the triangular meshes of the second projection data, and the coordinate value of the triangular mesh of the second projection data perpendicular to the predetermined plane is "d", the coordinate values ​​of the three triangular meshes of the first projection data perpendicular to the predetermined plane are "a", "b", and "c", and the degree of overlap between the triangular mesh of the second projection data and the three triangular meshes of the first projection data is "α%", "β%", and "γ%", then the difference data generation unit 54 calculates the difference in the coordinate values ​​perpendicular to the predetermined plane using "(da) × α / 100 + (db) × β / 100 + (dc) × γ / 100".

[0027] Returning to Figure 2, the image data generation unit 55 generates differential image data based on the differential data generated by the differential data generation unit 54. In this embodiment, the image data generation unit 55 generates differential image data for the display device 7 to display as an image color-coded according to the magnitude of the difference in coordinate values ​​perpendicular to the predetermined surface, in other words, image data having different color information according to the magnitude of the difference in coordinate values ​​perpendicular to the predetermined surface. Once the image data generation unit 55 has generated the differential image data, it outputs the generated differential image data to the display device 7.

[0028] Furthermore, if necessary, for example, if an enlargement display instruction is given by a monitor, the image data generation unit 55 multiplies the difference data generated by the difference data generation unit 54 by a predetermined magnification to generate enlarged difference data, and generates difference enlarged image data based on the generated enlarged difference data. In this embodiment, the image data generation unit 55 generates difference enlarged image data for the display device 7 to enlarge and display the difference data generated by the difference data generation unit 54 in three dimensions from a viewpoint obliquely above the predetermined surface. When the image data generation unit 55 generates difference enlarged image data, it outputs the generated difference enlarged image data to the display device 7.

[0029] Returning to Figure 1, the display device 7 is composed of a liquid crystal display or an organic EL display. When the display device 7 receives difference image data from the image data generation unit 55 of the point cloud data processing device 5, it displays the difference image data. That is, the display device 7 displays the difference data as an image color-coded according to the magnitude of the difference in coordinate values ​​in a direction perpendicular to the predetermined plane. Furthermore, when the display device 7 receives difference-enlarged image data from the image data generation unit 55 of the point cloud data processing device 5, it displays the difference-enlarged image data. That is, the display device 7 displays the difference data as an image that is three-dimensionally enlarged from a viewpoint obliquely above the predetermined plane.

[0030] Next, we will explain the visualization process of changes (movements) of the monitored object by Visualization System 1. Figure 4 is a flowchart showing an example of the process performed by Visualization System 1.

[0031] In step S1, the point cloud data acquisition device 3 acquires first three-dimensional point cloud data of the monitored object at a first time (for example, at the initial stage). Each point in the first three-dimensional point cloud data has a three-dimensional coordinate value relative to a predetermined reference point.

[0032] In step S2, the point cloud data acquisition device 3 acquires the second three-dimensional point cloud data of the monitored object at a second time (after a predetermined time has elapsed) that is different from the first time. Each point in the third three-dimensional point cloud data has a three-dimensional coordinate value relative to the reference point.

[0033] In step S3, the point cloud data processing device 5 receives the first three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device 3 at a first time, and generates first projection data of the first three-dimensional point cloud data of the monitored object onto a predetermined plane. Each point in the first projection data has coordinate values ​​in a direction perpendicular to the predetermined plane.

[0034] Specifically, in step S3, the point cloud data processing device 5 transforms the first three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device 3 at a first time into a local coordinate system for the monitored object, orthorectifies the transformed first three-dimensional point cloud data of the monitored object onto a predetermined plane, and generates first projected data by eliminating the data duplication caused by the orthorectification projection. The predetermined plane is the XY plane, XZ plane, and / or YZ plane of the local coordinate system.

[0035] Alternatively, in step S3, the point cloud data processing device 5 generates first projected data by unfolding and projecting the first three-dimensional point cloud data of the monitored object, acquired by the point cloud data acquisition device 3 at a first time, onto a plane. The plane is, for example, a horizontal plane.

[0036] In step S4, the point cloud data processing device 5 generates second projection data by projecting the second three-dimensional point cloud data of the monitored object, acquired by the point cloud data acquisition device 3 at a second time, onto the predetermined plane, in the same manner as in step S3. Each point in the second projection data has coordinate values ​​in a direction perpendicular to the predetermined plane.

[0037] In step S5, the point cloud data processing device 5 generates difference data between the corresponding first projection data and the second projection data. Specifically, the point cloud data processing device 5 overlays the corresponding first projection data and the second projection data, extracts the difference, and generates difference data showing the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the first projection data and the second projection data.

[0038] In step S6, the point cloud data processing device 5 generates differential image data based on the differential data. The differential image data is image data for displaying the differential data as an image color-coded according to the magnitude of the difference in coordinate values ​​in a direction perpendicular to the predetermined plane. The generated differential image data is output to the display device 7.

[0039] In step S7, the display device 7 receives the difference image data output from the point cloud data processing device 5 and displays the difference image data. That is, the display device 7 displays the difference data as an image color-coded according to the magnitude of the difference in coordinate values ​​in the direction perpendicular to the predetermined plane.

[0040] In step S8, the point cloud data processing device 5 determines whether or not an instruction to enlarge the display has been given by a monitor or the like. If an instruction to enlarge the display has been given, the process proceeds to step S9; otherwise, this flow ends.

[0041] In step S9, the point cloud data processing device 5 generates enlarged difference data by multiplying the difference data by a predetermined magnification factor.

[0042] In step S10, the point cloud data processing device 5 generates differential magnified image data based on the magnified differential data. The differential magnified image data is image data for displaying the differential data as an image that magnifies it three-dimensionally from a viewpoint obliquely above the predetermined plane. The generated differential magnified image data is output to the display device 7.

[0043] In step S11, the display device 7 receives the difference-enlarged image data from the point cloud data processing device 5 and displays the difference-enlarged image data. That is, the display device 7 displays the difference data as an image that is three-dimensionally enlarged from a viewpoint obliquely above the predetermined plane. Here, the display device 7 may display the difference-enlarged image data instead of the difference image data, or it may display the difference-enlarged image data in addition to the difference image data.

[0044] As described above, the visualization system 1 according to the embodiment acquires first three-dimensional point cloud data of the monitored object at a first time, acquires second three-dimensional point cloud data of the monitored object at a second time different from the first time, generates first projection data of the acquired first three-dimensional point cloud data onto a predetermined plane, and generates second projection data of the acquired second three-dimensional point cloud data onto the predetermined plane. The visualization system 1 also performs difference extraction by superimposing the first projection data and the second projection data, and generates difference data showing the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the first projection data and the second projection data. The visualization system 1 then displays the difference data as an image. Specifically, the visualization system 1 generates difference image data based on the difference data and displays the difference data as an image color-coded according to the magnitude of the difference in coordinate values ​​in the direction perpendicular to the predetermined plane.

[0045] The difference data represents the changes in the direction perpendicular to the predetermined surface that occurred in each part of the surface of the monitored object between the first period and the second period, that is, the "unidirectional" changes (movements) in each part of the surface of the monitored object, and this difference data is displayed as an image. In addition, different colors are assigned to the displayed image according to the magnitude of the "unidirectional" changes (movements). In other words, in the displayed image, the parts of the monitored object that have undergone "unidirectional" changes (movements) are displayed in a different color from the other parts (usually which make up the majority) that have not undergone such changes (movements). Therefore, by looking at the displayed image, monitors can easily understand whether or not there have been changes (movements) in the monitored object, and which parts of the monitored object have undergone changes (movements) in which direction.

[0046] Furthermore, when an enlargement display instruction is given, the visualization system 1 multiplies the difference data by a predetermined magnification to generate enlarged difference data, and displays the generated enlarged difference data as an image. Specifically, the visualization system 1 generates difference enlargement image data based on the enlarged difference data, and displays the difference data as an image that enlarges the difference data three-dimensionally from a viewpoint obliquely above the predetermined plane. In the displayed image, the parts where there has been a change (movement) in "one direction" are highlighted three-dimensionally, so monitors and others can easily grasp the state of the parts of the monitored object that have changed (moved) by looking at the displayed image.

[0047] Furthermore, by setting multiple predetermined surfaces on which the first and second three-dimensional point cloud data of the monitored object are projected, depending on the monitored object, monitors can easily and in detail grasp the changes (movements) in multiple directions of each part of the monitored object's surface, and consequently, how and which parts of the monitored object have changed (moved).

[0048] Furthermore, for example, by defining the first period as the start of monitoring and the second period as the first, second, third, and subsequent monitoring periods in a repeated monitoring process, monitors can gain a more detailed understanding of the changes (movements) of the monitored object based on the images formed at each stage. In addition, in some cases, it becomes possible to predict the changes (movements) of the monitored object.

[0049] Next, the present invention will be further explained by examples.

[0050] [First Embodiment] As a first embodiment, an example of the operation of the visualization system 1 when the object to be monitored is a retaining wall (including a part of the retaining wall) will be described. In this case, the point cloud data acquisition device 3 scans the retaining wall at the first and second time periods to acquire three-dimensional point cloud data of the surface of the retaining wall.

[0051] The point cloud data input unit 51 of the point cloud data processing device 5 receives the three-dimensional point cloud data of the retaining wall surface acquired by the point cloud data acquisition device 3.

[0052] The coordinate transformation unit 52 of the point cloud data processing device 5 transforms the three-dimensional point cloud data input to the point cloud data input unit 51, that is, the three-dimensional point cloud data of the retaining wall surface acquired by the point cloud data acquisition device 3, into a local coordinate system set for the retaining wall, such as the one shown in Figure 5. Through this coordinate transformation, each point in the three-dimensional point cloud data of the retaining wall surface will have coordinate values ​​(X coordinate value, Y coordinate value, Z coordinate value) in the local coordinate system. In the local coordinate system, the X axis represents the width (left-right) direction of the retaining wall, the Y axis represents the depth (front-back) direction of the retaining wall, and the Z axis represents the height (up-down) direction of the retaining wall.

[0053] The projection data generation unit 53 of the point cloud data processing device 5 generates XY plane projection data, XZ plane projection data, and / or YZ plane projection data from the coordinate-transformed three-dimensional point cloud data of the retaining wall surface. Here, each point in the XY plane projection data has a Z coordinate value, each point in the XZ plane projection data has a Y coordinate value, and each point in the YZ plane projection data has an X coordinate value. Figure 6 shows an overview of the XY plane projection data of the three-dimensional point cloud data of the retaining wall surface shown in Figure 5, and Figure 7 shows an overview of the XZ plane projection data of the three-dimensional point cloud data of the retaining wall surface shown in Figure 5.

[0054] The difference data generation unit 54 of the point cloud data processing device 5 generates Z-coordinate difference data showing the difference in Z-coordinate values, Y-coordinate difference data showing the difference in Y-coordinate values, and / or X-coordinate difference data showing the difference in X-coordinate values ​​as difference data.

[0055] The Z-coordinate difference data is generated based on the first XY-plane projection data generated from the first three-dimensional point cloud data of the retaining wall surface acquired in the first period, and the second XY-plane projection data generated from the second three-dimensional point cloud data of the retaining wall surface acquired in the second period. The Y-coordinate difference data is generated based on the first XZ-plane projection data generated from the first three-dimensional point cloud data of the retaining wall surface acquired in the first period, and the second XZ-plane projection data generated from the second three-dimensional point cloud data of the retaining wall surface acquired in the second period. The X-coordinate difference data is generated based on the first YZ-plane projection data generated from the first three-dimensional point cloud data of the retaining wall surface acquired in the first period, and the second YZ-plane projection data generated from the second three-dimensional point cloud data of the retaining wall surface acquired in the second period.

[0056] The image data generation unit 55 of the point cloud data processing device 5 generates a first differential image data based on Z coordinate difference data, a second differential image data based on Y coordinate difference data, and / or a third differential image data based on X coordinate difference data as differential image data.

[0057] The first differential image data shows the difference in Z coordinate values ​​(changes in the Z-axis direction) that occurred on the surface of the retaining wall between the first and second periods, and is used to display the data in different colors according to the magnitude of the difference in Z coordinate values ​​(amount of change in the Z-axis direction). The second differential image data shows the difference in Y coordinate values ​​(changes in the Y-axis direction) that occurred on the surface of the retaining wall between the first and second periods, and is used to display the data in different colors according to the magnitude of the difference in Y coordinate values ​​(amount of change in the Y-axis direction). The third differential image data shows the difference in X coordinate values ​​(changes in the X-axis direction) that occurred on the surface of the retaining wall between the first and second periods, and is used to display the data in different colors according to the magnitude of the difference in X coordinate values ​​(amount of change in the X-axis direction).

[0058] The display device 7 displays the first differential image data, the second differential image data, and / or the third differential image data generated by the image data generation unit 55 of the point cloud data processing device 5.

[0059] Figure 8 shows an example of the display of the first differential image data by the display device 7. Figure 8(a) shows an example of the display of the first differential image data when the second time period is the nth monitoring time, Figure 8(b) shows an example of the display of the first differential image data when the second time period is the (n+1)th monitoring time, and Figure 8(c) shows an example of the display of the first differential image data when the second time period is the (n+2)th monitoring time. Figure 9 shows an example of the display of the second differential image data by the display device 7. Figure 9(a) shows an example of the display of the second differential image data when the second time period is the nth monitoring time, Figure 9(b) shows an example of the display of the second differential image data when the second time period is the (n+1)th monitoring time, and Figure 9(c) shows an example of the display of the second differential image data when the second time period is the (n+2)th monitoring time. Note that in Figures 8 and 9, the display examples (display images) of the display device 7 are shown in black and white, but in reality, they are displayed in color.

[0060] By viewing the display images of the display device 7 shown in Figures 8 and 9, monitors can easily understand that there has been a change (movement) in the Z-axis direction (vertical direction) and the Y-axis direction (front-back direction) of "Part A" on the surface of the retaining wall, and that this change (movement) is continuing.

[0061] Furthermore, when an enlargement display instruction is given, the image data generation unit 55 of the point cloud data processing device 5 generates a first difference enlargement image data based on enlarged Z coordinate difference data obtained by multiplying the Z coordinate difference data by a predetermined magnification factor, a second difference enlargement image data based on enlarged Y coordinate difference data obtained by multiplying the Y coordinate difference data by a predetermined magnification factor, and / or a third difference enlargement image data based on enlarged X coordinate difference data obtained by multiplying the X coordinate difference data by a predetermined magnification factor.

[0062] The first difference-enlarged image data is image data for displaying the difference in Z coordinate values ​​(changes in the Z-axis direction) that occurred on the surface of the retaining wall between the first and second periods, in a three-dimensional, enlarged view from an oblique upward position relative to the XY plane. The second difference-enlarged image data is image data for displaying the difference in Y coordinate values ​​(changes in the Y-axis direction) that occurred on the surface of the retaining wall between the first and second periods, in a three-dimensional, enlarged view from an oblique upward position relative to the XZ plane. The third difference-enlarged image data is image data for displaying the difference in X coordinate values ​​(changes in the X-axis direction) that occurred on the surface of the retaining wall between the first and second periods, in a three-dimensional, enlarged view from an oblique upward position relative to the YZ plane.

[0063] The display device 7 displays the first difference-enlarged image data, the second difference-enlarged image data, and / or the third difference-enlarged image data generated by the image data generation unit 55 of the point cloud data processing device 5.

[0064] Figure 10 shows an example of the display of the first differential augmented image data by the display device 7. Figure 10(a) shows an example of the display of the first differential augmented image data when the second time period is the nth monitoring time, Figure 10(b) shows an example of the display of the first differential augmented image data when the second time period is the (n+1)th monitoring time, and Figure 10(c) shows an example of the display of the first differential augmented image data when the second time period is the (n+2)th monitoring time. Figure 11 shows an example of the display of the second differential augmented image data by the display device 7. Figure 11(a) shows an example of the display of the second differential augmented image data when the second time period is the nth monitoring time, Figure 11(b) shows an example of the display of the second differential augmented image data when the second time period is the (n+1)th monitoring time, and Figure 11(c) shows an example of the display of the second differential augmented image data when the second time period is the (n+2)th monitoring time. Although the display examples (display images) of the display device 7 are shown in black and white in Figures 10 and 11, the actual display is in color.

[0065] By viewing the display images of the display device 7 shown in Figures 10 and 11, monitors and other personnel can easily and accurately grasp the state of "A section" on the surface of the retaining wall where a change (movement) has occurred.

[0066] [Second Example] As a second embodiment, an example of the operation of the visualization system 1 when the object to be monitored is the tunnel (inner surface). In this case, the point cloud data acquisition device 3 scans the inner surface of the tunnel at the first and second time periods to acquire three-dimensional point cloud data of the inner surface of the tunnel.

[0067] The point cloud data input unit 51 of the point cloud data processing device 5 receives the three-dimensional point cloud data of the tunnel interior acquired by the point cloud data acquisition device 3.

[0068] The projection data generation unit 53 of the point cloud data processing device 5 generates planar unfolded data as projection data by unfolding (planar unfolding) the three-dimensional point cloud data input to the point cloud data input unit 51, that is, the three-dimensional point cloud data of the tunnel interior acquired by the point cloud data acquisition device 3, onto a horizontal plane (XY plane). Here, each point in the planar unfolded data has a coordinate value (Z coordinate value) in a direction perpendicular to the horizontal plane (XY plane). Figure 12 shows an example of planar unfolded data of three-dimensional point cloud data of the tunnel interior.

[0069] The difference data generation unit 54 of the point cloud data processing device 5 generates Z-coordinate difference data showing the difference in Z-coordinate values ​​based on the first planar unfolded data of the first three-dimensional point cloud data of the tunnel interior acquired in the first period and the second planar unfolded data of the second three-dimensional point cloud data of the tunnel interior acquired in the second period.

[0070] The image data generation unit 55 of the point cloud data processing device 5 generates differential image data based on Z coordinate value difference data. The differential image data is image data that shows the difference in Z coordinate values ​​(change in the Z-axis direction) that occurred on the inner surface of the tunnel between the first time period and the second time period, and is used for color-coding display according to the magnitude of the difference in Z coordinate values ​​(amount of change in the Z-axis direction). The Z-axis direction corresponds to the normal direction of the inner surface of the tunnel.

[0071] The display device 7 displays the differential image data generated by the image data generation unit 55 of the point cloud data processing device 5. Figure 13 shows examples of the display of differential image data by the display device 7. Figure 13(a) shows an example of the display of differential image data when the second time period is the nth monitoring time, Figure 13(b) shows an example of the display of differential image data when the second time period is the (n+1)th monitoring time, and Figure 13(c) shows an example of the display of differential image data when the second time period is the (n+2)th monitoring time. Note that although the display example (display image) of the display device 7 in Figure 13 is shown in black and white, it is actually displayed in color.

[0072] By viewing the display image of the display device 7 shown in Figure 13, monitors can easily grasp that there has been a change (movement) in the Z-axis direction (normal direction to the inner surface of the tunnel) in "Section B" on the inner surface of the tunnel, that the change (movement) in "Section B" has almost stopped, and that a new change (movement) in the Z-axis direction (normal direction to the inner surface of the tunnel) has occurred in "Section C".

[0073] Furthermore, when an enlarged display instruction is given, the image data generation unit 55 of the point cloud data processing device 5 generates differential enlarged image data based on enlarged Z coordinate difference data obtained by multiplying the Z coordinate difference data by a predetermined magnification. The differential enlarged image data is image data for displaying the difference in Z coordinate values ​​(changes in the Z-axis direction) that occurred on the inner surface of the tunnel between the first time period and the second time period in a three-dimensional enlarged view from a viewpoint obliquely above the XY plane.

[0074] The display device 7 displays the difference-enlarged image data generated by the image data generation unit 55 of the point cloud data processing device 5. Figure 14 shows an example of the display of difference image data by the display device 7. Figure 14(a) shows an example of the display of difference-enlarged image data (part B) when the second time period is the (n+2)th monitoring time, and Figure 14(b) shows an example of the display of difference-enlarged image data (part C) when the second time period is the (n+2)th monitoring time. Although the display example (display image) of the display device 7 in Figure 14 is shown in black and white, it is actually displayed in color.

[0075] By viewing the display image of the display device 7 shown in Figure 14, monitors and others can easily determine that a rectangular bulge has occurred in "Section B" on the inner surface of the tunnel, and that a rectangular depression has occurred in "Section C" on the inner surface of the tunnel.

[0076] Although embodiments and examples of the present invention have been described above, the present invention is not limited to the embodiments and examples described above, and can be appropriately modified and changed based on the technical concept of the present invention. [Explanation of symbols]

[0077] 1…Visualization system, 3…Point cloud data acquisition device, 5…Point cloud data processing device, 7…Display device, 51…Point cloud data input unit, 52…Coordinate transformation unit, 53…Projection data generation unit, 54…Difference data generation unit, 55…Image data generation unit

Claims

1. A method for visualizing changes in the monitored target, The first step is to acquire the first three-dimensional point cloud data of the monitored object in the first period, A second step involves acquiring second three-dimensional point cloud data of the monitored object at a second time different from the first time, A third step involves generating first projection data of the first three-dimensional point cloud data onto a predetermined plane, wherein each point in the first projection data has coordinate values ​​in a direction perpendicular to the predetermined plane. A fourth step involves generating second projection data of the second three-dimensional point cloud data onto the predetermined plane, wherein each point in the second projection data has coordinate values ​​perpendicular to the predetermined plane. A fifth step involves overlaying the first projection data and the second projection data to extract the difference and generate difference data showing the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the first projection data and the second projection data, The sixth step is to display the aforementioned difference data as an image, Methods that include...

2. The method according to claim 1, wherein the sixth step includes displaying the difference data as an image color-coded according to the magnitude of the difference.

3. The method according to claim 1, wherein the sixth step includes displaying the difference data as an image that is three-dimensionally enlarged from a viewpoint obliquely above the predetermined plane.

4. The third step includes transforming the first three-dimensional point cloud data into a local coordinate system for the object being monitored, and orthorectifying the transformed first three-dimensional point cloud data onto the XY plane, XZ plane, and / or YZ plane of the local coordinate system, and generating first projected data by eliminating the data duplication resulting from the orthorectification projection. The fourth step includes transforming the second three-dimensional point cloud data to the local coordinate system, and orthorectifying the transformed second three-dimensional point cloud data onto the XY plane, XZ plane, and / or YZ plane of the local coordinate system, and generating second projected data by eliminating the data duplication resulting from the orthorectification projection. The method according to claim 1.

5. The third step includes unfolding and projecting the first three-dimensional point cloud data onto a plane to generate first projection data, The fourth step includes unfolding and projecting the second three-dimensional point cloud data onto a plane to generate second projection data. The method according to claim 1.

6. The fifth step described above is: The first projection data is represented as a collection of multiple triangular meshes by forming a triangular mesh at three adjacent points in the first projection data, and the average of the coordinate values ​​of the three points constituting each triangular mesh in the direction perpendicular to the predetermined plane is taken as the coordinate value of each triangular mesh in the direction perpendicular to the predetermined plane. The second projection data is represented as a collection of multiple triangular meshes by forming a triangular mesh at three adjacent points in the second projection data, and the average of the coordinate values ​​of the three points constituting the triangular mesh in the direction perpendicular to the predetermined plane is taken as the coordinate value of each triangular mesh in the direction perpendicular to the predetermined plane. The difference is calculated based on the coordinate values ​​of the triangular mesh of the second projection data in a direction perpendicular to the predetermined plane, the coordinate values ​​of at least one triangular mesh of the first projection data in a direction perpendicular to the predetermined plane, which overlap with at least a portion of the triangular mesh of the second projection data, and the degree of overlap between the triangular mesh of the second projection data and the at least one triangular mesh of the first projection data, and the difference data is generated. The method according to claim 1, including the method described in claim 1.

7. A system that visualizes changes in the monitored target, A point cloud data acquisition device for acquiring three-dimensional point cloud data, a point cloud data processing device for processing three-dimensional point cloud data, and a display device are included. The point cloud data processing device is A projection data generation unit generates first projection data onto a predetermined plane of the first three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device at a first time, and generates second projection data onto the predetermined plane of the second three-dimensional point cloud data of the monitored object acquired by the point cloud data acquisition device at a second time different from the first time; A difference data generation unit overlays the first projection data and the second projection data to extract the difference and generates difference data showing the difference in coordinate values ​​in the direction perpendicular to the predetermined plane between the first projection data and the second projection data, An image data generation unit that generates differential image data based on the differential data, Includes, The display device displays the differential image data. system.

8. The system according to claim 7, wherein the difference image data is image data for the display device to display the difference data as an image color-coded according to the magnitude of the difference.

9. The image data generation unit generates magnified difference image data based on magnified difference data obtained by multiplying the difference data by a predetermined magnification factor. The display device displays the difference enlargement image data in place of or in addition to the difference image data. The system according to claim 7.

10. The system according to claim 9, wherein the difference-enlarged image data is image data for which the display device displays the difference data in a three-dimensional enlarged manner from a viewpoint obliquely above the predetermined surface.

11. The point cloud data processing device further includes a coordinate transformation unit that transforms the first three-dimensional point cloud data and the second three-dimensional point cloud data into a local coordinate system for the monitored object. The projection data generation unit generates first projection data by orthogonally projecting the coordinate-transformed first three-dimensional point cloud data onto the XY plane, XZ plane, and / or YZ plane of the local coordinate system and eliminating data duplication resulting from the orthogonal projection; and generates second projection data by orthogonally projecting the coordinate-transformed second three-dimensional point cloud data onto the XY plane, XZ plane, and / or YZ plane of the local coordinate system and eliminating data duplication resulting from the orthogonal projection. The system according to claim 7.

12. The system according to claim 7, wherein the projection data generation unit generates first projection data by unfolding and projecting the first three-dimensional point cloud data onto a plane, and generates second projection data by unfolding and projecting the second three-dimensional point cloud data onto a plane.

13. The aforementioned differential data generation unit, The first projection data is represented as a collection of many triangular meshes by forming a triangular mesh at three adjacent points in the first projection data, and the average of the coordinate values ​​of the three points constituting each triangular mesh in the direction perpendicular to the predetermined plane is taken as the coordinate value of each triangular mesh in the direction perpendicular to the predetermined plane. The second projection data is represented as a collection of many triangular meshes by forming a triangular mesh at three adjacent points in the second projection data, and the average of the coordinate values ​​of the three points constituting each triangular mesh in the direction perpendicular to the predetermined plane is taken as the coordinate value of each triangular mesh in the direction perpendicular to the predetermined plane. The difference is calculated based on the coordinate values ​​of the triangular mesh of the second projection data in a direction perpendicular to the predetermined plane, the coordinate values ​​of at least one triangular mesh of the first projection data in a direction perpendicular to the predetermined plane, which overlap with at least a portion of the triangular mesh of the second projection data, and the degree of overlap between the triangular mesh of the second projection data and the at least one triangular mesh of the first projection data, and the difference data is generated. The system according to claim 7.

Images