3D data generation device, 3D data generation method and program

Abstract

To provide a three-dimensional data generation apparatus capable of generating highly accurate three-dimensional data by combining three-dimensional data generated on the basis of measurement data measured using apparatuses in different measurement locations.SOLUTION: A three-dimensional data generation apparatus includes: an acquisition section that acquires first three-dimensional data indicating a predetermined region and being generated using a first three-dimensional data generation apparatus installed on a ground surface, and second three-dimensional data indicating the predetermined region and being generated using a second three-dimensional data generation apparatus from above the predetermined region; a correction section that corrects a position between first two-dimensional data generated by viewing the first three-dimensional data with a vertically oriented viewpoint from above and second two-dimensional data generated by viewing the second three-dimensional data with a vertically oriented viewpoint from above; and a synthesis section that synthesizes the first three-dimensional data and the second three-dimensional data using a correction result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a three-dimensional data generation device, a three-dimensional data generation method, and a program. [Background technology]

[0002] In order to remotely support work at facilities such as substations, there is a need to generate 3D data of the facilities. Patent Document 1 discloses generating 3D data by combining point cloud data acquired by portable scanners placed at multiple locations near the ground. Since the point cloud data acquired by each portable scanner includes the side of a common object, the point cloud data can be combined by aligning the positions of the side of the common object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-166933 Summary of the Invention [Problem to be solved by the invention]

[0004] When generating 3D data of equipment installed on a vast site, such as a substation, a laser scanner may be mounted on an aerial vehicle such as a drone, and the 3D data may be generated using measurement data measured by the laser scanner. It is also possible to generate 3D data of the entire facility by combining 3D data generated using measurement data measured using the aerial vehicle with 3D data generated using measurement data measured using equipment installed on the ground. However, the 3D data generated using measurement data measured using the aerial vehicle and the 3D data generated using measurement data measured using equipment installed on the ground differ in measurement locations, resulting in few common measurement locations. Therefore, there is a problem in that it is not possible to generate highly accurate 3D data by combining 3D data generated using measurement data measured using the aerial vehicle with 3D data generated using measurement data measured using equipment installed on the ground.

[0005] The object of the present disclosure is to provide a 3D data generation device, a 3D data generation method, and a program that can generate highly accurate 3D data by combining 3D data generated based on measurement data measured using devices with different measurement locations. [Means for solving the problem]

[0006] A three-dimensional data generating device according to a first aspect of the present disclosure includes an acquisition unit that acquires first three-dimensional data indicating a predetermined area generated using a first three-dimensional data generating device installed on the ground surface and second three-dimensional data indicating the predetermined area generated using a second three-dimensional data generating device from above the predetermined area; a correction unit that corrects the position of the first three-dimensional data between first two-dimensional data generated using a viewpoint facing vertically from above and second two-dimensional data generated using a viewpoint facing vertically from above; and a synthesis unit that synthesizes the first three-dimensional data and the second three-dimensional data using the correction results.

[0007] A three-dimensional data generation method according to a second aspect of the present disclosure acquires first three-dimensional data representing a predetermined area generated using a first three-dimensional data generation device installed on the ground surface and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generation device from above the predetermined area, corrects the position of the first three-dimensional data between first two-dimensional data generated using a viewpoint facing vertically from above and second two-dimensional data generated using a viewpoint facing vertically from above, and synthesizes the first three-dimensional data and the second three-dimensional data using the correction results.

[0008] A program according to a third aspect of the present disclosure causes a computer to acquire first three-dimensional data indicating a predetermined area generated using a first three-dimensional data generating device installed on the ground surface and second three-dimensional data indicating the predetermined area generated using a second three-dimensional data generating device from above the predetermined area, correct the position of the first three-dimensional data between first two-dimensional data generated using a viewpoint facing vertically from above and second two-dimensional data generated using a viewpoint facing vertically from above, and synthesize the first three-dimensional data and the second three-dimensional data using the correction results. [Effects of the Invention]

[0009] The present disclosure provides a three-dimensional data generation device, a three-dimensional data generation method, and a program that can generate highly accurate three-dimensional data by combining three-dimensional data generated based on measurement data measured using devices with different measurement locations. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a configuration diagram of a three-dimensional data generation device according to the present disclosure. [Figure 2] FIG. 2 is a diagram illustrating the flow of a three-dimensional data generation process according to the present disclosure. [Figure 3]1 is a configuration diagram of a three-dimensional data generation device according to the present disclosure. [Figure 4] 1 is an overall view of a facility to be monitored according to the present disclosure. [Figure 5] 1 is a diagram showing the downward point cloud data according to the present disclosure as viewed from viewpoints A to D. FIG. [Figure 6] 10 is a diagram of the lower point cloud data according to the present disclosure viewed from viewpoint E. FIG. [Figure 7] 1 is a diagram showing the upward point cloud data according to the present disclosure as viewed from viewpoints A to D. FIG. [Figure 8] 10 is a diagram of the lower point cloud data according to the present disclosure viewed from viewpoint E. FIG. [Figure 9] FIG. 10 is a diagram illustrating ground surface removed downward point cloud data according to the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating upward point cloud data from which the ground surface has been removed according to the present disclosure. [Figure 11] 1A and 1B are diagrams illustrating ground surface-removed lower point cloud data and ground surface-removed upper point cloud data according to the present disclosure. [Figure 12] 10 is a diagram showing the ground surface when the lower point cloud data and the upper point cloud data according to the present disclosure are viewed from viewpoint C. FIG. [Figure 13] FIG. 2 is a diagram illustrating the flow of a three-dimensional data generation process according to the present disclosure. [Figure 14] 1 is a configuration diagram of a three-dimensional data generation device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] (Embodiment 1) An example of the configuration of a three-dimensional data generation device 10 will be described below with reference to Fig. 1. The three-dimensional data generation device 10 may be a computer device that operates when a processor executes a program stored in a memory. For example, the three-dimensional data generation device 10 may be an information processing device or a server device.

[0012] The 3D data generating device 10 includes an acquisition unit 11, a correction unit 12, and a synthesis unit 13. The acquisition unit 11, the correction unit 12, and the synthesis unit 13 may be software or modules that are executed by a processor executing a program stored in a memory, or may be hardware such as a circuit or a chip.

[0013] The acquisition unit 11 acquires first three-dimensional data representing a predetermined area generated using a first three-dimensional data generation device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generation device from above the predetermined area. The acquisition unit 11 may be used as a means for acquiring the first and second three-dimensional data.

[0014] The earth's surface may be referred to as the ground or the ground surface, etc. The three-dimensional data generating device may be, for example, a device that measures the distance to an object. The device that measures the distance to an object may be, for example, a LiDAR device. The LiDAR device measures the distance to an object using a ToF (Time of Flight) method and generates points that indicate the shape of the object. A collection of points that indicate the shape of an object becomes point cloud data. The points that indicate the shape of an object may be specified using three-dimensional coordinates in a predetermined space. In other words, the points that indicate the shape of an object may be indicated using three-dimensional coordinates in a predetermined coordinate system. The three-dimensional data may be point cloud data, which is a collection of points specified using three-dimensional coordinates.

[0015] The predetermined area may be an area where a plurality of facilities with similar appearances are located, such as a substation or a power plant.

[0016] The first three-dimensional data may be generated, for example, based on a plurality of point cloud data indicating distances to objects within a predetermined area measured at a plurality of points on the ground surface. Alternatively, the first three-dimensional data may be generated based on a plurality of pieces of photographic data obtained by photographing a predetermined area at a plurality of points on the ground surface. Specifically, the first three-dimensional data may be generated by using Structure from Motion (SfM) to generate three-dimensional data of an object located within the predetermined area from the plurality of pieces of photographic data.

[0017] The second 3D data generation device may be mounted on an aircraft flying above a predetermined area, and may measure the distance to an object within the predetermined area or photograph the object within the predetermined area along the flight path of the aircraft.

[0018] The correction unit 12 corrects the position between first two-dimensional data generated from the first three-dimensional data using a viewpoint facing vertically from above, and second two-dimensional data generated from the second three-dimensional data using a viewpoint facing vertically from above. The correction unit 12 may be used as a means for correcting the position between the first and second two-dimensional data.

[0019] The first two-dimensional data may be generated, for example, by projecting three-dimensional data representing an object existing within a predetermined area onto a plane included in the first three-dimensional data. The second two-dimensional data is similar to the first two-dimensional data. The plane included in the first three-dimensional data and the second three-dimensional data may be, for example, a horizontal plane perpendicular to the vertical direction. For example, the first two-dimensional data and the second two-dimensional data may each be generated by projecting point cloud data of an object within a predetermined area onto the ground surface. The vertical direction in each of the first three-dimensional data and the second three-dimensional data is a direction perpendicular to the ground surface and is assumed to be the same.

[0020] For example, the correction unit 12 may correct the first two-dimensional data so as to align it with the position indicated by the second two-dimensional data. Specifically, the correction unit 12 may correct the position of the first two-dimensional data or the position of the second two-dimensional data so as to align the positions of objects commonly included in the first two-dimensional data and the second two-dimensional data.

[0021] The combining unit 13 combines the first three-dimensional data and the second three-dimensional data using the correction result. The combining unit 13 may be used as a means for correcting the first and second three-dimensional data. The correction result may be, for example, the amount of movement when at least one of the position of the first two-dimensional data and the position of the second two-dimensional data is moved. For example, moving the position of the first two-dimensional data may mean rotating or translating the first two-dimensional data so that it matches the second two-dimensional data. Alternatively, moving the position of the first two-dimensional data may mean rotating and translating the first two-dimensional data so that it matches the second two-dimensional data.

[0022] Next, the flow of the three-dimensional data generation process executed in the three-dimensional data generation device will be explained using Fig. 2. First, the acquisition unit 11 acquires first three-dimensional data and second three-dimensional data (S11). The first three-dimensional data is data representing a predetermined area generated using a first three-dimensional data generation device installed on the ground surface. The second three-dimensional data is data representing a predetermined area generated using a second three-dimensional data generation device from above the predetermined area.

[0023] Next, the correction unit 12 corrects the position between the first two-dimensional data generated from the first three-dimensional data using a viewpoint facing vertically from above, and the second two-dimensional data generated from the second three-dimensional data using a viewpoint facing vertically from above (S12).

[0024] Next, the synthesis unit 13 synthesizes the first three-dimensional data and the second three-dimensional data using the correction result (S13).

[0025] As described above, the three-dimensional data generation device 10 generates first two-dimensional data from first three-dimensional data and generates second two-dimensional data from second three-dimensional data using a common viewpoint, i.e., a viewpoint facing vertically from above a predetermined area. The three-dimensional data generation device 10 applies the correction results performed to align the positions of the first two-dimensional data and the second two-dimensional data to the synthesis of the first three-dimensional data and the second three-dimensional data. As a result, even if the first and second three-dimensional data have few similar feature points in appearance, the three-dimensional data generation device 10 can combine the first and second three-dimensional data with high accuracy by using two-dimensional data generated using a common viewpoint.

[0026] (Embodiment 2) Next, an example configuration of a three-dimensional data generation device 20 will be described with reference to Fig. 3. The three-dimensional data generation device 20 has a configuration in which a ground surface extraction unit 21, a ground surface removal unit 22, a horizontal alignment unit 23, a vertical alignment unit 24, and an output unit 25 are added to the three-dimensional data generation device 10 of Fig. 1. The horizontal alignment unit 23 and the vertical alignment unit 24 correspond to the correction unit 12 in the three-dimensional data generation device 10. In the following, detailed description of functions or operations similar to those in Fig. 1 will be omitted.

[0027] The acquisition unit 11 acquires point cloud data generated by a LiDAR device installed on the ground surface. The point cloud data is generated, for example, by the LiDAR device measuring the distance to an object. The acquisition unit 11 may acquire point cloud data that combines point cloud data generated by multiple LiDAR devices installed in different locations. Alternatively, the acquisition unit 11 may acquire point cloud data from multiple LiDAR devices. In this case, the acquisition unit 11 may generate point cloud data that combines multiple point cloud data. Combining point cloud data may mean superimposing points that have common features in each point cloud data. Alternatively, combining point cloud data may mean combining each point cloud data to complement points not represented in each point cloud data. In other words, by combining multiple point cloud data, the shape of an object existing in a specified area can be displayed with high accuracy. Combining point cloud data may mean joining multiple point cloud data. Displaying the shape of an object with high accuracy may mean accurately displaying the shape of the object.

[0028] Furthermore, the acquisition unit 11 acquires point cloud data generated by a LiDAR device mounted on an aircraft moving above a predetermined area. The LiDAR device generates point cloud data at multiple locations during movement. The acquisition unit 11 may acquire point cloud data that combines multiple point cloud data, or may acquire each point cloud data generated at each location. When the acquisition unit 11 acquires each point cloud data generated at each location, the acquisition unit 11 may combine the respective point cloud data.

[0029] The point cloud data obtained by the acquisition unit 11 by combining point cloud data acquired from a LiDAR device installed on the ground surface is referred to as downward point cloud data, and the point cloud data obtained by combining point cloud data acquired from a LiDAR device moving in the sky is referred to as upward point cloud data.

[0030] The downward point cloud data is assumed to be three-dimensional data having coordinate axes in a vertical direction relative to the ground surface. The LiDAR device installed on the ground surface may be installed so as to have coordinate axes in a vertical direction relative to the ground surface. The downward point cloud data may be three-dimensional data in a coordinate system that has the position of the LiDAR device installed on the ground surface as the origin and is configured with vertical coordinate axes and coordinate axes representing planes perpendicular to the vertical direction. If multiple LiDAR devices are installed on the ground surface, the position of any one of the LiDAR devices may be set as the origin.

[0031] The upward point cloud data is assumed to be three-dimensional data having a coordinate axis in a vertical direction relative to the ground surface. The LiDAR device installed on the aircraft can identify the vertical direction by using, for example, an acceleration sensor. The downward point cloud data may be three-dimensional data in a coordinate system that has the position of the LiDAR device installed on the aircraft as the origin and is configured with a vertical coordinate axis and a coordinate axis representing a plane perpendicular to the vertical direction. Because the aircraft is moving, the position of the LiDAR device at the time measurement is started may be assumed to be the origin, for example.

[0032] The acquisition unit 11 outputs the lower point cloud data and the upper point cloud data to the ground surface extraction unit 21 and the synthesis unit 13.

[0033] Here, the downward point cloud data and the upward point cloud data will be described using Figs. 4 to 8. Fig. 4 shows an overall view of the equipment to be monitored. The equipment to be monitored may be, for example, a substation. The equipment to be monitored includes equipment A101 to equipment A104 and equipment B105 to equipment B108. Equipment A and equipment B may be, for example, transformers, circuit breakers, or voltage transformers. It is assumed that equipment A101 to equipment A104 have similar appearances, and that equipment B105 to equipment B108 also have similar appearances. The coordinate axis perpendicular to the ground surface is the Z axis, and the coordinate axes representing the ground surface are the X axis and the Y axis. It is assumed that the X axis, the Y axis, and the Z axis are mutually orthogonal.

[0034] Viewpoints A to D are viewpoints when the equipment to be monitored is viewed from the side. For example, viewpoints A and C may be viewpoints in the Y-axis direction, and viewpoints B and D may be viewpoints in the X-axis direction. Viewpoint E is a viewpoint when the equipment to be monitored is viewed from above. For example, viewpoint E may be a viewpoint in the Z-axis direction.

[0035] Figure 5 shows the downward point cloud data as viewed from viewpoints A to D. The downward point cloud data and upward point cloud data are three-dimensional data that combine point cloud data generated by measuring the monitored equipment from multiple locations. Therefore, for example, by using software that displays three-dimensional data, a user can check the monitored equipment from various viewpoints.

[0036] For example, when the monitored equipment is viewed from viewpoint A, the areas below equipment A101, equipment A102, and equipment A103 are displayed. The dotted lines in Figure 5 indicate the hidden portions of each piece of equipment. A LiDAR device installed on the ground generates point cloud data for each piece of equipment by irradiating the equipment, which is located higher than the LiDAR device, with a laser at an angle looking up from below. The laser irradiation range is limited to the LiDAR's field of view. Therefore, if a LiDAR is installed near the equipment to acquire point cloud data with high accuracy or high point density, the area above the equipment will not fall within the field of view, and point cloud data above the equipment will not be acquired in the downward point cloud data. As a result, point cloud data above the equipment will not be displayed, as shown in Figure 5. In Figure 5, point cloud data is assumed to have been acquired for the shaded area surrounded by a solid line. The area from which point cloud data has been acquired is assumed to be the area visible via a display unit such as a monitor.

[0037] When the monitored equipment is viewed from viewpoint B, the areas below equipment A102, equipment A103, and equipment A104 are displayed. When the monitored equipment is viewed from viewpoint C, the areas below equipment A104, equipment A102, and equipment A101, as well as the areas below equipment B108 and equipment B106, are displayed. When the monitored equipment is viewed from viewpoint D, the areas below equipment A104, equipment A103, and equipment A101, as well as the areas below equipment B106 and equipment B105 are displayed.

[0038] Fig. 6 shows the downward point cloud data as viewed from viewpoint E. The radar emitted from the LiDAR device installed on the ground does not hit the upper surfaces of facilities A101 to A104 and facilities B105 to B108. Therefore, Fig. 6 shows that no point cloud data has been acquired on the upper surfaces of facilities A101 to A104 and facilities B105 to B108, and point cloud data has mainly been acquired on the ground surface.

[0039] FIG. 7 shows the upward point cloud data as viewed from viewpoints A to D. When observing the monitored equipment from viewpoint A, the areas above equipment A101, equipment A102, and equipment A103 are displayed. The dotted lines in FIG. 7 indicate the hidden portions of each equipment. A LiDAR device moving in the sky generates point cloud data for each equipment by irradiating equipment lower than the LiDAR device with a laser. When acquiring point cloud data from the sky, the laser is irradiated not only vertically downward but also in a direction obliquely incident on the ground surface within the LiDAR's field of view. Therefore, the laser hits not only the top surface of the equipment but also surfaces perpendicular to the ground surface, thereby acquiring point cloud data for both the top and upper surfaces of the equipment. On the other hand, depending on the equipment's placement, the laser may not hit the area below the equipment, making it impossible to acquire point cloud data below the equipment. As a result, the upward point cloud data does not show the point cloud data below the equipment, as shown in FIG. 7. In FIG. 7, it is assumed that point cloud data has been acquired for the shaded area surrounded by a solid line. The area from which point cloud data has been acquired is an area that can be viewed through a display unit such as a display.

[0040] When the equipment to be monitored is viewed from viewpoint B, the areas above equipment A102, equipment A103, and equipment A104 are displayed. When the equipment to be monitored is viewed from viewpoint C, the areas above equipment A104, equipment A102, and equipment A101, as well as the areas above equipment B108 and equipment B106, are displayed. When the equipment to be monitored is viewed from viewpoint D, the areas above equipment A104, equipment A103, and equipment A101, as well as the areas above equipment B106 and equipment B105, are displayed.

[0041] Fig. 8 shows a diagram of the upward point cloud data as viewed from viewpoint E. The radar emitted from the LiDAR device moving in the sky hits the upper surfaces and ground surface of facilities A101 to A104 and facilities B105 to B108. Therefore, Fig. 8 shows that point cloud data of the upper surfaces and ground surface of facilities A101 to A104 and facilities B105 to B108 is acquired.

[0042] Returning to FIG. 3, the ground surface extraction unit 21 extracts the ground surface from the lower point cloud data and the upper point cloud data. The ground surface extraction unit 21 identifies at least one plane in each of the lower point cloud data and the upper point cloud data. In other words, the ground surface extraction unit 21 identifies point cloud data constituting a plane in each of the lower point cloud data and the upper point cloud data. The point cloud data constituting a plane is a collection of points constituting the plane. For example, the ground surface extraction unit 21 may identify, for each point having three-dimensional information, multiple points having the same value on a specific coordinate axis as point cloud data constituting the plane.

[0043] The ground surface extraction unit 21 may extract, from each of the lower point cloud data and the upper point cloud data, the lowest plane among planes perpendicular to the vertical direction as the ground surface. The point cloud data constituting the plane perpendicular to the vertical direction may be, for example, a set of points with the same value on the vertical coordinate axis. If the value on the vertical coordinate axis decreases as the plane approaches the ground surface, the lowest plane among the planes perpendicular to the vertical direction may be the plane with the smallest value on the vertical coordinate axis. Furthermore, the point cloud data constituting the plane perpendicular to the vertical direction may be grouped using a method such as Euclidean clustering, and the group with the widest distribution of points or the group with the largest number of points may be the ground surface.

[0044] The ground surface removal unit 22 removes point cloud data constituting the ground surface included in the lower point cloud data from the lower point cloud data. Removal can also be said to be deletion. The lower point cloud data from which the point cloud data constituting the ground surface included in the lower point cloud data has been removed is defined as ground surface-removed lower point cloud data. Furthermore, the ground surface removal unit 22 removes point cloud data constituting the ground surface included in the upper point cloud data from the upper point cloud data. The upper point cloud data from which the point cloud data constituting the ground surface included in the upper point cloud data has been removed is defined as ground surface-removed upper point cloud data.

[0045] The horizontal alignment unit 23 projects the ground surface-removed downward point cloud data onto a horizontal plane. Projecting can also be referred to as projecting. The horizontal plane is any plane perpendicular to the vertical direction, and may be, for example, the ground surface. When the ground surface-removed downward point cloud data is projected onto the ground surface, the point cloud data that constituted the ground surface has been removed, and therefore the ground surface-removed downward point cloud data projected onto the ground surface represents two-dimensional data of the shape of an object placed on the ground surface.

[0046] The horizontal positioning unit 23 similarly projects the ground surface removed upward point cloud data onto the horizontal plane.

[0047] The lower point cloud data from which the ground surface has been removed and projected onto a horizontal plane is data in which the point cloud data constituting the ground surface has been removed from a view of the lower point cloud data in FIG. 6 as viewed from viewpoint E, and is displayed, for example, as shown in FIG. 9. FIG. 9 shows the outline of each piece of equipment as viewed from viewpoint E. The upper point cloud data from which the ground surface has been removed and projected onto a horizontal plane is data in which the point cloud data constituting the ground surface has been removed from a view of the upper point cloud data in FIG. 8 as viewed from viewpoint E, and is displayed, for example, as shown in FIG. 10. FIG. 10 shows the top surface of each piece of equipment as viewed from viewpoint E.

[0048] The ground surface-removed lower point cloud data and ground surface-removed upper point cloud data projected onto a horizontal plane have the same vertical coordinate axis (Z axis), but the coordinate axes (X axis and Y axis) perpendicular to the vertical direction do not match. Therefore, as shown in Figure 11, for example, the ground surface-removed upper point cloud data is point cloud data that is displayed at an angle relative to the ground surface-removed lower point cloud data.

[0049] The horizontal alignment unit 23 corrects the difference in position between the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data projected onto the horizontal plane. Specifically, the horizontal alignment unit 23 rotates at least one of the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data projected onto the horizontal plane around a vertical coordinate axis. Furthermore, the horizontal alignment unit 23 translates at least one of the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data projected onto the horizontal plane along a coordinate axis perpendicular to the vertical direction. In this way, the horizontal alignment unit 23 aligns the positions of the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data projected onto the horizontal plane.

[0050] When the position of the ground surface-removed lower point cloud data projected onto the horizontal plane is aligned with the position of the ground surface-removed upper point cloud data projected onto the horizontal plane, the horizontal alignment unit 23 determines the amount of rotation and translation of the ground surface-removed lower point cloud data. Alternatively, when the position of the ground surface-removed upper point cloud data projected onto the horizontal plane is aligned with the position of the ground surface-removed lower point cloud data projected onto the horizontal plane, the horizontal alignment unit 23 determines the amount of rotation and translation of the ground surface-removed upper point cloud data. Alternatively, when both the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data projected onto the horizontal plane are moved, the horizontal alignment unit 23 determines the amount of rotation and translation of each point cloud data.

[0051] The vertical alignment unit 24 corrects the difference in position between the ground surface included in the downward point cloud data and the ground surface included in the upward point cloud data. Specifically, the vertical alignment unit 24 vertically aligns the ground surface included in the downward point cloud data extracted by the ground surface extraction unit 21 with the ground surface included in the upward point cloud data. The downward point cloud data is three-dimensional data in a coordinate system based on the position of a LiDAR device installed on the ground surface. Therefore, a plane with a Z-axis value of 0 substantially coincides with the ground surface. On the other hand, the upward point cloud data is three-dimensional data in a coordinate system based on the position of a LiDAR device mounted on an aircraft. Therefore, a position where the Z-axis value is 0 is the position of the LiDAR device mounted on an aircraft, and if the vertical upward direction is a positive Z-axis value, the Z-axis value of the ground surface is a negative value.

[0052] The dotted line in Figure 12 indicates the ground surface when the downward point cloud data and the upward point cloud data are viewed from viewpoint C. Note that even when the downward point cloud data and the upward point cloud data are viewed from viewpoints A, B, and D, the ground surface is shown in the same manner as in Figure 12. Because the Z-axis value of the ground surface in the downward point cloud data is substantially equal to 0, the ground surface in the downward point cloud data is at a position where the Z-axis value is 0. On the other hand, in the upward point cloud data, the position of the LiDAR device mounted on the aircraft has a Z-axis value of 0. Therefore, the ground surface is assumed to be at a position of, for example, -30 meters. The Z-axis value of -30 meters is a value that changes depending on, for example, the altitude of the aircraft, and is not limited to -30 meters.

[0053] The vertical alignment unit 24 aligns the position or Z-axis value by vertically translating at least one of the ground surface of the lower point cloud data and the ground surface of the upper point cloud data. When the position of the ground surface of the lower point cloud data is aligned with the position of the ground surface of the upper point cloud data, the vertical alignment unit 24 determines the amount of translation of the ground surface of the lower point cloud data. Alternatively, when the position of the ground surface of the upper point cloud data is aligned with the position of the ground surface of the lower point cloud data, the vertical alignment unit 24 determines the amount of translation of the ground surface of the upper point cloud data. Alternatively, when both the position of the ground surface of the upper point cloud data and the position of the ground surface of the lower point cloud data are moved, the vertical alignment unit 24 determines the amount of translation of the ground surface of the upper point cloud data and the ground surface of the lower point cloud data.

[0054] The synthesis unit 13 applies the rotation and translation amounts determined by the horizontal alignment unit 23 and the vertical alignment unit 24 to the lower point cloud data and the upper point cloud data acquired by the acquisition unit 11. That is, the synthesis unit 13 rotates and translates the lower point cloud data and the upper point cloud data by the same amounts as the rotation and translation of the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data in the horizontal alignment unit 23 and the vertical alignment unit 24. As a result, the positions of the lower point cloud data and the upper point cloud data substantially coincide, and the undisplayed portions of the respective facilities shown in Figures 5 and 7 and Figures 6 and 8 are complemented, generating highly accurate point cloud data.

[0055] The output unit 25 outputs point cloud data obtained by combining the lower point cloud data and the upper point cloud data to a display unit such as a display. Furthermore, the output unit 25 may output the lower point cloud data and the upper point cloud data to the display unit, or may output the lower point cloud data and the upper point cloud data viewed from each viewpoint shown in Figures 5 to 8 to the display unit. Furthermore, the output unit 25 may output the lower point cloud data from which the ground surface has been removed and the upper point cloud data from which the ground surface has been removed to the display unit.

[0056] Next, the flow of the three-dimensional data generation process in the three-dimensional data generation device 20 will be described with reference to FIG.

[0057] First, the acquisition unit 11 acquires the lower point cloud data and the upper point cloud data (S21). Next, the ground surface extraction unit 21 extracts the ground surface included in each of the lower point cloud data and the upper point cloud data (S22). Extracting the ground surface may mean extracting or identifying the point cloud data that constitutes the ground surface.

[0058] Next, the ground surface removal unit 22 removes the point cloud data constituting the ground surface from the lower point cloud data and the upper point cloud data (S23). The point cloud data obtained by removing the point cloud data constituting the ground surface from the lower point cloud data is defined as ground surface-removed lower point cloud data, and the point cloud data obtained by removing the point cloud data constituting the ground surface from the upper point cloud data is defined as ground surface-removed upper point cloud data.

[0059] Next, the horizontal alignment unit 23 projects the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data onto a horizontal plane (S24). The horizontal plane may be any plane perpendicular to the vertical direction.

[0060] Next, the horizontal alignment unit 23 horizontally aligns the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data projected onto the horizontal plane (S25). The horizontal alignment unit 23 rotates and translates, on the horizontal plane, at least one of the ground surface-removed lower point cloud data and the ground surface-removed upper point cloud data projected onto the horizontal plane. The horizontal alignment unit 23 also specifies the amount of rotation and translation of the rotated and translated point cloud data.

[0061] Next, the vertical alignment unit 24 vertically aligns the ground surfaces of the lower point cloud data and the upper point cloud data extracted in step S22 (S26). The vertical alignment unit 24 translates at least one of the ground surfaces of the lower point cloud data and the upper point cloud data in the vertical direction. The vertical alignment unit 24 also identifies the amount of translation of the translated ground surfaces.

[0062] Next, the synthesis unit 13 applies the rotation amount and translation amount identified in steps S25 and S26 to the lower point cloud data and the upper point cloud data to synthesize the lower point cloud data and the upper point cloud data (S27).

[0063] As described above, the three-dimensional data generation device 20 projects the lower point cloud data and the upper point cloud data, from which the point cloud data constituting the ground surface has been removed, onto a horizontal plane. By removing the point cloud data constituting the ground surface, only the point cloud data indicating the appearance of the object is projected onto the horizontal plane. As a result, the three-dimensional data generation device 20 can perform horizontal alignment based on the shape of the object.

[0064] Furthermore, the 3D data generation device 20 can correct the difference in height of the location where the monitored equipment is measured by performing vertical alignment using the ground surface included in the lower point cloud data and the upper point cloud data. The 3D data generation device 20 can generate 3D data that complements each of the lower point cloud data and the upper point cloud data by using the horizontal and vertical movement amounts to align the entire lower point cloud data and the upper point cloud data.

[0065] FIG. 14 is a block diagram showing an example configuration of a three-dimensional data generation device 10 and a three-dimensional data generation device 20 (hereinafter referred to as the three-dimensional data generation device 10, etc.). Referring to FIG. 14, the three-dimensional data generation device 10, etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) that complies with the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0066] The processor 1202 reads and executes software (computer programs) from the memory 1203 to perform the processing of the RU device 10 and the like described using flowcharts in the above-described embodiments. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0067] The memory 1203 is configured by a combination of volatile memory and non-volatile memory. The memory 1203 may include storage located remotely from the processor 1202. In this case, the processor 1202 may access the memory 1203 via an I / O (Input / Output) interface (not shown).

[0068] 14, the memory 1203 is used to store software modules. The processor 1202 reads these software modules from the memory 1203 and executes them to perform the processing of the 3D data generation device 10 and the like described in the above-described embodiment.

[0069] As explained using Figure 14, each of the processors possessed by the three-dimensional data generation device 10, etc. executes one or more programs including a group of instructions for causing a computer to perform the algorithm explained using the drawings.

[0070] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0071] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0072] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0073] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) an acquisition unit that acquires first three-dimensional data representing a predetermined area generated using a first three-dimensional data generation device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generation device from above the predetermined area; a correction unit that corrects a position between first two-dimensional data generated from the first three-dimensional data using a viewpoint facing vertically from above, and second two-dimensional data generated from the second three-dimensional data using a viewpoint facing vertically from above; a synthesis unit that synthesizes the first three-dimensional data and the second three-dimensional data using a correction result. (Appendix 2) The correction unit 2. The three-dimensional data generating device according to claim 1, wherein the device corrects a difference in position between first projection data obtained by projecting the first three-dimensional data onto the earth's surface and second projection data obtained by projecting the second three-dimensional data onto the earth's surface. (Appendix 3) The correction unit 3. The 3D data generation device according to claim 2, wherein the device corrects a difference in position between the first projection data obtained by deleting point cloud data included in the Earth's surface from the first 3D data, and the second projection data obtained by deleting point cloud data included in the Earth's surface from the second 3D data. (Appendix 4) The correction unit 4. The three-dimensional data generating device according to claim 2, wherein the three-dimensional data generating device determines a movement amount for aligning a position of one of the first projection data and the second projection data with a position of the other of the first projection data and the second projection data. (Appendix 5) The correction unit 4. The three-dimensional data generation device according to claim 1, wherein the device corrects a difference between a value indicating a vertical height of the first two-dimensional data representing the ground surface and a value indicating a vertical height of the second two-dimensional data representing the ground surface. (Appendix 6) The correction unit 6. The three-dimensional data generation device according to claim 5, wherein the device specifies a movement amount for aligning one of a value indicating the vertical height of the first two-dimensional data indicating the ground surface and a value indicating the vertical height of the second two-dimensional data indicating the ground surface with the other of a value indicating the vertical height of the first two-dimensional data indicating the ground surface and a value indicating the vertical height of the second two-dimensional data indicating the ground surface. (Appendix 7) 7. The three-dimensional data generation device according to claim 5, wherein the first two-dimensional data representing the ground surface is the lowest plane among planes perpendicular to the vertical direction included in the first three-dimensional data, and the second two-dimensional data representing the ground surface is the lowest plane among planes perpendicular to the vertical direction included in the second three-dimensional data. (Appendix 8) The synthesis unit A three-dimensional data generation device according to any one of appendices 1 to 7, wherein point cloud data within the specified area that is not included in the first three-dimensional data is complemented using the second three-dimensional data. (Appendix 9) an acquisition unit that acquires first three-dimensional data representing a predetermined area generated using a first three-dimensional data generation device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generation device from above the predetermined area; a first correction unit that corrects a positional difference between first projection data obtained by projecting the first three-dimensional data onto the earth's surface and second projection data obtained by projecting the second three-dimensional data onto the earth's surface; a second correction unit that corrects a difference between the height of the ground surface included in the first three-dimensional data and the height of the ground surface included in the second three-dimensional data; a synthesis unit that synthesizes the first three-dimensional data and the second three-dimensional data using a correction result from the first correction unit and a second correction result from the second correction unit. (Appendix 10) The 3D data generating device described in Appendix 9, wherein the first projection data is 2D data from the first 3D data in which point cloud data included in the earth's surface has been deleted, and the second projection data is 2D data from the second 3D data in which point cloud data included in the earth's surface has been deleted. (Appendix 11) an acquisition unit that acquires first three-dimensional data representing a predetermined area generated using a first three-dimensional data generation device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generation device from above the predetermined area; a synthesis unit that synthesizes the first three-dimensional data and the second three-dimensional data by aligning the coordinate axes of the first three-dimensional data and the coordinate axes of the second three-dimensional data. (Appendix 12) acquiring first three-dimensional data representing a predetermined area generated using a first three-dimensional data generating device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generating device from above the predetermined area; correcting a position between first two-dimensional data generated from the first three-dimensional data using a viewpoint facing vertically from above and second two-dimensional data generated from the second three-dimensional data using a viewpoint facing vertically from above; A three-dimensional data generating method for synthesizing the first three-dimensional data and the second three-dimensional data using a correction result. (Appendix 13) acquiring first three-dimensional data representing a predetermined area generated using a first three-dimensional data generating device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generating device from above the predetermined area; correcting a position between first two-dimensional data generated from the first three-dimensional data using a viewpoint facing vertically from above and second two-dimensional data generated from the second three-dimensional data using a viewpoint facing vertically from above; a program for causing a computer to execute the steps of: synthesizing the first three-dimensional data and the second three-dimensional data using a correction result; (Appendix 14) acquiring first three-dimensional data representing a predetermined area generated using a first three-dimensional data generating device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generating device from above the predetermined area; correcting a positional difference between first projection data obtained by projecting the first three-dimensional data onto the earth's surface and second projection data obtained by projecting the second three-dimensional data onto the earth's surface; correcting a difference between the height of the ground surface included in the first three-dimensional data and the height of the ground surface included in the second three-dimensional data; and synthesizing the first three-dimensional data and the second three-dimensional data using the results. (Appendix 15) acquiring first three-dimensional data representing a predetermined area generated using a first three-dimensional data generating device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generating device from above the predetermined area; correcting a positional difference between first projection data obtained by projecting the first three-dimensional data onto the earth's surface and second projection data obtained by projecting the second three-dimensional data onto the earth's surface; correcting a difference between the height of the ground surface included in the first three-dimensional data and the height of the ground surface included in the second three-dimensional data; a program for causing a computer to execute the steps of: synthesizing the first three-dimensional data and the second three-dimensional data using a correction result;

[0074] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 8 that are dependent on Supplementary Notes 1 may also be dependent on Supplementary Notes 12 and 13 in the same dependent relationship as Supplementary Notes 2 to 8. Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 10 that are dependent on Supplementary Notes 9 may also be dependent on Supplementary Notes 14 and 15 in the same dependent relationship as Supplementary Notes 10. Some or all of the elements described in any Supplementary Notes may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0075] 10. 3D data generation device 11 Acquisition Department 12 Correction unit 13 Synthesis section 20 3D data generation device 21 Ground surface extraction part 22 Ground surface removal section 23 Horizontal alignment section 24 Vertical alignment section 25 Output section 101 Equipment A 102 Equipment A 103 Equipment A 104 Equipment A 105 Equipment B 106 Equipment B 107 Equipment B 108 Equipment B

Claims

1. an acquisition unit that acquires first three-dimensional data representing a predetermined area generated using a first three-dimensional data generation device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generation device from above the predetermined area; a correction unit that corrects a position between first two-dimensional data generated from the first three-dimensional data using a viewpoint facing vertically from above, and second two-dimensional data generated from the second three-dimensional data using a viewpoint facing vertically from above; a synthesis unit that synthesizes the first three-dimensional data and the second three-dimensional data using a correction result, The correction unit a first projection data obtained by projecting the first three-dimensional data onto the earth's surface and deleting point cloud data included in the earth's surface; and a second projection data obtained by projecting the second three-dimensional data onto the earth's surface and deleting point cloud data included in the earth's surface, by rotating at least one of the first projection data and the second three-dimensional data around a vertical coordinate axis.

2. The correction unit 2. The three-dimensional data generating device according to claim 1, further comprising: specifying a movement amount for aligning a position of one of the first projection data and the second projection data with a position of the other of the first projection data and the second projection data.

3. The correction unit 2. The three-dimensional data generation device according to claim 1, wherein a movement amount is specified to align one of a value indicating the vertical height of the first two-dimensional data indicating the ground surface and a value indicating the vertical height of the second two-dimensional data indicating the ground surface with the other of a value indicating the vertical height of the first two-dimensional data indicating the ground surface and a value indicating the vertical height of the second two-dimensional data indicating the ground surface.

4. 2. The three-dimensional data generation device according to claim 1, wherein the first two-dimensional data representing the ground surface is the lowest plane among planes perpendicular to the vertical direction included in the first three-dimensional data, and the second two-dimensional data representing the ground surface is the lowest plane among planes perpendicular to the vertical direction included in the second three-dimensional data.

5. The synthesis unit The three-dimensional data generating device according to claim 1 , wherein point cloud data within the predetermined region that is not included in the first three-dimensional data is complemented using the second three-dimensional data.

6. acquiring first three-dimensional data representing a predetermined area generated using a first three-dimensional data generating device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generating device from above the predetermined area; correcting a difference between a value indicating a vertical height of first two-dimensional data representing the ground surface, the value being generated from a viewpoint facing vertically from the sky, and a value indicating a vertical height of second two-dimensional data representing the ground surface, the value being generated from a viewpoint facing vertically from the sky, the second three-dimensional data; correcting a positional difference between the first projection data and the second projection data by rotating at least one of the first projection data, which is obtained by projecting the first three-dimensional data onto the ground surface and deleting point cloud data included in the ground surface, and the second projection data, which is obtained by projecting the second three-dimensional data onto the ground surface and deleting point cloud data included in the ground surface, around a vertical coordinate axis; a three-dimensional data generating method for synthesizing the first three-dimensional data and the second three-dimensional data using a correction result;

7. acquiring first three-dimensional data representing a predetermined area generated using a first three-dimensional data generating device installed on the ground surface, and second three-dimensional data representing the predetermined area generated using a second three-dimensional data generating device from above the predetermined area; correcting a difference between a value indicating a vertical height of first two-dimensional data representing the ground surface, the value being generated from a viewpoint facing vertically from the sky, and a value indicating a vertical height of second two-dimensional data representing the ground surface, the value being generated from a viewpoint facing vertically from the sky, the second three-dimensional data; correcting a positional difference between the first projection data and the second projection data by rotating at least one of the first projection data, which is obtained by projecting the first three-dimensional data onto the ground surface and deleting point cloud data included in the ground surface, and the second projection data, which is obtained by projecting the second three-dimensional data onto the ground surface and deleting point cloud data included in the ground surface, around a vertical coordinate axis; a program for causing a computer to execute the steps of: synthesizing the first three-dimensional data and the second three-dimensional data using a correction result;

Images