Three-dimensional display device, method, and computer-readable recording medium

Abstract

The present application provides a three-dimensional display device, method and program which display a three-dimensional model of a structure and enable easy confirmation of damage information and the like of each component constituting the structure. A CPU (20) functioning as a processor of the three-dimensional display device (10) maps damage information associated with a component of the structure stored in a storage section (16) onto a component corresponding to damage information of a three-dimensional model stored in the storage section (16). Then, when a display instruction of a component of an inspection object of the structure is accepted from an operation section (18), a three-dimensional model of only the component having damage information mapped thereto to which the display instruction is accepted is created, and the display section (30) is caused to display the three-dimensional model.

Description

TECHNICAL FIELD

[0001] The present application relates to a three-dimensional display device, method, and computer-readable recording medium, and particularly relates to a technology for displaying a three-dimensional model of a structure on a display. BACKGROUND

[0002] Social infrastructure structures such as bridges need to be periodically inspected for maintenance and repair.

[0003] In recent years, a technology is becoming widespread in which a plurality of images are captured in a manner in which a portion of each image is overlapped for a high bridge or the like that is difficult to reach by an unmanned aerial vehicle or a robot or the like equipped with an imaging device, a three-dimensional point cloud is generated using SfM (Structure from Motion), and a three-dimensional model of a structure based on the three-dimensional point cloud is generated.

[0004] In Patent Literature 1, a technology is disclosed in which a damage (damage map) detected by analyzing a photographic image of a structure is mapped on a surface of a three-dimensional model of the structure generated using SfM, and the three-dimensional model on which the damage map is mapped is displayed on a display.

[0005] PRIOR ART DOCUMENTS

[0006] PATENT LITERATURE

[0007] Patent Literature 1: International Publication No. 2019 / 198562 SUMMARY

[0008] PROBLEMS TO BE SOLVED BY THE INVENTION

[0009] In Patent Literature 1, the display method of the display of the three-dimensional model on which the damage map is mapped is not specifically described.

[0010] The inspection of a structure is performed in units of a management classification classified as a unit of inspection of a site and a member constituting the structure, and an evaluation of a damage degree, a record of an appearance characteristic, or the like is performed for each management classification according to an inspection evaluation criterion established by a state or an autonomous body or the like.

[0011] When a display displays a three-dimensional model, and a user wants to confirm a specific member constituting a structure and a damage in the member, in the display of the three-dimensional model, there is a problem that unnecessary information other than the information that the user wants to confirm is displayed, or a portion of the specific member becomes a blind spot of other members and is difficult to see.

[0012] The present application was completed in view of such a situation, and an object thereof is to provide a three-dimensional display device, method, and program that cause a display to display a three-dimensional model of a structure, and that can easily confirm damage information or the like of each member constituting the structure.

[0013] Means for solving technical problems

[0014] To achieve the above object, the invention according to a first aspect is a three-dimensional display device including a processor and a storage that stores a three-dimensional model of a structure and damage information associated with components of the structure, wherein the processor maps the damage information stored in the storage to the components corresponding to the damage information of the three-dimensional model stored in the storage, accepts an indication of display or non-display of the components of the structure from a user interface, causes a display to display a three-dimensional model of only the components to which the indication of display is accepted and to which the damage information is mapped when an indication of display of components of the structure to be inspected is accepted from the user interface, or causes the display to display a three-dimensional model of the components of the structure except for the components to which the indication of non-display is accepted and to which the damage information is mapped when an indication of non-display of desired components of the structure is accepted.

[0015] According to the invention according to the first aspect, the processor performs damage information mapping processing of mapping the damage information associated with the components of the structure to the components corresponding to the damage information of the three-dimensional model of the structure. Also, when an indication of display of components of the structure to be inspected is accepted from the user interface, the display is caused to display a three-dimensional model of only the components to which the indication of display is accepted and to which the damage information is mapped, or when an indication of non-display of desired components of the structure is accepted from the user interface, the display is caused to display a three-dimensional model of the components of the structure except for the components to which the indication of non-display is accepted and to which the damage information is mapped.

[0016] Thus, on the display that displays the three-dimensional model of the components to which the damage information is mapped, unnecessary information other than the information that the user wants to confirm is not displayed, or the components to which the indication of non-display is accepted are deleted, and the user can easily confirm the damage information of the desired components constituting the structure and the like.

[0017] In the three-dimensional display device according to the second aspect of the invention, it is preferable that the three-dimensional model is a three-dimensional point cloud model, a three-dimensional polygon model created based on the three-dimensional point cloud model, or a three-dimensional model formed by texture mapping an image obtained by photographing the structure on each polygon of the three-dimensional polygon model.

[0018] In the three-dimensional display device according to the third aspect of the invention, it is preferable that the processor identifies each component constituting the structure based on the three-dimensional model.

[0019] In the three-dimensional display device according to the fourth aspect of the invention, it is preferable that the storage stores a table indicating a relationship between each component of the structure and a damage that can occur in each component, and when a selection of an arbitrary one of a plurality of damage types of the structure is accepted from the user interface, the processor determines components corresponding to the selected damage type from the table and accepts display of the determined components.

[0020] Thus, the user can determine the component corresponding to the selected damage type with the processor and display only the three-dimensional model of the determined component (i.e., the component to which the damage information corresponding to the damage type to be confirmed is mapped).

[0021] In the three-dimensional display device according to the fifth aspect of the present application, it is preferable that the processor cause the display to display the three-dimensional model of the component to which the damage information is mapped, and accept a display instruction of the component of the structure to be inspected or a non-display instruction of the desired component of the structure indicated by the pointing device on the display on which the three-dimensional model is displayed.

[0022] Thus, the user can perform the display instruction of the component of the structure to be inspected while observing the three-dimensional model displayed on the display, or can perform the non-display instruction of the desired component of the structure.

[0023] In the three-dimensional display device according to the sixth aspect of the present application, it is preferable that, when the display instruction of the component of the structure to be inspected is accepted from the user interface, the processor set the line-of-sight direction to the normal direction of the surface of the component and cause the display to display the three-dimensional model.

[0024] Thus, the user can visually recognize the surface of the component of the structure to be inspected head-on, and can more accurately grasp the damage information mapped to the surface of the component.

[0025] In the three-dimensional display device according to the seventh aspect of the present application, it is preferable that the damage information is a damage map indicating the position or area of the damage.

[0026] In the three-dimensional display device according to the eighth aspect of the present application, the damage of the concrete component of the structure includes at least one of a crack, a spalling / steel bar exposure, and a water leakage / free lime, and the damage of the steel component of the structure includes at least one of corrosion, a crack, and a breakage.

[0027] The invention related to the ninth aspect is a three-dimensional display method of causing a display to display a three-dimensional model of a structure by a processor, wherein each process of the processor includes: a step of acquiring a three-dimensional model and damage information associated with a component of the structure from a storage that stores the three-dimensional model of the structure and the damage information; a step of mapping the acquired damage information to a component corresponding to the damage information of the three-dimensional model; a step of accepting an indication of display or non-display of a component of the structure from a user interface; and a step of causing the display to display a three-dimensional model of only a component to which a display indication is accepted and to which damage information is mapped when a display indication of a component to be inspected of the structure is accepted from the user interface, or a three-dimensional model of a component of the structure except for a component to which a non-display indication is accepted and to which damage information is mapped when a non-display indication of a desired component of the structure is accepted.

[0028] The invention related to the tenth aspect is a three-dimensional display program that causes a computer to execute a three-dimensional display method of causing a display to display a three-dimensional model of a structure, wherein the three-dimensional display method includes: a step of acquiring a three-dimensional model and damage information associated with a component of the structure from a storage that stores the three-dimensional model of the structure and the damage information; a step of mapping the acquired damage information to a component corresponding to the damage information of the three-dimensional model; a step of accepting an indication of display or non-display of a component of the structure from a user interface; and a step of causing the display to display a three-dimensional model of only a component to which a display indication is accepted and to which damage information is mapped when a display indication of a component to be inspected of the structure is accepted from the user interface, or a three-dimensional model of a component of the structure except for a component to which a non-display indication is accepted and to which damage information is mapped when a non-display indication of a desired component of the structure is accepted.

[0029] Effects of Invention

[0030] According to the present invention, it is possible to cause a display to display a three-dimensional model of a structure, and easily confirm damage information and the like of each component constituting the structure. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a block diagram showing an example of a hardware structure of a three-dimensional display apparatus related to the present invention.

[0032] Figure 2 is a chart showing an example of a table showing a relationship between each component of a structure and a damage that can occur in each component.

[0033] Figure 3 is an example of a chart showing a screen of a display section that displays an entire three-dimensional model of a bridge.

[0034] Figure 4is a drawing showing an example of a screen of a display section showing a part of a three-dimensional model of a bridge.

[0035] Figure 5 is a drawing showing a part of a flowchart showing an embodiment of a three-dimensional display method to which the present application pertains.

[0036] Figure 6 is a flowchart showing a flowchart shown in Figure 5

[0037] Figure 7 is a drawing showing an example of a screen of a display section showing a three-dimensional model of a floor on which a damage map is mapped.

[0038] Figure 8 is a drawing showing an example of a screen of a display section showing a three-dimensional model of a floor on which a damage map is mapped, that is, a three-dimensional model showing a surface of a structure represented by a collection of sides of polygons. DETAILED DESCRIPTION

[0039] Hereinafter, a preferred embodiment of a three-dimensional display apparatus, method and program to which the present application pertains will be described with reference to the accompanying drawings.

[0040] [Hardware structure of three-dimensional display apparatus]

[0041] Figure 1 is a block diagram showing an example of a hardware structure of a three-dimensional display apparatus to which the present application pertains.

[0042] As shown in Figure 1 The three-dimensional display apparatus 10 shown in the drawing can be used as a personal computer or a workstation. The three-dimensional display apparatus 10 of the present example is mainly composed of an image acquisition section 12, an image database 14, a storage section 16, an operation section 18, a CPU (Central Processing Unit) 20, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 24, and a display control section 26. The display control section 26 is composed of a GPU (Graphic Processing Unit) or the like, for example. Alternatively, the CPU 20 can also function as the display control section 26.

[0043] The image acquisition section 12 corresponds to an input / output interface, and in the present example, acquires a photographic image or the like obtained by photographing a structure of an inspection object. The structure of the inspection object includes a bridge, a tunnel or the like, for example.

[0044] ​The image acquired by the image acquisition unit 12 is, for example, a plurality of images (photograph image group) obtained by photographing a structure by a drone (unmanned aerial vehicle) or a robot on which an imaging device is mounted. It is preferable that the photograph image group cover the entire structure, and that adjacent photograph images overlap by images in a range of about 80%.

[0045] The photograph image group acquired by the image acquisition unit 12 is stored in the image database 14.

[0046] The storage unit 16 is a storage constituted by a hard disk device, a flash memory, or the like, and stores, in addition to an operating system and a three-dimensional display program, a three-dimensional model of a structure, damage information of each component associated with components of the structure, and a table indicating a relationship between each component of the structure and damage that can occur in each component, and the like.

[0047] Here, the three-dimensional model includes a three-dimensional point cloud model that extracts feature points between photograph images of the photograph image group stored in the image database 14 that overlap each other, estimates a position and a posture of the imaging device mounted on the drone based on the extracted feature points, and further estimates three-dimensional positions of the feature points simultaneously based on the estimation result of the position and the posture of the imaging device.

[0048] There is a Structure from Motion (SfM) method that tracks movement of a plurality of feature points from photograph images in which the imaging device is moved from a photograph position by the drone, and simultaneously estimates a three-dimensional structure (Structure) of the structure and a posture of the imaging device (Motion). In recent years, an optimization calculation method called bundle adjustment is being developed to enable high-precision output.

[0049] Further, parameters (focal length, image size of an image sensor, pixel pitch, and the like) of the imaging device required when the SfM method is applied can use parameters stored in the storage unit 16. In addition, in the SfM method, since an absolute scale cannot be solved, for example, an absolute scale (three-dimensional position) can be solved by indicating a known size (distance between two points, or the like) of the structure.

[0050] Here, it can be considered that the three-dimensional model is a three-dimensional point cloud model represented by a three-dimensional point cloud of a plurality of points on a surface of the structure, a three-dimensional polygonal model represented by a collection of polygons (for example, triangular patches) of polygons created based on the three-dimensional point cloud model, or a three-dimensional model on which an image (texture) obtained by photographing the structure is texture-mapped on the polygons of the polygons. The three-dimensional model of the structure of the present example is set to a three-dimensional model on which an image is texture-mapped on the polygons of the polygons of the three-dimensional polygonal model.

[0051] The damage information includes a damage map. In addition, as the damage information, a kind of damage shown in the damage map or a degree of damage (mild, severe, etc.) can be included.

[0052] The damage map can be generated by performing image processing on a photographic image obtained by photographing the structure, and manually correcting as necessary, in the image processing, for example, in the case of a concrete member, manually tracing a crack, a peeling, or the like of the concrete visually recognized on the photographic image, or automatically detecting damage from the photographic image.

[0053] The image processing of automatically detecting damage from the photographic image can be performed by artificial intelligence (AI).

[0054] Figure 2 is a chart showing an example of a table showing a relationship between each member of the structure and a kind of damage that can occur in each member.

[0055] In Figure 2 the table shown in FIG. 10, 26 kinds of damage corresponding to the inspection items of the structure are stored in association with the members of the structure (steel members, concrete members, etc.).

[0056] Hereinafter, as the structure, a bridge will be described as an example.

[0057] Figure 3 is a chart showing an example of a screen showing a display section that displays the entire three-dimensional model of the bridge, Figure 4 is a chart showing an example of a screen showing a display section that displays a part of the three-dimensional model of the bridge.

[0058] As shown in Figure 3 and Figure 4 , the bridge 1 displayed on the screen 30A of the display section 30 is composed of various members including a main girder 2, a cross girder 3, a cross brace 4, and a flat longitudinal tie 5, the main girder 2 is erected between piers 7, the cross girder 3 is provided in a direction orthogonal to the main girder 2 and links between the main girders, and the cross brace 4 and the flat longitudinal tie 5 link the main girders 2 to each other. A deck 6 for running of vehicles and the like is cast on the upper portion of the main girder and the like. The deck 6 is generally made of reinforced concrete.

[0059] The deck 6 is generally taken as a basic unit by a cofferdam in a rectangular shape divided by the main girder 2 and the cross girder 3, and when the damage (cracks, concrete peeling, etc.) of the deck is inspected, the cofferdam unit is taken.

[0060] Each cofferdam of the bottom plate is one of the components (inspection units) that constitute the structure (bridge). In addition, the inspection units of the bridge, in addition to the bottom plate (cofferdam), include a classification of the parts / components that constitute the structure (main girder 2, cross beam 3, cross strut 4, flat longitudinal link 5, bridge pier 7 (column portion / wall portion, beam portion, corner portion / joint portion), and the like.

[0061] The operation section 18 includes a keyboard, a mouse, and the like that are connected to the computer by wire or wirelessly. In this example, the operation section 18 functions as a user interface that indicates a viewing operation that instructs parallel movement, rotation, enlargement, reduction, and the like of the three-dimensional model displayed on the screen included in the display section 30, instructs display or non-display of the components of the structure, or instructs selection of any one of the plurality of damage types (in this example, 26 damage types shown in Table 1) of the structure. Figure 2

[0062] The CPU 20 reads out various programs stored in the storage section 16 or the ROM 24 and the like, performs unified control of each section, and executes various processes for causing the display section 30 to display the three-dimensional model of the structure.

[0063] The RAM 22 is used as a work area of the CPU 20 and is used as a storage section that temporarily stores the read-out programs or various data.

[0064] The display control section 26 is a section that creates display data for display by the display section 30 and outputs to the display section 30, and in this example, causes the display section 30 to display the three-dimensional model in accordance with the instruction from the CPU 20, and performs parallel movement, rotation, and enlargement / reduction of the three-dimensional model based on the viewing operation of the three-dimensional model performed by the operation section 18.

[0065] The display section 30 uses various displays such as a liquid crystal monitor that can be connected to the computer, and displays the three-dimensional model by the display data input from the display control section 26, and is used as a part of the user interface together with the operation section 18.

[0066] The processor of the CPU 20 of the three-dimensional display device 10 including the above-described structure reads out the programs stored in the storage section 16 or the ROM 24, and reads out the three-dimensional model and the damage information from the storage section 16, and causes the display section 30 to display the three-dimensional model of the structure in accordance with the operation performed by the operation section 18 by executing the programs.

[0067] [Three-dimensional display method]

[0068] Figure 5 and Figure 6 is a flowchart that shows an embodiment of the three-dimensional display method according to the present application.

[0069] In​Figure 5 and Figure 6 In China, Figure 1 The operation of each part of the three-dimensional display device 10 shown will be explained, and the three-dimensional display method involved in the present invention will also be explained.

[0070] The storage unit 16 of the 3D display device 10 stores a 3D model of a structure (a bridge in this example) and damage information associated with the components of the structure. When implementing the 3D display method of this example, it is preferable to prepare the storage unit 16 in advance, which stores the above information.

[0071] The CPU 20 of the three-dimensional display device 10, the three-dimensional display program stored in the storage unit 16, the RAM 22 and ROM 24, the display control unit 26, etc. constitute a processor, and the processor performs various processes as shown below.

[0072] The processor reads the 3D model and damage information from the storage unit 16 and performs damage information mapping processing, mapping the damage information to the components corresponding to the damage information in the 3D model (step S10). This creates a 3D model mapped with a damage map as damage information. If the damage information includes the degree of damage, for example, the degree of damage can be color-coded to create the damage map.

[0073] In this example, the 3D model is a 3D model in which the texture of an image obtained from photographing the structure is mapped onto each polygon of a 3D polygonal model. However, the damage map created based on the photographed image can be mapped onto the 3D model in the same way as the image mapping.

[0074] Next, the processor causes the display unit 30 to display a three-dimensional model mapped with damage information (step S11). Here, it is preferable that, when initially displaying the three-dimensional model on the display unit 30, the processor... Figure 3 As shown, the processor causes the display unit 30's screen 30A to display a three-dimensional model representing a panoramic view of the bridge 1, so that the overall structure of the bridge 1 can be understood. Furthermore, since... Figure 3 The 3D model representing the panoramic view of Bridge 1 shown is significantly reduced in size, making it almost impossible for users to visually identify the damage information mapped onto the 3D model.

[0075] Next, the processor determines whether a user has viewed the 3D model displayed on screen 30A (step S12). The viewing operation involves zooming in, zooming out, horizontally moving, or rotating the 3D model displayed in 3D (three dimensions) on screen 30A, and is performed using the operation unit 18. In this case, a 3D mouse is preferred as the operation unit for receiving the user's viewing operation, but a regular mouse or the like can also be used.

[0076] In step S12, when it is determined that a viewing operation has been performed (in the case of "Yes"), the processor determines whether the viewing operation is an operation to zoom in or out of the 3D model, an operation to move the 3D model in parallel, or an operation to rotate or move it (steps S14 and S16).

[0077] When the operation unit 18 performs a viewing operation to zoom in or out of the 3D model, the processor creates display data for zooming in or out of the 3D model according to the zoom-in or zoom-out instruction based on the viewing operation, and outputs it to the display unit 30 (step S18). Additionally, when the operation unit 18 performs a viewing operation to move the 3D model in parallel, the processor creates display data for moving the 3D model in parallel according to the parallel movement instruction based on the viewing operation, and outputs it to the display unit 30 (step S20). When the operation unit 18 performs a viewing operation to rotate the 3D model, the processor creates display data for rotating the 3D model according to the rotation movement instruction based on the viewing operation, and outputs it to the display unit 30 (step S22).

[0078] In step S12, if it is determined that no viewing operation was performed (in the case of "No"), or when the processing of steps S18, S20, and S22 is completed, proceed to step S23.

[0079] In step S23, the processor determines whether there is an indication input from the operation unit 18 indicating the end of the display of the 3D model. If there is no indication input (in the case of "No"), the process proceeds to step S24. If there is an indication input (in the case of "Yes"), the display of the 3D model ends.

[0080] Next, the processor determines whether it has received a display instruction for a component of the structure (a component of the specific inspection object) from the operation unit 18, which functions as a user interface (and performs acceptance processing) (step S24).

[0081] The acceptance process for the component displaying the structure using the operation unit 18 can be performed as follows.

[0082] For example, the processor causes the display unit 30 to display a sub-screen or another screen. Figure 2 The diagram shows 26 types of damage to the structure. Then, when the user selects the desired damage using the operation unit 18, the processor... Figure 2 The table shown identifies the component associated with the selected damage and accepts that component as the component for 3D display.

[0083] For example, when selecting at least one of "cracks," "stripping / exposed rebar," and "water leakage / free lime" as damage, the concrete component is identified. In the case of bridge 1 in this example, since the concrete component is the base slab 6 (Figure 4 ), the bottom plate 6 is determined.

[0084] Further, when at least one of "corrosion", "crack", and "break" is selected as the damage, a steel member of the structure is determined. In the case of the bridge 1 of the present example, since the steel members are the girder 2, the cross beam 3, the cross brace 4, and the flat longitudinal tie 5 ( Figure 4 ), these members are determined.

[0085] Further, in the case where "peeling / steel bar exposure" is selected, a bottom plate having "peeling / steel bar exposure" as the damage information in the bottom plate is preferably determined. In this case, when the information of the damage type is included as the damage information in addition to the damage map, the information of the damage type can be utilized.

[0086] Further, not limited to the example described above, one or more members can be directly selected from the members constituting the structure (the bridge 1) using the operation section 18.

[0087] For example, in Figure 4 , 32 is a cursor indicating the input position on the screen 30A of the display section 30, and the cursor 32 can be moved on the screen 30A by the operation of the operation section 18 (a pointing device such as a mouse).

[0088] When the user wants to confirm the damage of the member of the bridge which is the desired inspection object, the three-dimensional model is moved stereoscopically while the entire bridge is spatially grasped using the three-dimensional model, and the member of the desired inspection object is explored on the screen 30A of the display section 30. Then, the cursor 32 is moved within the display area of the desired member on the screen 30A of the display section 30, and a mouse-based single-click operation or an execution key-based input operation is performed. Thereby, the operation section 18 can specify the position on the three-dimensional model displayed on the screen 30A of the display section 30, and can accept the member corresponding to the specified position.

[0089] In Figure 4 , the position of the cursor 32 is located within a specific coffer included in the bottom plate 6 of the bridge 1. Here, when a mouse-based single-click operation or the like is performed, the processor can accept the display instruction of the bottom plate 6 of the bridge 1 or the specific coffer included in the bottom plate 6. In the case where the selection of the desired member is thus performed, the mark 34 indicating that the selection is performed is preferably displayed so as to be superimposed on the position indicated by the cursor 32 on the three-dimensional model.

[0090] In step S24, in the case where the display instruction of the member of the inspection object of the structure is accepted (in the case of "Yes"), the processing proceeds to step S26 of Figure 6 , and in the case where the display instruction of the member of the inspection object is not accepted (in the case of "No"), the processing returns to step S12.

[0091] In Figure 6 In step S26 shown in the drawing, the processor creates only a three-dimensional model of the component to which the display instruction is accepted (the component to which the damage information is mapped) (step S26).

[0092] In this case, the processor needs to identify the three-dimensional model corresponding to the component to which the display instruction is accepted and the three-dimensional model corresponding to the component other than this in the three-dimensional model. The processor preferably performs component identification processing of identifying each component constituting the structure based on the three-dimensional model. The component identification processing can be performed by AI.

[0093] In addition, the three-dimensional model can also be configured in advance to associate the portion corresponding to each component of the three-dimensional model and the component number indicating which component of the structure.

[0094] The processor performs display processing that causes the display section 30 to display the three-dimensional model of only the component to which the display instruction is accepted (the component to which the damage information is mapped) created in step S26 (step S28).

[0095] The display processing in step S28, when displaying the three-dimensional model of the component to which the display instruction is accepted (the component to which the damage information is mapped), sets the line-of-sight direction in the normal direction of the surface of the component and causes the display section 30 to display the three-dimensional model on the screen. Thereby, the user can visually recognize the face of the component of the inspection object of the structure directly, and can more accurately grasp the damage information mapped onto the surface of the component.

[0096] Figure 7 is a drawing showing an example of the screen of the display section displaying the three-dimensional model of the floor on which the damage map is displayed.

[0097] As Figure 7 shown, only the three-dimensional model of the floor on which the damage map is displayed is displayed on the screen 30A of the display section 30.

[0098] In Figure 7 , a plurality of cofferdams (including a portion of the cofferdam) constituting the floor are displayed, and as the damage map, six cracks C1 to C6, and concrete peeling / steel reinforcement exposure H1 are illustrated. In addition, the crack C6 is a crack of a cofferdam adjacent above with respect to the cofferdam having the cracks C1 to C5. Figure 7

[0099] The three-dimensional model of each cofferdam of the floor maps the captured image on which the damage map is mapped. In addition, the processor preferably performs display / non-display of the damage map by an instruction from the user. This is because when the user confirms the damage on the image, sometimes the user wants to set the damage map to non-display.

[0100] ​Next, the processor determines whether or not the viewing operation has been performed only on the three-dimensional model of the component to which the damage information is mapped, which is displayed by step S28 (step S30).

[0101] The viewing operation here is an operation for enlarging display, reducing display, or parallel moving of the three-dimensional model of the component to which the damage information is mapped. In this example, since the line-of-sight direction is set in the normal direction of the surface of the component, the viewing operation of rotating the three-dimensional model is not included.

[0102] In step S30, when it is determined that the viewing operation has been performed (in the case of "Yes"), the processor determines whether the viewing operation is an operation of enlarging or reducing the three-dimensional model, or an operation of parallel moving the three-dimensional model (step S32).

[0103] When the viewing operation of enlarging or reducing the three-dimensional model is performed by the operation section 18, the processor creates display data for enlarging or reducing the three-dimensional model in accordance with the instruction of enlargement or reduction based on the viewing operation, and outputs to the display section 30 (step S34). Also, when the viewing operation of parallel moving the three-dimensional model is performed by the operation section 18, the processor creates display data for parallel moving the three-dimensional model in accordance with the instruction of parallel moving based on the viewing operation, and outputs to the display section 30 (step S36).

[0104] In step S30, when it is determined that the viewing operation has not been performed (in the case of "No"), or when the processing of steps S34, S36 ends, the routine proceeds to step S38.

[0105] In step S38, the processor determines whether or not there is an instruction input from the operation section 18 for ending the display of the three-dimensional model, and in the case where there is no instruction input for ending (in the case of "No"), the routine proceeds to step S40, and in the case where there is an instruction input for ending (in the case of "Yes"), the display of the three-dimensional model is ended.

[0106] In step S40, the processor determines, based on the instruction input from the user, whether or not to display the three-dimensional model of a component different from the component of the three-dimensional model currently displayed.

[0107] The processor, in the case where there is no instruction input for displaying the three-dimensional model of another component (in the case of "No"), proceeds to step S30, and in the case where there is an instruction input for displaying the three-dimensional model of another component (in the case of "Yes"), proceeds to step S11. Figure 5

[0108] [Variation of Embodiment]

[0109] In the above embodiment, the three-dimensional model of the component to which the damage information is mapped is displayed in the case where the component to which the damage information is mapped is selected by the user. However, the three-dimensional model of the component to which the damage information is mapped can be displayed in the case where the component to which the damage information is mapped is selected by the user, or in the case where the component to which the damage information is mapped is selected by the user and the component is displayed in the display section 30. Figure 5 In the above embodiment, the three-dimensional model of the component to which the damage information is mapped is displayed in the case where the component to which the damage information is mapped is selected by the user. However, the three-dimensional model of the component to which the damage information is mapped can be displayed in the case where the component to which the damage information is mapped is selected by the user, or in the case where the component to which the damage information is mapped is selected by the user and the component is displayed in the display section 30. Figure 6 ​In the illustrated embodiment, in order to accept display instruction of a component of the inspection target of the structure, the display section displays a three-dimensional model of only the component to which display instruction is accepted and which has damage information mapped thereto. However, it is also possible to accept non-display instruction of a desired component of the structure, and cause the display section to display a three-dimensional model of a component of the structure other than the component to which non-display instruction is accepted and which has damage information mapped thereto.

[0110] For example, in a case where it is intended to confirm damage of the floor 6 of the bridge 1, a portion of the floor 6 sometimes becomes a blind spot of the cross beam 3, the cross brace 4, and the flat longitudinal link 5 present in front of the floor 6 (the viewpoint side), and it is difficult to confirm. In this case, the three-dimensional model of the component such as the cross beam 3, the cross brace 4, and the flat longitudinal link 5 is set to be non-displayed.

[0111] That is, instead of Figure 5 In step S24 illustrated above, the processor determines whether non-display instruction of a desired component of the structure (a component that obstructs visual recognition of a specific inspection target) is accepted from the operation section 18 that functions as a user interface (acceptance processing is performed).

[0112] Further, when non-display instruction of a desired component of the structure is accepted from the operation section 18, instead of Figure 6 In step S26 illustrated above, the processor creates a three-dimensional model of a component of the structure other than the component to which non-display instruction is accepted and which has damage information mapped thereto.

[0113] Further, the processing in steps other than step S24 and step S26 described above is the same as Figure 5 and Figure 6 The same as in the embodiment illustrated above, and detailed description thereof is omitted.

[0114] In addition, when a three-dimensional model of a component of the structure other than the component to which non-display instruction is accepted is initially displayed in the display section, it is preferable to display the three-dimensional model by removing the component to which non-display instruction is accepted, setting the line-of-sight direction in the normal direction of the surface of the component from which the entire component can be seen, and displaying the three-dimensional model.

[0115] [Others]

[0116] In the present embodiment, a three-dimensional model is constituted by mapping the image obtained by photographing the structure on the facets of the polygons of the three-dimensional polygon model of the structure. However, it is not limited thereto, and the three-dimensional model can also be a three-dimensional point cloud model represented by a three-dimensional point cloud of many points on the surface of the structure, or a three-dimensional model that represents the surface of the structure by a collection of facets of polygons created based on the three-dimensional point cloud model.

[0117] Figure 8This is an example of a display screen showing a three-dimensional model of a base plate with a damaged map, that is, a three-dimensional model of the surface of a structure represented by a collection of polygons.

[0118] That is, in Figure 7 The texture mapping on the shown 3D model contains the captured image, but... Figure 8 The three-dimensional model displayed on the screen 30A of the display unit 30 shown does not have a captured image mapped onto its texture; it only has a damage map mapped onto it.

[0119] In addition, Figure 5 and Figure 6 In the illustrated embodiment, when displaying a 3D model of a component with damage information that only receives a display instruction, the viewing direction is set in the normal direction of the component's surface, and the 3D model is displayed. Then, a viewing operation that only zooms in / out or moves the displayed 3D model is allowed. However, a rotation viewing operation can also be allowed. Alternatively, when initially displaying a 3D model of a component with damage information that only receives a display instruction, the viewing direction can be set in the normal direction of the component's surface, and the 3D model is not displayed.

[0120] The three-dimensional model of the structure and the damage information associated with the components of the structure stored in the storage unit 16 are not limited to being generated by the three-dimensional display device 10, but can also be generated by an external device. In this case, the three-dimensional model generated by the external device and the damage information associated with the components of the structure are stored in the storage unit 16.

[0121] Furthermore, 3D models are not limited to those generated using photographic images of structures and the SfM method; they can also be generated using various other methods.

[0122] For example, a 3D model can be generated by acquiring photographic images of a structure using a time-of-flight camera and obtaining the 3D coordinates of the structure corresponding to each pixel in the photographic image. Alternatively, a laser scanner, which functions as a camera, can generate a 3D model of the structure based on the 3D information acquired by the laser scanner.

[0123] The hardware of the three-dimensional display device according to the present application can be constituted by various processors. The various processors include a general-purpose processor, i.e., a CPU (Central Processing Unit), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD) in which the circuit structure can be changed after manufacturing, i.e., a Programmable Logic Device (PLD), and an ASIC (Application Specific Integrated Circuit) having a circuit structure specially designed for executing a specific process, i.e., an Application Specific Integrated Circuit (ASIC), and the like. One processing section constituting the three-dimensional display device can be constituted by one of the above-described various processors, or can be constituted by two or more processors of the same kind or different kinds. For example, one processing section can be constituted by a plurality of FPGAs or a combination of a CPU and an FPGA. In addition, a plurality of processing sections can be constituted by one processor. As an example in which a plurality of processing sections are constituted by one processor, first, there is a mode in which one processor is constituted by using one or more CPUs and software, and the processor functions as a plurality of processing sections, such as a computer typified by a client or a server. Second, there is a mode in which one processor is used to realize the functions of the entire system including a plurality of processing sections in one IC (Integrated Circuit) chip, such as a System On Chip (SoC). In this way, one or more of the above-described various processors are used as the hardware structure of the various processing sections. Furthermore, more specifically, the hardware structure of these various processors is a circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

[0124] In addition, the present application includes a three-dimensional display program and a storage medium in which the three-dimensional display program is recorded, the three-dimensional display program being installed on a computer that can access a three-dimensional model of a structure and damage information associated with a component of the structure, thereby causing the computer to function as the three-dimensional display device according to the present application.

[0125] Furthermore, needless to say, the present application is not limited to the above-described embodiments, and various modifications can be made within the scope of the present application.

[0126] Explanation of Symbols

[0127] 1 Bridge

[0128] 2 Girder

[0129] 3 Beam

[0130] 4 Cross support

[0131] 5 horizontal and vertical links

[0132] 6 base plate

[0133] 7 pier

[0134] 10 three-dimensional display device

[0135] 12 image acquisition section

[0136] 14 image database

[0137] 16 storage section

[0138] 18 operation section

[0139] 20 CPU

[0140] 22 RAM

[0141] 24 ROM

[0142] 26 display control section

[0143] 30 display section

[0144] 30A screen

[0145] 32 cursor

[0146] 34 mark

[0147] C1 to C6 cracks

[0148] H1 peeling / steel bar exposure

[0149] S10 to S40 steps

Claims

1. A three-dimensional display device comprising a processor and a memory, wherein the memory stores a three-dimensional model of a structure and damage information associated with components of the structure, wherein, The processor, The damage information stored in the memory is mapped to components corresponding to the damage information of the three-dimensional model stored in the memory. The user interface receives instructions on whether to display or not display the components of the structure. When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which has only received the instruction to display. Alternatively, when an instruction is received not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that have received the non-display instruction, and which is mapped with the damage information. The memory stores a table representing the relationship between the components of the structure and the damage that can occur to each component. When the processor accepts a selection of any one of the multiple damage types of the structure from the user interface, it determines the component corresponding to the selected damage type from the table and accepts the display of the determined component.

2. A three-dimensional display device comprising a processor and a memory, wherein the memory stores a three-dimensional model of a structure and damage information associated with components of the structure, wherein, The processor, The damage information stored in the memory is mapped to components corresponding to the damage information of the three-dimensional model stored in the memory. The user interface receives instructions on whether to display or not display the components of the structure. When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which has only received the instruction to display. Alternatively, when an instruction is received not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that have received the non-display instruction, and which is mapped with the damage information. When the processor receives an instruction from the user interface to display a component of the structure to be inspected, it sets the viewing direction in the normal direction of the surface of the component and displays the three-dimensional model on the display.

3. A three-dimensional display device comprising a processor and a memory, wherein the memory stores a three-dimensional model of a structure and damage information associated with components of the structure, wherein... The processor, The damage information stored in the memory is mapped to components corresponding to the damage information of the three-dimensional model stored in the memory. The user interface receives instructions on whether to display or not display the components of the structure. When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which has only received the instruction to display. Alternatively, when an instruction is received not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that have received the non-display instruction, and which is mapped with the damage information. The processor sets the component in front of the component that is mapped with the damage information to be not displayed.

4. The three-dimensional display device according to any one of claims 1 to 3, wherein, The three-dimensional model is a three-dimensional point cloud model, a three-dimensional polygon model created based on the three-dimensional point cloud model, or a three-dimensional model formed by mapping the texture of the image of the structure obtained by photographing it onto each polygon of the three-dimensional polygon model.

5. The three-dimensional display device according to any one of claims 1 to 3, wherein, The processor identifies the components that make up the structure based on the three-dimensional model.

6. The three-dimensional display device according to any one of claims 1 to 3, wherein, The processor causes the display to show the three-dimensional model of the component mapped with the damage information. The processor receives instructions to display components of the structure to be inspected, indicated by a pointing device, on the display showing the three-dimensional model, or instructions not to display desired components of the structure.

7. The three-dimensional display device according to any one of claims 1 to 3, wherein, The damage information is a damage map representing the location or area of ​​the damage.

8. The three-dimensional display device according to any one of claims 1 to 3, wherein, Damage to the concrete components of the structure includes at least one of cracks, peeling / exposed rebar, and water leakage / free lime, and damage to the steel components of the structure includes at least one of corrosion, cracking, and fracture.

9. A three-dimensional display method, wherein a processor enables a display to show a three-dimensional model of a structure, wherein, The processor's processes include: The step of retrieving the three-dimensional model and the damage information from a memory that stores the three-dimensional model of the structure and damage information associated with the components of the structure; The step of mapping the acquired damage information to the component corresponding to the damage information of the three-dimensional model; The steps of receiving instructions from the user interface regarding the display or non-display of components of the structure; and The steps are as follows: When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which only received the instruction to display; or when the user interface receives an instruction not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that received the instruction not to display, and which also shows the 3D model of the components mapped with the damage information. The memory stores a table representing the relationship between the components of the structure and the damage that can occur to each component. When the processor accepts a selection of any one of the multiple damage types of the structure from the user interface, it determines the component corresponding to the selected damage type from the table and accepts the display of the determined component.

10. A three-dimensional display method, wherein a processor causes a display to show a three-dimensional model of a structure, wherein, The processor's processes include: The step of retrieving the three-dimensional model and the damage information from a memory that stores the three-dimensional model of the structure and damage information associated with the components of the structure; The step of mapping the acquired damage information to the component corresponding to the damage information of the three-dimensional model; The steps of receiving instructions from the user interface regarding the display or non-display of components of the structure; and The steps are as follows: When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which only received the instruction to display; or when the user interface receives an instruction not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that received the instruction not to display, and which also shows the 3D model of the components mapped with the damage information. When the processor receives an instruction from the user interface to display a component of the structure to be inspected, it sets the viewing direction in the normal direction of the surface of the component and displays the three-dimensional model on the display.

11. A three-dimensional display method, wherein a processor enables a display to show a three-dimensional model of a structure, wherein, The processor's processes include: The step of retrieving the three-dimensional model and the damage information from a memory that stores the three-dimensional model of the structure and damage information associated with the components of the structure; The step of mapping the acquired damage information to the component corresponding to the damage information of the three-dimensional model; The steps of receiving instructions from the user interface regarding the display or non-display of components of the structure; and The steps are as follows: When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which only received the instruction to display; or when the user interface receives an instruction not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that received the instruction not to display, and which also shows the 3D model of the components mapped with the damage information. The processor sets the component in front of the component that is mapped with the damage information to be not displayed.

12. A computer-readable recording medium storing a three-dimensional display program, said three-dimensional display program causing a computer to execute a three-dimensional display method for displaying a three-dimensional model of a structure on a display screen, wherein, The three-dimensional display method includes: The step of retrieving the three-dimensional model and the damage information from a memory that stores the three-dimensional model of the structure and damage information associated with the components of the structure; The step of mapping the acquired damage information to the component corresponding to the damage information of the three-dimensional model; The steps of receiving instructions from the user interface regarding the display or non-display of components of the structure; and The steps are as follows: When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which only received the instruction to display; or when the user interface receives an instruction not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that received the instruction not to display, and which also shows the 3D model of the components mapped with the damage information. The memory stores a table representing the relationship between the components of the structure and the damage that can occur to each component. When the user interface accepts the selection of any one of the multiple damage types of the structure, the component corresponding to the selected damage type is determined from the table, and the determined component is displayed.

13. A computer-readable recording medium storing a three-dimensional display program, said three-dimensional display program causing a computer to execute a three-dimensional display method for displaying a three-dimensional model of a structure on a display screen, wherein... The three-dimensional display method includes: The step of retrieving the three-dimensional model and the damage information from a memory that stores the three-dimensional model of the structure and damage information associated with the components of the structure; The step of mapping the acquired damage information to the component corresponding to the damage information of the three-dimensional model; The steps of receiving instructions from the user interface regarding the display or non-display of components of the structure; and The steps are as follows: When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which only received the instruction to display; or when the user interface receives an instruction not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that received the instruction not to display, and which also shows the 3D model of the components mapped with the damage information. When an instruction to display a component of the structure to be inspected is received from the user interface, the viewing direction is set in the normal direction of the surface of the component and the display shows the three-dimensional model.

14. A computer-readable recording medium storing a three-dimensional display program, said three-dimensional display program causing a computer to execute a three-dimensional display method for displaying a three-dimensional model of a structure on a display screen, wherein... The three-dimensional display method includes: The step of retrieving the three-dimensional model and the damage information from a memory that stores the three-dimensional model of the structure and damage information associated with the components of the structure; The step of mapping the acquired damage information to the component corresponding to the damage information of the three-dimensional model; The steps of receiving instructions from the user interface regarding the display or non-display of components of the structure; and The steps are as follows: When the user interface receives an instruction to display a component of the structure to be inspected, the display shows the 3D model of the component mapped with the damage information, which only received the instruction to display; or when the user interface receives an instruction not to display a desired component of the structure, the display shows the 3D model of the components of the structure other than those that received the instruction not to display, and which also shows the 3D model of the components mapped with the damage information. Set the component on the front side of the component that maps the damage information to not be displayed.

Images