Image processing apparatus and image processing method

Abstract

To provide an image processing apparatus and an image processing method.SOLUTION: An image processing apparatus includes a processor, and a memory storing a plurality of images obtained by capturing images of a building and damage information of the building extracted based on the images, in association with each other. The processor is configured to perform: selection processing to refer to the memory and select a designated number of representative images corresponding to a designated member, based on member information which is information on a member constituting the building, from the plurality of images; and output processing to output the member information regarding the designated member, the representative images, and the damage information so that the association between them can be recognized.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for handling images of buildings and damage information. [Background technology]

[0002] Regarding technology for handling three-dimensional models and damage information of buildings, for example, Patent Document 1 describes a device for creating management drawings of structures (buildings, architectural structures) such as bridges. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-192270 Summary of the Invention

[0004] One embodiment of the technique of the present disclosure provides an image processing device and an image processing method that can easily handle images of buildings and damage information. [Means for solving the problem]

[0005] An image processing device according to a first aspect of the present invention is an image processing device comprising a processor, a memory in which a plurality of images of a building and a three-dimensional model of the building in which the components that make up the building are identified are stored, and the memory in which the plurality of images and the components are stored in correspondence with each other, and the processor performs an extraction process to extract damage information for the building based on the plurality of images, a selection process to select an image from the plurality of images that corresponds to a specified component in accordance with specified criteria, and an output process to output the specified component, the selected image, and the damage information in correspondence with each other.

[0006] In the image processing device of the second aspect, in the first aspect, the processor performs a generation process to generate a 3D model of a building based on a plurality of images, an identification process to identify components that make up the building in the generated 3D model, and a storage control process to store the 3D model in memory in association with the plurality of images and the identified components.

[0007] The image processing device according to the third aspect is the second aspect, in which the processor performs the identification process without relying on an operation by the user to identify the member.

[0008] An image processing device according to a fourth aspect is any one of the first to third aspects, wherein the processor performs a receiving process for receiving a specification of a criterion.

[0009] In the image processing device of the fifth aspect, in any one of the first to fourth aspects, the processor extracts at least one of the type, number, size, degree of damage, and change over time as damage information during the extraction process.

[0010] The image processing device according to a sixth aspect is any one of the first to fifth aspects, wherein the processor selects images for each type of damage in the selection process.

[0011] An image processing device according to a seventh aspect is any one of the first to sixth aspects, wherein the processor selects a designated number of images in the selection process.

[0012] In the image processing device according to the eighth aspect, in any one of the first to seventh aspects, the processor performs image placement processing to place a selected image in an area designated as an image area in a document file of a specified format.

[0013] An image processing device according to a ninth aspect is the eighth aspect, wherein the processor performs an information input process of inputting damage information into an area designated as an information area in a document file.

[0014] In the image processing device of the 10th aspect, in any one of the first to ninth aspects, the processor performs a first display process of associating a 3D model with position information indicating the position of a selected image in the 3D model and displaying them on the display device, and a second display process of displaying a selected image on the display device for specified position information from the displayed position information.

[0015] In an image processing device according to an eleventh aspect, in any one of the first to tenth aspects, the processor performs a first display process of associating a three-dimensional model with position information indicating the position of a selected image in the three-dimensional model and displaying them on a display device, and a third display process of displaying the selected image on the display device based on the displayed position information.

[0016] An image processing device according to a twelfth aspect is the image processing device of the tenth or eleventh aspect, wherein the processor, in at least the first display processing, causes the position information to be displayed in a distinctive manner according to the damage information.

[0017] In the image processing device of the 13th aspect, in any one of the 10th to 12th aspects, the processor, in at least the first display processing, synthesizes an image corresponding to a specified component among multiple images, maps the synthesized image to the specified component, and displays it on the display device.

[0018] An image processing device according to a fourteenth aspect is any one of the tenth to thirteenth aspects, wherein the processor highlights the damage information on the three-dimensional model in at least the first display processing.

[0019] In the image processing device of the 15th aspect, in any one of the first to fourteenth aspects, the processor performs an acquisition process to acquire multiple images of a building, the multiple images having different capture dates and times from the multiple images stored in the memory, and an association process to associate the multiple acquired images with components of a three-dimensional model stored in the memory.

[0020] An image processing device according to a 16th aspect is the 15th aspect, wherein the processor performs association processing based on the correlation between the plurality of acquired images and the plurality of images stored in the memory.

[0021] An image processing method according to a seventeenth aspect of the present invention is an image processing method by an image processing device including a processor and a memory storing a plurality of images of a building and a three-dimensional model of the building, the memory storing the plurality of images and components that constitute the building in the three-dimensional model in association with each other, wherein the processing performed by the processor includes an extraction step of extracting damage information of the building based on the plurality of images, a selection step of selecting, from the plurality of images, an image that corresponds to a component specified in the three-dimensional model in accordance with specified criteria, and an output step of outputting the specified component, the selected image, and the damage information in association with each other. The image processing method according to the seventeenth aspect may further have the same configuration as any of the second to sixteenth aspects.

[0022] An image processing program according to an 18th aspect of the present invention causes a computer to execute the image processing method according to the 17th aspect. A non-transitory recording medium having recorded thereon computer-readable code for the image processing program according to the 18th aspect can also be cited as an aspect of the present invention. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an image processing system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the functional configuration of the processing unit. [Figure 3] FIG. 3 is a diagram showing information stored in the storage device. [Figure 4] FIG. 4 is a flowchart showing the steps of the image processing method. [Figure 5] FIG. 5 is a diagram showing how an image group is acquired. [Figure 6] FIG. 6 is a diagram illustrating an example of three-dimensional point cloud data. [Figure 7]FIG. 7 is a diagram showing an example of a three-dimensional model. [Figure 8] FIG. 8 is a diagram showing an example of damage information. [Figure 9] FIG. 9 is a diagram showing how the selection criteria for the representative image are set. [Figure 10] FIG. 10 is a diagram showing how to set the output mode of the processing result. [Figure 11] FIG. 11 is a diagram showing a state in which a representative image and damage information are input into a two-dimensional inspection report. [Figure 12] FIG. 12 is a diagram showing how position information is displayed on a three-dimensional model. [Figure 13] FIG. 13 is a diagram showing a state in which the representative image at the specified position is displayed. [Figure 14] FIG. 14 is a diagram showing a state in which the representative image is displayed from the beginning. [Figure 15] FIG. 15 is a diagram showing a state in which a portion of the two-dimensional inspection record that includes a representative image at a specified position is displayed. [Figure 16] FIG. 16 is a diagram showing a state in which a composite image is mapped onto a three-dimensional model. DETAILED DESCRIPTION OF THE INVENTION

[0024] An embodiment of an image processing device and an image processing method according to the present invention is as follows: In the description, reference will be made to the accompanying drawings as necessary.

[0025] [First embodiment] [Image processing system configuration] FIG. 1 is a block diagram showing the schematic configuration of an image processing system 1 (image processing device). The image processing system 1 includes an image processing device 10 (image processing device) and a display device 20 (display device, monitor), and is a system that extracts damage information from multiple images acquired by photographing a subject in sections, creates a 3D model, and supports the creation of a 2D inspection report. The image processing system 1 can be configured using devices (information terminals) such as personal computers, tablet terminals, and smartphones. The elements of the image processing system 1 may be housed in a single housing or in separate housings. Alternatively, the elements may be located in separate locations and connected via a network.

[0026] [Configuration of image processing device] The image processing device 10 comprises a processing unit 100, a storage device 200, and an operation unit 300, and these units are interconnected to transmit and receive necessary information.

[0027] [Configuration of processing unit] 2 is a diagram showing the configuration of a processing unit (processor) 100. The processing unit 100 includes an input processing unit 102, an acquisition processing unit 103, an extraction processing unit 104, a generation processing unit 105, an identification processing unit 106, an association processing unit 107, a selection processing unit 108, a storage control processing unit 109, a reception processing unit 110, an image placement processing unit 112, an information input unit 114, a display processing unit 116, and a communication control unit 118, and performs the acquisition of captured images, the creation of 3D models, and support for the creation of 2D inspection reports. Details of the processing performed by each of these units will be described later.

[0028] The functions of the processing unit 100 described above can be realized using various processors and recording media. The various processors include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions, a GPU (Graphics Processing Unit), which is a processor specialized for image processing, and a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacturing. Each function may be realized by a single processor, or by multiple processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Furthermore, multiple functions may be realized by a single processor. The hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0029] When the above-mentioned processor or electrical circuit executes software (program), the computer-readable code of the software to be executed (for example, various processors and electrical circuits constituting the processing unit 100, and / or a combination thereof) is stored in a non-transitory recording medium (memory) such as a ROM, and the computer references the software. During execution, information stored in the storage device is used as necessary. During execution, for example, RAM (Random Access Memory) is used as a temporary storage area.

[0030] Note that some or all of the functions of the processing unit 100 may be realized by a server on the network, and the image processing device 10 may perform data input, communication control, display of results, etc. In this case, an Application Service Provider type system is constructed including the server on the network.

[0031] [Storage configuration] The storage device 200 (storage device, memory) is composed of non-transitory recording media such as a CD (Compact Disk), a DVD (Digital Versatile Disk), a hard disk (Hard Disk), and various semiconductor memories, as well as its control unit, and stores the information shown in FIG. 3 in a mutually associated manner. The captured images 202 are multiple images of a building, and the composite image 204 is a collection of images corresponding to specific components, synthesized from the captured images. The 3D model data 206 (3D model) is a 3D model of the building created based on the captured images, and identifies the components that make up the building. The 3D model data 206 is associated with the captured images, representative images, 2D reports, etc., and, as will be described in detail below, the user can display the representative image or 2D inspection report by specifying position information on the 3D model. The damage information 208 (damage information) is information indicating damage to the building, extracted from the captured images. The inspection report data 210 is a template of a 2D inspection report (a document file in a specified format), or data in which a representative image and damage information are arranged and input into a template (described later). The template may be in a format specified by the Ministry of Land, Infrastructure, Transport and Tourism or a local government.

[0032] In addition to this information, the storage device 200 may also store camera parameters (focal length, image size of the image sensor, pixel pitch, etc.) required when applying SfM (Structure from Motion), which will be described later.

[0033] [Configuration of the operation section] The operation unit 300 includes a keyboard 310 and a mouse 320, and a user can use these devices to perform operations necessary for image processing according to the present invention. By using a touch panel type device, the display device 20 may be used as the operation unit.

[0034] [Display device] The display device 20 (display device) is, for example, a device such as a liquid crystal display, and can display information such as acquired photographed images, damage information, a three-dimensional model, a two-dimensional inspection report, and a representative image.

[0035] [Image processing procedure] FIG. 4 is a flowchart showing the procedure of the image processing method according to the present invention.

[0036] [Image Input] The input processing unit 102 (processor) inputs multiple images of buildings as subjects (step S100: input processing, input step). The buildings (architectures, structures) are, for example, bridges, roads, etc., but may also be other buildings. The input processing unit 102 may input images stored in the storage device 200 as the captured images 202, or may input images via a recording medium (not shown) or a network. These images can be captured by a flying object such as a drone or a mobile robot while moving the viewpoint (or by a user). The captured images do not need to be stereo images. For creating a 3D model and compositing images, it is preferable that the images have many common feature points, and therefore it is preferable that adjacent images overlap sufficiently (for example, by 80% or more of their area). FIG. 5 is a diagram showing how such overlapping images are captured by the camera 30.

[0037] [Damage Extraction] The extraction processing unit 104 (processor) extracts damage information of the structure based on the input images (step S110: extraction processing, extraction step). In the extraction processing, the extraction processing unit 104 can extract at least one of the damage type, number, size, degree of damage, and change over time as damage information.

[0038] The extraction processing unit 104 can extract damage information using various methods. For example, it can use the crack detection method described in Japanese Patent No. 4006007 or the rust and peeling detection method described in Japanese Patent Publication No. 2010-538258. The extraction processing unit 104 can also extract damage information using a machine learning method. For example, it can provide images labeled with the type and size of damage as training data to generate a learning device such as a DNN (Deep Neural Network) through machine learning, and use the generated learning device to detect damage. It is possible.

[0039] The extraction processing unit 104 may extract damage information from each captured image and combine the corresponding information into one, or may extract damage information from a single image obtained by combining multiple captured images. Damage can be represented as a vector having a start point and an end point. In this case, a hierarchical structure of vectors may be taken into consideration, as described in WO2017 / 110279.

[0040] 8 is a diagram showing an example of the extracted damage information. The extraction processing unit 104 can store the extracted damage information in the storage device 200 as damage information 208.

[0041] [Creating a 3D model] The generation processing unit 105 (processor) creates a 3D model of the building based on the multiple input images (step S120: generation processing, generation step). There are various types of 3D models, such as a 3D point cloud model, a 3D surface model or a 3D polygon model created based on the 3D point cloud model, or a texture-mapped image. The generation processing unit 105 can create the 3D model using, for example, a Structure from Motion (SfM) technique. SfM is a technique for restoring a 3D shape from multi-viewpoint images. For example, feature points are calculated using an algorithm such as SIFT (Scale-Invariant Feature Transform), and the 3D position of the point cloud is calculated using the principle of triangulation using these feature points as clues. Specifically, a straight line is drawn from the camera to the feature point using the principle of triangulation, and the intersection of two lines passing through the corresponding feature point is the restored 3D point. Then, by performing this process for each detected feature point, the 3D position of the point cloud can be obtained. FIG. 6 is a diagram showing a point cloud 500 (an example of a point cloud).

[0042] Although size is not calculated in SfM, it is possible to associate it with the actual scale by, for example, placing a scaler with known dimensions on the subject and taking a photograph.

[0043] The generation processing unit 105 can apply, for example, a TIN (triangulated irregular network) model to the point cloud data obtained in this manner to approximate the building's surfaces with triangles, and obtain a solid model (3D model) based on the results. In a solid model, the building's 3D shape is configured as a combination of 3D components like solid building blocks or blocks. When obtaining a solid model, the user may specify "which ranges of the point cloud belong to the same surface" via the operation unit 300, and the generation processing unit 105 may use the result. Alternatively, the generation processing unit 105 may automatically generate a solid model without user operation using an algorithm such as RANSAC (Random Sample Consensus). When generating a solid model, the generation processing unit 105 may use information on the 3D position, color (R, G, B), and brightness of the point cloud to calculate changes in this information.

[0044] [Using pre-generated 3D models] Note that if a 3D model has already been generated or acquired by a past inspection or the like, the generation processing unit 105 may read that model. When using such a generated 3D model, the acquisition processing unit 103 can acquire a plurality of images of a building that have been photographed at different dates and times from the plurality of images stored in the storage device 200 (memory) (for example, images that have been photographed at newer dates and times than the stored images and have not been used to generate the 3D model) (acquisition processing, acquisition step). Furthermore, the association processing unit 107 can associate the acquired plurality of images with components of the 3D model stored in the storage device 200 (association processing, association step). The association processing unit 107 can perform association processing (association step) based on, for example, the correlation between the acquired plurality of images and the plurality of images stored in the storage device 200.

[0045] [Identifying components] The identification processing unit 106 (processor) identifies components constituting the building in the 3D model (step S130: identification processing, identification step). That is, the identification processing unit 106 identifies "which components of the building correspond to each region of the 3D model." The identification processing unit 106 may identify components based on a user's operation, or may identify components independently of the user's operation to identify the components. The identification processing unit 106 may use information about the shape and dimensions of the components when identifying the components. For example, information such as "a component that extends two-dimensionally in a horizontal plane and has an area equal to or greater than a threshold is a deck slab" or "a component attached to the deck slab and extending one-dimensionally is a main girder" may be used. The identification processing unit 106 may also identify components using a learning device such as a DNN configured by machine learning to which components constituting the 3D model are assigned as correct answer labels. FIG. 7 is a diagram showing an example of a 3D model in which components have been identified. In the example shown in the figure, a 3D model 510 of a bridge is composed of the following components: a deck slab 512, a wall 514, and a leg 516.

[0046] The storage control processing unit 109 can store data representing the generated three-dimensional model in the storage device 200 as three-dimensional model data 206 (storage control processing, storage control step).

[0047] [Select representative image] The point cloud described above is information indicating the positions of feature points contained in the captured images, and since the components have been identified in step S130, it is possible to identify the captured images corresponding to each component. However, since a large number of images are usually acquired for each component to create a 3D model, it is time-consuming for the user to select images. Therefore, in the image processing system 1, the reception processing unit 110 (processor) receives the specification of image selection criteria (step S140: reception processing, reception step), and the selection processing unit 108 (processor) selects images (representative images) corresponding to the identified components from the captured images in accordance with the criteria (step S140: selection processing, selection step).

[0048] FIG. 9 is a diagram showing how selection criteria for representative images are set, and the reception processing unit 110 displays such a screen on the display device 20. In the example of FIG. 9, the reception processing unit 110 sets priorities for information included in the damage information (number of damages, size, extent (degree) of damage, and change over time) in accordance with a user operation. The reception processing unit 110 also sets the number of representative images to be selected (which may be one or multiple images) in accordance with a user operation. In response to this setting, the selection processing unit 108 selects a specified number of images according to the set priorities. This allows the user to quickly and easily select representative images. Note that the aspect shown in FIG. 9 is an example of setting selection criteria, and representative images may also be selected based on other criteria (for example, the image quality of the captured image, the type of component, the identification number of the component, etc.).

[0049] [Output of processing results] FIG. 10 is a diagram illustrating how the output mode of the processing results is set. In the example shown in FIG. 10, the user can select the display of the 2D inspection report, the display of the 3D model, the timing of displaying the representative image, whether or not to map the composite image, highlight damage, and whether or not to display the 2D inspection report on the 3D model by selecting radio buttons and entering values ​​via the operation unit 300. Then, according to the settings, the image placement processing unit 112, the information input unit 114, and the display processing unit 116 independently or in cooperation with each other output the identified components, the selected images, and the damage information in association with each other (step S150: output processing, output step). The processing from step S110 to S150 can be repeated until the determination of termination is affirmative in step S160. Note that the setting of the output mode and the output in the set mode can be performed at a timing desired by the user. For example, the processing results can be displayed in one mode and then displayed again in another mode. The specific contents of each output mode are described below.

[0050] [Part 1: Output using 2D inspection report] FIG. 11 shows an example in which the image placement processing unit 112 places representative images (representative images 552, 554, 556, and 558) in areas (image areas) designated as areas for placing images in a two-dimensional inspection report (an example of a "document file of a specified format"). In the figure, the representative image 552 shows a float 552A, and the representative image 554 shows peeling 554A. A report in this format may span multiple pages. In the example of FIG. 11, the information input unit 114 also inputs damage information, such as the photo number, component name, element number, type and degree of damage, into areas designated as information areas in the document file (information input process, information input step). The contents of the "Memo" field may be automatically input by the information input unit 114 based on the damage information, or may be input based on user operation. The display processing unit 116 displays such a screen on the display device 20 when the "two-dimensional inspection report display mode" is on in the screen of FIG. 10. Moreover, the display processing unit 116 stores information corresponding to this screen as inspection paper data 210.

[0051] Various document formats and layouts of representative images are possible. For example, documents such as inspection reports may be in a format specified by the Ministry of Land, Infrastructure, Transport and Tourism, local governments, etc., or in other formats. Documents in other specified formats other than inspection reports may also be used. Furthermore, representative images may be sorted and arranged in a document using components as a key, or representative images of each component may be sorted and arranged (by damage type) using damage type as a key.

[0052] Through this processing, the user can quickly and easily select a representative image and create an inspection report in the image processing system 1. In other words, the user can easily handle images of a structure, a 3D model of the structure, and damage information.

[0053] [Part 2: Output using 3D models] FIG. 12 is a diagram showing a state (result of the first display process) in which a 3D model 520 and pins 532, 534 (position information) indicating the position of a selected image (representative image) in the 3D model are associated and displayed on the display device. When the display timing of the representative image is set to "at pin designation" in FIG. 10, a pin is displayed at the position of the representative image as shown. In the example of FIG. 12, the 3D model displays the vicinity of the wall surface of a bridge leg. The display processing unit 116 can enlarge or reduce the 3D model and change the viewpoint and line of sight in response to user operations via the operation unit 300. Furthermore, when displaying the 3D model, the display processing unit 116 may turn on or off the display of a specific component (e.g., a component designated by the user). The display processing unit 116 performs such a display when the "3D model display mode" in FIG. 10 is on. Although FIGS. 12 to 15 show a 3D model without texture, similar processing can be performed on a 3D model with texture (see FIG. 16 and the description related to the same figure).

[0054] The display processing unit 116 may highlight the damage information on the three-dimensional model. For example, the damage, such as cracks, swelling, or peeling, or its outline may be displayed by tracing it with a thick line or a line in a conspicuous color.

[0055] In the example of Fig. 11, the display processing unit 116 displays the pins in a manner that corresponds to the damage information. Specifically, the display processing unit 116 changes the symbol of the pin depending on the type of damage, but the color or number may be changed in addition to the symbol, or the pin may be displayed as an icon. The display manner may also be changed depending on the degree of damage. In the example of Fig. 12, the user can select (click, etc.) a pin by moving the cursor 540 on the screen using the mouse 320, for example.

[0056] 13 is a diagram showing a state (result of the second display process) in which the display processing unit 116 has caused the display device 20 to display representative images of designated pins (position information) displayed on the screen. Part (a) of FIG. 13 shows a state in which a representative image 552 of "float" is displayed as designated by pin 532, and part (b) of the same figure shows a state in which a representative image 554 of "peel" is displayed as designated by pin 534. The display processing unit 116 may display the representative images in a region separate from the three-dimensional model, on a separate screen, or on a separate display device.

[0057] FIG. 14 is a diagram showing a state in which a representative image is displayed on a three-dimensional model from the beginning (results of the first display process and the third display process). The display processing unit 116 performs such a display when the display of the representative image is set to "from the beginning" in FIG. 10. In the example shown in FIG. 14, the pin and the representative image are connected by a leader line, so that the user can easily understand the relationship between the pin and the representative image. The representative image may also be displayed in a balloon-shaped figure connected to the pin.

[0058] 15 is a diagram showing a state in which a portion of the 2D inspection report including a representative image corresponding to a selected pin is displayed (the result of the fourth display process). The display processing unit 116 can perform such a display when "Display 2D inspection report on 3D model" is turned on in FIG. 10. The display processing unit 116 may display such a partial 2D inspection report in a different area, on a different screen, or on a different display device from the 3D model.

[0059] Through this processing, the user can quickly and easily view representative images and create inspection reports in the image processing system 1. In other words, the user can easily handle images of buildings, 3D models of buildings, and damage information.

[0060] [Part 3: Mapping synthetic images onto 3D models] FIG. 16 is a diagram showing a three-dimensional model 522, in which a composite image is mapped onto the three-dimensional model 520 (see FIG. 12), displayed on the display device 20. This display is performed when "Mapping of composite image" is turned on in FIG. 10. In this mode, the image composition unit 117 (processor) composites an image corresponding to a specified component from among multiple captured images, and the display processing unit 116 maps the composite image onto the specified component and displays it on the display device 20 (fifth display process). Image composition and mapping may be performed for some or all of the components of the building. Note that in the example shown in FIG. 16, the display processing unit 116 performs identification display of location information (displaying pins 532, 534, and 536 with symbols corresponding to the type of damage), as in the above-described mode, and may display a representative image or a two-dimensional inspection report depending on the selected location information. Damage information may also be highlighted.

[0061] While FIG. 16 shows an example in which a separately generated composite image is mapped onto a 3D model of only a three-dimensional shape, the generation processing unit 105 (processor) may directly generate a textured 3D model using the above-mentioned SfM or the like. In this case, the generation processing unit 105 may use other algorithms such as MVS (Multi-View Stereo). MVS can generate a more detailed 3D model because it has more information than a single camera. For such textured 3D models, position information, representative images, partial 2D inspection reports, and highlighted damage information may be displayed, similar to the aspects described above with reference to FIGS. 12 to 15.

[0062] By performing such processing, the image processing system 1 allows the user to easily handle images of buildings, three-dimensional models of buildings, and damage information.

[0063] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described aspects, and various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0064] 1. Image processing system 10 Image processing device 20 Display device 30 Camera 100 Processing section 102 Input processing section 103 Acquisition processing unit 104 Extraction processing section 105 Generation processing unit 106 Specialized Processing Department 107 Correspondence processing unit 108 Selection processing section 109 Storage control processing unit 110 Reception department 112 Image placement processing unit 114 Information input section 116 Display processing unit 117 Image synthesis unit 118 Communication Control Unit 200 Storage device 202 images 204 Composite Images 206 3D model data 208 Damage information 210 Inspection report data 300 Operation section 310 keyboard 320 Mouse 500 point cloud 510 3D Models 512 Floor slab 514 Wall 516 Legs 520 3D Models 522 3D Models 532 pins 534 pins 536 pins 540 cursor 552 Representative images 552A Float 554 Representative images 554A Peeling 556 Representative images 558 Representative Image S100~S160 Image processing steps

Claims

1. a processor; a memory in which a plurality of images of a building and damage information of the building extracted based on the plurality of images are stored in association with each other; The processor refers to the memory and a selection process for selecting a designated number of representative images corresponding to designated components from the plurality of images based on component information that is information on components constituting the building; an image processing device that performs an output process of outputting the component information, the representative image, and the damage information for the specified component in a manner that enables the correspondence between the component information, the representative image, and the damage information to be recognized.

2. The image processing device according to claim 1 , wherein the processor selects the representative image by further considering, in addition to the component information, at least one of the priority of information items included in the damage information and the image quality of the image.

3. 3. The image processing apparatus according to claim 1, wherein the processor inputs the damage information and the member information into an area designated as an information area in a document file of a designated format.

4. The image processing device according to claim 3 , wherein the processor arranges the representative image in an area designated as an image area in the document file.

5. The image processing device according to claim 4 , wherein the processor inputs the damage information and the component information as a document file in the format of a building inspection report specified by the Ministry of Land, Infrastructure, Transport and Tourism or a local government.

6. The image processing device according to claim 3 , wherein the processor, in the output process, associates the three-dimensional model of the building with the document file in the specified format and displays them on a display device.

7. The image processing device according to claim 6 , wherein the processor causes the display device to display a portion of the document file that includes the representative image for a specified position in the three-dimensional model of the building.

8. The image processing device according to claim 1 , wherein the component information is information about components that constitute the building, which are identified in a three-dimensional model of the building.

9. The image processing apparatus according to claim 1 , wherein the member information includes at least one of a type or a name of the member and an element number of the member.

10. The image processing apparatus according to claim 1 , wherein the member information includes at least one of information about the shape of the member and information about the dimensions of the member.

11. a processor; a memory in which a plurality of images of a building and damage information of the building extracted based on the plurality of images are stored in association with each other; An image processing method executed by an image processing device comprising: The processor refers to the memory and a selection step of selecting a designated number of representative images corresponding to designated components from the plurality of images based on component information that is information about components constituting the building; an output step of outputting the component information, the representative image, and the damage information for the specified component in a manner that enables the correspondence between the component information, the representative image, and the damage information to be recognized.

Images