How to create a damage diagram

Abstract

To provide a novel image synthesis method which improves damage shape measurement accuracy in overlapping areas.SOLUTION: An image synthesis method is provided, for generating a synthesized image using a plurality of images obtained by split-imaging an object such that some areas overlap. In an overlapping area, one having a smaller synthesis parameter is selected for synthesis.SELECTED DRAWING: Figure 30

Description

[Technical Field]

[0001] The present invention relates to a damage diagram creation method, a damage diagram creation device, a damage diagram creation system, and a recording medium, and more particularly to a technique for detecting damage based on a plurality of images acquired by photographing a subject in sections. [Background technology]

[0002] Systems that analyze the deterioration and damage of large structures such as bridges based on photographed images of the structures have been known for some time. For example, Patent Document 1 describes a method of photographing a structure such as a bridge in multiple divided areas, stitching the images together to generate a single composite image, and then identifying cracks and other damaged areas and creating related data (such as the length and width of the cracks). To create the composite image, the divided images are photographed so that overlapping areas are generated, and then projective transformation is performed on the images based on the characteristics of the overlapping areas to determine the relative positions of the images so that the feature positions match, thereby generating a single composite image. To prevent image degradation in the overlapping areas, the method does not combine two images to generate an image of the overlapping areas, but instead uses (adopts) only one of the images to obtain the overlapping area image. It also describes a method of selecting the image to be adopted based on image brightness, subject clarity, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO / 2019 / 003796 Summary of the Invention [Problem to be solved by the invention]

[0004] However, if the selection of the image in the overlapping area is determined based on the brightness or sharpness of the image, the accuracy of the data on damage (cracks, etc.) may be reduced. [Means for solving the problem]

[0005] Therefore, the present invention, which solves the above-mentioned problems, includes the steps of inputting a plurality of images acquired by photographing a subject in divided portions so that some areas overlap, and calculating synthesis parameters (e.g., projective transformation parameters) for synthesizing the plurality of images based on corresponding points between the images, detecting damage to the subject from the images constituting the plurality of images; and synthesizing the detection results for the plurality of images based on the synthesis parameters; In the combining step, in the overlapping area, one of the overlapping images is selected and combined, and the selected image has a smaller value of the combining parameter than the other image. [Effects of the Invention]

[0006] According to the present invention, the measurement accuracy of the shape of the damage in the overlap region is improved. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of a damage diagram creation system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating a configuration of a processing unit. [Figure 3] FIG. 4 is a diagram illustrating information stored in a storage unit. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a server. [Figure 5] FIG. 2 is a flowchart showing a damage diagram creation method according to the first embodiment. [Figure 6] FIG. 4 is another flowchart showing the damage diagram creation method according to the first embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a procedure for photographing a deck. [Figure 8] FIG. 10 is a diagram illustrating an example of a procedure for photographing a case. [Figure 9] FIG. 2 is a diagram showing the shooting range of each image. [Figure 10]FIG. [Figure 11] FIG. 2 is a diagram showing a hierarchical structure of folders. [Figure 12] FIG. 10 is a diagram showing a state in which photographed images are stored in a folder. [Figure 13] 10 is a flowchart showing a process of determining an image layout. [Figure 14] FIG. 10 is a diagram showing a state of setting a reference image. [Figure 15] FIG. 10 is a diagram illustrating an example of image arrangement for each image group. [Figure 16] FIG. 10 is another diagram showing an example of image arrangement for each image group. [Figure 17] FIG. 10 is a diagram showing a state in which vectorized detection results are superimposed on an image. [Figure 18] FIG. 10 is a diagram showing how the relative arrangement of each image group is changed. [Figure 19] FIG. 10 is a diagram showing how corresponding points are set in an image group. [Figure 20] FIG. 10 is a diagram showing a synthesis of detection results. [Figure 21] FIG. 10 is another diagram showing the synthesis of detection results. [Figure 22] FIG. 10 illustrates the merging of detection results in an area where images overlap. [Figure 23] FIG. 10 is a diagram showing a composite image. [Figure 24] FIG. 10 is another diagram showing a composite image. [Figure 25] FIG. 10 is a diagram showing a state in which detection results and the like are stored in the same folder as images. [Figure 26] FIG. 10 is a block diagram showing the configuration of a damage diagram creation device according to a second embodiment. [Figure 27] 10A and 10B are diagrams illustrating another example of combining detection results in an area where images overlap. [Figure 28] 10A and 10B are diagrams illustrating another example of combining detection results in an area where images overlap. [Figure 29] 10A and 10B are diagrams illustrating an example of calculation of synthesis parameters in an area where images overlap. [Figure 30]FIG. 10 is a flowchart of a damage diagram creation method according to a third embodiment. [Figure 31] FIG. 10 is a conceptual diagram of a damage diagram creation method according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of a damage diagram creation method, a damage diagram creation device, a damage diagram creation system, and a recording medium according to the present invention will be described in detail with reference to the accompanying drawings.

[0009] First Embodiment <Bridge structure> A bridge is a preferred example of a structure to which the damage mapping method and damage mapping device of the present invention can be applied. For details of this structure, see FIG. 1 of Patent Document 1. Note that, although the following embodiment will be described with a bridge as the target (subject), the target structure is not limited to a bridge; it may also be a tunnel, a building, a road, etc.

[0010] <Image acquisition> When photographing images of a bridge to detect damage, an inspector uses a digital camera 100 (see Figure 1) to photograph the bridge from below and acquires multiple images of the inspection area (multiple images each capturing a different part of the bridge). The photographs are taken while moving appropriately in the direction of the bridge extension and in a direction perpendicular to the direction. If the conditions around the bridge make it difficult for the inspector to move, the digital camera 100 may be mounted on a mobile object that can move along the bridge to take photographs. Such a mobile object may be equipped with an elevation mechanism and a rotation mechanism (a mechanism for panning and / or tilting) for the digital camera 100. Examples of mobile objects include, but are not limited to, vehicles, robots, and flying objects (drones, etc.).

[0011] <Configuration of the damage diagram creation system> 1 is a block diagram showing a schematic configuration of a damage diagram creation system 10 (damage diagram creation system). The damage diagram creation system 10 includes a digital camera 100, a client 200, and a server 300, and creates a damage diagram by detecting damage in multiple images acquired by photographing a subject in segments and synthesizing the detection results. In the damage diagram creation system 10, a device (information terminal) such as a personal computer, tablet terminal, or smartphone that inputs images and receives results from the server 300 can be used as the client 200, and a computer connected to the client 200 via a network can be used as the server 300.

[0012] <Digital camera configuration> The digital camera 100 acquires an image using an imaging optical system 110 equipped with a photographing lens and an imaging element (not shown). Examples of the imaging element include a CCD (Charge Coupled Device) type imaging element and a CMOS (Complementary Metal-Oxide Semiconductor) type imaging element. R (red), G (green), or B (blue) color filters are provided on the light-receiving surface of the imaging element, and a color image of the subject can be acquired based on the signals of each color. The digital camera 100 communicates wirelessly with the client 200 via the wireless communication unit 130 and antenna 132, and the captured image is input to the processing unit 210 for processing, which will be described later. The digital camera 100 may be incorporated into a housing separate from the client 200, or may be integrated with the client 200.

[0013] <Overall client configuration> The client 200 comprises a processing unit 210, a storage unit 220, a display unit 230, and an operation unit 240, and these units are interconnected to transmit and receive necessary information. The client 200 also communicates wirelessly with the digital camera 100 via an antenna 212, and acquires images captured by the digital camera 100. The client 200 is also connected to a server 300 via a network NW, and transmits and receives, to and from the server 300, acquired images, processing results for the transmitted images (synthesis detection results, composite images, etc.), processing requests and their responses, etc.

[0014] <Configuration of the processing unit> FIG. 2 is a diagram showing the configuration of the processing unit 210. The processing unit 210 includes an image input unit 210A (image input unit), a file management unit 210B, a display control unit 210C, and a communication control unit 210D, and performs functions such as transmitting captured images acquired by the digital camera 100 to the server 300, receiving processed results, and controlling the display of the processed results on the monitor 232. The image input unit 210A inputs captured images of a bridge (multiple images of the bridge captured in segments) from the digital camera 100 (or a recording medium, a network, etc.). The file management unit 210B creates folders in response to user operations via the keyboard 242 and / or mouse 244. The display control unit 210C controls the display of captured images, received processed results, etc. on the monitor 232. The communication control unit 210D transmits and receives images and information to and from the digital camera 100 via the antenna 212, and also transmits and receives images and information to and from the server 300 via the network NW. The ROM210E (ROM: Read Only Memory, non-transitory recording medium) stores computer-readable code of a program necessary for image acquisition, transmission and reception, and other processes (including a program or part thereof for executing the damage diagram creation method according to the present invention).

[0015] Each function of the processing unit 210 described above can be realized by various processors or electrical circuits referencing software recorded on a recording medium, in the same manner as the server 300 will be described in detail later.

[0016] <Storage unit configuration> The storage unit 220 is composed of a non-transitory recording medium such as a CD (Compact Disk), a DVD (Digital Versatile Disk), a hard disk, or various semiconductor memories, and its control unit, and stores the images and information shown in FIG. 3 in association with each other. The captured image 220A is a plurality of images of the subject, a bridge (deck portion), photographed in sections by the digital camera 100 and input by the image input unit 210A. Note that instead of images input by the digital camera 100 and the image input unit 210A, images acquired via a network or a recording medium may be stored. The detection result 220B includes damage detection results for each image constituting the captured image 220A and a detection result obtained by combining the detection results for each image. The composite image 220C is an image obtained by combining the captured images (including a group of partially combined images). The damage mapping image 220D is an image in which information indicating damage (such as detection results) is mapped. These images and information can be stored in folders (see FIGS. 11 and 12).

[0017] <Display and operation unit configuration> The display unit 230 is equipped with a monitor 232 (display device), and can display input images, images and information stored in the storage unit 220, results of processing by the server 300, etc. The operation unit 240 includes a keyboard 242 and a mouse 244 as input devices and / or pointing devices, and a user can perform operations required to execute the damage diagram creation method according to the present invention, such as creating folders, storing images in folders, and specifying corresponding points, via these devices and the screen of the monitor 232 (described later).

[0018] <Server configuration> FIG. 4 is a diagram showing the configuration of server 300. Server 300 includes image acquisition unit 300A (image acquisition unit), synthesis parameter calculation unit 300B (synthesis parameter calculation unit), image synthesis unit 300C, damage detection unit 300D (damage detection unit), detection result synthesis unit 300E (detection result synthesis unit), and corresponding point designation unit 300F (corresponding point designation unit). Server 300 further includes damage mapping unit 300G, detection result output unit 300H (detection result output unit), display control unit 300I (display control unit), communication control unit 300J, and ROM 300K (non-transitory recording medium). Server 300 is connected to client 200 via network NW, acquires captured images (captured images 220A in FIG. 3) from client 200, and performs damage detection, synthesis of detection results, etc.

[0019] The image acquisition unit 300A inputs photographed images (photographed images 220A in FIG. 3) from the client 200. The synthesis parameter calculation unit 300B calculates synthesis parameters for synthesizing the photographed images based on corresponding points between the images. The image synthesis unit 300C synthesizes the photographed images based on the synthesis parameters. The damage detection unit 300D detects (extracts and measures) damage (cracks, peeling, corrosion, etc.) of the subject (bridge) from the photographed images. The detection result synthesis unit 300E synthesizes the damage detection results (detection results) for the photographed images based on the synthesis parameters calculated by the synthesis parameter calculation unit 300B. The corresponding point designation unit 300F designates corresponding points between one image group among the displayed images and another image group among the displayed images based on a user instruction input. The damage mapping unit 300G maps information indicating damage onto the composite image. The detection result output unit 300H outputs damage detection results, composite detection results, identification information, composite images, damage mapping images, etc. to the client 200. The display control unit 300I displays captured images, detection results, etc. on the monitor 232 (display device). The communication control unit 300J transmits and receives images and information to and from the client 200 via the network NW. The ROM 300K (non-transitory recording medium) stores computer-readable codes of various programs for operating the damage diagram creation system 10, such as a damage diagram creation program for executing the damage diagram creation method according to the present invention. In addition to the above-mentioned components, the server 300 also includes a recording device (e.g., a magneto-optical recording medium, such as a hard disk) (not shown) that records images and information acquired from the client 200 and processing results (e.g., damage detection results) by each component of the server 300. The recorded images, etc. can be transmitted to the client 200 upon request.

[0020] The functions of each unit of the server 300 described above can be realized using various processors. The various processors include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions. The various processors described above also include a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacturing. Furthermore, the various processors described above also include dedicated electrical circuits, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically to execute specific processing.

[0021] The functions of each section may be realized by a single processor or by a combination of multiple processors. Furthermore, multiple functions may be realized by a single processor. Examples of multiple functions configured by a single processor include, first, a configuration in which one processor is configured by a combination of one or more CPUs and software, as typified by computers such as client and server computers, and this processor realizes multiple functions. Second, a configuration in which a processor is used to realize the functions of the entire system on a single IC (Integrated Circuit) chip, as typified by a System on Chip (SoC). In this way, various functions are configured as a hardware structure using one or more of the various processors described above. Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit formed by combining circuit elements such as semiconductor elements.

[0022] When the above-described processor or electrical circuit executes software (programs), processor-readable code for the software to be executed (including a program for executing the damage map creation method according to the present invention) is stored in a non-transitory recording medium such as ROM 300K (see FIG. 4), and the processor references the software. Instead of ROM 300K, the code may be recorded in a non-transitory recording medium such as various types of magneto-optical recording devices or semiconductor memory. When processing using software, for example, RAM (Random Access Memory) is used as a temporary storage area, and data stored in, for example, EEPROM (Electronically Erasable and Programmable Read Only Memory) is referenced. Note that devices such as RAM and EEPROM are not shown in FIG. 4.

[0023] Similarly to the server 300, each function of the client 200 described above can be realized by various processors, electric circuits, and software.

[0024] <Image processing procedure> The image processing by the damage diagram creation system 10 will now be described. Figures 5 and 6 are flowcharts showing the procedure of image processing (including each step of the damage diagram creation method according to the present invention). In these figures, steps S100 to S112 indicate the processing by the client 200, and steps S200 to S236 indicate the processing by the server 300.

[0025] <photograph> In the procedure shown in FIGS. 5 and 6, the bridge 1 (structure) is photographed in sections by the digital camera 100 to obtain a plurality of photographed images (step S100).

[0026] In this embodiment, a case where a deck slab 6 is photographed will be described. FIG. 7 is a diagram showing an example of the procedure for photographing the deck slab 6. FIG. 7 shows how photographing is performed in units of area A including the lattice spacing GO defined by the main girders (components extending in the x direction) and cross girders (components extending in the y direction), and photographing is repeated while the photographing area is moved sequentially in the y direction and x direction (in the direction of the arrows). As long as an image of the entire photographing range can be obtained, photographing may be performed using other procedures. In FIG. 7, the extension direction of the bridge (deck slab) is defined as x, the direction perpendicular to x within the plane of the deck slab is defined as y, and the direction perpendicular to the deck slab (vertical downward direction) is defined as z, and the coordinates are formed as (x, y, z).

[0027] FIG. 8 is a diagram showing an example of the photographing procedure for one grid GO. In the example of FIG. 8, photographs are taken starting from area A1 at the +x end of the grid GO and moving to area Ai at the -x end, then returning to the +x end and starting from area Aj at the -x end, area An, for a total of n images (n is an integer equal to or greater than 2). A different pattern (e.g., the order of areas A1-Ai-An-Aj) may also be used. When photographing, the photographing position may be changed for each image so that a straight-on image is always photographed, or multiple images may be taken at one photographing position while changing the photographing direction (in this case, an image photographed from an oblique direction may be included). Furthermore, it is preferable to appropriately set the photographing position and photographing direction to create sufficient overlap (e.g., about 30%) between adjacent images, making it easy and accurate to detect and set corresponding points.

[0028] Figure 9 is an example of captured images, showing 10 images i1 to i10 taken with overlapping images ensured. Figure 10 shows images i1 to i10 individually. Note that Figures 9 and 10 show the frame F of the coffer GO (a rectangle defined by the main girders and cross beams), and omit the illustration of damage to other components and floorboards.

[0029] The client 200 inputs the above-mentioned multiple captured images via the digital camera 100 (image capturing optical system 110, wireless communication unit 130, antenna 132) and the processing unit 210 (communication control unit 210D, image input unit 210A, antenna 212) (step S102).

[0030] <Image and information storage> In the damage diagram creation system 10, folders are created in the storage unit 220 of the client 200 to store the captured images. Fig. 11 is a diagram showing an example of a folder structure. In the example of Fig. 11, a main folder MF is created for the entire bridge, and subfolders SF1 and SF2 are created within this main folder for each capture area (inspection area 1A, inspection area 1B). Within the subfolders for each capture area, subfolders SS1 to SS5 are further created for each lattice spacing GO defined by the main girder and cross girder, and the captured images are stored therein. Fig. 12 shows the state in which 10 captured images are stored for the lattice spacing (subfolder SS1) with lattice spacing number A001.

[0031] 11 and 12 can be created by the file management unit 210B in response to operations on the operation unit 240 (keyboard 242, mouse 244). The folder structure may be different from the examples in FIGS. 11 and 12, and for example, subfolders may be created for each component number. As will be described in detail later, the damage diagram creation system 10 combines the damage detection results (detection results) for each folder (subfolders are also included in the term "folder") and stores the combined detection results in the same folder as the captured image.

[0032] <Image acquisition> The operation unit 240 of the client 200 receives a damage detection and synthesis instruction operation via the keyboard 242 and / or mouse 244 (step S104), and the server 300 (image acquisition unit 300A) acquires the captured images in response to this operation (step S200). The image acquisition unit 300A acquires images stored in the same folder in the client 200 (storage unit 220) as images belonging to the same group. For example, the 10 captured images stored in the subfolder SS1 shown in FIGS. 11 and 12 are acquired as images belonging to the same group.

[0033] <Calculation of synthesis parameters> After the captured images are acquired in step S200, server 300 (combining parameter calculation unit 300B) calculates combining parameters for combining multiple images based on corresponding points between the images (step S202). For example, the composite parameters can be calculated as a projective transformation matrix of each captured image relative to a reference image. Note that while the flowchart in FIG. 5 illustrates an example in which the computation of the combining parameters in step S202 is performed first, damage detection in step S204 may be performed first, or steps S202 and S204 may be performed in parallel.

[0034] Damage detection The server 300 (damage detection unit 300D) detects (extracts and measures) damage from the acquired captured image (step S204). Damage can be classified as peeling, water leakage, cracking, rust, etc., but the specific type of damage to be detected may be set according to conditions such as the type and characteristics of the building (subject), the purpose of the inspection, etc. Furthermore, items to be detected include position, size, direction, range, shape, etc., but the detection items may also be set according to the damage classification, the type and characteristics of the building, the purpose of the inspection, etc. In damage detection, the damage detection unit 300D vectorizes the detection result and represents it as a line segment having a start point and an end point or a set of lines (in the case of linear damage such as cracks), or as a polygon or other figure composed of such line segments (in the case of spreading damage such as peeling or corrosion).

[0035] Damage detection can be performed using various techniques depending on the classification. For cracks, for example, the crack detection method described in Japanese Patent Publication No. 4006007 can be used. This method involves two steps: creating a wavelet image and determining crack areas based on the wavelet image. In the wavelet image creation step, wavelet coefficients corresponding to the two densities to be compared are calculated, and wavelet coefficients for each of the two densities are calculated to create a wavelet coefficient table. An input image of the concrete surface to be detected for crack detection is then subjected to wavelet transformation. In the crack area determination step, the wavelet coefficient corresponding to the average density of neighboring pixels in a local area and the density of the pixel of interest in the wavelet coefficient table is used as a threshold, and the wavelet coefficient of the pixel of interest is compared with the threshold to determine whether the area is cracked or not.

[0036] Furthermore, as a method for detecting rust and peeling, for example, the processing method of the steel bridge paint film inspection system described in JP-A-2010-538258 can be used. This processing method detects rust and peeling using color information from the image file of the steel bridge paint film, image processing, watershed, and Parzen window.

[0037] In this way, the damage diagram creation system 10 according to the first embodiment detects damage from captured images before compositing. Therefore, damage detection performance does not deteriorate due to image quality degradation in overlapping areas of the images, and damage can be detected with high accuracy based on multiple images acquired by photographing the subject in separate images. Note that the damage detection results are composited using a composition parameter between the images, as will be described later.

[0038] <Determining whether or not to combine> The server 300 (combining parameter calculation unit 300B) determines whether all captured images can be combined into one image based on the combining parameters calculated in step S202 (step S206). Whether combining is possible can be determined based on the number of corresponding points, whether the reliability of the corresponding points is sufficient (whether the corresponding points are characteristic points), whether a projective transformation matrix can be calculated from the corresponding points, and so on. In this case, the determination may be made by repeatedly calculating the projective transformation matrix and its evaluation value by changing the combination of corresponding points using a RANSAC (RANdom Sample Consensus) algorithm or the like. If all images can be combined into one image (YES in step S206), the process proceeds to step S222, where the detection results are combined. If all images cannot be combined into one image (NO in step S206), the process performs the processes of steps S208 to S220 as described below, and then proceeds to step S222. Steps S208 to S220 are processes performed when all images cannot be combined into a single image (automatic combination). In these steps, for image groups that can be combined, combination parameters are calculated for each image group, and damage detection results are combined for each image group. On the other hand, for image groups that cannot be combined, combination parameters are calculated by specifying corresponding points based on a user operation, as described below, and detection results are combined based on the calculated combination parameters. For example, in the case of image groups G1 and G2 (see FIGS. 15 to 18), which will be described later, the detection results for the images that make up image groups G1 and G2 are combined, and between image groups G1 and G2, the detection results combined for image groups G1 and G2 are combined into one based on combination parameters calculated based on a user operation (see FIGS. 19 to 21).

[0039] <Image Classification> If it is determined in step S206 that "all captured images cannot be combined into one image," the combining parameter calculation unit 300B divides the captured images into image groups that can be combined (step S208), and determines the image layout (calculates the combining parameters) for each image group based on corresponding points between the images (step S210). Once the image layout is determined, the detection results can be combined based on that layout.

[0040] <Determining image placement> Fig. 13 is a flowchart showing the details of the image arrangement determination process in step S210 of Fig. 5. The synthesis parameter calculation unit 300B sets a reference image from among the multiple captured images, which serves as a basis for projective transformation (step S210A). The reference image can be set (selected) based on image characteristics such as orientation and clarity, but an image in a specific shooting order (for example, the first image captured) may also be used as the reference image. In the example of Figs. 9 and 10, as shown in part (a) of Fig. 14, image i1 can be set as the reference image for images i1 and i2.

[0041] Once the reference image has been set, synthesis parameter calculation unit 300B calculates a projective transformation matrix for each image other than the reference image relative to the reference image based on corresponding points between the images (step S210B). In the example of FIGS. 9 and 10, as shown in part (a) of FIG. 14, image i2 (an image other than the reference image) is projected onto the same plane as image i1 (reference image) and a projective transformation matrix for matching corresponding points is calculated. Once the projective transformation matrix for image i2 relative to image i1 (reference image) has been calculated, as shown in part (b) of FIG. 14, the transformed image i2 is set as the reference image and the projective transformation matrix for image i3 relative to the transformed image i2 is calculated. In this way, the reference image is changed and projective transformation matrices are calculated for all images in the image group (until step S210C returns YES).

[0042] After calculating the projective transformation matrices for all images, synthesis parameter calculation unit 300B performs movements, rotations, enlargements or reductions, deformations, etc. on each image based on the calculated projective transformation matrices, and determines the arrangement of each image in the image group (step S210D). In the following description, it is assumed that images i1 to i4 and images i6 to i9 form a compositeable image group G1, and images i5 and i10 form a compositeable image group G2. Note that in determining the image arrangement described above, each image is moved, rotated, deformed, etc. using the projective transformation matrix, but the illustrated example conceptually illustrates the arrangement of the images and does not accurately represent the movement, rotation, deformation, etc. of the images.

[0043] The image layout determined in step S210D may be an image layout in which overlapping areas of the images overlap (see FIG. 15), or an image layout in which the images do not overlap (see FIG. 16). In the case of an image layout in which the images do not overlap, the images can be laid out spaced apart in the x and y directions as shown in FIG. 16, and the orientation of the images can be left unchanged from the orientation determined by the projective transformation matrix (the images are not rotated).

[0044] <Image display> Once the image layout is determined in step S210 (steps S210A to S210D), the server 300 (display control unit 300I, communication control unit 300J) instructs the client 200 to display the group of images that can be combined (step S212). The instruction to the client 200 includes the images to be displayed, information on the layout of the images, and information on the vectorized damage detection results obtained in step S204. In response to the display instruction, the client 200 (display control unit 210C) superimposes the vectorized information on the damage detection results on the images and displays them on the monitor 232 for each group of images that can be combined (step S106). In the example described above, the display control unit 210C displays the image groups G1 and G2 in the image layout shown in FIG. 15 or FIG. 16. FIG. 17 shows how the cracks in images i4, i5, i9, and i10 in the image layout of FIG. 15 are vectorized and superimposed. When displaying, the display control unit 300I and / or the display control unit 210C may perform processing such as surrounding each of the image groups G1 and G2 with a frame, displaying each image group in a different color, or displaying the number of the image group, so that the image groups can be easily identified.

[0045] When displaying images in step S106, the arrangement between the image groups may be inappropriate. For example, as shown in FIG. 18, image group G2, which should be arranged below image group G1 (in the −x direction), may be arranged to the side of image group G1. In this case, the server 300 (combination parameter calculation unit 300B) determines whether to change the arrangement of the image groups (step S214). If the arrangement is to be changed (YES in step S214), the process returns to step S212, and the client 200 redisplays the image groups G1 and G2 in the changed arrangement. The determination in step S214 may be made based on a user instruction input (e.g., moving image group G2 via the keyboard 242 and / or mouse 244, i.e., changing the relative arrangement). In the example of FIG. 18, the combination parameter calculation unit 300B and the display control unit 300I move image group G2 below image group G1 (in the direction of the arrow) based on a user operation (e.g., dragging with the mouse 244), resulting in a display as shown in FIG. 15.

[0046] If the relative positioning of the image groups is inappropriate when they are displayed, it may take time to specify corresponding points. However, the damage diagram creation system 10 can change the positioning in this way, allowing corresponding points to be specified quickly and easily.

[0047] <Specifying corresponding points> Once the arrangement of the images has been determined by the processing up to step S214, the server 300 (corresponding point designation unit 300F) designates corresponding points between one of the displayed images and the other (step S216). For example, if the image groups G1 and G2 are arranged and displayed as shown in FIG. 15, point P1a is designated in image group G1 by clicking the mouse 244 or the like (the corresponding point designation operation in step S108), and point P1b, which corresponds to point P1a, is designated in image group G2, as shown in FIG. 19. Corresponding points can be, for example, the start point, end point, or branch point of a crack determined to be "identical," and / or characteristic points such as the end, side, or joint of a component. In this case, the corresponding point designation unit 300F and the display control unit 300I display the designated points P1a and P1b in a distinguishable manner by connecting the designated points P1a and P1b with a straight line or the like (see FIG. 19), making it easy to understand that the designated points correspond to each other. Similarly, points P2a and P2b, points P3a and P3b, points P4a and P4b, points P5a and P5b, and points P6a and P6b are also specified.

[0048] 19 shows an example in which six corresponding points are specified for each of the image groups G1 and G2, but the number of corresponding points to be specified is not particularly limited. Furthermore, even when the image groups G1 and G2 are arranged and displayed as shown in FIG. 16, corresponding points can be similarly specified between images i4 and i5 and images i9 and i10.

[0049] As described above, in the damage diagram creation system 10 according to the first embodiment, the image groups that can be combined (image groups G1 and G2) are displayed for each image group, so that it is possible to easily grasp which image groups were (automatically) combined and which were not. Furthermore, since it is only necessary to specify corresponding points for the image groups that could not be combined (image group G1 and image group G2), it is not necessary to specify corresponding points for all images, and it is possible to quickly and easily specify corresponding points and calculate combining parameters based on the specified corresponding points.

[0050] If the synthesis parameters cannot be calculated accurately using the corresponding points specified as described above, the server 300 (synthetic parameter calculation unit 300B, corresponding point specification unit 300F, display control unit 300I, etc.) and the client 200 (display control unit 210C, etc.) may display a warning message on the monitor 232 to prompt the user to perform the corresponding point specification operation again.

[0051] <Image alignment based on specified corresponding points> Once the corresponding points are specified in step S216, server 300 (composite parameter calculation unit 300B) calculates composite parameters for each compositeable image group based on the specified corresponding points (step S218). In the example of FIG. 19, composite parameter calculation unit 300B calculates a projective transformation matrix for image group G2 using image group G1 as a reference (or a projective transformation matrix for image group G1 using image group G2 as a reference) based on points P1a to P6b, which are the corresponding points. Image composite unit 300C and composite parameter calculation unit 300B determine the image arrangement by moving, rotating, transforming, etc., the images (images i5, i10) that make up image group G2 using the projective transformation matrix calculated in this manner (step S220). The image arrangement determination in step S220 can be performed using the same procedure as in step S210.

[0052] <Combining detection results> The server 300 (detection result synthesis unit 300E) synthesizes the detection results (detection results) based on the synthesis parameters (projection transformation matrix) calculated in steps S202 and S220 (step S222). For example, suppose different portions (partial overlap) of damage vectors V1 and V2 are detected in images i1 and i2 as shown in parts (a) and (b) of FIG. 20, and the start and end points of the damage vectors V1 and V2 are points P7 to P12. Parts (a) and (b) of FIG. 21 are tables showing the start and end points of the damage vectors V1 and V2 corresponding to parts (a) and (b) of FIG. 20, respectively. In this case, the detection results after synthesis are as shown in part (c) of FIG. 20 (image i2 is synthesized by moving, rotating, etc. using the projective transformation matrix), with the start and end points of the damage vector V1 being points P7 and P8, respectively, and the start and end points of the damage vector V2 being points P8 and P12, respectively. Part (c) of FIG. 21 corresponds to part (c) of FIG. 20, and shows the start and end points of the damage vectors V1 and V2 after synthesis.

[0053] <Combining detection results in overlapping image areas> When images are acquired so that part of the imaging range overlaps as described above, in the overlapping area of ​​multiple images, the accuracy of synthesis may be degraded, for example, detection results that should be synthesized into one may be multiple due to misalignment of the images. Therefore, in the damage diagram creation system 10, in the overlapping area of ​​multiple images, one of the overlapping images is selected and the detection results are synthesized, thereby enabling the detection results to be synthesized with high accuracy. Such synthesis is shown in FIG. 22. In the example of FIG. 22, since images i1 and i2 overlap in the area OL, image i2, which has a smaller value of the synthesis parameter, is selected in this area OL, and the detection results for image i2 are used for synthesis (see (b) of FIG. 22).

[0054] This makes it possible to suppress deformation and displacement of damage, thereby improving the accuracy of damage detection results and data calculations.

[0055] In the case of the above-mentioned projective transformation, the value of the synthesis parameter is a value that represents the degree of projective transformation (the manner of deformation due to projective transformation), and the greater the amount of movement of coordinates due to projective transformation in the synthesis area, the larger the value of the synthesis parameter.

[0056] A specific explanation will be given using FIG. 29. Consider the case where two original images i1 and i2 in FIG. 29(a) are synthesized. For i1, a projective transformation matrix is ​​calculated using i0 (not shown) as the reference image by the method described above, and i1 is projectively transformed as shown in FIG. 29(b). In this case, specific coordinates P1a, P2a, P3a, and P4a in i1 are moved to P1a', P2a', P3a', and P4a' by projective transformation. Next, to calculate the projective transformation matrix for i2 using i1 after projective transformation as the reference image, corresponding points P1a to P4a and P1b to P4b in each image are specified (Pxa and Pxb are corresponding points, respectively). A projective transformation matrix for i2 is calculated based on these corresponding points, and i2 is projectively transformed as shown in FIG. 29(b). At this time, P1b, P2b, P3b, and P4b in i2 are moved to P1b', P2b', P3b', and P4b' by projective transformation. Then, the image layout is determined so that the corresponding points P1a' to P4a' in i1 and P1b' to P4b' in i2 coincide with each other.

[0057] Here, the amount by which P1a to P4a of i1 have moved due to the projective transformation is calculated. Specifically, the distance between P1a and P1a' (|P1a - P1a'|) is calculated, and this is repeated for P2a to P4a, and finally the sum is taken. This is the synthesis parameter for i1. Similarly, for i2, the distance between P1b and P1b' (|P1b - P1b'|) is calculated, and this is repeated for P2b to P4b, and finally the sum is taken. This is the synthesis parameter for i2. Using the synthesis parameter calculated in this way, the image with the smallest synthesis parameter (i2 in the example of Figure 29(c)) is selected in the overlapping area of ​​i1 and i2, and they are synthesized.

[0058] As described above, the synthesis parameter represents the amount of coordinate movement due to projective transformation, so the greater the degree of transformation (degree of deformation), the greater the value of the synthesis parameter. In this embodiment, the synthesis parameter is defined as the sum of the coordinate movement distances, but other calculation methods may be used as long as the parameter represents the degree of deformation. For example, there is a method of calculating the shape similarity between a figure formed by connecting P1a to P4a and a figure formed by connecting P1a' to P4a'. In this case, by normalizing the size of the figures to be compared in advance, the difference in shape of the figures can be ignored and the synthesis parameter can be calculated by focusing only on the degree of deformation. In this case, the image with the higher shape similarity is considered to have a smaller degree of deformation (smaller synthesis parameter), and the overlapping area is selected.

[0059] The above-mentioned combination of detection results is performed for images stored in the same folder (multiple images divided into groups). In the example shown in Fig. 12, the detection results for 10 images stored in subfolder SS1 are combined (the detection results are combined for each group).

[0060] <Image Composition> The server 300 (image synthesis unit 300C) synthesizes the images based on the synthesis parameters (projection transformation matrix) calculated in steps S202 and S218 (step S224). Fig. 23 shows the synthesized image G3. Fig. 24 shows an image G3a synthesized by selecting one image in the overlapping area as in Fig. 22. Note that since a damage diagram can be created once the detection results are synthesized, the synthesis of the images may be omitted.

[0061] <Post-composite facing correction> In the damage diagram creation system 10, as described above, the projective transformation matrix of other images relative to the reference image is calculated to determine the image arrangement. However, if the reference image is not directly facing the imaging direction, an area that should be rectangular in the combined image may not be rectangular. For example, the frame F between the images may become trapezoidal in the combined image. In this case, the server 300 (the synthesis parameter calculation unit 300B, the image synthesis unit 300C, etc.) specifies points that form the rectangle (for example, the four corner points of the frame F) based on user operation via the keyboard 242 and / or the mouse 244, and forms a rectangle using these four points through projective transformation. This makes it possible to obtain an image in which the subject is directly facing (a frontal image) even after the images are synthesized.

[0062] <Mapping of measurement results> The server 300 (damage mapping unit 300G) may map the damage detection results onto the composite image. Mapping can be performed, for example, by displaying characters, figures, symbols, etc. associated with the detection results on the composite image. The characters, figures, symbols, etc. to be displayed can be selected via the operation unit 240 (keyboard 242 and / or mouse 244), and in accordance with the selection, the server 300 (damage mapping unit 300G, display control unit 300I, etc.) and the client 200 (display control unit 210C) display the mapped image on the monitor 232. The characters, figures, symbols, etc. may be simplified or emphasized versions of the actual damage, or may be displayed in different formats depending on the type, size, etc. of the damage. The image onto which the measurement results are mapped is stored in the memory unit 220 (damage mapping image 220D in FIG. 3) and displayed on the monitor 232 under the control of the display control unit 210C. Damage information may be input into the damage mapping image.

[0063] Such mapping of the measurement results may be performed on drawing data (e.g., CAD data, CAD: Computer-Aided Design) including diagram information showing the shape of the bridge 1. In this case, if the coordinate system defining the CAD data is different from the coordinate system shown in Figures 23 and 24, etc., coordinate transformation (movement, rotation, mirroring, etc.) is performed according to the relationship between the coordinate systems. Such transformation can be performed by the server 300 (damage mapping unit 300G).

[0064] <Result display> The server 300 (detection result output unit 300H, display control unit 300I, communication control unit 300J, etc.) instructs the client 200 to display the detection results (step S226), and in response to this instruction, the client 200 (display control unit 210C, etc.) displays the detection results on the monitor 232 (step S110). The detection results can be displayed using letters, numbers, symbols, etc., and the detection results for each captured image may be displayed, or a composite detection result may be displayed (see FIGS. 20 and 21). Also, captured images, composite images, damage mapping images, etc. may be displayed together with or instead of the detection results. Vectorized information of the detection results (see FIGS. 20 and 21) may be displayed superimposed on the image. The display control unit 210C can select the content and format to be displayed on the monitor 232 in response to operation of the operation unit 240. The display control unit 210C measures the time since the results were displayed and determines whether an output instruction has been issued after the results were displayed. If the display time is long without an output instruction, or if the screen transitions to another screen without an output instruction, it is possible that fraud such as visual copying is being attempted, and such fraud can be detected by measuring the display time and determining whether an output instruction was given. If fraud is detected, measures can be taken, such as stopping the display.

[0065] <Result output> The server 300 (detection result output unit 300H, etc.) determines whether or not an output instruction operation (e.g., an output instruction operation via the operation unit 240) has been performed for the detection results displayed in step S110 (step S228). Only if an output instruction operation has been performed (YES in step S228), the process proceeds to step S230 to instruct the client 200 to output the detection results (detection results for each image and the combined detection result), and the client 200 (file management unit 210B) outputs the detection results in response to the output instruction (step S112). In the example of FIG. 12, the detection results for the images stored in subfolder SS1 are stored in the same subfolder SS1 as the images (the detection results are output in association with the group; see FIG. 25). The detection results may be output in the same format as the drawing data (e.g., CAD data) containing line drawing information indicating the shape of the bridge 1 (subject). The detection results may also be printed by a printer not shown.

[0066] In this way, in the first embodiment, the detection results are stored in the same folder as the folder in which the images are stored, so the correspondence between the input images and the combined detection results becomes clear, and the images and detection results can be easily managed and used. Note that the detection results are output only when an output operation instruction is given (when YES is determined in step S228), and when no output operation instruction is given, identification information is assigned to the image (step S232), and the assigned identification information is notified to client 200 and the image is stored in the same folder as the folder in which the images are stored (subfolder SS1 in the examples of FIGS. 12 and 25).

[0067] Once the detection result or identification information has been output, server 300 determines whether processing has been completed for all folders (step S234). If the determination is affirmative, server 300 notifies client 200 of completion in step S236 (step S236) and terminates processing. If processing has not been completed for all folders, server 300 returns to step S202 and repeats steps S202 to S234 for other folders.

[0068] As described above, the damage diagram creation system 10 according to the first embodiment can detect damage with high accuracy based on a plurality of images acquired by photographing a subject in sections.

[0069] <Second embodiment> In the first embodiment described above, a damage diagram creation system 10 including a server 300 and a client 200 has been described, but in the second embodiment, a damage diagram creation device 20 will be described. FIG. 26 is a diagram showing the configuration of the damage diagram creation device 20. The damage diagram creation device 20 is composed of a digital camera 100 and a device main body 500. Since the configuration of the digital camera 100 is the same as the configuration of the digital camera 100 in the first embodiment, the same reference numerals are used and detailed description will be omitted. Note that the digital camera 100 may be configured integrally with the device main body 500.

[0070] <Configuration of the damage diagram creation device> The damage diagram creation system 10 according to the first embodiment includes a server 300 and a client 200, and the server 300 performs the majority of the processing, such as damage detection and synthesis. However, in the damage diagram creation device 20 according to the second embodiment, a processing unit 510 of the device main body 500 performs the processing. Specifically, the processing unit 510 has the functions of the client 200 shown in FIG. 2 and the functions of the server 300 shown in FIG. 4. The storage unit 520 stores information similar to that of the storage unit 220 according to the first embodiment (see FIG. 3). The configuration and function of the operation unit 540 (keyboard 542, mouse 544) are similar to those of the operation unit 240 (keyboard 242, mouse 244) according to the first embodiment. The configuration and function of the display unit 530 (monitor 532) are similar to those of the display unit 230 (monitor 232) according to the first embodiment. In the damage diagram creation device 20, a device (information terminal) such as a personal computer, tablet terminal, or smartphone that inputs images and performs processing, such as damage detection and synthesis, can be used as the device main body 500.

[0071] <Processing for creating damage diagrams> The processing in the damage diagram creation device 20 (processing of the damage diagram creation method according to the present invention) is similar to the flowcharts in Figures 5, 6, and 13. For example, a plurality of images acquired by photographing an object in separate images are input, a synthesis parameter (projection transformation matrix) for synthesizing the plurality of images is calculated based on corresponding points between the images, damage to the object is detected from the images constituting the plurality of images, and the detection results for the plurality of images are synthesized based on the synthesis parameter. Such processing is performed by the processing unit 510 of the device main body 500. Note that, while communication is performed between the client 200 and the server 300 in the first embodiment, communication is performed within the device main body 500 in the damage diagram creation device 20.

[0072] In the damage diagram creation device 20 according to the second embodiment, damage is detected from the captured images before compositing, as in the damage diagram creation system 10 according to the first embodiment. As in the first embodiment, images with small values ​​of the compositing parameter are used for the images in the overlapping region. This makes it possible to suppress deformation and positional deviation of damage, thereby improving the accuracy of damage detection results and their data calculation.

[0073] Therefore, damage detection performance does not deteriorate due to degradation in image quality in overlapping image regions, and damage can be detected with high accuracy based on multiple images acquired by photographing the subject in sections.

[0074] In both the first and second embodiments described above, the images to be used may be selected based on the positional relationship between the overlapping area and the damage (cracks, etc.). For example, as shown in FIG. 27(a), there may be a case where damage is present in the overlapping area, and the damage in image i2 straddles the overlapping area from inside to outside the overlapping area, while the damage in image i1 does not straddle the overlapping area. In such a case, as shown in FIG. 27(b), one image i2 may be used as the image of the overlapping area (regardless of the value of the synthesis parameter). In other words, if one image i2 has damage that extends from inside to outside the overlapping area, and the other image i1 has damage that is contained within the overlapping area (but does not extend outside), one image i2 may be used as the image of the overlapping area (regardless of the value of the synthesis parameter). This ensures the continuity of the damage, improving the accuracy of the damage detection results and the data calculation. In addition, as shown in (a) of Figure 28, when damage is located both inside and outside the overlapping region in both image i2 and image i1, it is recommended to use image i2, which has the greater number of damages located inside and outside the overlapping region, in the overlapping region, as shown in (b) of Figure 28. This reduces the number of damages that lose continuity, improving the accuracy of damage detection results and data calculation.

[0075] Furthermore, for overlapping areas where no damage exists, an image with good image quality may be used as the image of the overlapping area regardless of the value of the synthesis parameter. In this case, the quality of image quality can be determined based on image quality assessment information such as the appropriateness of brightness, the appropriateness of saturation, and the degree of focus. This not only improves the accuracy of damage detection results and data calculation, but also improves the quality of the synthesized image, making it possible to provide users with better damage diagrams. It is preferable to determine the quality of image quality based on two or more pieces of assessment information rather than using only one piece of image quality assessment information (such as the appropriateness of brightness, the accuracy of saturation, or the degree of focus). This makes it possible to provide users with better damage diagrams. In this way, for overlapping areas that do not fall into the above-mentioned cases, an image with a small value of the synthesis parameter is selected and synthesized, and for overlapping areas that do fall into the above-mentioned cases, an image is selected and synthesized according to the above-mentioned conditions, thereby improving the accuracy of damage detection results and data calculation, and achieving the various effects described above.

[0076] <Third embodiment> A third embodiment will be described with respect to the processing in the damage diagram creation devices 10 and 20 described above.

[0077] Here, the configuration of the damage diagram creation system 10 in the first embodiment will be used for explanation, but it can also be applied to the configuration of the damage diagram creation device 20 in the second embodiment. In addition, the processing in the damage diagram creation device is similar to the flowcharts in Figures 5 and 6, but the image layout determination processing in steps S210 and S220 is different.

[0078] Specifically, the image layout determination process in the third embodiment will be described with reference to Figures 30 and 31. Here, as an example, a method for creating a damage diagram from a group of images of an arch-shaped bridge photographed from below as shown in Figure 31(a) will be described.

[0079] <Determining image placement> Fig. 30 is a flowchart showing details of the image arrangement determination process in steps S210 and S220 in Fig. 5. Synthesis parameter calculation unit 300B sets a reference image that serves as a basis for projective transformation from among multiple captured images (step S210E). The reference image can be set (selected) based on image characteristics such as orientation and clarity, but an image in a specific shooting order (for example, the first image captured) may also be used as the reference image. In the example of Fig. 31, image i1 can be set as the reference image among images captured in the order i1, i2, i3, ...

[0080] Once the reference image is set, synthesis parameter calculation unit 300B calculates a projective transformation matrix for the reference image of an image adjacent to the reference image based on corresponding points between the images (step S210F). In the example of Fig. 31, as shown in Fig. 31(b), image i2 (an image adjacent to the reference image) is projected onto the same plane as image i1 (reference image) and a projective transformation matrix for matching corresponding points is calculated.

[0081] After calculating the projective transformation matrix for image i2, the synthesis parameter calculation unit 300B calculates synthesis parameters for i2 based on the calculated projective transformation matrix. The synthesis parameter here is a value that represents the degree of projective transformation (the degree of deformation due to projective transformation), and the value of the synthesis parameter increases as the angle between the captured image plane (a plane normal to the optical axis) and the projection plane increases. Specifically, as described in the first embodiment, the synthesis parameter can be calculated based on the amount of movement of the coordinates of each corresponding point due to projective transformation. While the first embodiment calculates the synthesis parameters focusing on the overlapping region between i1 and i2, the third embodiment desirably calculates the degree of projective transformation focusing on the entire image i2, and therefore the synthesis parameter may be calculated based on the amount of movement of the coordinates of the four vertices of image i2.

[0082] After calculating the synthesis parameters for image i2, synthesis parameter calculation unit 300B determines whether or not to permit projective transformation of image i2 based on the calculated synthesis parameters (step S210H). Here, if the calculated synthesis parameters are equal to or smaller than a predetermined value, projective transformation is permitted, and if they are greater than the predetermined value, projective transformation is not permitted.

[0083] If projective transformation is permitted, the synthesis parameter calculation unit 300B performs projective transformation on image i2 based on the calculated projective transformation matrix (step S210I) and arranges it next to image i1 (step S210J). At this time, the images may be arranged so that the overlapping areas of the images overlap, or so that the images do not overlap.

[0084] If projective transformation is not permitted, the synthesis parameter calculation unit 300B does not perform projective transformation on the image i2, but arranges it next to the image i1 (step S210J). At this time, the images are arranged so that overlapping areas between the images do not overlap.

[0085] Then, image i2 is set as the reference image, and the above-described process (steps S210E to S210J) is similarly performed on image i3 adjacent to image i2. In this way, the process is repeated for all images in the image group (until step S210K returns YES) while changing the reference image.

[0086] In the example of FIG. 31, because the bridge is arch-shaped, repeated projective transformation using image i1 as the initial reference image results in a gradually increasing degree of deformation due to projective transformation (because the images are projected onto the same plane as image i1). Therefore, as shown in FIG. 31(b), images i2 and i3 can be projected and combined without any problems, but images i4 and beyond experience a greater degree of deformation, making them unsuitable for use as damage diagrams. Therefore, by setting an appropriate value in step S210H of the third embodiment, projective transformation of image i4 can be prohibited (FIG. 31(c)). In this case, image i4 cannot be combined with image i3, but each image can maintain an image quality (shape) suitable for use as a damage diagram.

[0087] As shown in FIG. 31(c), if projective transformation is not permitted in step S210H, the images i3 and i4 may be displayed in a manner that allows the user to recognize this, such as by adding a dotted line between them.

[0088] Furthermore, when determining whether to permit projective transformation in step S210H, the determination may be made based on the facing degree of the captured image. For example, if the captured image is facing forward and projective transformation would deform it in a direction that is not facing forward, projective transformation may not be permitted. However, if the captured image is not facing forward originally and projective transformation would deform it in a direction that is facing forward, the determination may be made to permit projective transformation.

[0089] As described above, according to the third embodiment, it is possible to create an appropriate damage diagram even for structures that are not on the same plane, such as arched bridges.

[0090] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described aspects, and various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0091] 200 clients 210 Processing section 210A Image input unit 220 Storage section 220A photographed image 300 servers 300A Image acquisition unit 300B Synthesis parameter calculation section 300C Image composition section 300D Damage detection unit 300E Detection result synthesis unit

Claims

1. inputting a plurality of images acquired by dividing and photographing an object so that some areas of the object overlap; calculating synthesis parameters for synthesizing the plurality of images based on corresponding points between the images; detecting damage to the subject from the images constituting the plurality of images; synthesizing the detection results for the plurality of images based on the synthesis parameters; and In the combining step, in the overlapping region, one image is selected from the plurality of overlapping images and combined, and the selected one image has a smaller value of the combining parameter than the other images; In the combining step, if there is no damage in the overlapping area, the selected one image has a better judgment result of image quality judgment information than the other images. A damage diagram creation method characterized by:

2. 2. The damage diagram creation method according to claim 1, wherein the image quality judgment information is any one of brightness suitability, saturation suitability, and focusing suitability.

3. 3. The damage diagram creation method according to claim 2, wherein the judgment result is determined based on at least two pieces of judgment information.

4. 2. The damage diagram creation method according to claim 1, wherein, in the combining step, when an image in which damage is contained within the overlapping area and an image in which damage straddles the inside and outside of the overlapping area are combined, the selected one image is the image in which damage straddles the inside and outside of the overlapping area.

5. 2. The damage diagram creation method according to claim 1, wherein, in the combining step, when images in which damage straddles both inside and outside of an overlapping area are combined, the selected image is an image in which a larger number of damages straddle both inside and outside of the overlapping area than the other images.

6. A step of inputting a plurality of images obtained by photographing a subject in divided areas so that some areas overlap; calculating synthesis parameters for synthesizing the plurality of images based on corresponding points between the images; detecting damage to the subject from the images constituting the plurality of images; synthesizing the detection results for the plurality of images based on the synthesis parameters; and In the combining step, in the overlapping region, one image is selected from the plurality of overlapping images and combined, and the selected one image has a smaller value of the combining parameter than the other images; In the combining step, when combining an image in which damage is contained within the overlapping area and an image in which damage straddles the inside and outside of the overlapping area, the selected one image is an image in which damage straddles the inside and outside of the overlapping area. A damage diagram creation method characterized by:

7. A step of inputting a plurality of images obtained by photographing a subject in divided areas so that some areas overlap; calculating synthesis parameters for synthesizing the plurality of images based on corresponding points between the images; detecting damage to the subject from the images constituting the plurality of images; synthesizing the detection results for the plurality of images based on the synthesis parameters; and In the combining step, in the overlapping region, one image is selected from the plurality of overlapping images and combined, and the selected one image has a smaller value of the combining parameter than the other images; In the combining step, when images having damage straddling both inside and outside of an overlapping area are combined, the selected image is an image having a larger number of damages straddling both inside and outside of the overlapping area than the other images. A damage diagram creation method characterized by:

8. an image input unit that inputs a plurality of images obtained by dividing and photographing a subject so that some areas of the image overlap; a synthesis parameter calculation unit that calculates synthesis parameters for synthesizing the plurality of images based on corresponding points between the images; a damage detection unit that detects damage to the subject from the images that constitute the plurality of images; a detection result synthesis unit that synthesizes the detection results for the plurality of images based on the synthesis parameters; and The detection result synthesis unit selects one image from the plurality of overlapping images in the overlapping area and synthesizes the selected image, and if the selected image has a smaller value of the synthesis parameter than the other images and there is no damage in the overlapping area, the selected image has a better judgment result of image quality assessment information than the other images. A damage diagram creation device characterized by:

9. A damage diagram creation system including a server and a client, The client: an image input unit for inputting a plurality of images obtained by dividing and photographing a subject so that some areas of the image overlap; The server an image acquisition unit that acquires the plurality of images from the client; a synthesis parameter calculation unit that calculates synthesis parameters for synthesizing the acquired plurality of images based on corresponding points between the images; a damage detection unit that detects damage to the subject from the images that constitute the plurality of images; a detection result synthesis unit that synthesizes the detection results for the plurality of images based on the synthesis parameters; a detection result output unit that outputs the detection result to the client; Equipped with The detection result synthesis unit selects one image from the plurality of overlapping images in the overlapping area and synthesizes the selected image, and if the selected image has a smaller value of the synthesis parameter than the other images and there is no damage in the overlapping area, the selected image has a better judgment result of image quality assessment information than the other images. A damage diagram creation system characterized by:

10. A recording medium on which computer-readable code of a program for causing a computer to execute the damage diagram creation method described in any one of claims 1 to 7 is recorded.

Images