Image processing device, image processing method, and program

Abstract

Provided are an image processing device, an image processing method, and a program capable of generating a composite image that has relatively little distortion and is such that an arrangement of a plurality of the composite images exhibits no noticeable discontinuity. An image processing device (20): acquires (60) an image group including a plurality of images, the order of which in a first direction is known, and which are obtained by capturing images of an image capture target; divides (62) the image group into a plurality of image sets, wherein both of each two adjacent image sets in the first direction have one or more first overlapping images; generates (64) a plurality of composite images for each image set by combining, in the first direction, a plurality of images included in the image set; and removes (66) portions of the first overlapping images for each composite image.

Description

Image processing device, image processing method and program

[0001] The present invention relates to an image processing device, an image processing method, and a program.

[0002] Patent Document 1 describes an inspection device that acquires images for visual inspection of an inspection object. The device described in this document acquires image data including a plurality of images captured by scanning the surface of the object, synthesizes a series of images in the short direction to acquire a first composite image, and synthesizes the plurality of first composite images in the long direction to acquire a second composite image.

[0003] Patent Document 2 describes an image reading device that includes a first sensor that reads a first region of a document, a second sensor that reads a second region that partially overlaps with the first region, a third sensor that reads a third region that partially overlaps with the second region, a first processor that generates a first composite image that includes the first region and the second region, and a second processor that generates a second composite image that includes the second region and the third region, and that performs a process of linking the first composite image and the second composite image to generate an output image.

[0004] JP 2021-196735 A JP 2019-161635 A

[0005] One embodiment of the technology disclosed herein provides an image processing device, an image processing method, and a program that can generate a composite image with relatively little distortion and in which discontinuities are not noticeable when multiple composite images are arranged side by side.

[0006] An image processing device according to a first aspect of the present disclosure includes one or more processors and one or more memories for storing instructions to be executed by the one or more processors, wherein the one or more processors acquire an image group including a plurality of images acquired by photographing an object and whose order in a first direction is known, divide the image group into a plurality of image sets, each of which has one or more first overlapping images in each of adjacent image sets in the first direction, combine the images included in the image sets in the first direction to generate a plurality of composite images for each image set, and remove a portion of the first overlapping images for each composite image.

[0007] An image processing device according to a second aspect is the image processing device of the first aspect, wherein the image group may include a plurality of images generated by capturing images of a subject extending in a first direction.

[0008] In the image processing device of the third aspect, in the image processing device of the first or second aspect, one or more processors acquire an image group including a plurality of images whose order in a second direction intersecting the first direction is known, divide the image group into a plurality of image sets, each of which has one or more second overlapping images in each of adjacent image sets in the second direction, combine the images included in the image sets in the second direction to generate a plurality of composite images for each image set, and remove some of the second overlapping images for adjacent composite images in the second direction.

[0009] An image processing device according to a fourth aspect may be the image processing device of the third aspect, wherein the image group includes a plurality of images generated by photographing an object extending in a first direction and a second direction.

[0010] An image processing device according to a fifth aspect is an image processing device according to any one of the first to fourth aspects, wherein one or more processors detect damage in the object to be photographed based on at least one of the image and the composite image, and may use a display device to display the damage detection results on a screen on which the composite image is displayed.

[0011] In an image processing device according to a sixth aspect, in an image processing device according to any one of the first to fifth aspects, one or more processors may use a display device to display a plurality of composite images arranged along a first direction.

[0012] In the image processing device of the seventh aspect, in the image processing device of the sixth aspect, one or more processors may use a display device to display a group of composite images, in which multiple composite images are arranged along a first direction, arranged along a second direction that intersects the first direction.

[0013] An image processing device according to an eighth aspect is an image processing device according to any one of the first to seventh aspects, wherein one or more processors detect damage in the object to be photographed based on at least one of the image and the composite image, and use a display device to display the damage detection results on a screen on which the composite image is displayed, and the display device may switch between displaying the composite image and displaying the damage detection results.

[0014] An image processing method according to a ninth aspect of the present disclosure is an image processing method in which one or more processors provided in a computer acquire an image group including a plurality of images acquired by photographing an object, the order of which in a first direction is known, divide the image group into a plurality of image sets, each of which has one or more first overlapping images in both of adjacent image sets in the first direction, combine the images included in the image sets in the first direction to generate a plurality of composite images for each image set, and remove a portion of the first overlapping image for each composite image.

[0015] A program according to a tenth aspect of the present disclosure is a program that enables a computer to realize the following functions: acquiring an image group including a plurality of images acquired by photographing an object and whose arrangement in a first direction is known; dividing the image group into a plurality of image sets, each of which has one or more first overlapping images in both of adjacent image sets in the first direction; combining the images included in the image sets in the first direction to generate a plurality of composite images for each image set; and removing a portion of the first overlapping image for each composite image.

[0016] FIG. 1 is a schematic diagram of imaging inside a tunnel performed using an imaging system according to an embodiment. FIG. 2 is an explanatory diagram of distortion occurring at the edges of a composite image. FIG. 3 is a diagram showing a specific example of a composite image in which distortion has occurred. FIG. 4 is a schematic diagram showing a group of images whose order is known. FIG. 5 is a schematic diagram showing image set division. FIG. 6 is a schematic diagram showing another example of image set division. FIG. 7 is an explanatory diagram of image synthesis. FIG. 8 is a schematic diagram of edge removal from a composite image. FIG. 9 is a schematic diagram of edge removal from a composite image according to another example. FIG. 10 is a diagram showing a composite image before edge distortion is removed. FIG. 11 is a diagram showing a composite image after edge distortion is removed. FIG. 12 is a diagram showing another example of a composite image before edge distortion is removed. FIG. 13 is a diagram showing another example of a composite image after edge distortion is removed. FIG. 14 is a diagram showing an example of a user interface applied to display a group of composite images. FIG. 15 is a diagram showing an example of a camera applied to an imaging system according to an embodiment. FIG. 16 is a schematic diagram showing a traveling state of the camera unit shown in FIG. 15. Fig. 17 is a functional block diagram showing an example of the electrical configuration of an imaging system according to an embodiment. Fig. 18 is a functional block diagram showing the electrical configuration of an image processing device shown in Fig. 17. Fig. 19 is a block diagram showing the hardware configuration of the image processing device shown in Fig. 18. Fig. 20 is a flowchart showing the procedure of an image processing method according to an embodiment.

[0017] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification, identical components are designated by the same reference numerals, and duplicate descriptions will be omitted where appropriate. Furthermore, when multiple components are listed in the following embodiments, it can be interpreted as including at least one of the multiple components.

[0018] 1 is a schematic diagram of an imaging system according to an embodiment for imaging the interior of a tunnel, which uses a camera unit 50 including five cameras 50A, 50B, 50C, 50D, and 50E to image an inner wall IW of a tunnel TU.

[0019] The photography system moves the camera unit 50 in the longitudinal direction of the tunnel TU, and photographs the inner wall IW of the tunnel TU at each of a plurality of predefined photography positions. The plurality of photography positions defined in the longitudinal direction of the tunnel TU are defined so that edges of images captured at adjacent photography positions overlap. The longitudinal direction of the tunnel TU is the direction penetrating the plane of the paper in FIG. 1 and is an example of a first direction.

[0020] The symbols SA1, SA2, SA3, SA4, and SA5 shown in FIG. 1 represent the respective imaging areas of camera 50A, camera 50B, camera 50C, camera 50D, and camera 50E. For example, adjacent imaging areas in the short-side direction of tunnel TU are defined so that their ends overlap. For example, imaging area SA2 is defined so that its left end in FIG. 1 overlaps with the end of imaging area SA1, and its right end in FIG. 1 overlaps with the end of imaging area SA3. The short-side direction of tunnel TU is a direction perpendicular to the longitudinal direction of tunnel TU and is a direction along the inner wall IW of tunnel TU. The longitudinal direction of tunnel TU described in the embodiment is an example of the second direction.

[0021] FIG. 1 also shows a composite image ICA obtained by combining a plurality of captured images taken by the camera 50B included in the camera unit 50 in the longitudinal direction of the tunnel TU.

[0022] It is not realistic to represent the entirety of a long structure such as a tunnel using a single composite image. Therefore, a method has been proposed in which the structure to be represented is divided into multiple sections, a composite image is generated for each section, and the composite images for the multiple sections are arranged to represent the entire long structure.

[0023] However, when a composite image is generated for a long and narrow object such as a tunnel, distortions are likely to occur, particularly at the edges of the composite image, due to the accumulation of errors and the imbalance of corresponding points with adjacent images. The corresponding points referred to here may also be referred to as constraint points. When multiple composite images are displayed side by side, distortions at the edges of the composite images make the discontinuity between adjacent composite images more noticeable.

[0024] Fig. 2 is an explanatory diagram of distortion occurring at the edges of a composite image. Fig. 3 is a diagram showing a specific example of a composite image in which distortion has occurred. The multiple captured images IMA1 to IMC5 shown in Fig. 2 are components of the composite image ICB shown in Fig. 3.

[0025] Each of the multiple captured images IMA1 to IMC5 shown in Figure 2 was captured and acquired at a combination of five capture positions along the longitudinal direction of the tunnel and three capture positions along the lateral direction of the tunnel. Each of the captured images IMA1 to IMC5 has an overlapping area where it overlaps with an adjacent captured image.

[0026] For example, the photographed image IMB2 has an overlapping area OA1 that overlaps with the adjacent photographed image IMA2 at its edge on the side of the photographed image IMB2. Similarly, the photographed image IMA2 has an overlapping area OA11 that has the same area as the overlapping area OA1 at its edge on the side of the photographed image IMB2.

[0027] Furthermore, photographed image IMB2 has overlapping areas OA2, OA3, and OA4 at the respective ends of photographed images IMB1, IMC2, and IMB3. Photographed images IMB1, IMC2, and IMB3 also have overlapping areas OA12, OA13, and OA14 corresponding to overlapping areas OA2, OA3, and OA4, respectively. Hereinafter, when there is no need to distinguish between photographed images IMA1, etc., photographed images IMA1, etc. will be collectively referred to as photographed images IM. Note that the photographed image IM described in the embodiment is an example of a plurality of images that make up an image group.

[0028] 2, for example, the edge image IMA5 has fewer corresponding points with the adjacent images IM than the central image IMB3, and the corresponding points are unevenly distributed. Specifically, the image IMA5 has three corresponding points with the image IMA4 and five corresponding points with the image IMB5, but has no corresponding points on the opposite side of the image IMA4 or the opposite side of the image IMB5.

[0029] The tip of the arrow line connecting the photographed images IMA5 and IMA4 indicates a corresponding point between the photographed images IMA5 and IMA4. Similarly, the tip of the arrow line connecting the photographed images IMA5 and IMB5 indicates a corresponding point between the photographed images IMA5 and IMB5. The same applies to the arrow line for the photographed image IMB3.

[0030] On the other hand, non-edge photographed images IM such as the central photographed image IMB3 have more corresponding points than edge photographed images IM, and the corresponding points are evenly distributed around the photographed image IM.

[0031] In end images IMA5 and the like, which have fewer corresponding points and a bias in the corresponding points compared to non-end images IM, the effects of distortions caused by calculation errors in the transformation relationship that matches the corresponding points, and distortions caused by differences between the shape model and the actual photographed object, such as differences between the cylindrical model and the actual shape of the tunnel, are clearly visible at the ends of composite image ICB shown in Figure 3. In composite image IC shown in Figure 3, distortions are clear at both ends EN in the longitudinal direction.

[0032] Furthermore, when the distortion at the edge of the composite image is relatively large, not only does this cause the appearance problem of the composite image as described above, but it can also cause relatively large errors in the damage size, etc., that are determined based on the composite image. Relatively large errors in the damage size, etc., can be an obstacle to determining the condition of an object based on the captured image of the object.

[0033] Therefore, even when a composite image is generated for a long and narrow object such as a tunnel, it is preferable that distortion at the edges of the composite image be as small as possible.Furthermore, not only for long and narrow objects such as tunnels, but also for huge structures such as dams, even when composite images are generated for multiple regions and multiple composite images are arranged to represent the entire structure, it is preferable that distortion in each composite image be as small as possible.

[0034] [Explanation of a group of images with a known order] Fig. 4 is a schematic diagram showing a group of images with a known order. The order of the multiple captured images IM shown in the figure is defined using a camera number and a traveling direction number. In Fig. 4, a symbol IM representing the captured image is assigned to any one of the multiple captured images IM.

[0035] The direction in which the travel direction number is specified is the longitudinal direction of the tunnel shown in Fig. 2, which is the direction in which the camera unit 50 shown in Fig. 1 travels. The direction in which the camera number is specified is the lateral direction of the tunnel shown in Fig. 2, which is the direction in which the cameras 50A and the like are lined up.

[0036] The camera number is a unique number assigned to each of the cameras 50A, 50B, 50C, 50D, and 50E shown in Fig. 1 according to their order. For example, the camera 50A may be assigned camera number 1, the camera 50B may be assigned camera number 2, the camera 50C may be assigned camera number 3, the camera 50D may be assigned camera number 4, and the camera 50E may be assigned camera number 5.

[0037] The traveling direction number is a number indicating the photographing position, which is assigned in order from the photographing start position to the photographing end position along the traveling direction of the camera 50A, etc. The maximum value of the traveling direction number is determined according to the total length of the photographed object, the angle of view of the camera 50A, etc. Although FIG. 4 shows examples of photographed images IM with traveling direction numbers 1 to 13, there may also be photographed images IM with traveling direction numbers 14 or higher. The traveling direction number may be assigned based on the measurement results obtained by measuring the position at the time of photographing.

[0038] The multiple captured images IM are given unique file names that use the camera number and the traveling direction number. That is, the multiple captured images IM are each given a file name that allows the order of the images to be determined. An example of a file name is one that uses the format [camera number]_[traveling direction number].

[0039] In addition, information is prepared that indicates the correspondence between the file names of each of the multiple captured images IM and the combinations of camera numbers and traveling direction numbers. An example of the information indicating the correspondence is a correspondence table between file names such as "IMG_0001.extension" and combinations of camera numbers and traveling direction numbers. The correspondence table may be in CSV format.

[0040] [Image Set Division] Fig. 5 is a schematic diagram showing image set division. This figure shows an example in which the multiple captured images IM shown in Fig. 4 are divided into three image sets IS1, IS2, and IS3 in relation to the traveling direction for which the traveling direction number is specified.

[0041] The image sets IS1 and IS2 are adjacent to each other, and one captured image IM at the boundary between them overlaps. In the example shown in Figure 5, the image set IS1 and the image set IS2 have the captured image IM with a traveling direction number of 5 defined as the overlapping image ID.

[0042] Similarly, for image sets IS2 and IS3, the captured image IM with a traveling direction number of 9 is defined as the overlapping image ID. Note that the number of image sets IS, the number of captured images IM included in one image set IS, and the number of overlapping image IDs included in each image set IS are not limited to the example shown in the figure, and can be defined as appropriate.

[0043] For example, if the number of image sets IS is 3 and the number of overlapping image IDs is 2, the image sets IS and overlapping image IDs are defined as follows: Image set IS1 applies to captured images IM with traveling direction numbers 1 to 5. Image set IS2 applies to captured images IM with traveling direction numbers 4 to 8. Image set IS3 applies to captured images IM with traveling direction numbers 7 to 11.

[0044] The overlapping image IDs at the boundary between image set IS1 and image set IS2 are the captured image IM with a traveling direction number of 4 and the captured image IM with a traveling direction number of 5. The overlapping image IDs at the boundary between image set IS2 and image set IS3 are the captured image IM with a traveling direction number of 7 and the captured image IM with a traveling direction number of 8.

[0045] When the number of image sets IS is 3 and the number of overlapping image IDs is 3, the image sets IS and overlapping image IDs are defined as follows: Image set IS1 applies to captured images IM with traveling direction numbers 1 to 7. Image set IS2 applies to captured images IM with traveling direction numbers 5 to 11. Image set IS3 applies to captured images IM with traveling direction numbers 9 to 15.

[0046] For the overlapping image ID at the boundary between image set IS1 and image set IS2, the captured images IM with travel direction numbers 5 to 7 are applied. For the overlapping image ID at the boundary between image set IS2 and image set IS3, the captured images IM with travel direction numbers 9 to 11 are applied.

[0047] In this way, when an image set IS having multiple overlapping image IDs is defined, there is more room to remove distortion if it is noticeable in the composite image IC generated for each image set IS, which is preferable.

[0048] 6 is a schematic diagram showing another example of image set division, in which a plurality of captured images IM, the order of which is defined in a first traveling direction and a second traveling direction that are orthogonal to each other, are divided into image sets in each of the first traveling direction and the second traveling direction.

[0049] The first traveling direction or the second traveling direction shown in Fig. 6 is an example of the first direction. Furthermore, of the first traveling direction and the second traveling direction, a direction different from the first direction is an example of the second direction. The second traveling direction perpendicular to the first traveling direction shown in Fig. 6 or the first traveling direction perpendicular to the second traveling direction is an example of the second direction intersecting the first direction.

[0050] The image set division shown in Fig. 6 is applied when a camera is scanned over a photographic object in a first traveling direction and a second traveling direction, photographing is performed at multiple photographing positions, and multiple photographed images IM are acquired. An example of a photographic object is a structure such as a dam, which requires the camera to move two-dimensionally when photographing the entire structure.

[0051] Figure 6 shows an example in which multiple captured images IM, the order of which is known for each of the first and second traveling directions, are divided into six image sets IS11, IS12, IS13, IS21, IS22, and IS23, and one overlapping image ID1 or ID2 is defined for the boundary between adjacent image sets IS.

[0052] The number of overlapping image IDs 1 and 2 may be 2 or more. The number of overlapping image IDs 1 in the first traveling direction and the number of overlapping image IDs 2 in the second traveling direction may be the same or different.

[0053] [Image Composition] Fig. 7 is an explanatory diagram of image composition. In image composition, composition parameters are determined from the results of corresponding point matching between adjacent captured images IM, and one composite image is generated for the multiple captured images IM included in the image set IS. Furthermore, the composite images for each image set IS are combined to generate a group of composite images corresponding to the entire measurement object. Note that each image set IS described in the embodiment is an example of each image set.

[0054] The synthesis parameters include pose parameters of the shape model, pose parameters for each camera for each of the multiple captured images IM, and lens distortion parameters for each camera for each of the multiple captured images IM.

[0055] When the shape model is a planar model, the orientation parameters of the shape model include a rotation matrix and a translation vector. When the shape model is a cylindrical model, the orientation parameters of the shape model include a rotation matrix, a translation vector, and the radius of the cylinder. When a planar model is applied, the synthesis parameters may be a projective transformation matrix of each captured image IM.

[0056] A non-rigid transformation may be applied, which includes scaling as a pose parameter, such as an affine transformation.

[0057] The captured image IM is projected onto the shape model based on the determined synthesis parameters, and a synthetic image ICC is generated. If the shape model is a cylindrical model, planar development is further performed.

[0058] 7 illustrates a three-dimensional coordinate system having mutually orthogonal Xs, Ys, and Zs axes as a coordinate system applied to the pose parameters of the shape model. Also, FIG. 7 illustrates a three-dimensional coordinate system having mutually orthogonal Xi, Yi, and Zi axes as a coordinate system applied to the pose parameters for each camera CAM and the lens distortion parameters for each camera CAM. Note that i is a sequential number starting from 1 that indicates the shooting position. The Xs and Xi axes illustrated in FIG. 7 correspond to the longitudinal direction of the tunnel illustrated in FIG. 2. The Ys and Yi axes illustrated in FIG. 7 correspond to the lateral direction of the tunnel illustrated in FIG. 2.

[0059] In the image synthesis, synthesis and optimization may be repeated for some of the multiple captured images IM included in the image set IS, and all of the multiple captured images IM included in the image set IS may be synthesized.

[0060] The composite image IC may be generated by combining multiple captured images IM from each camera 50A, etc. in the Xs direction to generate a composite image element, and then combining the composite image elements from each camera 50A, etc. in the Ys direction.

[0061] [Edge Removal of Composite Image] Fig. 8 is a schematic diagram of edge removal of a composite image. Fig. 8 schematically illustrates the correspondence between image set IS1 and composite image IC1, the correspondence between image set IS2 and composite image IC2, and the correspondence between image set IS3 and composite image IC3. Image set IS1, image set IS2, and image set IS3 illustrated in the upper part of Fig. 8 are the same as image set IS1, image set IS2, and image set IS3 illustrated in the lower part of Fig. 5.

[0062] The captured images IM included in the image set IS1 are combined to generate a composite image IC1, the captured images IM included in the image set IS2 are combined to generate a composite image IC2, and the captured images IM included in the image set IS3 are combined to generate a composite image IC3.

[0063] Although actual composite images IC1, IC2, and IC3 each have distortion, in FIG. 8, composite images IC1, IC2, and IC3 are each represented as a rectangular shape.

[0064] At the ends of each of composite images IC1, IC2, and IC3, a portion of the area corresponding to the overlapping image ID in the image set IS is removed. The removal area CA removed from composite image IC1, etc. is schematically illustrated using dashed lines. Figure 8 shows an example in which half of the area corresponding to the overlapping image ID is removed for each of composite images IC1, IC2, and IC3. The removal area CA may be greater than 0 percent and less than 50 percent of the area corresponding to the overlapping image ID, or greater than 50 percent and less than 100 percent of the area corresponding to the overlapping image ID.

[0065] Composite images IC1 and IC2 illustrate a position P15 corresponding to the captured image IM with camera number 1 and traveling direction number 5. Composite images IC2 and IC3 also illustrate a position P19 corresponding to the captured image IM with camera number 1 and traveling direction number 9.

[0066] 9 is a schematic diagram showing edge removal of a composite image according to another example, which shows how edge removal is performed on composite images IC11, IC12, and IC13 when the number of overlapping image IDs set in each of image sets IS11, IS12, and IS13 is two.

[0067] 9, the captured image IM with the traveling direction number 4 and the captured image IM with the traveling direction number 5 are defined as overlapping image IDs. In the image set IS12, the captured image IM with the traveling direction number 4 and the captured image IM with the traveling direction number 5 are defined as overlapping image IDs, and the captured image IM with the traveling direction number 7 and the captured image IM with the traveling direction number 8 are defined as overlapping image IDs. In the image set IS13, the captured image IM with the traveling direction number 7 and the captured image IM with the traveling direction number 8 are defined as overlapping image IDs.

[0068] The composite image IC11 is generated by combining the multiple captured images IM included in the image set IS11. Similarly, the composite image IC12 is generated by combining the multiple captured images IM included in the image set IS12, and the composite image IC13 is generated by combining the multiple captured images IM included in the image set IS13.

[0069] A removal area CA that is part of the overlapping image ID of the image set IS11 is removed from the composite image IC11. Specifically, from the composite image IC11, of a position P14 corresponding to the captured image IM whose traveling direction number is 4 and a position P15 corresponding to the captured image IM whose traveling direction number is 5, a part of the position P14 and a part of the position P15 are removed as the removal area CA.

[0070] In the composite image IC12, the removal area CA that is part of the overlapping image ID of the image set IS12 is removed, and the removal area CA that is part of the overlapping image ID of the image set IS12 is also removed.

[0071] Specifically, in the composite image IC12, of position P14 corresponding to the photographed image IM with traveling direction number 4 and position P15 corresponding to the photographed image IM with traveling direction number 5, a part of position P14 and a part of position P15 are removed as removal areas CA. Also, in the composite image IC12, of position P17 corresponding to the photographed image IM with traveling direction number 7 and position P18 corresponding to the photographed image IM with traveling direction number 8, a part of position P17 and a part of position P18 are removed as removal areas CA.

[0072] A removal area CA that is part of the overlapping image ID of the image set IS13 is removed from the composite image IC13. Specifically, from the composite image IC13, of a position P17 corresponding to the captured image IM whose traveling direction number is 7 and a position P18 corresponding to the captured image IM whose traveling direction number is 8, a part of the position P17 and a part of the position P18 are removed as the removal area CA.

[0073] Fig. 10 shows a composite image before the distortion at the ends is removed. In the composite image ICD shown in Fig. 10, distortion DI occurs at both one end EN and the other end EN in the longitudinal direction of the tunnel, similar to the composite image ICB shown in Fig. 3.

[0074] Fig. 11 is a diagram showing a composite image after the distortion at the ends has been removed. In the composite image ICE shown in Fig. 11, the distortion DI at one end EN and the other end EN in the composite image ICD shown in Fig. 10 has been removed. One end ENA and the other end ENB of the composite image ICE shown in Fig. 11 are straight sides that are parallel to the short direction of the tunnel.

[0075] 12 is a diagram showing another example of a composite image before the distortion at the edges is removed, in which the number of captured images IM constituting the composite image ICF is greater than that of the composite image ICD shown in FIG.

[0076] Fig. 13 is a diagram showing another example of a composite image after the distortion at the ends has been removed. In the composite image ICG shown in Fig. 13, the distortion DI at one end EN and the other end EN in the composite image ICF shown in Fig. 12 has been removed, similar to the composite image ICE shown in Fig. 12. One end ENA and the other end ENB of the composite image ICG shown in Fig. 13 are linear sides parallel to the short direction of the tunnel.

[0077] A plurality of composite images ICE in which the distortion DI of the end EN shown in Figure 11 has been trimmed and a plurality of composite images ICG in which the distortion DI of the end EN shown in Figure 13 has been trimmed are generated corresponding to the total length of the tunnel, which is the object to be photographed.

[0078] The process of combining a plurality of composite images ICE, etc., is the same as the process applied to the captured image IM in generating the composite image ICD, etc. shown in FIG. 10, such as the process of matching corresponding points of adjacent composite images.

[0079] [User Interface Applied to Display of Composite Image Group] Figure 14 is a diagram showing an example of a user interface applied to display of a composite image group. This figure shows an example of a display screen DS on which a plurality of composite images IC are arranged and displayed as a composite image group GIC. In the composite image group GIC shown in Figure 14, the positions of the overlapping image IDs of adjacent composite images IC among the plurality of composite images IC are aligned, and if adjacent composite images IC have overlapping areas, the overlapping areas are overlapped. On the other hand, if adjacent composite images IC do not have overlapping areas, the adjacent composite images IC are arranged without overlapping.

[0080] The display screen DS is configured to be freely switchable between a normal display screen NDS and an enlarged display screen EDS. The switching between the normal display screen NDS and the enlarged display screen EDS is performed based on a user instruction.

[0081] 14 shows an example in which ten composite images IC corresponding to 10 meters in the longitudinal direction of a tunnel to be photographed, each of which has a total length of several kilometers, are arranged in the longitudinal direction of the tunnel. In other words, the normal display screen NDS displays 100 meters, which is a portion of the longitudinal direction of a tunnel having a total length of several kilometers.

[0082] 14 displays one composite image IC included in the ten composite images IC displayed on the normal display screen NDS. The enlarged display screen EDS shown in the same figure is obtained by reducing the normal display screen NDS and superimposing it on the upper right corner of the enlarged display screen EDS.

[0083] The display screen DS includes a first area AR1, a second area AR2, and a third area AR3. In the first area AR1, the composite image group GIC is displayed on the normal display screen NDS, and a portion of the composite image group GIC displayed on the normal display screen NDS, such as one composite image IC, is displayed in an enlarged form on the enlarged display screen EDS.

[0084] The second area AR2 displays a toolbar containing icons representing various tools. Examples of the various tools include zoom in and zoom out. The third area AR3 displays a position bar. The position bar displays a reduced version of the entire composite image group GIC. The position bar includes a mark PM that indicates the range of the entire composite image group GIC to be displayed in the first area AR1. Figure 14 shows an example in which a rectangular shape is applied as the mark PM.

[0085] The position bar displayed on the normal display screen NDS includes marks PM representing the area of ​​ten composite images IC, and the position bar displayed on the enlarged display screen EDS includes a mark PM representing the area of ​​one composite image IC.

[0086] For the first and second travel directions shown in Figure 6, when multiple captured images IM are divided into multiple image sets IS and a composite image is generated for each of the multiple image sets IS, multiple composite images IC for each image set IS are arranged for the first travel direction, and multiple composite images IC for each image set IS are arranged for the second travel direction.

[0087] When multiple composite images IC are arranged in the first traveling direction, the positions of the overlapping image IDs in the first traveling direction are aligned, and the multiple composite images IC are arranged so that overlapping image ID1 overlaps. Similarly, when multiple composite images IC are arranged in the second traveling direction, the positions of the overlapping image ID2 in the second traveling direction are aligned, and the multiple composite images IC are arranged so that overlapping image ID2 overlaps. Note that the overlapping image ID1 in the first traveling direction described in the embodiment is an example of a first overlapping image, and the overlapping image ID2 in the second traveling direction described in the embodiment is an example of a second overlapping image.

[0088] When any two points on the composite image IC are designated on the enlarged display screen EDS, the length of the designated two points is measured and the measurement result is displayed. For example, when points P1 and P2 are designated on the composite image IC on the enlarged display screen EDS shown in Figure 14, the designated points P1 and P2 are displayed, and a line segment LS connecting points P1 and P2 is displayed.

[0089] Additionally, the enlarged display screen EDS displays the measurement values ​​of the length L, width W, and height H of the line segment LS as the measurement results of points P1 and P2. The width W of the line segment LS is the component of the length of the line segment LS in the longitudinal direction of the tunnel, and the height H of the line segment LS is the component of the length of the line segment LS in the lateral direction of the tunnel.

[0090] Figure 14 illustrates an example in which the measurement results of points P1 and P2 are superimposed on the composite image IC, but the measurement results of points P1 and P2 may also be displayed in an area different from the first area AR1, the second area AR2, and the third area AR3.

[0091] Measurement of any two points specified on the composite image IC may be applied to measuring damage to the photographed object. For example, if the photographed object is the inner wall IW of the tunnel TU, the positions of both ends of damage such as scratches, chips, breaks, and discoloration on the inner wall IW of the tunnel TU may be specified to measure the size of the damage. Calculations for any two points specified on the composite image IC may be performed using the photographed images IM that constitute the composite image IC.

[0092] The switching between the normal display screen NDS described with reference to Fig. 14 and the enlarged display screen EDS on which the measurement results of any two points on the composite image IC are displayed is an example of switching between displaying the composite image and displaying the damage detection results. Also, the specification of points P1 and P2 shown in Fig. 14 is an example of damage detection on the measurement object, and the measurement results of points P1 and P2 are an example of damage detection results.

[0093] As described above, the image processing device may manually detect damage to the inner wall IW of the tunnel TU from the composite image IC, or may automatically detect damage to the inner wall IW of the tunnel TU from the captured image IM or the composite image IC. The image processing device may also display the damage to the inner wall IW of the tunnel TU.

[0094] The image processing device may switch between superimposing the results of damage detection on the inner wall IW of the tunnel TU on the composite image IC, or displaying the composite image IC without displaying the results of damage detection on the inner wall IW of the tunnel TU.

[0095] [Configuration example of imaging system according to embodiment] [Configuration example of camera unit] Fig. 15 is a diagram showing an example of a camera applied to the imaging system according to the embodiment. Camera unit 50 includes camera 50A, camera 50B, camera 50C, camera 50D, and camera 50E. Cameras 50A, 50B, 50C, 50D, and 50E are arranged in an arc equidistant from a reference position O of camera support base 52. Camera support base 52 is attached to a dolly 54.

[0096] The optical axes of cameras 50A, 50B, 50C, 50D, and 50E are arranged radially from reference position O of camera support base 52. The cameras 50A, etc. each have a different shooting direction, and simultaneously capture images of the inner wall IW of tunnel TU shown in FIG.

[0097] The camera support base 52 includes an illumination device, a range finder, and a position meter. The illumination device, range finder, and position meter are not shown in Fig. 15. The illumination device, range finder, and position meter are denoted by reference numerals 55, 56, and 57, respectively, and are shown in Fig. 17. The illumination device irradiates illumination light onto the shooting area of ​​the camera 50A, etc. The illumination device may be provided individually for each camera 50A, etc.

[0098] The range finder measures the distance from the position of the range finder to the inner wall IW of the tunnel TU. A laser range finder that uses laser light is applied as the range finder. An example of a laser range finder is LiDAR. LiDAR is an abbreviation for Light Detection And Ranging. The laser range finder rotates the laser light emitted from the measurement head, irradiates the laser light onto the inner wall IW of the tunnel TU, and measures the distance between the measurement head and the inner wall IW of the tunnel TU.

[0099] The distance meter preferably converts the measured distance into a distance in the same direction as the optical axis of the camera 50A, etc., based on the angle between the optical axis of the camera 50A, etc. and the laser light. The distance meter may also irradiate the laser light in the same direction as the optical axis of the camera 50A, etc., without rotating the laser light.

[0100] The positioning meter outputs a positioning signal that indicates the position in the traveling direction of the carriage 54. The longitudinal direction of the tunnel shown in Fig. 2 is applied as the traveling direction of the carriage 54. That is, the positioning meter measures the traveling distance from the travel start position of the carriage 54, and generates a positioning signal that indicates the position in the traveling direction of the carriage 54 based on the measurement result.

[0101] Fig. 16 is a schematic diagram showing the traveling state of the camera unit shown in Fig. 15. Fig. 16 shows a state in which the camera unit 50 travels along the traveling direction and captures an image of the inner wall IW of the tunnel TU using the camera 50A etc. The camera unit 50 stops at a predetermined photographing position along the traveling direction to photograph the inner wall IW of the tunnel TU.

[0102] 17 is a functional block diagram showing an example of the electrical configuration of the imaging system according to the embodiment. The imaging system 10 includes an image processing device 20, an imaging control device 30, and a camera unit 50. The camera unit 50 uses a camera 50A or the like to capture an image of the inner wall IW of the tunnel TU and generate a captured image.

[0103] The imaging control device 30 is a control device that controls the camera unit 50, and operates the camera unit 50 to capture an image of the inner wall IW of the tunnel TU using the camera 50A or the like, and generates a captured image. The imaging control device 30 transmits the captured image to the image processing device 20. A computer is applied to the imaging control device 30.

[0104] The image processing device 20 generates a plurality of composite images IC using the photographed images IM of the inner wall IW of the tunnel TU transmitted from the photographing control device 30. The image processing device 20 uses the display 22 to display a composite image group GIC in which the plurality of composite images IC are arranged.

[0105] The image processing device 20 measures the inner wall IW of the tunnel TU using the composite image IC. An example of measuring the distance between two points on the inner wall IW of the tunnel TU is illustrated in Fig. 14. When the image processing device 20 acquires signals that define the positions of the two points transmitted from the input device 24, it measures the distance between the two points.

[0106] The imaging control device 30 includes an image acquisition unit 32. The image acquisition unit 32 acquires an electrical signal representing a captured image IM transmitted from the camera 50A, etc. The captured image IM acquired using the image acquisition unit 32 is transmitted to the image processing device 20.

[0107] The imaging control device 30 includes a camera control unit 34. The camera control unit 34 controls the operation of each of the cameras 50A, etc. when imaging the inner wall IW of the tunnel TU. The camera control unit 34 transmits an imaging signal indicating the start of imaging to the cameras 50A, etc. at a specified imaging timing. Each of the cameras 50A, etc. images the inner wall IW of the tunnel TU based on the imaging signal.

[0108] The camera control unit 34 transmits information on the distance from each of the cameras 50A, etc. to the inner wall IW of the tunnel TU, measured using the distance meter 56, to each of the cameras 50A, etc. The cameras 50A, etc. can perform focusing based on the distance to the inner wall IW of the tunnel TU.

[0109] The camera control unit 34 sets the shooting conditions for each of the cameras 50A, etc. For example, the camera control unit 34 sets the shooting resolution as the shooting condition for each of the cameras 50A, etc.

[0110] The imaging control device 30 includes an illumination control unit 36. The illumination control unit 36 ​​sets illumination conditions to be applied to the illumination device 55 and controls the operation of the illumination device 55 based on the illumination conditions. An example of the illumination conditions is the irradiation intensity of the illumination light.

[0111] The photography control device 30 includes a ranging information acquisition unit 38. The ranging information acquisition unit 38 acquires ranging information, which indicates the distance from each of the cameras 50A, etc. to the inner wall IW of the tunnel TU, and is transmitted from a rangefinder 56. The camera control unit 34 transmits the ranging information acquired using the ranging information acquisition unit 38 to each of the cameras 50A, etc.

[0112] The imaging control device 30 includes a positioning information acquisition unit 40. The positioning information acquisition unit 40 acquires positioning information transmitted from a positioning meter 57, the positioning information indicating the position of the camera unit 50 in the traveling direction of the camera unit 50. Based on the positioning information acquired using the positioning information acquisition unit 40, the camera control unit 34 uses the camera 50A or the like to capture an image of the inner wall IW of the tunnel TU at a specified measurement position.

[0113] The imaging control device 30 includes a dolly control unit 42. The dolly control unit 42 sets the travel conditions of the dolly 54 and controls the travel of the dolly 54 based on the travel conditions of the dolly 54. The travel conditions of the dolly 54 include the travel speed and stopping timing of the dolly 54, etc.

[0114] [Configuration example of image processing device] Fig. 18 is a functional block diagram showing an example of the electrical configuration of the image processing device shown in Fig. 17. The image processing device 20 includes an image group acquisition unit 60. The image group acquisition unit 60 acquires a plurality of captured images obtained by measuring the inner wall IW of the tunnel TU, the captured images having a known order. The captured images having a known order are shown in Fig. 4 as a plurality of captured images IM whose order is defined using a travel direction number and a camera number.

[0115] The image processing device 20 includes an image set dividing unit 62. The image set dividing unit 62 divides a plurality of captured images, the order of which is known and acquired using the image group acquiring unit 60, into a plurality of image sets IS shown in Fig. 5. Note that the image set IS is a general term for the image set IS1 and the like shown in Fig. 5.

[0116] The image processing device 20 includes an image synthesis unit 64. The image synthesis unit 64 performs synthesis processing for each image set IS defined using the image set division unit 62 to synthesize the multiple captured images IM included in each of the multiple image sets IS, thereby generating a synthesized image IC. As an example of the synthesized image IC, a synthesized image ICD is illustrated in FIG. 10.

[0117] The image processing device 20 includes an edge removal unit 66. The edge removal unit 66 removes distortion DI from the edge EN of the composite image IC for each image set IS. As an example of the composite image IC from which the distortion DI from the edge EN has been removed, a composite image ICE is shown in FIG. 11 .

[0118] The image processing device 20 includes a display control unit 68. The display control unit 68 generates a display signal representing a composite image group GIC in which a plurality of composite images IC are arranged, and transmits the generated display signal to the display 22 shown in Fig. 17. The composite image group GIC is displayed on the display 22. An example of the display of the composite image group GIC is illustrated in Fig. 14.

[0119] The display control unit 68 generates a display signal representing a portion of the composite image group GIC for enlarged display, and transmits the generated display signal to the display 22. The portion of the composite image group GIC is enlarged and displayed on the display 22. An example of an enlarged display of a portion of the composite image group GIC is shown in FIG. 14. The display 22 shown in FIG. 18 is an example of a display device.

[0120] The image processing device 20 includes a processing condition acquisition unit 70. The processing condition acquisition unit 70 acquires processing conditions to be applied to various processes. The processing condition acquisition unit 70 sets processing conditions, such as the number of image sets, the number of captured images IM included in each image set, and the number of overlapping images, in the image set division process performed using the image set division unit 62.

[0121] The processing condition acquisition unit 70 sets processing conditions to be applied to the generation of a composite image IC for each image set IS, which is performed using the image synthesis unit 64. The processing condition acquisition unit 70 sets processing conditions for edge removal processing to remove distortion DI for each composite image IC, which is performed using the edge removal unit 66. The processing condition acquisition unit 70 sets processing conditions for the generation processing of a composite image group GIC, which is performed using the composite image group generation unit 67.

[0122] Fig. 19 is a block diagram showing the hardware configuration of the image processing device shown in Fig. 18. The image processing device 20 is implemented by a computer. The computer may be a personal computer or a workstation. The computer may also be a virtual machine.

[0123] The image processing device 20 includes a processor 102, a computer-readable medium 104, a communication interface 106, an input / output interface 108, and a bus 110. The processor 102 is connected to the computer-readable medium 104, the communication interface 106, and the input / output interface 108 via the bus 110. The display 22 and the input device 24 are connected to the image processing device 20 via the input / output interface 108.

[0124] The image processing device 20 includes one or more processors 102 and one or more memories. The processor 102 executes various programs stored in the memory of a computer-readable medium 104 to realize various functions of the image processing device 20.

[0125] The processor 102 includes a CPU. The processor 102 may also include a GPU. The processor 102 is connected to a computer-readable medium 104, a communication interface 106, and an input / output interface 108 via a bus 110. Note that CPU is an abbreviation for Central Processing Unit, and GPU is an abbreviation for Graphics Processing Unit.

[0126] The computer-readable medium 104 may include a memory serving as a primary storage device and a storage device serving as an auxiliary storage device. The computer-readable medium 104 may include a semiconductor memory, a hard disk drive, a solid-state drive, etc. The computer-readable medium 104 may also include any combination of multiple devices.

[0127] A hard disk drive may be referred to as an HDD, which is an abbreviation of the English term Hard Disk Drive, and a solid state drive may be referred to as an SSD, which is an abbreviation of the English term Solid State Drive.

[0128] Various programs stored in the memory of the computer-readable medium 104 include one or more instructions. Various data, various parameters, etc. are stored in the computer-readable medium 104. Note that the term "program" is synonymous with the term "software."

[0129] The hardware structure of the processor 102 is various processors as follows: The various processors include a CPU, which is a general-purpose processor that executes software and functions as various functional units, a GPU, which is a processor specialized for image processing, a PLD (Programmable Logic Device), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor having a circuit configuration designed specifically for executing specific processing such as an ASIC (Application Specific Integrated Circuit).

[0130] The hardware configuration of the image processing device shown in FIG. 19 can also be applied to a computer that functions as the imaging control device 30 shown in FIG.

[0131] [Procedure of Image Processing Method According to Embodiment] Fig. 20 is a flowchart showing the procedure of the image processing method according to the embodiment. In the image group acquisition step S10, the image group acquisition unit 60 shown in Fig. 18 acquires a plurality of captured images IM whose arrangement order is known. After the image group acquisition step S10, the process proceeds to the processing condition setting step S12.

[0132] In the processing condition setting step S12, various processing conditions are set that are applied to the image set dividing step S14, the image combining step S16, and the edge removing step S18. After the processing condition setting step S12, the process proceeds to the image set dividing step S14.

[0133] In the image set dividing step S14, the image set dividing unit 62 divides the plurality of captured images IM, the order of which is known and which have been acquired using the image group acquiring unit 60, into a plurality of image sets IS. After the image set dividing step S14, the process proceeds to the image combining step S16.

[0134] In the image synthesis step S16, the image synthesis unit 64 generates a synthesized image IC for each of the plurality of image sets IS defined in the image set division step S14. After the image set division step S14, the process proceeds to an edge removal step S18.

[0135] In the edge removal step S18, the edge removal unit 66 removes distortion DI from the edge EN of each of the multiple composite images IC. After the edge removal step S18, the process proceeds to the display signal generation step S20.

[0136] In the display signal generation step S20, the display control unit 68 generates a display signal for a composite image group GIC in which a plurality of composite images IC are arranged. After the display signal generation step S20, the process proceeds to a display signal output step S22. In the display signal output step S22, the display control unit 68 outputs a display signal for the composite image group GIC to the display 22. The composite image group GIC is displayed on the display 22.

[0137] An enlarged display determination step S24 is executed while the composite image group GIC is being displayed on the display 22. In the enlarged display determination step S24, the display control unit 68 determines whether or not to acquire a signal indicating switching to enlarged display.

[0138] In the enlarged display determination step S24, if the display control unit 68 does not receive a signal indicating switching to enlarged display, the determination is No. If the determination is No, the process proceeds to the end determination step S26.

[0139] In the termination determination step S26, the image processing device 20 determines whether or not to terminate image processing. If the image processing device 20 determines to continue image processing in the termination determination step S26, the determination is No. If the determination is No, the process proceeds to the image group acquisition step S10, and each step from the image group acquisition step S10 to the termination determination step S26 is repeatedly executed until the determination is Yes in the termination determination step S26.

[0140] On the other hand, in the termination determination step S26, if the image processing device 20 determines that the image processing is to be terminated, the determination is Yes. If the determination is Yes, a specified termination process is performed, and the procedure of the image processing method is terminated.

[0141] In the enlarged display determination step S24, if the display control unit 68 acquires a signal indicating switching to enlarged display, the determination is Yes. If the determination is Yes, the process proceeds to an enlarged display signal generation step S28.

[0142] In the enlargement display signal generating step S28, the display control unit 68 generates an enlargement display signal that enlarges a portion of the composite image group GIC. For example, in the enlargement display signal generating step S28, an enlargement display signal that represents the enlarged display screen EDS shown in Fig. 14 is generated. After the enlargement display signal generating step S28, the process proceeds to an enlargement signal output step S30.

[0143] In the enlargement signal output step S30, the display control unit 68 outputs the enlargement display signal generated in the enlargement display signal generation step S28 to the display 22. For example, an enlarged display screen EDS is displayed on the display 22. After the enlargement signal output step S30, the process proceeds to the termination determination step S26.

[0144] When the enlarged display is performed, a measurement step of measuring the measurement object may be executed. The measurement step may include a measurement point designation step of designating measurement points, a calculation step of performing calculations based on the designated measurement points to derive measurement results, and a measurement result display step of displaying the measurement results.

[0145] [Operational Effects of the Embodiments] The image processing device, image processing method, and imaging system according to the embodiments can achieve the following operational effects.

[0146] [1] An image group including multiple captured images IM whose arrangement in the traveling direction of photographing the object is known is acquired. The multiple captured images IM are divided into multiple image sets each including multiple captured images IM, with one or more overlapping image IDs in both adjacent image sets in the traveling direction. The multiple captured images IM included in each of the multiple image sets are combined in the traveling direction to generate a composite image IC. Distortion DI at the end EN of each composite image IC is removed.

[0147] As a result, when a plurality of composite images IC are arranged in the direction of travel as a composite image group, discontinuity in the composite image group caused by distortion DI occurring at the end EN of each composite image IC in the direction of travel is not noticeable.

[0148] [2] A plurality of captured images IM whose arrangement in the direction in which the object to be captured is known can be generated by moving a camera along the direction in which the object to be captured extends and capturing images of the object to be captured using the camera at a plurality of capturing positions.

[0149] [3] An image group including multiple captured images IM whose arrangement in a camera direction perpendicular to the direction in which the object is photographed is acquired. The multiple captured images IM are an image set including multiple captured images IM, and are divided into multiple image sets in which one or more overlapping image IDs are included in both adjacent image sets in the camera direction. The multiple captured images IM included in each of the multiple image sets are composited in the camera direction to generate a composite image IC. Distortion DI at the end EN of each composite image IC is removed.

[0150] As a result, when a plurality of composite images IC are arranged in the camera direction as a composite image group, discontinuity in the composite image group caused by distortion DI occurring at the end EN of each composite image IC in the camera direction is not noticeable.

[0151] [4] An image group including a plurality of captured images IM whose arrangement order in the camera direction is known can be generated by capturing images of a subject in the camera direction using a plurality of cameras arranged along the camera direction.

[0152] [5] An image group including multiple captured images IM whose order in the camera direction is known can be generated by moving a camera along the camera direction and capturing images of the subject using the camera at multiple capturing positions along the camera direction.

[0153] [6] A composite image group GIC in which a plurality of composite images IC are arranged along the traveling direction, and a composite image group GIC in which a plurality of composite images IC are arranged along the camera direction are displayed on the display 22. This visualizes the photographing results of the measurement object.

[0154] [7] Any two points on the composite image IC are designated on the enlarged display screen EDS, and measurement of the designated two points is performed. This allows measurement results to be obtained in which the influence of distortion DI at the end EN in the composite image IC is mitigated when measuring damage by designating two points included in the damage of the measurement object.

[0155] [8] The measurement results of the two specified points are displayed on the enlarged display screen EDS, thereby visualizing the measurement results.

[0156] The above-described embodiments of the present invention may be modified, added, or deleted as appropriate within the scope of the spirit of the present invention. The present invention is not limited to the above-described embodiments, and many modifications may be made by a person skilled in the art within the technical concept of the present invention. Furthermore, the embodiments, modifications, and applications may be implemented in appropriate combinations.

[0157] DESCRIPTION OF SYMBOLS 10 Photography system 20 Image processing device 22 Display 24 Input device 30 Photography control device 32 Image acquisition unit 34 Camera control unit 36 ​​Lighting control unit 38 Distance measurement information acquisition unit 40 Positioning information acquisition unit 42 Cart control unit 50 Camera unit 50A Camera 50B Camera 50C Camera 50D Camera 50E Camera 52 Camera support stand 54 Cart 55 Lighting device 56 Distance measurement device 57 Positioning device 60 Image acquisition unit 62 Image set division unit 64 Image synthesis unit 66 Edge removal unit 68 Display control unit 70 Processing condition acquisition unit 102 Processor 104 Computer readable medium 106 Communication interface 108 Input / output interface 110 Bus AR1 First area AR2 Second area AR3 Third area CA Removal area CAM Camera DI Distortion DS Display screen EDS Enlarged display screen EN End ENA One end ENB Other end GIC Composite image group IC1 Composite image IC2 Composite image IC3 Composite image IC11 Composite image IC12 Composite image IC13 Composite image ICA Composite image ICB Composite image ICC Composite image ICD Composite image ICE Composite image ICF Composite image ICG Composite image ID Overlapping image ID1 Overlapping image ID2 Overlapping image IM Photographed image IMA1 Photographed image IMA2 Photographed image IMA3 Photographed image IMA4 Photographed image IMA5 Photographed image IMB1 Photographed image IMB2 Photographed image IMB3 Photographed image IMB4 Photographed image IMB5 Photographed image IMC1 Photographed image IMC2 Photographed image IMC3 Captured image IMC4 Captured image IMC5 Captured image IS1 Image set IS2 Image set IS3 Image set IS11 Image set IS12 Image set IS13 Image set IS21 Image set IS22 Image set IS23 Image set IW Inner wall LS Line segment NDS Normal display screen O Reference position OA1 Overlapped area OA2 Overlapped area OA3 Overlapped area OA4 Overlapped area OA11 Overlapped area OA12 Overlapped area OA13 Overlapped area OA14 Overlapped area P1 Point P2 Point P14 Position P15 Position P17 Position P18 Position P19 Position PM Mark S10 to S30 Each step of the image processing method

Claims

1. An image processing device comprising: one or more processors; and one or more memories for storing instructions to be executed by the one or more processors, wherein the one or more processors acquire an image group including a plurality of images acquired by photographing an object and whose arrangement in a first direction is known; divide the image group into a plurality of image sets, each of which has one or more first overlapping images in adjacent image sets in the first direction; composite the images included in the image sets in the first direction to generate a plurality of composite images for each image set; and remove a portion of the first overlapping images for each composite image.

2. The image processing device according to claim 1, wherein the image group includes a plurality of images generated by photographing the object extending in the first direction.

3. The image processing device described in claim 1, wherein the one or more processors: acquire the image group including a plurality of the images whose order in a second direction intersecting the first direction is known; divide the image group into a plurality of image sets having one or more second overlapping images in both of the image sets adjacent to each other in the second direction; combine the images included in the image sets in the second direction to generate a plurality of composite images for each of the image sets; and remove a portion of the second overlapping images for the composite images adjacent to each other in the second direction.

4. The image processing device according to claim 3, wherein the image group includes a plurality of images generated by photographing the object extending in the first direction and the second direction.

5. The image processing device according to claim 1, wherein the one or more processors detect damage in the photographed object based on at least one of the image and the composite image, and use a display device to display the damage detection result on a screen on which the composite image is displayed.

6. The image processing device according to claim 1, wherein the one or more processors cause a display device to display a plurality of the composite images side-by-side along a first direction.

7. The image processing device according to claim 6, wherein the one or more processors use the display device to display a group of composite images in which a plurality of the composite images are arranged along a first direction, the composite images being arranged along a second direction intersecting the first direction.

8. The image processing device described in claim 1, wherein the one or more processors: detect damage in the photographed object based on at least one of the image and the composite image; use a display device to display the damage detection result on a screen on which the composite image is displayed; and use the display device to switch between displaying the composite image and displaying the damage detection result.

9. An image processing method in which one or more processors in a computer acquire an image group including a plurality of images acquired by photographing an object, the order of which in a first direction is known; divide the image group into a plurality of image sets, each of which has one or more first overlapping images in adjacent image sets in the first direction; combine the images included in the image sets in the first direction to generate a plurality of composite images for each image set; and remove a portion of the first overlapping images for each composite image.

10. A program that causes a computer to realize the following functions: acquiring an image group including a plurality of images acquired by photographing an object and whose arrangement in a first direction is known; dividing the image group into a plurality of image sets, each of which has one or more first overlapping images in adjacent image sets in the first direction; combining the images included in the image sets in the first direction to generate a plurality of composite images for each image set; and removing a portion of the first overlapping images for each composite image.

11. A non-transitory computer-readable recording medium having the program according to claim 10 recorded thereon.