Image processing device, image processing method, and program

Abstract

Provided are an image processing device, an image processing method, and a program with which it is possible to generate a composite image having relatively little distortion, that is, a composite image in which discontinuity is not significant when a plurality of composite images are arranged. An image processing device (20) performs: (60) acquiring an image group including a plurality of images acquired by capturing an image of an object to be captured, the plurality of images being arranged in a first direction in a known order; (62) dividing the image group into a plurality of image sets, each of which has one or more first duplicate images, among image sets adjacent to each other in the first direction; (64) synthesizing the plurality of images included in the image sets in the first direction and generating a plurality of synthesized images for each image set; and (66) removing a portion of the first repeating image for each composite image.

Description

Technical Field

[0001] This invention relates to an image processing apparatus, an image processing method, and a program. Background Technology

[0002] Patent Document 1 describes an inspection device that acquires images for visually inspecting an object. The device described in this document implements an image synthesis method that acquires image data including multiple images captured by scanning and photographing the surface of the object, synthesizes a column of images along the short side to obtain a first synthesized image, and synthesizes multiple first synthesized images along the long side to obtain a second synthesized image.

[0003] Patent Document 2 describes an image reading device comprising: a first sensor for reading a first region of an original document; a second sensor for reading a second region that overlaps with a portion of the first region; a third sensor for reading a third region that overlaps with a portion of the second region; a first processor for generating a first composite image including the first region and the second region; and a second processor for generating a second composite image including the second region and the third region, performing a linking process between the first composite image and the second composite image, and generating an output image.

[0004] Patent documents Patent Document 1: Japanese Patent Application Publication No. 2021-196735 Patent Document 2: Japanese Patent Application Publication No. 2019-161635 Summary of the Invention

[0005] The technical problem to be solved by the invention One embodiment of the present invention provides an image processing apparatus, image processing method, and program capable of generating composite images with relatively little distortion and with minimal discontinuity when arranging multiple composite images.

[0006] means for solving technical problems An image processing apparatus according to a first aspect of the present invention includes: one or more processors; and one or more memory storing commands for the one or more processors to execute. The one or more processors perform the following processing: acquiring an image group including multiple images of a subject acquired by photographing the subject in a known arrangement order in a first direction; dividing the image group into multiple image sets having one or more first duplicate images on both sides of adjacent image sets in the first direction; compositing the multiple images included in the image sets in the first direction and generating multiple composite images for each image set; and removing a portion of the first duplicate image from each composite image.

[0007] In the image processing apparatus of the second method, the image group may include multiple images generated by photographing an object extending along the first direction.

[0008] In the image processing apparatus of the third method, which is in the image processing apparatus of the first or second method, one or more processors can perform the following processing: Obtain an image group comprising multiple images whose arrangement order is known in a second direction intersecting the first direction; The image group is divided into multiple image sets that are adjacent to each other in the second direction and have more than one second repeated image on both sides; Multiple images included in the image set are synthesized in the second direction, generating multiple synthesized images for each image set; and For the composite images adjacent in the second direction, remove a portion of the second duplicate image.

[0009] The image processing apparatus of the fourth method, in the image processing apparatus of the third method, may include multiple images generated by capturing images of a photographing object extending along the first and second directions.

[0010] In the image processing apparatus of any one of the first to fourth methods, the fifth method involves one or more processors that can perform the following processing: detecting damage in the photographed object based on at least one of the image and the composite image; and using a display device to display the damage detection result on a screen displaying the composite image.

[0011] In the image processing apparatus of any of the first to fifth embodiments, one or more processors may use a display device to arrange and display multiple composite images along the first direction.

[0012] In the image processing apparatus of the seventh method, in the image processing apparatus of the sixth method, one or more processors use a display device to arrange and display a composite image group in which multiple composite images are arranged along the first direction, along a second direction that intersects the first direction.

[0013] In the image processing apparatus of any one of the first to seventh methods, one or more processors can perform the following processing: detecting damage in the photographed object based on at least one of the image and the composite image; displaying the damage detection result on the screen displaying the composite image using a display device; and switching between displaying the composite image or displaying the damage detection result using the display device.

[0014] In the image processing method according to the ninth aspect of the present invention, one or more processors in a computer perform the following processing: acquiring an image group including multiple images of a subject acquired by photographing the subject and whose arrangement order in a first direction is known; dividing the image group into multiple image sets having one or more first duplicate images on both sides of adjacent image sets in the first direction; compositing the multiple images included in the image sets in the first direction and generating multiple composite images for each image set; and removing a portion of the first duplicate image from each composite image.

[0015] The program according to the tenth aspect of the present invention enables a computer to perform the following functions: acquiring an image group including multiple images of a subject acquired by photographing the subject in a known arrangement order in a first direction; dividing the image group into multiple image sets having one or more first repeating images on both sides of adjacent image sets in the first direction; compositing the multiple images included in the image set in the first direction and generating multiple composite images for each image set; and removing a portion of the first repeating image from each composite image. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a tunnel shot taken using the shooting system described in the implementation method.

[0017] Figure 2 This is an illustration of the distortion that occurs at the ends of a composite image.

[0018] Figure 3 This is a diagram representing a specific example of a composite image that has undergone distortion.

[0019] Figure 4 This is a schematic diagram representing a group of images whose arrangement order is known.

[0020] Figure 5 This is a schematic diagram illustrating the segmentation of an image set.

[0021] Figure 6 This is a schematic diagram illustrating another example of image set segmentation.

[0022] Figure 7 This is an illustration of image synthesis.

[0023] Figure 8 This is a schematic diagram of end removal in a composite image.

[0024] Figure 9 This is a schematic diagram of end removal of a synthetic image involved in another example.

[0025] Figure 10 This is a diagram representing the composite image before the distortion at the ends is removed.

[0026] Figure 11 This is a diagram showing the composite image after the distortion at the ends has been removed.

[0027] Figure 12 This is another example of a composite image before the distortion at the ends is removed.

[0028] Figure 13 This is another example of a composite image after the distortion at the ends has been removed.

[0029] Figure 14 This is a diagram illustrating an example of a user interface suitable for displaying a composite image set.

[0030] Figure 15 This is a diagram illustrating an example of a camera applicable to the imaging system described in the embodiment.

[0031] Figure 16 It means Figure 15 This is a schematic diagram showing the movement of the camera unit.

[0032] Figure 17 This is a functional block diagram illustrating an example of the electrical configuration of the imaging system involved in the implementation method.

[0033] Figure 18 It means Figure 17 The diagram shows the functional block diagram of the electrical components of the image processing device.

[0034] Figure 19 It means Figure 18 The diagram shows a block diagram of the hardware structure of the image processing device.

[0035] Figure 20 This is a flowchart illustrating the sequence of image processing methods involved in the implementation. Detailed Implementation

[0036] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification, the same reference numerals are used to denote the same constituent elements, and repeated descriptions are omitted where appropriate. Furthermore, in the following embodiments where multiple constituent elements are exemplified, it can be interpreted as including at least one of the multiple constituent elements.

[0037] [Description of the topic] Figure 1 This is a schematic diagram illustrating filming inside a tunnel using the filming system described in the implementation method. Figure 1 The diagram illustrates a camera unit 50 that uses five cameras—50A, 50B, 50C, 50D, and 50E—to photograph the inner wall IW of the tunnel TU.

[0038] The imaging system moves the camera unit 50 along the long side of the tunnel TU, extending from which the tunnel TU extends, and captures images of the inner wall IW of the tunnel TU at each of a predetermined plurality of shooting positions. The plurality of shooting positions along the long side of the tunnel TU are defined in such a manner that the images captured at adjacent shooting positions are repeated at the ends. Furthermore, the long side of the tunnel TU is through... Figure 1 The orientation of the paper is an example of the first orientation.

[0039] Figure 1 The symbols SA1, SA2, SA3, SA4, and SA5 shown each represent the shooting area of ​​each of cameras 50A, 50B, 50C, 50D, and 50E. For example, adjacent shooting areas in the short-side direction of tunnel TU are defined by repeating the end of each. For example, shooting area SA2 is... Figure 1 The left end of the image overlaps with the end of the shooting area SA1 and Figure 1 The right end of the tunnel is specified to repeat the end of the shooting area SA3. Furthermore, the short side direction of the tunnel TU is orthogonal to the long side direction of the tunnel TU, and is the direction along the inner wall IW of the tunnel TU. The long side direction of the tunnel TU described in the embodiment is an example of the second direction.

[0040] Furthermore, in Figure 1 The middle figure shows a composite image ICA formed by combining multiple captured images taken using the camera 50B included in the camera unit 50 along the long side of the tunnel TU.

[0041] It is impractical to represent a large structure such as a tunnel using a single composite image. Therefore, a method is proposed that divides the structure into multiple segments, generates composite images for each segment, and arranges the composite images of each segment to represent the large structure as a whole.

[0042] However, when generating composite images of elongated objects such as tunnels, distortion easily occurs, especially at the ends of the composite image, due to the accumulation of errors and deviations from corresponding points in adjacent images. These corresponding points can also be referred to as constraint points, etc. When multiple composite images are arranged and displayed, distortion at the ends of the composite image leads to noticeable discontinuities with adjacent composite images.

[0043] Figure 2 This is an illustration of the distortion that occurs at the ends of a composite image. Figure 3 This is a diagram representing a specific example of a composite image that has undergone distortion. Figure 2 The multiple captured images shown, IMA1 to IMC5, are... Figure 3The constituent elements of the composite image ICB shown.

[0044] Figure 2 Each of the multiple captured images IMA1 to IMC5 shown was captured and acquired at each of the following positions: a combination of five capture positions along the long side of the tunnel and three capture positions along the short side of the tunnel. Each captured image IMA1 to IMC5 has a repeating region that overlaps with the adjacent captured image.

[0045] For example, the captured image IMB2 has a repeating region OA1 at the end of the adjacent captured image IMA2 side, which is the same as the captured image IMA2. Similarly, the captured image IMA2 has a repeating region OA11 at the end of the captured image IMB2 side, which has the same area as the repeating region OA1.

[0046] Furthermore, the captured image IMB2 has repeating regions OA2, OA3, and OA4 at each end of the captured images IMB1, IMC2, and IMB3. Each of the captured images IMB1, IMC2, and IMB3 also has repeating regions OA12, OA13, and OA14 corresponding to the repeating regions OA2, OA3, and OA4. Hereinafter, without needing to distinguish each captured image IMA1, etc., the captured images IMA1, etc., will be collectively referred to as captured images IM. Additionally, the captured images IM described in this embodiment are an example of multiple images constituting an image group.

[0047] exist Figure 2 Among the multiple captured images IM shown, for example, the end captured image IMA5 has fewer corresponding points and is deviated from the central captured image IMB3 compared to its adjacent captured images IM. Specifically, captured image IMA5 has 3 corresponding points with captured image IMA4 and 5 corresponding points with captured image IMB5, but has no corresponding points on the side opposite to captured image IMA4 and the side opposite to captured image IMB5.

[0048] Furthermore, the leading edge of the arrow connecting image IMA5 and image IMA4 indicates the corresponding point between image IMA5 and image IMA4. Similarly, the leading edge of the arrow connecting image IMA5 and image IMB5 indicates the corresponding point between image IMA5 and image IMB5. The same applies to the arrow for image IMB3.

[0049] On the other hand, compared with the end-captured image IM, the central captured image IMB3 and other non-end-captured images IM have more corresponding points, and corresponding points are evenly distributed around the captured image IM.

[0050] Compared to non-endpoint images (IM), endpoint images (IMA5, etc.) have fewer corresponding points and exhibit deviations in these points. The distortion caused by computational errors in the transformation relationships used to align corresponding points, and distortion caused by differences between the shape model and the actual object being photographed, are influenced by factors such as: differences between the cylindrical model and the actual tunnel shape; and differences between the shape model and the actual object being photographed. Figure 3 The ends of the composite image ICB are clearly visible. Figure 3 The composite image IC shown has obvious distortion at both ends EN in the long side direction.

[0051] Furthermore, when the distortion at the ends of the synthesized image is relatively large, not only will the aforementioned appearance problems of the synthesized image occur, but also relatively large errors may occur regarding the damage size and other parameters obtained based on the synthesized image. These relatively large errors in damage size and other parameters can become obstacles to understanding the state of an object based on its captured image.

[0052] Therefore, even when generating composite images of slender objects such as tunnels, it is preferable to minimize the distortion at the ends of the composite image. Furthermore, not limited to slender objects such as tunnels, even when generating composite images of massive structures such as dams by dividing them into multiple regions and arranging multiple composite images to represent the structure as a whole, it is preferable to minimize the distortion of each composite image.

[0053] [Description of image groups with known arrangement] Figure 4 This is a schematic diagram representing a group of images whose arrangement order is known. Figure 4 The multiple captured images shown are arranged in a specific order using camera number and direction of travel number. Additionally, in Figure 4 In this context, any one of the multiple captured images IM is assigned the symbol IM to represent the captured image.

[0054] Here, the direction of the travel direction number is defined as follows: Figure 2 The tunnel shown is along its long side, and is... Figure 1 The direction of travel for camera unit 50 is shown. Furthermore, the direction of camera numbering is defined as follows: Figure 2 The tunnel shown is oriented along its short side, which is also the direction in which cameras such as the 50A are arranged.

[0055] Camera number is a pair Figure 1Each of the cameras 50A, 50B, 50C, 50D, and 50E shown has a unique number assigned according to the order of their arrangement. For example, camera 50A can be designated as camera number 1, camera 50B as camera number 2, camera 50C as camera number 3, camera 50D as camera number 4, and camera 50E as camera number 5.

[0056] The direction of travel number is a number assigned sequentially to the shooting positions along the direction of travel of the camera 50A, etc., from the starting position to the ending position. The maximum value of the direction of travel number is determined based on the total length of the subject being photographed, the field of view of the camera 50A, etc. Figure 4 Examples of images IM with travel direction numbers 1 to 13 are shown, but images IM with travel direction numbers 14 and above are also possible. The position at the time of shooting can be determined, and the travel direction number can be assigned based on the determination results.

[0057] Multiple captured images (IMs) are assigned unique filenames using the camera ID and direction of travel ID. That is, each of the multiple captured images (IMs) is assigned a filename that indicates its order. An example of a filename using the extension [camera ID]_[direction of travel ID].

[0058] Furthermore, information is prepared to represent the correspondence between the filenames, camera numbers, and direction-of-flight numbers for each of the multiple captured images (IMs). As an example of this correspondence information, a table showing the correspondence between filenames with extensions like IMD_0001 and their corresponding camera numbers and direction-of-flight numbers can be provided. This correspondence table can be in CSV format.

[0059] Image set segmentation Figure 5 This is a schematic diagram illustrating image set segmentation. In Figure 5 shown in the Figure 4 The example shown illustrates a scenario where multiple captured images IM are divided into three image sets IS1, IS2, and IS3 along a specified direction of travel.

[0060] One captured image IM is repeated at the boundary between adjacent image sets IS1 and IS2. Figure 5 In the example shown, in image set IS1 and image set IS2, the image IM with the direction of travel number 5 is designated as a duplicate image ID.

[0061] Similarly, in image set IS2 and image set IS3, the image IM captured in the direction of travel number 9 is designated as a repeating image ID. Furthermore, the number of image sets IS, the number of captured images IM included in one image set IS, and the number of repeating image IDs included in each image set IS are not limited to the illustrated example and can be appropriately specified.

[0062] For example, when the number of image sets IS is set to 3 and the number of duplicate image IDs is set to 2, the image sets IS and duplicate image IDs are defined as follows: Image set IS1 applies to images IM captured in the direction of travel numbered 1 to 5. Image set IS2 applies to images IM captured in the direction of travel numbered 4 to 8. Image set IS3 applies to images IM captured in the direction of travel numbered 7 to 11.

[0063] The repeating image IDs at the boundary between image set IS1 and image set IS2 apply to image IM taken in the direction of travel number 4 and image IM taken in the direction of travel number 5. The repeating image IDs at the boundary between image set IS2 and image set IS3 apply to image IM taken in the direction of travel number 7 and image IM taken in the direction of travel number 8.

[0064] With the number of image sets (IS) and the number of duplicate image IDs set to 3, the image sets (IS) and duplicate image IDs are defined as follows: Image set IS1 applies to images IM captured in the direction of travel numbered 1 to 7. Image set IS2 applies to images IM captured in the direction of travel numbered 5 to 11. Image set IS3 applies to images IM captured in the direction of travel numbered 9 to 15.

[0065] The repeating image IDs at the boundary between image set IS1 and image set IS2 apply to images IM numbered 5 to 7 in the direction of travel. The repeating image IDs at the boundary between image set IS2 and image set IS3 apply to images IM numbered 9 to 11 in the direction of travel.

[0066] Thus, if an image set IS with multiple duplicate image IDs is specified, then in the case of significant distortion in the composite image IC generated according to each image set IS, the scope for distortion removal becomes larger, which is therefore preferred.

[0067] [Another example of image set segmentation] Figure 6 This is a schematic diagram illustrating another example of image set segmentation. In Figure 6 The following example is shown: For multiple captured images IM with a specified orthogonal arrangement in a first and second traveling direction, image set segmentation is performed in each of the first and second traveling directions.

[0068] in addition, Figure 6The first or second direction of travel shown is an example of the first direction. Furthermore, any direction in the first or second direction of travel that differs from the first direction is an example of the second direction. Figure 6 The second direction of travel shown is orthogonal to the first direction of travel, or the first direction of travel is orthogonal to the second direction of travel, which is an example of a second direction that intersects with the first direction.

[0069] Figure 6 The image set segmentation shown is applicable to situations where, in the first and second travel directions, the camera scans the object to be photographed and takes pictures at multiple shooting positions, acquiring multiple captured images (IM). An example of an object to be photographed is a structure where the camera needs to move in a two-dimensional manner when photographing a dam or similar structure.

[0070] exist Figure 6 The following example is shown: For the first and second directions of travel, multiple captured images IM with known order are divided into 6 image sets IS11, IS12, IS13, IS21, IS22 and IS23, and a repeating image ID1 or ID2 is specified at the boundary of adjacent image sets IS.

[0071] The number of repeating image ID1 and the number of repeating image ID2 can be more than 2. The number of repeating image ID1 in the first moving direction and the number of repeating image ID2 in the second moving direction can be the same or different.

[0072] [Image Composition] Figure 7 This is an explanatory diagram of image synthesis. In image synthesis, synthesis parameters are determined based on the corresponding point matching results between adjacent captured images IM, and a single synthesized image is generated from the multiple captured images IM included in the image set IS. Furthermore, synthesized images from each image set IS are synthesized, and a group of synthesized images corresponding to the entire object being measured is generated. Additionally, each image set IS described in this embodiment is an example of a single image set.

[0073] The synthesis parameters include pose parameters of the shape model, pose parameters of each camera for each of the multiple captured images IM, and lens distortion parameters of each camera for each of the multiple captured images IM.

[0074] When the shape model is a planar model, the pose parameters of the shape model include the rotation matrix and translation vector. When the shape model is a cylindrical model, the pose parameters of the shape model include the rotation matrix, translation vector, and the radius of the cylinder. When a planar model is used, the synthesis parameters can be the projection transformation matrix of each captured image IM.

[0075] Non-rigid transformations, including scaling and reduction, can be used as pose parameters. Examples of non-rigid transformations include affine transformations.

[0076] Based on the determined compositing parameters, an image IM is projected onto the shape model, and a composite image ICC is generated. In the case of a cylindrical shape model, further planar unfolding is performed.

[0077] exist Figure 7 The middle figure illustrates a three-dimensional coordinate system with mutually orthogonal Xs, Ys, and Zs axes as a coordinate system applicable to the pose parameters of the shape model. Furthermore, in Figure 7 The diagram illustrates a three-dimensional coordinate system with mutually orthogonal Xi, Yi, and Zi axes, serving as the coordinate system for the pose parameters and lens distortion parameters of each camera CAM. Additionally, 'i' represents a consecutive number starting from 1, indicating the shooting position. Figure 7 The Xs axis and Xi axis shown are... Figure 2 The tunnel shown corresponds to the direction of its long side. Figure 7 The Ys axis and Yi axis shown are... Figure 2 The tunnel shown corresponds to the direction of its short side.

[0078] In image synthesis, a portion of multiple captured images IM included in the image set IS can be repeatedly synthesized and optimized to synthesize all of the multiple captured images IM included in the image set IS.

[0079] The generation of the composite image IC can be achieved by compositing multiple captured images IM from each camera 50A in the Xs direction to generate a composite image element, and by compositing the composite image elements from each camera 50A in the Ys direction.

[0080] [End-cap removal from composite images] Figure 8 This is a schematic diagram of end-cap removal in a composite image. Figure 8 The diagram illustrates the correspondence between image set IS1 and composite image IC1, image set IS2 and composite image IC2, and image set IS3 and composite image IC3. Figure 8 The image sets IS1, IS2, and IS3 shown in the upper paragraph are... Figure 5 The image sets IS1, IS2, and IS3 shown in the lower paragraph are the same.

[0081] Multiple captured images IM included in image set IS1 are synthesized to generate composite image IC1. Furthermore, multiple captured images IM included in image set IS2 are synthesized to generate composite image IC2, and multiple captured images IM included in image set IS3 are synthesized to generate composite image IC3.

[0082] Furthermore, each of the actual composite images IC1, IC2, and IC3 exhibits distortion, but... Figure 8 In the image, each of the composite image IC1, composite image IC2 and composite image IC3 is represented by a rectangular shape.

[0083] At each end of the composite images IC1, IC2, and IC3, a portion of the region corresponding to the duplicate image ID of the image set IS is removed. The removed region CA from the composite images IC1, etc., is schematically illustrated using dashed lines. Figure 8 The following example is shown: For each of the composite images IC1, IC2, and IC3, half of the region corresponding to the duplicate image ID is removed. The removed region CA can be more than 0% and less than 50% of the region corresponding to the duplicate image ID, or it can be more than 50% and less than 100% of the region corresponding to the duplicate image ID.

[0084] The composite images IC1 and IC2 illustrate position P15, which corresponds to image IM captured by camera number 1 and direction of travel number 5. Furthermore, the composite images IC2 and IC3 illustrate position P19, which corresponds to image IM captured by camera number 1 and direction of travel number 9.

[0085] Figure 9 This is a schematic diagram of end-cap removal in another example involving a synthetic image. Figure 9 The following method is shown: End removal is performed on composite image IC11, composite image IC12, and composite image IC13 when the number of duplicate image IDs set for each of image sets IS11, IS12, and IS13 is 2.

[0086] Figure 9 In image set IS11, images IM with direction-of-travel number 4 and IM with direction-of-travel number 5 are designated as duplicate image IDs. In image set IS12, images IM with direction-of-travel number 4 and IM with direction-of-travel number 5 are designated as duplicate image IDs, and images IM with direction-of-travel number 7 and IM with direction-of-travel number 8 are also designated as duplicate image IDs. In image set IS13, images IM with direction-of-travel number 7 and IM with direction-of-travel number 8 are designated as duplicate image IDs.

[0087] Composite image IC11 is generated from multiple captured images IM included in composite image set IS11. Similarly, composite image IC12 is generated from multiple captured images IM included in composite image set IS12, and composite image IC13 is generated from multiple captured images IM included in composite image set IS13.

[0088] In the composite image IC11, a portion of the duplicate image ID in the image set IS11 is removed, i.e., the removal region CA. Specifically, in the composite image IC11, the removal region CA removes a portion of position P14 and a portion of position P15 from position P14 corresponding to the image IM captured in the direction of travel number 4 and position P15 corresponding to the image IM captured in the direction of travel number 5.

[0089] In the composite image IC12, a portion of the duplicate image ID of the image set IS12 is removed, which is the removal region CA.

[0090] Specifically, in the composite image IC12, the removal region CA removes a portion of position P14 and a portion of position P15 from position P14 corresponding to the image IM captured in the travel direction number 4 and position P15 corresponding to the image IM captured in the travel direction number 5. Furthermore, in the composite image IC12, the removal region CA removes a portion of position P17 and a portion of position P18 from position P17 corresponding to the image IM captured in the travel direction number 7 and position P18 corresponding to the image IM captured in the travel direction number 8.

[0091] In the composite image IC13, a portion of the duplicate image ID of the image set IS13 is removed, namely the removal region CA. Specifically, in the composite image IC13, the removal region CA removes a portion of position P17 and a portion of position P18 from position P17 corresponding to the image IM captured in the direction of travel number 7 and position P18 corresponding to the image IM captured in the direction of travel number 8.

[0092] Figure 10 This is a diagram representing the composite image before the distortion at the ends is removed. Figure 10 The composite image shown is ICD and Figure 3 Similarly, the synthetic image ICB shown also produces distortion DI at both one end EN and the other end EN in the long side direction of the tunnel.

[0093] Figure 11 This is a diagram showing the composite image after the distortion at the ends has been removed. Figure 11 In the composite image ICE shown, the image has been removed. Figure 10The distortion DI of one end EN and the other end EN in the synthetic image ICD shown. Figure 11 The composite image ICE shown has one end ENA and the other end ENB set as straight lines parallel to the short side of the tunnel.

[0094] Figure 12 This is another example of a composite image before the distortion at the ends is removed. Figure 12 The middle shows with Figure 10 The example shown is where the number of captured images (IMs) that make up the composite image (ICF) is greater than that of the composite image (ICD).

[0095] Figure 13 This is another example of a composite image after the distortion at the ends has been removed. Figure 13 The composite image shown is ICG and Figure 12 The composite image ICE shown also has the following features removed: Figure 12 The distortion DI of one end EN and the other end EN in the synthetic image ICF shown. Figure 13 The composite image ICG shown has one end ENA and the other end ENB set as straight lines parallel to the short side of the tunnel.

[0096] Figure 11 The composite image ICE shows the distortion of the end EN by DI and is cropped. Figure 13 The distortion DI of the end EN shown is cropped into a composite image ICG, which is generated in multiple ways corresponding to the total length of the object being photographed, i.e., the tunnel.

[0097] Processing techniques such as ICE (Integrated Image Processing) for synthesizing multiple composite images are applicable to processes that ensure corresponding points in adjacent composite images are consistent. Figure 10 The processing applicable to captured images (IM) is shown in the generation of synthetic images such as ICDs.

[0098] [User interface for displaying composite image groups] Figure 14 This is a diagram illustrating an example of a user interface suitable for displaying a composite image set. Figure 14 The image shows an example of a display screen DS in which multiple composite image ICs are arranged and displayed as a composite image group GIC. Figure 14 In the illustrated composite image group GIC, when the positions of the repeating image IDs of adjacent composite image ICs are aligned and adjacent composite image ICs have repeating regions, the repeating regions overlap. On the other hand, when adjacent composite image ICs do not have repeating regions, adjacent composite image ICs are arranged without overlap.

[0099] The display screen DS is configured to freely switch between the normal display screen NDS and the magnified display screen EDS. Switching between the normal display screen NDS and the magnified display screen EDS is performed according to the user's instructions.

[0100] exist Figure 14 The NDS typically displays an example where 10 composite images (ICs) corresponding to 10 meters along the long side of a tunnel that is several kilometers long are arranged along the long side of the tunnel. That is, the NDS typically displays a portion of the long side of a tunnel with a total length of several kilometers, specifically 100 meters.

[0101] Figure 14 The enlarged display shown in the EDS shows one of the ten composite image ICs displayed on the NDS screen. Figure 14 The enlarged display screen EDS shown is reduced to the normal display screen NDS and overlaid on the enlarged display screen EDS in the upper right corner.

[0102] The display screen DS includes a first area AR1, a second area AR2, and a third area AR3. In the first area AR1, the composite image group GIC is displayed on the normal display screen NDS, and in the enlarged display screen EDS, a composite image IC and a portion of the composite image group GIC displayed on the normal display screen NDS are enlarged and displayed.

[0103] Area 2 (AR2) displays a toolbar with icons representing various tools. Examples of these tools include zoom in and zoom out. Area 3 (AR3) displays a position bar. A zoomed-out position bar indicates the entire composite image group (GIC). The position bar includes markers (PM) indicating the extent of the GIC displayed in Area 1 (AR1). Figure 14 The image shown is an example of a rectangle used as a marker PM.

[0104] The position bar displayed on the NDS (Non-Display DS) includes a marker PM indicating an area representing 10 composite image ICs. Furthermore, the position bar displayed on the zoomed-in EDS (Electronic Display DS) includes a marker PM indicating an area representing 1 composite image IC.

[0105] exist Figure 6 In the case where multiple captured images IM are divided into multiple image sets IS and a composite image is generated according to the multiple image sets IS in the first and second travel directions shown, multiple composite images IC of each image set IS are arranged in the first travel direction, and multiple composite images IC of each image set IS are arranged in the second travel direction.

[0106] When multiple composite image ICs are arranged in the first moving direction, they are aligned with the positions of repeating image IDs in the first moving direction, and the repeating images ID1 overlap. Similarly, when multiple composite image ICs are arranged in the second moving direction, they are aligned with the positions of repeating image ID2 in the second moving direction, and the repeating images ID2 overlap. Furthermore, the repeating image ID1 in the first moving direction described in this embodiment is an example of a first repeating image, and the repeating image ID2 in the second moving direction described in this embodiment is an example of a second repeating image.

[0107] If any two points on the composite image IC are specified in the magnified display screen EDS, the lengths of the two specified points are measured and the measurement results are displayed. For example, in Figure 14 In the composite image IC of the enlarged display screen EDS shown, if points P1 and P2 are specified, the specified points P1 and P2 are displayed, and the line segment LS connecting points P1 and P2 is also displayed.

[0108] Furthermore, the measured values ​​of the length L, width W, and height H of line segment LS are displayed in the magnified display screen EDS as the measurement results for points P1 and P2. The width W of line segment LS is the component of the length of line segment LS in the long side direction of the tunnel, and the height H of line segment LS is the component of the length of line segment LS in the short side direction of the tunnel.

[0109] exist Figure 14 The diagram illustrates how the measurement results of points P1 and P2 are superimposed on the composite image IC, but the measurement results of points P1 and P2 can also be displayed in regions different from the first region AR1, the second region AR2, and the third region AR3.

[0110] Measurements of any two points specified on the composite image IC can be applied to the measurement of damage to the object being photographed. For example, when the object being photographed is the inner wall IW of a tunnel TU, the positions of the two ends of damage such as scratches, notches, breaks, and discoloration on the inner wall IW of the tunnel TU can be specified, and the size of the damage can be measured. The captured image IM constituting the composite image IC can be used to perform calculations for any two points specified on the composite image IC.

[0111] refer to Figure 14 The switching between the normally displayed NDS screen and the magnified EDS screen showing the measurement results of any two points on the composite image IC is an example of switching between displaying the composite image and displaying the detection results of the damage. Furthermore, Figure 14 The designation of points P1 and P2 shown is an example of damage detection in the object being measured, and the measurement results of points P1 and P2 are an example of damage detection results.

[0112] As described above, the image processing apparatus can manually detect damage in the inner wall (IW) of the tunnel TU from the composite image IC, but it can also automatically detect damage in the inner wall (IW) of the tunnel TU from the captured image IM or the composite image IC. The image processing apparatus can display the damage in the inner wall (IW) of the tunnel TU.

[0113] The image processing device can switch between displaying the results of damage detection in the inner wall IW of the tunnel TU overlaid on the synthetic image IC, or displaying the synthetic image IC instead of the results of damage detection in the inner wall IW of the tunnel TU.

[0114] [Structural Example of the Imaging System According to the Embodiment] [Example of camera unit structure] Figure 15 This diagram illustrates an example of a camera used in the imaging system described in the embodiment. The camera unit 50 includes cameras 50A, 50B, 50C, 50D, and 50E. Cameras 50A, 50B, 50C, 50D, and 50E are arranged in an arc shape at equal intervals from a reference position O on the camera support platform 52. The camera support platform 52 is mounted on a trolley 54.

[0115] The optical axes of cameras 50A, 50B, 50C, 50D, and 50E are arranged radially from the reference position O of the camera support 52. Each camera (50A, etc.) has a different shooting direction, and cameras 50A, etc., shoot simultaneously. Figure 1 The inner wall IW of the tunnel TU shown.

[0116] The camera support platform 52 is equipped with a lighting device, a rangefinder, and a positioning device. Figure 15 The illustrations of the lighting device, rangefinder, and position measuring instrument are omitted. Each of the lighting device, rangefinder, and position measuring instrument is assigned the symbols 55, 56, and 57 respectively. Figure 17 The diagram shows an illumination device that illuminates the shooting area of ​​the camera 50A, etc. The illumination device can be individually configured for each camera 50A, etc.

[0117] A rangefinder measures the distance from its position to the inner wall (IW) of the tunnel TU. A laser rangefinder using a laser beam is suitable. An example of a laser rangefinder is LiDAR. LiDAR is an abbreviation for Light Detection and Ranging. The laser rangefinder rotates a laser beam emitted from a measuring head and illuminates the inner wall (IW) of the tunnel TU, measuring the distance between the measuring head and the inner wall (IW).

[0118] The rangefinder preferably converts the measured distance into a distance in the same direction as the optical axis of the camera 50A, etc., based on the angle between the optical axis of the camera 50A, etc. and the laser beam. The rangefinder can direct the laser beam in the same direction as the optical axis of the camera 50A, etc., without rotating the laser beam.

[0119] The position sensor outputs a positioning signal indicating the position of the trolley 54 in its direction of travel. The direction of travel of the trolley 54 is applicable to... Figure 2 The tunnel shown is oriented along its long side. That is, the positioning instrument measures the distance traveled from the starting position of the trolley 54 and generates a positioning signal indicating the direction of travel of the trolley 54 based on the measurement results.

[0120] Figure 16 It means Figure 15 This is a schematic diagram showing the movement of the camera unit. Figure 16 The schematic diagram shows the state of the inner wall IW of the tunnel TU being photographed by camera 50A and other cameras while the camera unit 50 is moving along the direction of travel. The camera unit 50 stops at a pre-defined shooting position along the direction of travel and performs the photographing of the inner wall IW of the tunnel TU.

[0121] [Electrical Configuration of the Camera System] Figure 17 This is a functional block diagram illustrating an example of the electrical configuration of the imaging system according to the embodiment. The imaging system 10 includes an image processing device 20, an imaging control device 30, and a camera unit 50. The camera unit 50 uses a camera 50A or the like to photograph the inner wall IW of the tunnel TU and generates an image.

[0122] The shooting control device 30 is a control device for the camera unit 50, enabling the camera unit 50 to operate and use the camera 50A, etc., to capture images of the inner wall IW of the tunnel TU and generate captured images. The shooting control device 30 sends the captured images to the image processing device 20. The shooting control device 30 is compatible with a computer.

[0123] The image processing apparatus 20 uses the captured image IM of the inner wall IW of the tunnel TU sent from the image control device 30 to generate multiple composite image ICs. The image processing apparatus 20 uses the display 22 to display a composite image group GIC in which multiple composite image ICs are arranged.

[0124] Image processing device 20 uses synthetic image IC to measure the inner wall IW of tunnel TU. An example of measuring the distance between two points on the inner wall IW of tunnel TU is shown in... Figure 14 As shown in the diagram. If the image processing device 20 receives a specified signal indicating the positions of two points sent from the input device 24, it then performs a distance measurement between the two points.

[0125] The shooting control device 30 includes an image acquisition unit 32. The image acquisition unit 32 acquires an electrical signal representing a captured image IM transmitted from a camera 50A or the like. The captured image IM acquired by the image acquisition unit 32 is then sent to the image processing device 20.

[0126] The shooting control device 30 includes a camera control unit 34. When shooting the inner wall IW of the tunnel TU, the camera control unit 34 controls the operation of each camera, such as 50A. The camera control unit 34 sends a shooting signal indicating the start of shooting to the cameras, such as 50A, at a predetermined shooting time. Each camera, such as 50A, performs shooting of the inner wall IW of the tunnel TU according to the shooting signal.

[0127] The camera control unit 34 sends information about the distance from each camera 50A to the inner wall IW of the tunnel TU, measured using the rangefinder 56, to each camera 50A. The cameras 50A can then focus based on the distance to the inner wall IW of the tunnel TU.

[0128] The camera control unit 34 sets shooting conditions for each camera, such as the 50A. For example, the camera control unit 34 sets the shooting resolution as a shooting condition for each camera, such as the 50A.

[0129] The shooting control device 30 includes a lighting control unit 36. The lighting control unit 36 ​​sets lighting conditions suitable for the lighting device 55 and controls the operation of the lighting device 55 according to the lighting conditions. As an example of lighting conditions, the intensity of the illumination light can be given.

[0130] The shooting control device 30 includes a ranging information acquisition unit 38. The ranging information acquisition unit 38 acquires ranging information sent from the rangefinder 56, which indicates the distance from each of the cameras 50A, etc., to the inner wall IW of the tunnel TU. The camera control unit 34 sends the ranging information acquired by the ranging information acquisition unit 38 to each of the cameras 50A, etc.

[0131] The shooting control device 30 includes a position information acquisition unit 40. The position information acquisition unit 40 acquires position information sent from the position measuring instrument 57, indicating the position of the camera unit 50 in the direction of travel of the camera unit 50. The camera control unit 34, based on the position information acquired by the position information acquisition unit 40, uses the camera 50A or the like to shoot the inner wall IW of the tunnel TU at a predetermined measurement position.

[0132] The shooting control device 30 includes a trolley control unit 42. The trolley control unit 42 sets the travel conditions of the trolley 54 and implements travel control of the trolley 54 according to the travel conditions of the trolley 54. The travel conditions of the trolley 54 include the travel speed of the trolley 54 and the stopping time, etc.

[0133] [Example of the structure of an image processing device] Figure 18 It means Figure 17 The diagram shows a functional block diagram of an example of the electrical configuration of an image processing apparatus. The image processing apparatus 20 includes an image group acquisition unit 60. The image group acquisition unit 60 acquires multiple images obtained by measuring the inner wall IW of the tunnel TU, i.e., images with a known arrangement order. These images with a known arrangement order are presented as multiple images IM arranged in a specific order using a travel direction number and a camera number. Figure 4 The diagram in the middle is shown.

[0134] The image processing apparatus 20 includes an image set segmentation unit 62. The image set segmentation unit 62 segments multiple captured images, whose arrangement order is known, acquired by the image group acquisition unit 60 into... Figure 5 The image set IS is shown. Additionally, image set IS is... Figure 5 The collective name for the image set IS1, etc. shown.

[0135] The image processing apparatus 20 includes an image compositing unit 64. The image compositing unit 64, according to the image set segmentation unit 62, defines each image set IS, performs compositing processing on multiple captured images IM included in each of the multiple image sets IS, and generates a composite image IC. As an example of a composite image IC, a composite image ICD is... Figure 10 The diagram in the middle is shown.

[0136] The image processing apparatus 20 includes an end-removal unit 66. The end-removal unit 66 removes the end distortion DI of the composite image IC from each image set IS. As an example of a composite image IC with the end distortion DI removed, the composite image IC... Figure 11 The diagram in the middle is shown.

[0137] The image processing apparatus 20 includes a display control unit 68. The display control unit 68 generates a display signal representing a composite image group GIC with multiple composite image ICs arranged in a row, and sends the generated display signal to... Figure 17 As shown in display 22. The composite image group (GIC) is displayed on display 22. An example of the display of the composite image group (GIC) is shown in... Figure 14 The diagram in the middle is shown.

[0138] The display control unit 68 generates a display signal representing a portion of the composite image group GIC for magnified display, and sends the generated display signal to the display 22. A portion of the composite image group GIC is magnified and displayed on the display 22. An example of magnified display of a portion of the composite image group GIC is shown below. Figure 14 The image is shown in the middle. Additionally... Figure 18 The display 22 shown is an example of a display device.

[0139] The image processing apparatus 20 includes a processing condition acquisition unit 70. The processing condition acquisition unit 70 acquires processing conditions applicable to various processes. The processing condition acquisition unit 70 sets processing conditions such as the number of image sets, the number of captured images 1M included in the image set in the image set segmentation process performed by the image set segmentation unit 62, and the number of duplicate images in the image set.

[0140] The processing condition acquisition unit 70 sets processing conditions applicable to the generation of composite image ICs for each image set IS implemented using the image compositing unit 64. The processing condition acquisition unit 70 sets processing conditions for end-removal processing, which removes distortion DI from each composite image IC, implemented using the end-removal unit 66. The processing condition acquisition unit 70 sets processing conditions for the generation processing of composite image group GICs implemented using the composite image group generation unit 67.

[0141] Figure 19 It means Figure 18 The diagram shows a block diagram of the hardware structure of the image processing apparatus. The image processing apparatus 20 is compatible with a computer. The computer can be a personal computer or a workstation. The computer can be a virtual machine.

[0142] The image processing apparatus 20 includes a processor 102, a computer-readable medium 104, a communication interface 106, an input / output interface 108, and a bus 110. The processor 102 is connected to the computer-readable medium 104, the communication interface 106, and the input / output interface 108 via the bus 110. The display 22 and the input device 24 are connected to the image processing apparatus 20 via the input / output interface 108.

[0143] The image processing apparatus 20 includes one or more processors 102 and one or more memories. The image processing apparatus 20 executes various programs stored in the memory of the computer-readable medium 104 through the processor 102 to realize various functions of the image processing apparatus 20.

[0144] Processor 102 includes a CPU. Processor 102 may include a GPU. Processor 102 is connected to computer-readable medium 104, communication interface 106, and input / output interface 108 via bus 110. Additionally, CPU is an abbreviation for Central Processing Unit, and GPU is an abbreviation for Graphics Processing Unit.

[0145] Computer-readable medium 104 can include a memory as a primary storage device and a storage device as an auxiliary storage device. Computer-readable medium 104 is applicable to semiconductor memories, hard disk drives, and solid-state drives, etc. Computer-readable medium 104 can be used in any combination of multiple devices.

[0146] Additionally, hard disk drives can be referred to as HDD (Hard Disk Drive), and solid state drives can be referred to as SSD (Solid State Drive).

[0147] Various programs stored in the memory of computer-readable medium 104 include one or more commands. Computer-readable medium 104 stores various data and parameters, etc. Furthermore, the term "program" has the same meaning as the term "software".

[0148] The hardware structure of processor 102 consists of various processors as shown below. These processors include general-purpose processors (CPUs) that execute software and function as various functional units; processors specifically designed for image processing (GPUs); processors with customizable circuit structures, such as FPGAs (Field Programmable Gate Arrays), such as PLDs (Programmable Logic Devices); and processors with circuit structures specifically designed for performing specific processes, such as ASICs (Application Specific Integrated Circuits).

[0149] in addition, Figure 19 The hardware structure of the image processing device shown can also be used as... Figure 17 The shooting control device 30 shown in the middle diagram is in the computer in operation.

[0150] [The order of image processing methods involved in the implementation] Figure 20 This is a flowchart illustrating the sequence of image processing methods involved in the implementation. In the image group acquisition step S10, Figure 18 The image group acquisition unit 60 shown acquires multiple captured images IM with a known arrangement order. After the image group acquisition step S10, the processing condition setting step S12 is entered.

[0151] In the processing condition setting step S12, various processing conditions applicable to the image set segmentation step S14, the image synthesis step S16, and the end-cap removal step S18 are set. After the processing condition setting step S12, the image set segmentation step S14 begins.

[0152] In the image set segmentation step S14, the image set segmentation unit 62 segments multiple captured images IM, whose arrangement order is known, obtained by the image group acquisition unit 60, into multiple image sets IS. After the image set segmentation step S14, the image synthesis step S16 is performed.

[0153] In the image compositing step S16, the image compositing unit 64 generates a composite image IC according to the multiple image sets IS specified in the image set segmentation step S14. After the image set segmentation step S14, the end removal step S18 is performed.

[0154] In the end-removal step S18, the end-removal unit 66 removes the distortion DI of each end EN of the plurality of composite image ICs. After the end-removal step S18, the display signal generation step S20 is performed.

[0155] In the display signal generation step S20, the display control unit 68 generates a display signal for a composite image group GIC formed by arranging multiple composite image ICs. After the display signal generation step S20, the process proceeds to the display signal output step S22. In the display signal output step S22, the display control unit 68 outputs the display signal of the composite image group GIC to the display 22. The composite image group GIC is then displayed on the display 22.

[0156] During the display of the composite image group GIC on the display 22, the zoom-in display determination step S24 is executed. In the zoom-in display determination step S24, the display control unit 68 determines whether to acquire a signal indicating switching to zoom-in display.

[0157] In the magnification display determination step S24, if the display control unit 68 does not receive a signal indicating switching to magnification display, it determines that the result is negative. If the result is negative, the process proceeds to the termination determination step S26.

[0158] In the termination determination step S26, the image processing device 20 determines whether to terminate image processing. In the termination determination step S26, if the image processing device 20 determines that image processing should continue, it determines that it should not. If it determines that it should not, it proceeds to the image group acquisition step S10 and repeatedly executes each step from the image group acquisition step S10 to the termination determination step S26 until it determines that it should.

[0159] On the other hand, in the termination determination step S26, if the image processing device 20 determines that image processing should be terminated, the determination is made as such. If the determination is made as such, the prescribed termination process is performed to terminate the sequence of the image processing method.

[0160] In the magnification display determination step S24, if the display control unit 68 receives a signal indicating switching to magnification display, it determines that this is the case. If this is the case, the process proceeds to the magnification display signal generation step S28.

[0161] In the magnified display signal generation step S28, the display control unit 68 generates a magnified display signal for a portion of the magnified composite image group GIC. For example, in the magnified display signal generation step S28, a signal representing... Figure 14 The magnified display signal of the enlarged display screen EDS is shown. After the magnified display signal generation step S28, the process proceeds to the magnified signal output step S30.

[0162] In the amplified signal output step S30, the display control unit 68 outputs the amplified display signal generated in the amplified display signal generation step S28 to the display 22. For example, the amplified display screen EDS is displayed on the display 22. After the amplified signal output step S30, the process proceeds to the termination determination step S26.

[0163] When a magnified display is implemented, a measurement procedure for measuring the object being measured can be performed. This measurement procedure can include a measurement point specification step, a calculation step to derive the measurement result based on the specified measurement point, and a measurement result display step to display the measurement result.

[0164] [Effects of the Implementation Method] The image processing apparatus, image processing method, and shooting system involved in the embodiments can achieve the following effects.

[0165] [1] Acquire an image set comprising multiple captured images IM, whose arrangement order along the direction of travel is known. Each set of captured images IM is further divided into multiple image sets whose adjacent image sets along the direction of travel have at least one duplicate image ID. The multiple captured images IM included in each of the multiple image sets are combined along the direction of travel to generate a composite image IC. Distortion DI at the ends EN of each composite image IC is removed.

[0166] Therefore, when multiple composite image ICs are arranged along the traveling direction as a composite image group, the discontinuity of the composite image group caused by the distortion DI of the end EN of each composite image IC in the traveling direction is not obvious.

[0167] [2] Multiple captured images (IM) with a known order of orientation for capturing the subject can be generated by moving the camera along the direction in which the subject extends and capturing the subject from multiple shooting positions.

[0168] [3] A group of images is acquired, comprising multiple captured images IM in a known order along a camera direction orthogonal to the direction in which the subject is being photographed. Each group of captured images IM is a set of images, further divided into multiple image sets whose adjacent image sets along the camera direction have at least one duplicate image ID. The multiple captured images IM included in each of the multiple image sets are combined along the camera direction to generate a composite image IC. The distortion DI at the ends EN of each composite image IC is removed.

[0169] Therefore, when multiple synthetic image ICs are arranged along the camera direction as a synthetic image group, the discontinuity of the synthetic image group caused by the distortion DI of the end EN of each synthetic image IC in the camera direction is not obvious.

[0170] [4] An image set, including multiple captured images whose arrangement order in the camera direction is known, can be generated by using multiple cameras arranged in the camera direction to capture images of the subject in the camera direction.

[0171] [5] An image set including multiple captured images in a known order along the camera direction can be generated by moving the camera along the camera direction and taking pictures of the subject from multiple shooting positions along the camera direction.

[0172] [6] A composite image group GIC, consisting of multiple composite image ICs arranged along the direction of travel, and a composite image group GIC, consisting of multiple composite image ICs arranged along the camera direction, are displayed on the display 22. Thus, the image capture results of the measured object can be seen.

[0173] [7] In the magnified display screen EDS, any two points on the synthetic image IC are specified, and measurements are performed at the specified two points. Thus, in the damage measurement performed by specifying two points included in the damage of the object to be measured, a measurement result in which the influence of the distortion DI of the end EN in the synthetic image IC is mitigated can be obtained.

[0174] [8] The measurement results for the two specified points are displayed in the magnified EDS screen. Thus, the measurement results are visible.

[0175] The embodiments of the present invention described above can be appropriately modified, added to, and deleted from their constituent elements without departing from the spirit and scope of the present invention. The present invention is not limited to the embodiments described above; various modifications can be made by those skilled in the art within the technical concept of the present invention. Furthermore, embodiments, modifications, and application examples can be appropriately combined and implemented.

[0176] Symbol Explanation 10-Shooting system, 20-Image processing device, 22-Display, 24-Input device, 30-Shooting control device, 32-Image acquisition unit, 34-Camera control unit, 36-Illumination control unit, 38-Distance information acquisition unit, 40-Positioning information acquisition unit, 42-Cart control unit, 50-Camera unit, 50A-Camera, 50B-Camera, 50C-Camera, 50D-Camera, 50E-Camera, 52-Camera support platform, 54-Cart, 55-Illumination device, 56-Distance meter, 57-Positioning meter, 60-Image acquisition unit, 62-Image set segmentation unit, 64-Image synthesis unit, 66-End-of-image removal unit, 68-Display control unit, 70-Processing condition acquisition unit 102 - Processor, 104 - Computer-readable medium, 106 - Communication interface, 108 - Input / output interface, 110 - Bus, AR1 - Region 1, AR2 - Region 2, AR3 - Region 3, CA - Region removal, CAM - Camera, DI - Distortion, DS - Display screen, EDS - Magnified display screen, EN - End, ENA - One end, ENB - The other end, GIC - Composite image group, IC1 - Composite image, IC2 - Composite image, IC3 - Composite image, IC11 - Composite image, IC12 - Composite image, IC13 - Composite image, ICA - Composite image, ICB - Composite image, ICC - Composite image, IC D - Composite image, ICE - Composite image, ICF - Composite image, ICG - Composite image, ID - Repeating image, ID1 - Repeating image, ID2 - Repeating image, IM - Captured image, IMA1 - Captured image, IMA2 - Captured image, IMA3 - Captured image, IMA4 - Captured image, IMA5 - Captured image, IMB1 - Captured image, IMB2 - Captured image, IMB3 - Captured image, IMB4 - Captured image, IMB5 - Captured image, IMC1 - Captured image, IMC2 - Captured image, IMC3 - Captured image, IMC4 - Captured image, IMC5 - Captured image, IS1 - Image set, IS2 - Image set, IS 3-Image set, IS11-Image set, IS12-Image set, IS13-Image set, IS21-Image set, IS22-Image set, IS23-Image set, IW-Inner wall, LS-Line segment, NDS-Normal display screen, O-Reference position, OA1-Repeating area, OA2-Repeating area, OA3-Repeating area, OA4-Repeating area, OA11-Repeating area, OA12-Repeating area, OA13-Repeating area, OA14-Repeating area, P1-Point, P2-Point, P14-Position, P15-Position, P17-Position, P18-Position, P19-Position, PM-Marker, S10~S30-Steps of the image processing method.

Claims

1. An image processing apparatus, comprising: More than one processor; and One or more memories store commands that cause the one or more processors to execute. The one or more processors perform the following processing: Acquire an image group consisting of multiple images of a known arrangement in the first direction, obtained by photographing the object being photographed; The image group is divided into multiple image sets that are adjacent to each other in the first direction and have more than one first repeating image on both sides; The multiple images included in the image set are combined in the first direction to generate multiple composite images for each image set; and Remove a portion of the first duplicate image from each of the synthesized images.

2. The image processing apparatus according to claim 1, wherein, The image group includes a plurality of images generated by photographing the subject object extending along the first direction.

3. The image processing apparatus according to claim 1, wherein, The one or more processors perform the following processing: Obtain the image group comprising a plurality of images arranged in a known order in a second direction intersecting the first direction; The image group is divided into multiple image sets in which two sides of the adjacent image sets in the second direction have more than one second repeated image; The multiple images included in the image set are combined in the second direction to generate multiple composite images for each image set; and Regarding the composite images adjacent in the second direction, a portion of the second duplicate image is removed.

4. The image processing apparatus according to claim 3, wherein, The image group includes a plurality of images generated by photographing the subject extending along the first direction and the second direction.

5. The image processing apparatus according to claim 1, wherein, The one or more processors perform the following processing: Damage in the photographed object is detected based on at least one of the image and the synthesized image; Using a display device, the detection results of the damage are displayed on a screen that displays the synthetic image.

6. The image processing apparatus according to claim 1, wherein, The one or more processors use a display device to arrange and display multiple of the composite images along a first direction.

7. The image processing apparatus according to claim 6, wherein, The one or more processors use the display device to arrange and display a composite image group in which multiple composite images are arranged in the first direction along a second direction that intersects the first direction.

8. The image processing apparatus according to claim 1, wherein, The one or more processors perform the following processing: Damage in the photographed object is detected based on at least one of the image and the synthesized image; Using a display device, the detection results of the damage are displayed on a screen displaying the synthesized image; and Using the display device, switch between displaying the synthetic image or displaying the detection results of the damage.

9. An image processing method, wherein, A computer with one or more processors performs the following processing: Acquire an image group consisting of multiple images of a known arrangement in the first direction, obtained by photographing the object being photographed; The image group is divided into multiple image sets that are adjacent to each other in the first direction and have more than one first repeating image on both sides; The multiple images included in the image set are combined in the first direction to generate multiple composite images for each image set; and Remove a portion of the first duplicate image from each of the synthesized images.

10. A program that enables a computer to perform the following functions: The function of acquiring an image group consisting of multiple images of a known arrangement in the first direction, obtained by photographing the object being photographed; The function of dividing the image group into multiple image sets that have one or more first repeating images on both sides of adjacent image sets in the first direction; The function of combining multiple images included in the image set in the first direction and generating multiple composite images for each image set; and The function of removing a portion of the first duplicate image from each of the synthesized images.

11. A recording medium that is non-transitory and computer-readable, and which records the program of claim 10.

Images