Information processing apparatus, information processing method, program, and recording medium

Abstract

The present application provides an information processing apparatus, an information processing method, a program, and a recording medium capable of generating a composite image with high precision from a plurality of divided images. The information processing apparatus includes a processor that performs the following processing: acquires a plurality of divided images obtained by dividing a photographic subject; acquires a composite target region of each divided image; acquires a corresponding point extraction region from each divided image according to the structure of the subject; calculates a composite parameter according to the corresponding points of each corresponding point extraction region between the divided images; and generates at least one composite image using the images within each composite target region according to the composite parameter.

Description

Technical Field

[0001] This invention relates to an information processing device, an information processing method, a program, and a recording medium. Background Technology

[0002] A technique for panoramic synthesis of multiple segmented images is known. For example, the image processing apparatus described in Patent Document 1 extracts feature points by removing obstacle objects that would hinder panoramic synthesis, and matches these feature points, thereby generating a composite image by panoramic synthesis of multiple segmented images.

[0003] Previous technical documents Patent documents Patent Document 1: International Publication No. 2020 / 137405 Summary of the Invention

[0004] One embodiment of the present invention provides an information processing apparatus, information processing method, program, and recording medium capable of generating synthetic images with minimal influence from the structure of the object.

[0005] means for solving technical problems The information processing apparatus of the first method includes a processor that performs the following processing: acquiring multiple segmented images obtained by segmenting a photographed object; acquiring a composite object region of each segmented image; extracting corresponding point regions from each segmented image according to the structure of the object; calculating composite parameters based on the corresponding points of the corresponding point regions between each segmented image; and generating at least one composite image using images within each composite object region according to the composite parameters.

[0006] In the first method, the information processing device of the second method performs the following processing: determining the region located at a distance set by the imaging device that captures the segmented image as the corresponding point extraction region.

[0007] In the first method, the information processing device of the third method performs the following processing: determining the region with a pre-set ratio α relative to the synthetic object region within a range greater than 0 and less than 1 as the corresponding point extraction region.

[0008] In the first method, the processor of the fourth method performs the following processing: determining the corresponding point extraction region based on the evaluation results of the generated synthetic image.

[0009] In the first method, the information processing device of the fifth method performs the following processing: displays the composite object area and determines the corresponding point extraction area according to the instructions from the user.

[0010] In the first method, the information processing device of the sixth method performs the following processing: determining the corresponding point extraction region based on the image quality distribution of the segmented image.

[0011] In any of the six methods, the processor of the information processing apparatus of the seventh method performs the following processing: determining the synthetic object region from the segmented image through image recognition.

[0012] In any of the six modes, the information processing apparatus of the eighth mode performs the following processing: displays the segmented image and acquires the composite object region according to the instructions from the user.

[0013] In any of the first to eighth methods, the information processing apparatus of the ninth method, where at least one composite image is two or more composite images, performs the following processing: arranging the two or more composite images side by side.

[0014] In any of the 1st to 9th modes, the information processing apparatus of the 10th mode performs the following processing: extracting damage from multiple segmented images and overlaying damage-related information onto a synthetic image.

[0015] In any of the 1st to 10th modes, the object of the information processing device is a cylinder.

[0016] In any of the 1st to 10th modes, the object of the information processing device in the 12th mode is the wall of the tunnel.

[0017] The information processing method of the 13th method is an information processing method based on an information processing device equipped with a processor, wherein the processor performs the following processing: acquiring multiple segmented images obtained by segmenting the photographed object; acquiring a composite object region of each segmented image; extracting corresponding point regions from each segmented image according to the structure of the object; calculating composite parameters based on the corresponding points of each corresponding point region extracted between the segmented images; and generating at least one composite image using the images within each composite object region according to the composite parameters.

[0018] The program of the 14th method causes the processor of the information processing device to perform information processing, wherein the program causes the processor to perform the following processing: acquiring multiple segmented images obtained by segmenting the photographed object; acquiring a composite object region of each segmented image; extracting corresponding point regions from each segmented image according to the structure of the object; calculating composite parameters according to the corresponding points of each corresponding point region extracted between the segmented images; and generating at least one composite image using the images in each composite object region according to the composite parameters.

[0019] The recording medium of the 15th method is a non-transitory and computer-readable recording medium, and records the program described above. Attached Figure Description

[0020] Figure 1 It is a diagram that conceptually represents a shooting system.

[0021] Figure 2 This is an illustration used to explain the photograph of the subject.

[0022] Figure 3 This is a block diagram illustrating an example of the hardware structure of an information processing device.

[0023] Figure 4 This is a diagram representing an example of multiple segmented images.

[0024] Figure 5 This is a diagram representing a failed example of panoramic compositing.

[0025] Figure 6 It is a diagram showing the functional structure of an information processing device.

[0026] Figure 7 It is a flowchart representing the steps of an information processing method.

[0027] Figure 8 This is a diagram used to illustrate the process of obtaining the composite object region.

[0028] Figure 9 This is a diagram representing the method for obtaining the region of the composite object.

[0029] Figure 10 This is a diagram illustrating another method for obtaining the region of a composite object.

[0030] Figure 11 This is a diagram used to illustrate the process of obtaining the region extracted from the corresponding points.

[0031] Figure 12 This is a diagram illustrating the method for obtaining the extraction region for the corresponding point.

[0032] Figure 13 This is a diagram illustrating another method for obtaining the region extracted from the corresponding points.

[0033] Figure 14 This is a diagram illustrating another method for obtaining the region extracted from the corresponding points.

[0034] Figure 15 This is a diagram illustrating another method for obtaining the region extracted from the corresponding points.

[0035] Figure 16 This is a diagram illustrating another method for obtaining the region extracted from the corresponding points.

[0036] Figure 17 This is a diagram illustrating the method for obtaining the best synthesized image.

[0037] Figure 18 This is a diagram illustrating the extraction of corresponding points.

[0038] Figure 19 This is a diagram representing an example of the synthesis parameters.

[0039] Figure 20 This is a diagram representing an example of a composite image.

[0040] Figure 21 This is a diagram illustrating a preferred embodiment.

[0041] Figure 22 This is a diagram illustrating another preferred embodiment.

[0042] Figure 23 This is an illustration of a photograph taken of the tunnel wall.

[0043] Figure 24 This diagram illustrates the composite object region and the corresponding point extraction region.

[0044] Figure 25 This diagram illustrates the extraction of corresponding points between the composite unfolded image and adjacent images in the direction of travel.

[0045] Figure 26 This diagram illustrates tunnels with different wall surfaces.

[0046] Figure 27 This diagram illustrates tunnels with different wall surfaces. Detailed Implementation

[0047] Hereinafter, the information processing apparatus and information processing method involved in the embodiments will be described with reference to the accompanying drawings.

[0048] Figure 1 This diagram conceptually illustrates a photographing system 1 consisting of a moving body 100 equipped with a camera 200, a controller 250, and an information processing device 300. The information processing device 300 in this embodiment is used for photographing an object 500 (see reference 200) captured by the camera 200. Figure 2 An apparatus for generating a composite image from segmented images is an example of the imaging apparatus of the present invention.

[0049] The mobile body 100 includes a mobile body main body 102, a propulsion unit 104 disposed on the mobile body main body 102, and a control device 120 disposed on the mobile body main body 102. The mobile body main body 102 is the main component of the mobile body 100. The mobile body main body 102 is controlled by the control device 120. The mobile body main body 102 is, for example, an unmanned aerial vehicle, such as a drone. The mobile body main body 102 has multiple propellers and propeller drive motors. The propellers and propeller drive motors constitute the propulsion unit 104.

[0050] The mobile body 102 is exemplified as an unmanned aerial vehicle, but it can also be a mobile robot, vehicle, or vessel. The mobile body 102 can be configured for remote operation or autonomous operation. Remote operation refers to a user (the user) operating the mobile body 102 from a location separate from it, by instructing the control device 120 from the controller 250. Autonomous operation refers to the control device 120 operating the mobile body 102 through a pre-created program, without the user's intervention. The program, etc., can be appropriately modified depending on the location where the mobile body 100 is used.

[0051] A camera 200 is mounted on a mobile body 100. The camera 200 is mounted to the main body 102 of the mobile body, for example, via a gimbal 110. The camera 200 is controlled by a control device 120 located on the main body 102 of the mobile body. While the mobile body 100 is flying in the atmosphere, the camera 200 mounted on the mobile body 100 photographs objects. The camera 200 is, for example, a digital camera, capable of capturing still images and moving images.

[0052] The information processing device 300 includes, for example, an operation unit 310, a display device 320, and a CPU (Central Processing Unit) 330. The CPU 330 generates a composite image after panoramic synthesis based on the segmented images. The display device 320 displays the segmented images or the composite image after panoramic synthesis. The user confirms the segmented images or composite images displayed on the display device 320 and, as needed, inputs correction commands via the operation unit 310.

[0053] Next, according to Figure 2 An example of a method for photographing an object 500 based on the photographing system 1 will be described.

[0054] like Figure 2 As shown, user U photographs object 500 using camera 200 mounted on mobile body 100. Object 500 may be, for example, the outer surface of a bridge, dam, chimney, building, house, or storage tank, or the inner surface of a tunnel. However, object 500 is not limited to these.

[0055] The mobile body 100 flies around the object 500 according to control signals sent from the controller 250. The camera 200 coordinates with the movement of the mobile body 100 and, according to control signals from the control device 120, captures images of the outer surface of the object 500. The camera 200 acquires an image within a viewing angle 210 each time it captures an image. Since the object 500 is larger than the viewing angle 210 of the camera 200, the camera 200 captures images of the object 500 in segments (segmented capture). The camera 200 obtains multiple segmented images. The multiple segmented images acquired during the flight of the mobile body 100 are stored as an image group, for example, in the memory (not shown) of the control device 120. In this example, the mobile body 100 moves from bottom to top in the height direction, and the camera 200 acquires a segmented image of one side of the object 500. The camera 200 is a digital camera, and the camera 200 acquires a two-dimensional color image of the object 500 as a segmented image.

[0056] The control device 120 stores each segmented image and the file name, file ID, arbitrarily assigned photo number, and other identification information or metadata associated with each segmented image in the memory.

[0057] <Information Processing Device> Figure 3 This is a block diagram illustrating an example of the hardware structure of the information processing device 300. The information processing device 300 can be a personal computer or a workstation.

[0058] The information processing device 300 includes an operation unit 310, a display device 320, a CPU 330, an input / output interface (I / F) 350, a storage unit 360, a RAM (Random Access Memory) 370, and a ROM 380.

[0059] Images obtained by segmented shooting (multiple segmented shooting) of object 500 are input to information processing device 300 via input / output I / F350 in a wired or wireless manner.

[0060] The CPU 330 controls the entire information processing device 300. Furthermore, the CPU 330 generates a composite image based on the segmented image and through panoramic synthesis.

[0061] The display device 320 is, for example, a liquid crystal display (LCD) or similar device, capable of displaying various information.

[0062] The operation unit 310 includes a keyboard and mouse, allowing the user to control the CPU 330 to perform necessary processing. The display device 320 can also function as the operation unit 310 by using a touch panel device. Furthermore, this example shows a system where the controller 250 and information processing device 300 are separate, but the controller 250 and information processing device 300 can also be integrated.

[0063] The storage unit 360 is a memory device composed of a hard disk drive, flash memory, etc. The storage unit 360 stores the operating system, programs, and data that enable the information processing device 300 to operate.

[0064] Figure 4 This is a diagram representing multiple segmented images acquired via the input / output I / F350. Figure 5 This is an example of a failed panoramic composite.

[0065] The acquired segmented images 401 and 403 are obtained by moving vertically along a direction and photographing the outer surface of the object 500 with repeating regions. Segmented images 401 and 403 repeat through repeating regions 401A and 403A. Furthermore, repeating regions are areas that reflect the same subject, ideally identical images. The CPU 330 uses the repeating regions of multiple segmented images to extract feature points from each image. Corresponding points are extracted by matching feature points between different images, and a composite image is generated through two-dimensional panoramic synthesis.

[0066] However, when the outer surface of the object 500 is not planar (e.g., a cylindrical structure), the depth in the image differs. That is, in the case of a cylindrical structure, the area near the center of the structure is closer to the camera 200, while the areas near the ends farther from the center are farther from the camera 200. As a result, the depth in the image differs. When the depth in the image differs, the amount of translation in the two-dimensional image differs relative to the vertical translation of the camera 200. For example... Figure 4 As shown, the arrow Ar1, located near the center, has a different length than the arrows Ar2 and Ar3, located near the ends.

[0067] Therefore, when attempting to synthesize multiple segmented images using 2D panoramic synthesis, if corresponding points near the center are integrated, corresponding points near the ends will not be integrated. Conversely, if corresponding points near the ends are integrated, corresponding points near the center will not be integrated.

[0068] Since the corresponding points cannot be integrated, therefore, Figure 5As shown in Figure 5-1, panoramic composite images sometimes split. This splitting results in images that cannot be combined into a single composite image, thus generating two composite images. Furthermore, to merge the non-integrated corresponding points, one must... Figure 5 As shown in Figure 5-2, distortion sometimes occurs in the results of panoramic synthesis. Distortion is a phenomenon that causes the image to deform unnaturally. In this case, a composite image is generated, but the distortion of each image is large, and the connection with the whole is not smooth. Thus, if corresponding points are extracted by matching feature points in regions of different depths in the image, the result of panoramic synthesis will be distorted (split, distortion, etc.).

[0069] Therefore, in this example, by limiting the range of corresponding points when extracting corresponding points between different images, a synthesized image (synthetic result) with suppressed distortion in two-dimensional panoramic synthesis can be obtained.

[0070] Figure 6 This diagram illustrates the functional structure of the information processing device 300, and this block diagram illustrates the functions implemented by the CPU 330. Additionally, the storage unit 360, RAM 370, and ROM 380 also have the function of implementing the aforementioned functions.

[0071] The CPU 330 of the information processing device 300 includes an image acquisition unit 331, a composite object region acquisition unit 332, a corresponding point extraction region acquisition unit 333, a composite parameter calculation unit 334, and a composite image generation unit 335.

[0072] The image acquisition unit 331 acquires multiple segmented images of the object 500 obtained by segmented shooting (segmented shooting) via the input / output I / F 350. The image acquisition unit 331 can acquire segmented images from the camera 200. Furthermore, the image acquisition unit 331 can acquire multiple segmented images stored in the storage unit 360.

[0073] The object region acquisition unit 332 determines the object region to be synthesized for each of the multiple segmented images and acquires the synthesized object region. The synthesized object region acquired by the object region acquisition unit 332 is finally synthesized into a panoramic image, forming part of the synthesized image. The processing of the object region acquisition unit 332 will be described later.

[0074] The corresponding point extraction region acquisition unit 333 determines the region for extracting corresponding points from the synthesis object regions of each segmented image acquired by the synthesis object region acquisition unit 332 based on the structure of the object 500, and acquires the corresponding point extraction region. The corresponding point extraction region refers to an object region that includes multiple feature points and extracts corresponding points representing the correspondence between feature points used to calculate synthesis parameters through feature point matching. For example, as a method, feature points can be extracted from each synthesis object region, and corresponding points can be extracted between different images through matching based on the feature points contained in the corresponding point extraction region. Furthermore, as another method, corresponding point extraction regions can be acquired from each synthesis object region, feature points can be extracted from the corresponding point extraction regions, and corresponding points can be extracted between different images through matching based on the feature points contained in the corresponding point extraction regions. The processing of the corresponding point extraction region acquisition unit 333 will be described later.

[0075] The compositing parameter calculation unit 334 calculates the compositing parameters required to generate the composite image. The compositing parameter calculation unit 334 determines the compositing parameters based on the results of feature point matching (corresponding point extraction) between images. In this example, the compositing parameters are calculated based on corresponding points extracted from the corresponding point extraction region corresponding to the structure of the object 500. This avoids calculating compositing parameters that would cause distortion in the panoramic composite image due to the structure of the object 500. The compositing parameters include, for example, pose parameters of the shape model, pose parameters of the camera corresponding to each image, and lens distortion parameters of the camera corresponding to each image. The processing of the compositing parameter calculation unit 334 will be described later.

[0076] The composite image generation unit 335 generates at least one composite image based on the composite parameters. The processing of the composite image generation unit 335 will be described later.

[0077] Next, an example of an information processing method (in this example, the generation of a composite image) executed by the information processing device 300 will be described.

[0078] Figure 7 It is a flowchart representing the steps of an information processing method. Figure 8 This is a diagram used to illustrate the process of obtaining the composite object region. Figure 9 This is a diagram representing the method for obtaining the region of the composite object. Figure 10 This is a diagram illustrating another method for obtaining the region of a composite object.

[0079] First, multiple segmented images obtained by segmenting the photographed object 500 are acquired (step S10). As described above, the image acquisition unit 331 acquires multiple segmented images. The multiple segmented images are acquired as a series of images obtained by continuously photographing the outer surface of the object 500. Specifically, they are segmented images of the upper and lower directions of the outer surface of the object 500 captured by the camera 200 of the moving body 100 (see reference). Figure 2 Furthermore, in this example, object 500 has a cylindrical shape that extends in the vertical direction.

[0080] Next, the composite object region of each segmented image is acquired (step S11). The composite object region acquisition unit 332 determines and acquires the composite object region based on each segmented image acquired by the image acquisition unit 331. Specifically, as follows Figure 8 As shown in Figure 8-1, the image acquisition unit 331 acquires multiple segmented images 801 and 802. The segmented images 801 and 802 shown in Figure 8-1 also include images other than the object 500. Figure 8 As shown in Figure 8-2, the composite object region acquisition unit 332 acquires the composite object region 801A surrounded by black lines from the segmented image 801 in a manner that includes the object 500 and, as far as possible, excludes areas other than the object 500. Similarly, the composite object region 802A surrounded by black lines is acquired from the segmented image 802. For example, the composite object regions 801A and 802A are regions extending from the left and right ends of the object 500 to the right ends (including a small portion of the outer side or a small portion of the inner side). Furthermore, regions unrelated to the object 500, such as distant backgrounds, can be excluded, and this range is defined as the composite object region. As described later, the images of the composite object regions 801A and 802A are used to generate a composite image.

[0081] Figure 9 This is a diagram representing a specific example when acquiring the region of a composite object. Figure 9 In the method described in 9-1, the user determines the compositing object region, and the compositing object region acquisition unit acquires the compositing object region. As shown in 9-1, the compositing object region acquisition unit 332 displays the segmented image 901 of the object 500 on the display device 320. For example, the user specifies four vertices P for the displayed segmented image 901. The compositing object region acquisition unit 332 determines the inner side as the compositing object region and acquires the compositing object region 901A.

[0082] As another method, in Figure 9In the method described in 9-2, the composite object region acquisition unit automatically determines and acquires the composite object region. As shown in 9-2, the composite object region acquisition unit 332 automatically extracts the concrete region as the object 500 from the segmented image 902. Then, the composite object region acquisition unit 332 acquires the concrete region as the composite object region 902A. For example, by using a learning model based on an image recognition AI (Artificial Intelligence) algorithm, the composite object region acquisition unit 332 can automatically extract the concrete region. Depending on the material of the object 500, the composite object region acquisition unit 332 can not only automatically extract the concrete region but also automatically extract the metal region and acquire the metal region as the composite object region.

[0083] In addition, Figure 9 The image shows the process of acquiring the composite object region one by one from multiple segmented images.

[0084] Figure 10 This is a diagram illustrating another method for obtaining the region of a composite object. Figure 10 In the method, with Figure 9 In different cases, the composite object region is obtained from multiple segmented images. For example... Figure 10 As shown in Figure 10-1, multiple segmented images 1001, 1002, and 1003 are arranged vertically, with the center of the image of object 500 aligned. For example... Figure 10 As shown in Figure 10-2, the composite object region acquisition unit 332 acquires the composite object region 1000A from the arranged segmented images 1001, 1002, and 1003 simultaneously. The composite object region acquisition unit 332 can acquire the composite object region 1000A simultaneously by receiving the user's specification of four vertices P, or it can acquire the composite object region 1000A simultaneously by utilizing a learning model based on an image recognition AI algorithm.

[0085] Next, based on the structure of the object, corresponding point extraction regions are obtained from each segmented image (step S12). The corresponding point extraction region acquisition unit 333 determines and acquires the corresponding point extraction regions based on the composite object region acquired by the composite object region acquisition unit 332. As an example, the corresponding point extraction region is a defined region further inside the composite object region, which is considered to be a roughly planar region.

[0086] Specifically, such as Figure 11-1 As shown, the composite object region acquisition unit 332 acquires composite object regions 801A and 802A from the segmented images 801 and 802. Figure 11As shown in Figure 11-2, the corresponding point extraction region acquisition unit 333 acquires the corresponding point extraction regions 801B and 802B from the range of the composite object regions 801A and 802A of each segmented image 801 and 802 according to the structure of the object 500.

[0087] For example, when the object 500 is cylindrical, as described, the depth varies in images far from the center. The inventors discovered that if corresponding points are extracted and synthesis parameters are calculated including image regions near the ends, panoramic synthesis becomes distorted, thus completing this invention. When the object 500 is cylindrical, it is preferable to obtain regions within the object 500 that are nearly planar in shape, as the corresponding point extraction regions 801B and 802B. If the corresponding point extraction regions 801B and 802B are nearly planar, they are less susceptible to the influence of image regions near the ends, thus suppressing distortion in panoramic synthesis.

[0088] Figure 12 This is a diagram representing a specific example of obtaining the extraction region for a corresponding point. Figure 12 In method 12-1, the user determines the corresponding point extraction region, and the corresponding point extraction region acquisition unit 333 acquires the corresponding point extraction region. For example... Figure 12 As shown in Figure 12-1, the corresponding point extraction region acquisition unit 333 displays the segmented image 1201 of the object 500 on the display device 320. For the segmented image 1201, the composite object region 1201A, indicated by the dashed line, is acquired. The user specifies four vertices P for the displayed segmented image 1201. The corresponding point extraction region acquisition unit 333 determines the inner side as the corresponding point extraction region and acquires the corresponding point extraction region 1201B.

[0089] As another method, in Figure 12 In method 12-2, a region setting parameter α is set, and the corresponding point extraction region is obtained from the composite object region based on the region setting parameter α. For example... Figure 12 As shown in Figure 12-2, the composite object region 1202A was obtained from the segmented image 1202. The range of the composite object region 1202A in the left and right directions is defined as range W.

[0090] The corresponding point extraction region acquisition unit 333 determines the corresponding point extraction region from the segmented image 1202 that has acquired the synthetic object region 1202A according to the region setting parameter α, and acquires the corresponding point extraction region 1202B.

[0091] like Figure 12 As shown in Figure 12-2, the range W1 of the corresponding point extraction region 1202B is the range obtained by the following formula (1) based on the range W of the composite object region 1202A and the region setting parameter α (0 < α < 1). The region setting parameter α is an example of the ratio α of the present invention.

[0092] W1 = W × α……(1) The region setting parameter α can be predetermined, for example, to a value of 0.3 to 0.5. It can be arbitrarily determined based on the magnitude of the region setting parameter α or the results of past panoramic composites.

[0093] Figure 13 This is a diagram illustrating another method for obtaining the extraction region of corresponding points. Figure 13 In this method, a portion of a segmented image is used from multiple segmented images. The corresponding point extraction region acquisition unit 333 acquires multiple corresponding point extraction regions using a region setting parameter α as a variable. Next, the synthesis parameter calculation unit 334 calculates synthesis parameters based on the multiple corresponding point extraction regions, and the synthesis image generation unit 335 generates a synthesis image for verification. In step 13-1, the corresponding point extraction region acquisition unit 333 sets the region setting parameter α to 0.2; in step 13-2, it sets it to 0.3; in step 13-3, it sets it to 0.4; and in step 13-4, it sets it to 0.5. The region setting parameters α in steps 13-1 to 13-4 become candidate parameters before final determination. Steps 13-1 to 13-4 represent the synthesis images for verification generated from the corresponding point extraction regions acquired according to their respective region setting parameters α (candidate parameters).

[0094] Image 13-1 cannot be synthesized from three segmented images into one, resulting in splitting. Image 13-2 can be synthesized from three segmented images into one, and the proportion of the filled region (PD) is small, indicating the best synthesis result. In images 13-3 and 13-4, distortion exists and the proportion of the filled region is large. The filled region refers to the blank area (black area) added to the boundary of the synthesized image, and it is preferable to have a small size.

[0095] Based on the results of the composite images used for verification (pages 13-1 to 13-4), the corresponding point extraction region acquisition unit 333 determines the final region setting parameter α based on the evaluation results and acquires the corresponding point extraction region. Subsequent processing is then performed.

[0096] As an evaluation method for determining the optimal region setting parameter α, the result of the synthesized image can be displayed, and the region setting parameter α can be determined by the user from the candidate parameters. Furthermore, the candidate parameter that results in a synthesized image with the minimum proportion of the filled region PD (minimum distortion) can be automatically determined as the region setting parameter α. For example, by using a learning model based on an image recognition AI algorithm, the region setting parameter α can be automatically determined.

[0097] Figure 14 This is a diagram illustrating another method for obtaining the region extracted from the corresponding points. Figure 14 The method determines and obtains the corresponding point extraction region based on the image quality distribution in the composite object region of the segmented image. For example...Figure 14 As shown in 14-1, the synthetic object region 1401A was obtained from the segmented image 1401.

[0098] like Figure 14 As shown in Figure 14-2, when the object 500 is cylindrical, if the focus is on the center of the cylinder for shooting, the peripheral part S (near the end) is located further away than the central part C, and therefore becomes blurry or has lower resolution. The corresponding point extraction region acquisition unit 333 determines the high-quality area of ​​the central part C as the corresponding point extraction region in the composite object region 1401A, and acquires the corresponding point extraction region 1401B.

[0099] As a method for extracting high-quality regions, image recognition AI can be utilized. Alternatively, image recognition AI can be used that learns to distinguish between frontal regions (high quality) and peripheral regions (low quality), and can differentiate between high-quality and low-quality regions. The corresponding point extraction region acquisition unit 333 can determine and acquire the corresponding point extraction region through image recognition AI. The judgment threshold based on image recognition AI can be appropriately changed.

[0100] The corresponding point extraction region acquisition unit 333 can acquire the bounding rectangle of the region determined by the threshold as the corresponding point extraction region 1401B. It can also determine the region setting parameter α to become the region bounding the region determined by the threshold. If the left and right directions of the bounding region are set to W1 and the left and right directions of the composite object region 1401A are set to W, the region setting parameter α can be obtained by the following formula (2).

[0101] α = W1 / W……(2) Another method is based on information obtained from camera 200. The distribution of defocus amount (defocus map) obtained from the image plane phase difference imaging sensor is correlated with the image and acquired. The corresponding point extraction region acquisition unit 333 sets the region with defocus amount less than a threshold as a high-quality region and acquires the region as the corresponding point extraction region 1401B.

[0102] When obtaining the corresponding point extraction region 1401B, the bounding rectangle of the region smaller than the threshold can be obtained as the corresponding point extraction region 1401B. Furthermore, the region setting parameter α can be determined to be the region bounded by the region smaller than the threshold. The region setting parameter α can be obtained by applying equation (2).

[0103] Figure 15 This is a diagram illustrating another method for obtaining the region extracted from the corresponding points. Figure 15 The method automatically determines and obtains the corresponding point extraction region from the synthetic object region of the segmented image based on the shooting distance.

[0104] Figure 15 15-1 and 15-2 represent the states of object 500 being photographed by camera 200 from different shooting distances. In 15-1, the distance between object 500 and camera 200 is greater than that in 15-2. Figure 15 15-3 represents the segmented image 1501, the composite object region 1501A, and the corresponding point extraction region 1501B obtained by shooting at a shooting distance of 15-1. Figure 15 15-4 represents the segmented image 1502, the composite object region 1502A, and the corresponding point extraction region 1502B obtained by shooting at a shooting distance of 15-2.

[0105] As shown in 15-1, when the shooting distance is far, the range (W) in the segmented image 1501 of the cylindrical wall of the object 500 becomes wider. On the other hand, the proportion of the range (W1) that can be regarded as a roughly planar area in the composite object region 1501A becomes smaller.

[0106] As shown in 15-2, when the shooting distance is close, the range (W) in the segmented image 1502 reflecting the cylindrical wall of the object 500 becomes narrower. On the other hand, the proportion of the area that can be regarded as a roughly flat plane (W2) in the composite object region 1502A becomes larger.

[0107] The corresponding point extraction area acquisition unit 333 changes and determines the area setting parameter α according to the shooting distance. For example, in the cases of 15-3 and 15-4, the area setting parameters α1 and α2 can be obtained by the following formula (3) respectively.

[0108] α1=W1 / W, α2=W2 / W……(3) The corresponding point extraction region acquisition unit 333 acquires the corresponding point extraction region 1501B from the composite object region 1501A according to the region setting parameter α1. Furthermore, the corresponding point extraction region acquisition unit 333 acquires the corresponding point extraction region 1502B from the composite object region 1502A according to the region setting parameter α2.

[0109] Figure 16 This is a diagram illustrating another method for obtaining the region extracted from the corresponding points. Figure 16 Methods and Figures 11 to 15 The methods are different. Figures 11 to 15 In this method, the image range in the width direction of the object 500 is limited to exclude the image near the ends of the object 500, thereby obtaining the corresponding point extraction region. On the other hand, Figure 16 In this method, the direction of movement of the camera 200 relative to the object 500 is used (see reference). Figure 2The distance limits the image range and obtains the corresponding point extraction region. Within the corresponding point extraction region with a limited width (left-right) direction, the movement (up-down) direction is also restricted, thereby improving the computational speed when extracting corresponding points and avoiding mismatches (matching different feature points). As a result, distortion in the synthesized image can be reduced.

[0110] When the position of the image of the captured object 500 or the distance of movement between images can be obtained, the approximate overlapping area of ​​the movement direction in the image can be estimated based on information such as shooting distance, focal length, sensor size, and pixel count. That is, in addition to the width direction, a corresponding point extraction area is also determined based on the movement distance, which is also defined in the movement direction (vertical). Taking into account deviations in shooting direction or shooting distance, it is preferable to set the corresponding point extraction area with a margin in the movement direction.

[0111] In the following explanation, Figure 16 Both 16-1 and 16-2 obtain the corresponding point extraction region based on the region setting parameter α.

[0112] Figure 16 The 16-1 represents three segmented images 1601, 1602, and 1602. (This is achieved through...) Figures 11 to 15 The method obtained corresponding point extraction regions 1601A, 1602A, and 1603A from three segmented images 1601, 1602, and 1603A. The vertical lengths of the corresponding point extraction regions 1601A, 1602A, and 1603A are approximately the same as the vertical lengths of the segmented images 1601, 1602, and 1602.

[0113] Similarly, Figure 16 16-2 represents three segmented images 1601, 1602, and 1603. Based on the movement distance of the camera 200, the corresponding point extraction region 1601B was obtained from segmented image 1601. Two corresponding point extraction regions 1602B and 1602B were obtained from segmented image 1602. The corresponding point extraction region 1603B was obtained from segmented image 1603. In segmented image 1601 in 16-2, the region higher than the corresponding point extraction region 1601B was not obtained as a corresponding point extraction region. This is because the upper region hardly overlaps with segmented image 1602.

[0114] Furthermore, in segmented image 1602, the intermediate region between two corresponding point extraction regions 1602B was not acquired as a corresponding point extraction region. This is because the intermediate region hardly overlaps with segmented images 1601 and 1603. Also, in segmented image 1603, the region lower than corresponding point extraction region 1603B was not acquired as a corresponding point extraction region. This is because the lower region hardly overlaps with segmented image 1602.

[0115] This enables high-speed computation and avoids mismatches.

[0116] The actual movement distance (Lrx, Lry) and the image movement distance (Lpx, Lpy) are related as follows.

[0117] If we set the shooting distance (D), focal length (F), sensor size (Sx, Sy) and image pixel count (Px, Py), then the relationship becomes the following equation (4).

[0118] x Shooting range (Ax) = D × Sx / F, y Shooting range (Ay) = D × Sy / F……(4) According to equation (4), the moving distance and the image moving distance are related as shown in equation (5).

[0119] Lpx=Lrx×Px / Ax, Lpy=Lry×Py / Ay……(5) Figure 17 Is with Figure 13 Similar to the method. Figure 13 In this process, a portion of the segmented images is used from multiple segmented images, and the region setting parameter α is used as a variable and varied to generate a composite image. The region setting parameter α is then determined based on the result.

[0120] exist Figure 17 In this method, multiple segmented images are captured, and the region setting parameter α is used as a variable and varied to generate a composite image, from which the best composite image is selected.

[0121] exist Figure 17 In this process, the corresponding point extraction region acquisition unit 333 sets the region setting parameter α to 0.2 in 17-1, 0.3 in 17-2, 0.5 in 17-3, and 0.8 in 17-4. 17-1 to 17-4 represent composite images generated based on the corresponding point extraction regions acquired according to the set region setting parameter α. The optimal composite image is selected from these.

[0122] In step 17-1 with α=0.2, a single composite image is generated. In step 17-2 with α=0.3, no composite image is generated; instead, two split composite images are produced. In steps 17-3 and 17-4 with α=0.5 and α=0.8, no single composite image is generated; instead, three split composite images are produced. Furthermore, in steps 17-3 and 17-4, distortion exists, and the proportion of filled regions (PDs) is high.

[0123] As a method for selecting the best synthesized image, the results of the synthesized images can be displayed, allowing the user to select the best one. Furthermore, the synthesized image can be automatically selected as a single image with the lowest proportion of the filled region PD (minimum distortion). For example, by using a learning model based on an image recognition AI algorithm, the best synthesized image can be automatically selected.

[0124] Next, feature points and corresponding points are extracted from the corresponding point extraction regions (step S13). If corresponding point extraction regions are obtained for each segmented image, the feature point extraction unit (not shown) of CPU330 extracts feature points from the corresponding point extraction regions. Feature point extraction is performed using known techniques such as SIFT (scale invariant feature transform), SURF (speeded up robust features), AKAZE (Accelerated-KAZE), or ORB (Oriented FAST and Rotated BRIEF).

[0125] The corresponding point extraction unit (not shown) of CPU330 matches feature points within the corresponding point extraction region between different segmented images and extracts corresponding points between different segmented images. Furthermore, the feature point extraction and matching (corresponding point extraction) utilizes known techniques.

[0126] Specifically, such as Figure 18 As shown, feature points P1 are extracted from corresponding point extraction regions 801B and 802B of segmented image 801 and segmented image 802, respectively. The extracted feature points P1 are matched with each other as indicated by arrow S1. That is, the matching of feature points P1 is performed between corresponding point extraction regions 801B and 802B, which are considered to be approximately planar. Therefore, in the calculation of the synthesis parameters described later, synthesis parameters that will not result in splitting or distortion when generating the synthesized image can be calculated.

[0127] Next, corresponding points in the regions are extracted based on the corresponding points between each segmented image, and synthesis parameters are calculated (step S14). The synthesis parameter calculation unit 334 calculates the synthesis parameters based on the correlation of the matched corresponding points. The synthesis parameters are parameters representing the movement, rotation, and deformation of the image during synthesis, including the projection transformation matrix and lens distortion parameters.

[0128] Figure 18 This represents an example of synthesis parameters. Synthesis parameters may include (1) the pose parameters of the shape model, (2) the pose parameters of the camera corresponding to each image, and (3) the lens distortion parameters of the camera corresponding to each image, etc.

[0129] (1) The pose parameters of the shape model can include planar models and cylindrical models (curved surface tunnels). The planar model includes the rotation matrix R. s and translation vector t s The cylindrical model includes the rotation matrix R. s Translation vector t s And the radius of the cylinder, etc. (2) The camera corresponding to each image i The attitude parameters include the rotation matrix R i Translation vector t i (3) Includes the camera corresponding to each image. i The lens distortion parameters (k1, k2) are used. These parameters are not limited to lens distortion; they function as parameters for various nonlinear distortions, such as those caused by absorbing walls. i represents all images. Furthermore, the synthesis parameters for planar compositing can be the projection transformation matrices of each image and the lens distortion parameters. The synthesis parameters are calculated using well-known techniques.

[0130] Next, a composite image is generated based on the compositing parameters (step S15). The composite image generation unit 335 generates at least one composite image using images within each composite object region based on the compositing parameters. Figure 20 This represents the composite image 2001 after panoramic synthesis. In this example, because the synthesis parameters were calculated based on the corresponding points of the regions extracted from the corresponding points between the segmented images, a composite image 2001 can be generated. A high-precision composite image 2001 with no splits and almost no filled areas can be generated. The composite image 2001 is displayed on the display device 320, allowing the user to view it.

[0131] <Preferred Implementation> Next, the preferred embodiments will be described.

[0132] The CPU330 can extract damage from multiple segmented or synthesized images and overlay damage-related information onto the synthesized image.

[0133] Specifically, the CPU330 performs damage detection processing based on multiple segmented images (so-called original images). For example, the CPU330 uses learned image recognition AI to determine whether a pixel (or group of pixels) in the image is damaged. Based on the pixel groups extracted as "damage", the CPU330 overlays an image generated using polylines, etc., onto the segmented images, thereby enabling the user to identify the damage condition of the object 500.

[0134] according to Figure 21 Let's take one example as an illustration. Figure 2121-1 represents the composite image 2001. The composite image 2101, for example, contains consecutive pixel groups 2101A and 2101B. On the other hand, in this state, it is unclear whether pixel groups 2101A and 2101B are damaged.

[0135] CPU330 analyzes the synthesized image 2101 to determine whether pixel group 2101A and pixel group 2101B correspond to "damage". In this example, CPU330 determines pixel group 2101A as damage and pixel group 2101B as not damage.

[0136] Figure 21 21-2 represents the composite image 2101 after image analysis. In the composite image 2101, an image containing damage information generated using polylines, etc., is overlaid on the pixel group 2101A identified as damaged. This is displayed in a way that allows it to be distinguished from the composite image 2101 of 21-1 before image analysis. In the composite image 2101 of 21-2, for example, the color and / or thickness of the damaged image 2101C are changed, or it is made semi-transparent, allowing pixel group 2101A to be transmitted. In addition to simulating damaged polylines, it is also possible to assign a unique number to the damage and display only that number.

[0137] On the other hand, pixel group 2101B, which is determined not to be damaged, displays an image identical to pixel group 2101B in 21-1. Furthermore, in 21-1, a composite image is shown as the image for analyzing damage; however, it could be a segmented image or a composite object region within a segmented image.

[0138] Next, another preferred embodiment will be described.

[0139] The CPU330 can perform processing such as cropping on the generated composite image and arrange them side by side to create an image that is a pseudo-unfolded image.

[0140] Figure 22 22-1 represents a composite image 2201 generated from a segmented image captured from direction D1 as shown in 22-4. To be presented as the unfolded diagram described later, the composite image 2201 is processed by cropping and other methods. Here, a portion of the composite image 2201 in the vertical and horizontal directions is cropped.

[0141] like Figure 22As shown in Figure 22-3, similar to composite image 2201, composite images 2202, 2203, and 2204 generated from segmented images captured from the remaining directions D2, D3, and D4 are processed by cropping and other methods. By arranging the cropped composite images 2201, 2202, 2203, and 2204 side by side, a pseudo-unfolded image can be created. The CPU 330 creates the pseudo-unfolded image and displays it, for example, on the display device 320. Figure 22 In section 22-4, an example is shown of shooting the object 500 from four directions D1 to D4, but it can also be shot from six or eight directions.

[0142] <Application to other objects> Next, the case in which the information processing apparatus and method of the embodiment are applied to the wall of a tunnel, rather than a cylindrical shape, will be described.

[0143] Figure 23 Figure 23-1 is a cross-sectional view of the tunnel, showing the state of the tunnel wall 602 being photographed by camera 200. When photographing the wall 602, the camera 200 is rotated circumferentially R while continuously photographing the wall 602. This yields multiple segmented images. However, the segmented images sometimes contain lower surface regions whose shapes deviate from the tunnel composite model (cylindrical model). Specifically, if photographed from the shooting direction SD indicated by the dashed line, then... Figure 23 As shown in Figure 23-2, segmented image 2301 includes the region of wall 602 and the region of lower surface 604. Wall 602 has a shape consistent with the cylindrical model, while lower surface 604 has a shape deviating from the cylindrical model. On the other hand, other segmented images (images taken from the shooting direction SD indicated by solid lines: not shown) only contain the region of wall 602, and therefore have a shape consistent with the cylindrical model. If this segmented image 2301 and other segmented images are synthesized to generate a composite image, distortion sometimes occurs in the composite image due to the influence of lower surface 604.

[0144] Therefore, in this example, as Figure 24 As shown in 24-1, the region containing the wall 602 and the region of the lower surface 604P, which is part of the lower surface 604, are obtained from the segmented image 2301 as the composite object region 2301A. On the other hand, as Figure 24As shown in Figure 24-2, the region containing the wall 602 is obtained from the segmented image 2301 as the corresponding point extraction region 2301B. Regions containing the wall 602 are also obtained from other segmented images (not shown) as corresponding point extraction regions. That is, corresponding point extraction regions are obtained based on the structure of the object. Since feature points and corresponding points are extracted from the corresponding point extraction region 2301B, and synthesis parameters are calculated, a high-precision synthetic image with minimal splitting or distortion in the circumferential direction can be generated.

[0145] The determination of the composite object region 2301A and the corresponding point extraction region 2301B can be made by the user from the displayed screen, or by image recognition AI, etc. Furthermore, in determining the corresponding point extraction region 2301B, a region with a constant shooting distance from camera 200 can be used as the corresponding point extraction region 2301B. The tunnel wall 602 can be considered to be approximately arc-shaped.

[0146] Figure 25 This diagram illustrates the extraction of corresponding points between the composite unfolded image and adjacent images in the direction of travel. Figure 25 25-1 represents the unfolded image of the synthetic image generated in this example onto a plane. Based on this result, it can be understood that no splitting or distortion occurred, and the synthetic image was appropriately generated.

[0147] Figure 25 Figure 25-2 is a diagram used to illustrate the extraction of corresponding points along the tunnel's direction of travel. For example... Figure 24 As shown, the composite image of the tunnel wall 602 requires both a composite image in the circumferential direction and a composite image in the direction of travel. In this example, in adjacent segmented images 2501 and 2502, feature points P1 in corresponding point extraction regions 2501A and 2502A are matched to extract corresponding points, as indicated by arrow S1. Therefore, when generating the composite image in the direction of travel, it is also possible to generate a high-precision composite image with less splitting or distortion.

[0148] Figure 26 26-1 is with Figure 23 Different cross-sectional views of the tunnel 600 are shown, illustrating the state of the tunnel wall 602 as photographed by camera 200. The tunnel wall 602 of 26-1 includes an arc-shaped top surface 602A and a planar side surface 602B. Therefore, in the case of shooting from the shooting direction SD indicated by the dashed line, as... Figure 26 As shown in Figure 26-2, the segmented image 2601 includes a side surface 602B that deviates from the top surface 602A of the cylindrical shape. When a composite image is generated from multiple segmented images including this segmented image 2601, splitting or distortion occurs in the composite image.

[0149] Therefore, in this example, the region containing the top surface 602A and the side surface 602B within the segmented image 2601 is obtained as the synthesis object region 2601A. On the other hand, the region containing the top surface 602A within the segmented image 2601 is obtained as the corresponding point extraction region 2601B. That is, the corresponding point extraction region is obtained according to the structure of the object. In this example, feature points and corresponding points are also extracted from the corresponding point extraction region 2601B, which is consistent with the circumferential model, and synthesis parameters are calculated. Therefore, a high-precision synthetic image with less splitting or distortion in the circumferential direction can be generated.

[0150] In addition, the composite object region 2601A and the corresponding point extraction region 2601B can be obtained using the acquisition method described above.

[0151] Figure 27 27-1 is with Figure 23 and Figure 26 Images of different tunnels. As shown in 27-1, tunnel 600 has a wall 602. On the other hand, sometimes accessories 606 (pipes, cables, lighting, reinforcement materials and fences installed on the wall 602) are laid on the wall 602.

[0152] Therefore, when the wall 602 of tunnel 600 is photographed, as shown in 27-2, the appendage 606 is projected into the segmented image 2701. When a composite image is generated from multiple segmented images including this segmented image 2701, splitting or distortion occurs in the composite image.

[0153] Therefore, in this example, the region containing the wall 602 and the appendage 606 within the segmented image 2701 is selected as the synthesis object region 2701A, and the region containing the wall 602 excluding the appendage 606 within the segmented image 2701 is selected as the corresponding point extraction region 2701B. That is, the corresponding point extraction region is obtained based on the structure of the object. In this example, feature points and corresponding points are extracted from the corresponding point extraction region 2701B, which does not contain regions unrelated to the synthesis image, and synthesis parameters are calculated. Therefore, a high-precision synthesis image with less splitting or distortion in the circumferential direction can be generated.

[0154] In addition, the composite object region 2701A and the corresponding point extraction region 2701B can be obtained using the acquisition method described above.

[0155] The present invention has been described above, but the present invention is not limited to the above examples. Various improvements or modifications can be made without departing from the spirit of the present invention.

[0156] (Regarding the processor's architecture) The CPU330 is implemented through various processors. These processors include general-purpose processors (CPUs and / or GPUs, Graphics Processing Units) that execute programs to perform various processing functions, FPGAs (Field Programmable Gate Arrays), programmable logic devices (PLDs) whose circuit structure can be modified after manufacturing, and application-specific integrated circuits (ASICs) with circuit structures specifically designed to perform specific processes. The terms "program" and "software" have the same meaning.

[0157] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types. For example, a processing unit can be composed of multiple FPGAs, or a combination of a CPU and an FPGA. Furthermore, a single processor can also constitute multiple processing units. As examples of a single processor constituting multiple processing units, firstly, as represented by computers used in clients, servers, etc., there are ways in which a single processor is composed of a combination of one or more CPUs and software, and that processor functions as multiple processing units. Secondly, as represented by Systems-On-Chip (SoC), there are ways in which a processor is used to implement the functions of the entire system, including multiple processing units, using a single IC (Integrated Circuit) chip. Thus, for various processing units, one or more of the aforementioned processors are used as the hardware structure. When the aforementioned processor or circuit executes software (programs), it stores the processor (computer) readable code of the executed software in a non-transitory recording medium such as ROM. Then, the processor refers to the software. The software stored in the non-transitory recording medium contains a program for executing the information processing method involved in this invention. Code can also be recorded in non-temporary recording media such as various optical and magnetic recording devices and semiconductor memories instead of in ROM. When processing with software, RAM is used as a temporary storage area, and data stored in EEPROM (Electrically Erasable and Programmable Read Only Memory) (not shown) can also be referenced.

[0158] Symbol Explanation 1-Shooting system, 100-Moving body, 102-Main body of the moving body, 104-Propulsion unit, 110-Universal joint, 120-Control device, 200-Camera, 210-Viewing angle, 250-Controller, 300-Information processing device, 310-Operating unit, 320-Display device, 330-CPU, 331-Image acquisition unit, 332-Composite object area acquisition unit, 333-Corresponding point extraction area acquisition unit, 334-Composite parameter calculation unit, 335-Composite image generation unit, 350-Input / output interface (I / F), 360-Storage unit, 370-RAM, 380-ROM, 401-Segmented image, 401A-Repeating region, 403-Segmented image, 403 A - Repeating region, 500 - Object, 600 - Tunnel, 602 - Wall, 602A - Top surface, 602B - Side surface, 604 - Lower surface, 604P - Lower surface, 606 - Accessory, 801 - Segmented image, 801A - Composite object region, 801B - Corresponding point extraction region, 802 - Segmented image, 802A - Composite object region, 802B - Corresponding point extraction region, 901 - Segmented image, 901A - Composite object region, 902 - Segmented image, 902A - Composite object region, 1000A - Composite object region, 1001 - Segmented image, 1002 - Segmented image, 1003 - Segmented image, 1201 - Segmented image, 1201A - Composite object region, 12 01B - Corresponding point extraction region, 1202 - Segmented image, 1202A - Composite object region, 1202B - Corresponding point extraction region, 1401 - Segmented image, 1401A - Composite object region, 1401B - Corresponding point extraction region, 1501 - Segmented image, 1501A - Composite object region, 1501B - Corresponding point extraction region, 1502 - Segmented image, 1502A - Composite object region, 1502B - Corresponding point extraction region, 1601 - Segmented image, 1601A - Corresponding point extraction region, 1601B - Corresponding point extraction region, 1602 - Segmented image, 1602A - Corresponding point extraction region, 1602B - Corresponding point extraction region, 1603 - Segmented image, 16 03A - Corresponding point extraction region, 1603B - Corresponding point extraction region, 2001 - Composite image, 2101 - Composite image, 2101A - Pixel group, 2101B - Pixel group, 2101C - Damaged image, 2201 - Composite image, 2202 - Composite image, 2203 - Composite image, 2204 - Composite image, 2301 - Segmented image, 2301A - Composite object region, 2302B - Corresponding point extraction region, 2501 - Segmented image, 2501A - Corresponding point extraction region, 2502 - Segmented image, 2502A - Corresponding point extraction region, 2601 - Segmented image, 2601A - Composite object region, 2601B - Corresponding point extraction region, 2701 - Segmented image.2701A - Composite object region, 2701B - Corresponding point extraction region.

Claims

1. An information processing apparatus, comprising a processor, The processor performs the following processing: Acquire multiple segmented images obtained by segmenting the photographed object; Obtain the composite object region of each segmented image; Based on the structure of the object, the corresponding point extraction region is obtained from each of the segmented images; Based on the corresponding points of the regions extracted from the corresponding points of the segmented images, the synthesis parameters are calculated. and Based on the synthesis parameters, at least one synthesized image is generated using images within each of the said synthesis object regions.

2. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The region located at a distance set by the imaging device that captures the segmented image is defined as the corresponding point extraction region.

3. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The region with a pre-defined ratio α relative to the synthetic object region within the range of greater than 0 and less than 1 is determined as the corresponding point extraction region.

4. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: Based on the evaluation results of the generated synthetic image, the corresponding point extraction region is determined.

5. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The synthesized object region is displayed, and the corresponding point extraction region is determined according to the instructions from the user.

6. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The corresponding point extraction region is determined based on the image quality distribution of the segmented image.

7. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The synthesized object region is determined from the segmented image through image recognition.

8. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The segmented image is displayed, and the synthesized object region is obtained according to instructions from the user.

9. The information processing apparatus according to claim 1, wherein, The at least one composite image is composed of two or more composite images. The processor performs the following processing: Arrange the two or more composite images side by side.

10. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: Damage is extracted from the multiple segmented images, and information related to the damage is overlaid on the synthesized image.

11. The information processing apparatus according to claim 1, wherein, The object is a cylinder.

12. The information processing apparatus according to claim 1, wherein, The object in question is the tunnel wall.

13. An information processing method, which is an information processing method based on an information processing device equipped with a processor, wherein, The processor performs the following processing: Acquire multiple segmented images obtained by segmenting the photographed object; Obtain the composite object region of each segmented image; Based on the structure of the object, the corresponding point extraction region is obtained from each of the segmented images; Based on the corresponding points of the regions extracted from the corresponding points of the segmented images, the synthesis parameters are calculated. and Based on the synthesis parameters, at least one synthesized image is generated using images within each of the said synthesis object regions.

14. A program that causes a processor of an information processing device to perform information processing, wherein, The program causes the processor to perform the following processing: Acquire multiple segmented images obtained by segmenting the photographed object; Obtain the composite object region of each segmented image; Based on the structure of the object, the corresponding point extraction region is obtained from each of the segmented images; Based on the corresponding points of the regions extracted from the corresponding points of the segmented images, the synthesis parameters are calculated. and Based on the synthesis parameters, at least one synthesized image is generated using images within each of the said synthesis object regions.

15. A recording medium that is non-transitory and computer-readable, and which records the program of claim 14.

Images