Information processing device, information processing method, program, and recording medium

Abstract

The present invention provides an information processing device, an information processing method, a program, and a recording medium with which it is possible to generate a synthesized image from a plurality of divided images with good accuracy. The information processing device includes a processor. The processor acquires a plurality of divided images obtained by dividedly imaging an object, acquires the region of each divided image to be synthesized, acquires a corresponding point extraction region from each divided image according to the structure of the object, calculates a synthesis parameter on the basis of a corresponding point in each corresponding point extraction region between each divided image, and generates on the basis of the synthesis parameter at least one synthesized image by using an image in each region to be synthesized.

Description

Information processing device, information processing method, program, and recording medium

[0001] The present invention relates to an information processing device, an information processing method, a program, and a recording medium.

[0002] There are known techniques for synthesizing a plurality of divided images into a panorama. For example, an image processing device described in Patent Literature 1 extracts feature points by excluding obstacles that hinder panorama synthesis, and then synthesizes the plurality of divided images into a panorama to generate a composite image by matching the feature points.

[0003] International Publication No. 2020 / 137405

[0004] One embodiment of the technique of the present disclosure provides an information processing device, an information processing method, a program, and a recording medium that can generate a composite image while being affected as little as possible by the structure of an object.

[0005] The information processing device of the first aspect is an information processing device that includes a processor, which acquires a plurality of divided images obtained by photographing an object in sections, acquires a synthesis target area for each divided image, acquires a corresponding point extraction area from each divided image according to the structure of the object, calculates synthesis parameters based on corresponding points in each corresponding point extraction area between each divided image, and generates at least one synthetic image using images within each synthesis target area based on the synthesis parameters.

[0006] In the information processing device of the second aspect, in the first aspect, the processor determines, as the corresponding point extraction area, an area located at a set distance from the image capturing device that captured the segmented image.

[0007] In the information processing apparatus of the third aspect, the processor is the same as in the first aspect, and determines, as the corresponding point extraction area, an area having a ratio α, which is set in advance in the range of greater than 0 and less than 1, relative to the synthesis target area.

[0008] In the information processing apparatus of a fourth aspect, the processor determines a corresponding point extraction region based on the evaluation result of the generated composite image in the first aspect.

[0009] In the information processing apparatus of the fifth aspect, in the first aspect, the processor displays a synthesis target region and determines a corresponding point extraction region based on an instruction from a user.

[0010] In the information processing device of a sixth aspect, in the first aspect, the processor determines a corresponding point extraction region for the divided image in accordance with the image quality distribution.

[0011] In a seventh aspect of the information processing device, in any one of the first to sixth aspects, the processor determines a synthesis target region from the divided images by image recognition.

[0012] In an information processing device of an eighth aspect, in any one of the first to sixth aspects, the processor displays the divided images and acquires the synthesis target region based on an instruction from the user.

[0013] In a ninth aspect of the information processing device, in any one of the first to eighth aspects, the at least one composite image is two or more composite images, and the processor arranges the two or more composite images side by side.

[0014] In a tenth aspect of the information processing device, in any one of the first to ninth aspects, the processor extracts damage based on the plurality of segmented images and superimposes information about the damage on the composite image.

[0015] An information processing device of an eleventh aspect is based on any one of the first to tenth aspects, wherein the object is a cylinder.

[0016] The information processing device of a twelfth aspect is the information processing device of any one of the first to tenth aspects, wherein the object is a wall surface of a tunnel.

[0017] An information processing method of a thirteenth aspect is an information processing method by an information processing device having a processor, in which the processor acquires a plurality of divided images obtained by dividing an object into photographs, acquires a synthesis target area for each divided image, acquires a corresponding point extraction area from each divided image according to the structure of the object, calculates synthesis parameters based on corresponding points in each corresponding point extraction area between each divided image, and generates at least one synthetic image using images within each synthesis target area based on the synthesis parameters.

[0018] The program of the fourteenth aspect is a program that causes a processor of an information processing device to perform information processing, and causes the processor to acquire a plurality of divided images obtained by photographing an object in sections, acquire a synthesis target area for each divided image, acquire a corresponding point extraction area from each divided image according to the structure of the object, calculate synthesis parameters based on corresponding points in each corresponding point extraction area between each divided image, and generate at least one synthetic image using images within each synthesis target area based on the synthesis parameters.

[0019] A fifteenth aspect of the recording medium is a non-transitory computer-readable recording medium on which the above-described program is recorded.

[0020] FIG. 1 is a conceptual diagram of an imaging system. FIG. 2 is a diagram illustrating imaging of an object. FIG. 3 is a block diagram illustrating an example of the hardware configuration of an information processing device. FIG. 4 is a diagram illustrating an example of a plurality of divided images. FIG. 5 is a diagram illustrating an example of a failed panoramic synthesis. FIG. 6 is a diagram illustrating the functional configuration of an information processing device. FIG. 7 is a flowchart illustrating the procedure of an information processing method. FIG. 8 is a diagram illustrating the process of acquiring a synthesis target region. FIG. 9 is a diagram illustrating a method of acquiring a synthesis target region. FIG. 10 is a diagram illustrating another method of acquiring a synthesis target region. FIG. 11 is a diagram illustrating the process of acquiring a corresponding point extraction region. FIG. 12 is a diagram illustrating a method of acquiring a corresponding point extraction region. FIG. 13 is a diagram illustrating another method of acquiring a corresponding point extraction region. FIG. 14 is a diagram illustrating another method of acquiring a corresponding point extraction region. FIG. 15 is a diagram illustrating another method of acquiring a corresponding point extraction region. FIG. 16 is a diagram illustrating another method of acquiring a corresponding point extraction region. FIG. 17 is a diagram illustrating a method of acquiring the best synthesized image. FIG. 18 is a diagram illustrating the extraction of corresponding points. FIG. 19 is a diagram illustrating an example of synthesis parameters. FIG. 20 is a diagram illustrating an example of a synthesized image. FIG. 21 is a diagram illustrating a preferred embodiment. FIG. 22 is a diagram illustrating another preferred embodiment. Fig. 23 is a diagram explaining the photographing of the wall surface of a tunnel. Fig. 24 is a diagram explaining the synthesis target area and the corresponding point extraction area. Fig. 25 is a diagram explaining the synthesis unfolded image and the extraction of corresponding points between adjacent images in the traveling direction. Fig. 26 is a diagram explaining a tunnel with different wall surfaces. Fig. 27 is a diagram explaining a tunnel with different wall surfaces.

[0021] An information processing apparatus and an information processing method according to an embodiment will be described below with reference to the drawings.

[0022] 1 is a conceptual diagram illustrating an imaging system 1 including a moving object 100 equipped with a camera 200, a controller 250, and an information processing device 300. The information processing device 300 according to the embodiment is a device for generating a composite image from segmented images of an object 500 (see FIG. 2 ) captured by the camera 200, and is an example of the imaging device of the present invention.

[0023] The mobile body 100 has a mobile body main body 102, a propulsion unit 104 provided in the mobile body main body 102, and a control device 120 provided in the mobile body main body 102. The mobile body main body 102 is a main component of the mobile body 100. The mobile body main body 102 is controlled by the control device 120. The mobile body main body 102 is, for example, an unmanned aerial vehicle, such as a drone. The mobile body main body 102 has multiple propellers and a propeller drive motor. The propellers and the propeller drive motor form the propulsion unit 104.

[0024] Although an unmanned aerial vehicle is exemplified as the mobile body 102, the mobile body 102 may also be a mobile robot, a vehicle, or a ship. The mobile body 102 may be configured to be remotely controlled or autonomous. Remote control refers to a user operating the mobile body 102 by issuing instructions to the control device 120 from the controller 250 from a location remote from the mobile body 102. Autonomous refers to a user operating the mobile body 102 using a pre-created program or the like without user intervention. The program or the like is changed as appropriate depending on the location where the mobile body 100 is used.

[0025] The moving body 100 is equipped with a camera 200. The camera 200 is attached to the moving body body 102 via, for example, a gimbal 110. The camera 200 is controlled by a control device 120 provided in the moving body body 102. While the moving body 100 flies in the atmosphere, the camera 200 mounted on the moving body 100 captures an image of an object. The camera 200 is, for example, a digital camera, and is capable of capturing still images and moving images.

[0026] The information processing device 300 includes, for example, an operation unit 310, a display device 320, and a CPU (Central Processing Unit) 330. The CPU 330 generates a composite image by panoramic synthesis from the divided images. The display device 320 displays the divided images or the panoramic synthesis composite image. The user checks the divided images or the composite image displayed on the display device 320 and inputs commands such as corrections via the operation unit 310 as necessary.

[0027] Next, an example of a method for photographing the object 500 using the photographing system 1 will be described with reference to FIG.

[0028] 2, a user U captures an object 500 using a camera 200 mounted on a mobile object 100. The object 500 may be, for example, the outer surface of a bridge, a dam, a chimney, a building, a house, or a tank, or the inner surface of a tunnel. However, the object 500 is not limited to these.

[0029] The mobile object 100 flies around the object 500 based on a control signal transmitted from the controller 250. The camera 200 photographs the outer surface of the object 500 based on a control signal from the control device 120 in accordance with the movement of the mobile object 100. The camera 200 acquires a photographed image of the angle of view range 210 for each photograph. Because the object 500 is larger than the angle of view range 210 of the camera 200, the camera 200 photographs the object 500 in segments (segmented photographing). The camera 200 acquires multiple segmented images. The multiple segmented images acquired during the flight of the mobile object 100 are stored as a single image group, for example, in a memory (not shown) of the control device 120. In this example, the mobile object 100 moves vertically from bottom to top, and the camera 200 acquires segmented images of one side of the object 500. The camera 200 is a digital camera, and acquires a two-dimensional color image of the object 500 as the segmented image.

[0030] The control device 120 stores in memory each divided image, and identification information associated with each divided image, such as a file name, a file ID, and a photo number arbitrarily assigned to a position, or meta information.

[0031] 3 is a block diagram showing an example of the hardware configuration of an information processing device 300. As the information processing device 300, a personal computer or a workstation can be used.

[0032] The information processing device 300 includes an operation unit 310 , a display device 320 , a CPU 330 , an input / output interface (input / output I / F) 350 , a storage unit 360 , a RAM (Random Access Memory) 370 , and a ROM 380 .

[0033] Images obtained by photographing the object 500 in sections (multiple divided photographs) are input to the information processing device 300 via the input / output I / F 350 in a wired or wireless manner.

[0034] The CPU 330 controls the entire information processing device 300. The CPU 330 also generates a composite image from the divided images by panoramic synthesis.

[0035] The display device 320 is, for example, a device such as a liquid crystal display, and can display various types of information.

[0036] The operation unit 310 includes a keyboard and a mouse, and a user can cause the CPU 330 to perform necessary processing via the operation unit 310. By using a touch panel type device, the display device 320 can also function as the operation unit 310. Note that, although this example shows a system in which the controller 250 and the information processing device 300 are separate, the controller 250 and the information processing device 300 may be integrated.

[0037] The storage unit 360 is a memory configured from a hard disk drive, a flash memory, etc. The storage unit 360 stores an operating system that operates the information processing device 300, programs, data, etc.

[0038] Fig. 4 is a diagram showing a plurality of divided images acquired via the input / output I / F 350. Fig. 5 is a diagram showing an example of a failed panoramic synthesis.

[0039] The acquired divided images 401 and 403 were captured by moving the object 500 in the vertical direction so as to have an overlapping area of ​​the outer surface. The divided images 401 and 403 overlap with each other due to overlapping areas 401A and 403A. The overlapping areas are areas that show the same subject, and ideally, are the same image. The CPU 330 extracts feature points from each of the multiple divided images using the overlapping areas. Corresponding points are extracted by matching feature points between different images, and a composite image is generated by two-dimensional panoramic synthesis.

[0040] However, if the outer surface of the object 500 is not flat (for example, a cylindrical structure), the depth in the image will be different. That is, in the case of a cylindrical structure, the area near the center of the structure is close to the camera 200, and the area near the ends away from the center is farther away from the camera 200. As a result, the depth in the image will be different. When the depth in the image is different, the amount of translation on the two-dimensional image will be different in response to the translation of the camera 200 in the up and down direction. As shown in FIG. 4, the arrow Ar1 near the center and the arrows Ar2 and Ar3 near the ends have different lengths.

[0041] Therefore, when combining multiple divided images using two-dimensional panoramic synthesis, matching corresponding points near the center will result in mismatching of corresponding points near the edges, and conversely, matching corresponding points near the edges will result in mismatching of corresponding points near the center.

[0042] Because corresponding points no longer match, splitting may occur in the panoramic synthesis result, as shown in 5-1 in Figure 5. Splitting is a phenomenon in which panoramic synthesis does not result in a single composite image; in this case, two composite images are generated. Furthermore, when attempting to match inconsistent corresponding points, distortion may occur in the panoramic synthesis result, as shown in 5-2 in Figure 5. Distortion is a phenomenon in which an image is distorted unnaturally; in this case, a single composite image is generated, but the deformation of each image is significant and the overall image does not smoothly connect. In this way, extracting corresponding points by matching from feature points in areas of different depths in the image will result in a failure (splitting, distortion, etc.) in the panoramic synthesis result.

[0043] Therefore, in this example, when extracting corresponding points between different images, the range is limited so that a composite image (composite result) with reduced breakdown in two-dimensional panoramic synthesis can be obtained.

[0044] 6 is a diagram showing the functional configuration of the information processing device 300, and is a block diagram showing the functions realized by the CPU 330. Note that the storage unit 360, RAM 370, and ROM 380 also have the functions to realize the above functions.

[0045] The CPU 330 of the information processing device 300 includes an image acquisition unit 331 , a synthesis target region acquisition unit 332 , a corresponding point extraction region acquisition unit 333 , a synthesis parameter calculation unit 334 , and a synthetic image generation unit 335 .

[0046] The image acquisition unit 331 acquires a plurality of divided images obtained by dividing and photographing the object 500 (divided photography) via the input / output I / F 350. The image acquisition unit 331 can acquire the divided images from the camera 200. The image acquisition unit 331 can also acquire a plurality of divided images stored in the storage unit 360.

[0047] The synthesis target area acquisition unit 332 determines a target area to be synthesized for each of the multiple divided images and acquires the synthesis target area. The synthesis target areas acquired by the synthesis target area acquisition unit 332 are ultimately synthesized into a panoramic image to form part of the synthesized image. The processing of the synthesis target area acquisition unit 332 will be described later.

[0048] The corresponding point extraction region acquisition unit 333 determines a region from which corresponding points are extracted according to the structure of the object 500 from the compositing target region of each divided image acquired by the compositing target region acquisition unit 332, and acquires the corresponding point extraction region. The corresponding point extraction region refers to a region that includes multiple feature points and from which corresponding points that represent the correspondence between feature points used to calculate compositing parameters are extracted by feature point matching. For example, one method is to extract feature points from each compositing target region, and then extract corresponding points between different images by matching based on the feature points included in the corresponding point extraction region. Another method is to acquire a corresponding point extraction region from each compositing target region, extract feature points from the corresponding point extraction region, and then extract corresponding points between different images by matching based on the feature points included in the corresponding point extraction region. The processing of the corresponding point extraction region acquisition unit 333 will be described later.

[0049] The synthesis parameter calculation unit 334 calculates synthesis parameters necessary for generating a composite image. The synthesis parameter calculation unit 334 determines synthesis parameters from the results of feature point matching (corresponding point extraction) between images. In this example, the synthesis parameters are calculated based on corresponding points extracted from a corresponding point extraction region according to the structure of the object 500. This makes it possible to avoid calculating synthesis parameters that will cause the panoramic composite image to fail due to the structure of the object 500. The synthesis parameters include, for example, the attitude parameters of the shape model, the attitude parameters of the cameras corresponding to each image, and the lens distortion parameters of the cameras corresponding to each image. The processing of the synthesis parameter calculation unit 334 will be described later.

[0050] The composite image generating unit 335 generates at least one composite image based on the synthesis parameters. The processing of the composite image generating unit 335 will be described later.

[0051] Next, an example of an information processing method (generation of a composite image in this example) executed by the information processing device 300 of this example will be described.

[0052] Fig. 7 is a flowchart showing the procedure of an information processing method. Fig. 8 is a diagram for explaining the process of acquiring a compositing target region. Fig. 9 is a diagram showing a method of acquiring a compositing target region. Fig. 10 is a diagram showing another method of acquiring a compositing target region.

[0053] First, a plurality of divided images are acquired by dividing and photographing the object 500 (step S10). As described above, the image acquisition unit 331 acquires the plurality of divided images. The plurality of divided images are acquired as a series of images obtained by successively photographing the outer surface of the object 500. Specifically, the divided images are divided images of the outer surface of the object 500 in the vertical direction, photographed by the camera 200 of the moving body 100 (see FIG. 2). In this example, the object 500 has a cylindrical shape extending in the vertical direction.

[0054] Next, a compositing target region of each divided image is acquired (step S11). The compositing target region acquisition unit 332 determines a compositing target region from each divided image acquired by the image acquisition unit 331, and acquires the compositing target region. Specifically, as shown in 8-1 of FIG. 8, the image acquisition unit 331 acquires multiple divided images 801 and 802. The divided images 801 and 802 shown in 8-1 include images other than the object 500. As shown in 8-2 of FIG. 8, the compositing target region acquisition unit 332 acquires a compositing target region 801A surrounded by a black line from the divided image 801 so as to include the object 500 and to minimize the inclusion of anything other than the object 500. Similarly, a compositing target region 802A surrounded by a black line from the divided image 802. For example, the compositing target regions 801A and 802A are regions extending from one end to the other in the left-right direction of the object 500 (including slightly outside or slightly inside). Furthermore, it is possible to exclude areas such as the distant background that are not related to the object 500, and set this range as the synthesis target area. A synthetic image is generated as described below using the images of the synthesis target areas 801A and 802A.

[0055] FIG. 9 is a diagram showing a specific example of acquiring a compositing target area. In the method of 9-1 in FIG. 9, the user determines the compositing target area, and the compositing target area acquisition unit acquires the compositing target area. As shown in 9-1, the compositing target area acquisition unit 332 displays a divided image 901 of the object 500 on the display device 320. For example, the user specifies four vertices P for the displayed divided image 901. The compositing target area acquisition unit 332 determines the area inside as the compositing target area, and acquires the compositing target area 901A.

[0056] As another method, in the technique of 9-2 in FIG. 9 , the compositing target region acquisition unit automatically determines and acquires the compositing target region. As shown in 9-2, the compositing target region acquisition unit 332 automatically extracts the concrete region of the object 500 from the segmented image 902. Next, the compositing target region acquisition unit 332 acquires the concrete region as the compositing target region 902A. For example, the compositing target region acquisition unit 332 can automatically extract the concrete region by using a learning model based on an image recognition AI (Artificial Intelligence) algorithm. Depending on the material of the object 500, the compositing target region acquisition unit 332 can automatically extract not only the concrete region but also the metal region and acquire the metal region as the compositing target region.

[0057] FIG. 9 shows a case where a synthesis target region is acquired for each of a plurality of divided images.

[0058] FIG. 10 is a diagram showing another method for acquiring a compositing target area. Unlike the method shown in FIG. 9, the method shown in FIG. 10 acquires compositing target areas for multiple segmented images at once. As shown in 10-1 of FIG. 10, multiple segmented images 1001, 1002, and 1003 are arranged vertically so that the centers of the images of the object 500 are aligned. As shown in 10-2 of FIG. 10, the compositing target area acquisition unit 332 acquires a compositing target area 1000A from the arranged segmented images 1001, 1002, and 1003 at once. The compositing target area acquisition unit 332 may acquire the compositing target area 1000A at once in response to a user's instruction of four vertices P, or may acquire the compositing target area 1000A at once using a learning model based on an image recognition AI algorithm.

[0059] Next, a corresponding point extraction region is acquired from each divided image according to the structure of the object (step S12). The corresponding point extraction region acquisition unit 333 determines and acquires a corresponding point extraction region from the compositing target region acquired by the compositing target region acquisition unit 332. The corresponding point extraction region is, for example, a limited region further inside the compositing target region, and is an area that can be considered to be approximately flat.

[0060] 11-1, the synthesis target region acquisition unit 332 acquires synthesis target regions 801A and 802A from the divided images 801 and 802. As shown in 11-2 of FIG. 11, the corresponding point extraction region acquisition unit 333 acquires corresponding point extraction regions 801B and 802B from within the synthesis target regions 801A and 802A of the divided images 801 and 802, respectively, in accordance with the structure of the object 500.

[0061] For example, if the object 500 is cylindrical, as described above, the depth in the image varies as the distance from the center increases. The inventors discovered that panoramic synthesis fails if corresponding points are extracted including image regions near the edges to calculate synthesis parameters, leading to the present invention. When the object 500 is cylindrical, it is preferable to obtain regions of the object 500 that are close to a planar shape as the corresponding point extraction regions 801B and 802B. If the corresponding point extraction regions 801B and 802B are close to a planar shape, they are less likely to be affected by image regions near the edges, and failure of panoramic synthesis can be suppressed.

[0062] FIG. 12 is a diagram showing a specific example of acquiring a corresponding point extraction region. In the method of 12-1 in FIG. 12, the user determines the corresponding point extraction region, and the corresponding point extraction region acquisition unit 333 acquires the corresponding point extraction region. As shown in 12-1 in FIG. 12, the corresponding point extraction region acquisition unit 333 displays a divided image 1201 of the object 500 on the display device 320. A synthesis target region 1201A, indicated by a dashed line, is acquired for the divided image 1201. The user specifies four vertices P for the displayed divided image 1201. The corresponding point extraction region acquisition unit 333 determines the area inside as the corresponding point extraction region, and acquires the corresponding point extraction region 1201B.

[0063] As another method, in the technique of 12-2 in Fig. 12, a region setting parameter α is set, and a corresponding point extraction region is acquired from the synthesis target region based on the region setting parameter α. As shown in 12-2 in Fig. 12, a synthesis target region 1202A is acquired from a divided image 1202. The horizontal range of the synthesis target region 1202A is range W.

[0064] The corresponding point extraction region acquisition unit 333 determines a corresponding point extraction region from the divided image 1202 from which the synthesis target region 1202A has been acquired, based on the region setting parameter α, and acquires the corresponding point extraction region 1202B.

[0065] As shown in 12-2 of FIG. 12, the range W1 of the corresponding point extraction region 1202B is a range that can be calculated from the range W of the synthesis target region 1202A and a region setting parameter α (0<α<1) using the following formula (1). The region setting parameter α is an example of the ratio α of the present invention. W1=W×α (1)

[0066] The region setting parameter α is determined in advance to a value of, for example, 0.3 to 0.5, and can be arbitrarily determined based on the size of the region setting parameter α or the results of past panoramic synthesis.

[0067] FIG. 13 illustrates another method for acquiring corresponding point extraction regions. The method illustrated in FIG. 13 uses some of the multiple segmented images. The corresponding point extraction region acquisition unit 333 acquires multiple corresponding point extraction regions using the region setting parameter α as a variable. Next, the synthesis parameter calculation unit 334 calculates synthesis parameters based on the multiple corresponding point extraction regions, and the synthetic image generation unit 335 generates a synthetic image for verification. The corresponding point extraction region acquisition unit 333 sets the region setting parameter α to 0.2 in 13-1, 0.3 in 13-2, 0.4 in 13-3, and 0.5 in 13-4. The region setting parameters α in 13-1 to 13-4 are candidate parameters before being finally determined. References 13-1 to 13-4 illustrate synthetic images for verification generated from corresponding point extraction regions acquired based on the respective region setting parameters α (candidate parameters).

[0068] In image 13-1, the three divided images could not be combined into one, resulting in splitting. In image 13-2, the three divided images could be combined into one, and the proportion of padding area PD was small, showing the best combination result. In images 13-3 and 13-4, there is distortion and the proportion of padding area is large. The padding area is a margin area (black area) added to the boundary part of the combined image, and it is desirable that it is small.

[0069] From the results of the verification composite images 13-1 to 13-4, the corresponding point extraction region acquisition unit 333 determines the final region setting parameter α from the evaluation results and acquires the corresponding point extraction region, and then proceeds with the subsequent processing.

[0070] As a method for determining (evaluating) the most preferable region setting parameter α, the result of the composite image may be displayed, and the user may determine the region setting parameter α from among candidate parameters. Alternatively, the candidate parameter that results in a single composite image with the smallest proportion of padding region PD (least distortion) may be automatically determined as the region setting parameter α. For example, the region setting parameter α can be determined automatically by using a learning model based on an image recognition AI algorithm.

[0071] Fig. 14 shows another method for acquiring corresponding point extraction regions. In the method shown in Fig. 14, corresponding point extraction regions are determined and acquired according to the image quality distribution in the synthesis target regions of the divided images. As shown in 14-1 in Fig. 14, a synthesis target region 1401A is acquired from a divided image 1401.

[0072] 14-2, when the object 500 has a cylindrical shape, if an image is captured with the focus set on the center of the cylinder, the peripheral area S (near the edge) will be blurred and have low resolution because it is farther away than the central area C. The corresponding point extraction area acquisition unit 333 determines the high-image-quality area in the central area C within the synthesis target area 1401A as the corresponding point extraction area, and acquires the corresponding point extraction area 1401B.

[0073] Image recognition AI can be used as a method for extracting high-image-quality areas. Image recognition AI may be used that is trained to distinguish between high-image-quality areas and low-image-quality areas by learning the frontal area (high image quality) and the peripheral area (low image quality). The corresponding point extraction area acquisition unit 333 can determine and acquire the corresponding point extraction area by discrimination using the image recognition AI. The judgment threshold used by the image recognition AI may be changeable as appropriate.

[0074] The corresponding point extraction region acquisition unit 333 may acquire a circumscribing rectangle of the region determined by the threshold value as the corresponding point extraction region 1401B. The region setting parameter α may be determined so that the region circumscribing the region determined by the threshold value is circumscribing. If the horizontal range of the circumscribing region is W1 and the horizontal range of the synthesis target region 1401A is W, the region setting parameter α can be calculated by the following equation (2): α=W1 / W (2)

[0075] Another method is based on information obtained from the camera 200. A distribution of defocus amounts (defocus map) obtained from an image plane phase difference imaging sensor is acquired in association with the image. The corresponding point extraction region acquisition unit 333 determines a region where the defocus amount is smaller than a threshold value as a high-image-quality region, and acquires this region as the corresponding point extraction region 1401B.

[0076] When acquiring the corresponding point extraction region 1401B, a circumscribing rectangle of a region smaller than the threshold value may be acquired as the corresponding point extraction region 1401B. Alternatively, the region setting parameter α may be determined so that the region circumscribing the region smaller than the threshold value. The region setting parameter α can be calculated by applying equation (2).

[0077] Fig. 15 is a diagram showing another method for acquiring a corresponding point extraction region, in which a corresponding point extraction region is automatically determined and acquired from the synthesis target region of the divided images according to the shooting distance.

[0078] 15-1 and 15-2 in Fig. 15 show states in which an object 500 is photographed by a camera 200 from different shooting distances. The distance between the object 500 and the camera 200 in 15-1 is greater than the distance between the object 500 and the camera 200 in 15-2. 15-3 in Fig. 15 shows a divided image 1501, a synthesis target region 1501A, and a corresponding point extraction region 1501B photographed at the shooting distance of 15-1. 15-4 in Fig. 15 shows a divided image 1502, a synthesis target region 1502A, and a corresponding point extraction region 1502B photographed at the shooting distance of 15-2.

[0079] As shown in 15-1, when the shooting distance is long, the range (W) reflected in the divided image 1501 of the cylindrical wall surface of the object 500 becomes wider. On the other hand, the proportion of the range (W1) that can be regarded as a substantially flat surface in the synthesis target area 1501A becomes smaller.

[0080] As shown in 15-2, when the shooting distance is short, the range (W) reflected in the divided image 1502 of the cylindrical wall surface of the object 500 becomes narrow. On the other hand, the proportion of the area that can be regarded as a substantially flat surface (W2) in the synthesis target area 1502A becomes large.

[0081] The corresponding point extraction region acquisition unit 333 determines the region setting parameter α by changing it according to the shooting distance. For example, in the cases of 15-3 and 15-4, the region setting parameters α1 and α2 can be calculated using the following formula (3): α1=W1 / W, α2=W2 / W (3)

[0082] The corresponding point extraction region acquisition unit 333 acquires a corresponding point extraction region 1501B from the synthesis target region 1501A based on the region setting parameter α1. The corresponding point extraction region acquisition unit 333 also acquires a corresponding point extraction region 1502B from the synthesis target region 1502A based on the region setting parameter α2.

[0083] FIG. 16 is a diagram showing another method for acquiring corresponding point extraction regions. The method of FIG. 16 differs from the methods of FIGS. 11 to 15. The methods of FIGS. 11 to 15 acquire corresponding point extraction regions by restricting the image range in the width direction of the object 500 so as not to include images near the edges of the object 500. On the other hand, the method of FIG. 16 acquires corresponding point extraction regions by restricting the image range according to the distance in the movement direction (see FIG. 2) of the camera 200 relative to the object 500. By restricting the movement (up and down) direction in the corresponding point extraction region that limits the width (left and right) direction, the calculation speed when extracting corresponding points can be increased and erroneous matching of corresponding points (matching of different feature points) can be avoided. As a result, distortion of the composite image can be reduced.

[0084] If the position where the images of the object 500 were captured or the distance traveled between the images can be obtained, a rough overlap area in the direction of movement in the images can be estimated from information on the shooting distance, focal length, sensor size, and number of pixels. That is, a corresponding point extraction area is determined that is limited not only in the width direction but also in the direction of movement (vertical direction) according to the distance traveled. It is desirable to set the corresponding point extraction area with a margin in the direction of movement, taking into account variations in the shooting direction and shooting distance.

[0085] In the following description, 16-1 and 16-2 in FIG. 16 both acquire corresponding point extraction regions based on the region setting parameter α.

[0086] 16-1 in Fig. 16 shows three divided images 1601, 1602, and 1603. Corresponding point extraction regions 1601A, 1602A, and 1603A are obtained from the three divided images 1601, 1602, and 1602 by the techniques shown in Figs. 11 to 15. The vertical lengths of the corresponding point extraction regions 1601A, 1602A, and 1603A are approximately the same as the vertical lengths of the divided images 1601, 1602, and 1602.

[0087] Similarly, 16-2 in Fig. 16 shows three divided images 1601, 1602, and 1603. A corresponding point extraction region 1601B is acquired from divided image 1601 in accordance with the movement distance of camera 200. Two corresponding point extraction regions 1602B and 1602B are acquired from divided image 1602. A corresponding point extraction region 1603B is acquired from divided image 1603. In divided image 1601 in 16-2, the region above corresponding point extraction region 1601B is not acquired as a corresponding point extraction region. This is because the upper region barely overlaps with divided image 1602.

[0088] Furthermore, in divided image 1602, the intermediate region between the two corresponding point extraction regions 1602B is not acquired as a corresponding point extraction region because the intermediate region barely overlaps with divided image 1601 and divided image 1603. Furthermore, in divided image 1603, the region below corresponding point extraction region 1603B is not acquired as a corresponding point extraction region because the lower region barely overlaps with divided image 1602.

[0089] This makes it possible to increase the calculation speed and avoid erroneous matching.

[0090] The actual movement distance (Lrx, Lry) and the movement distance on the image (Lpx, Lpy) have the following relationship:

[0091] Given the shooting distance (D), focal length (F), sensor size (Sx, Sy), and number of image pixels (Px, Py), the relationship expressed by the following formula (4) is established: x shooting range (Ax) = D x Sx / F, y shooting range (Ay) = D x Sy / F (4) From formula (4), the relationship between the movement distance and the movement distance on the image is established by the following formula (5): Lpx = Lrx x Px / Ax, Lpy = Lry x Py / Ay (5)

[0092] Fig. 17 is a method similar to the method in Fig. 13. In Fig. 13, some divided images are used from among a plurality of divided images, and the region setting parameter α is changed as a variable to generate a composite image, and the region setting parameter α is determined from the result.

[0093] In the technique of FIG. 17, a plurality of captured divided images are used, and the region setting parameter α is changed as a variable to generate composite images, and the best composite image is selected from among them.

[0094] 17, the corresponding point extraction region acquisition unit 333 sets the region setting parameter α to 0.2 in 17-1, 0.3 in 17-2, 0.5 in 17-3, and 0.8 in 17-4. Reference numerals 17-1 to 17-4 show composite images generated based on the corresponding point extraction regions acquired based on the set region setting parameter α. The best composite image is selected from these.

[0095] In 17-1 where α = 0.2, one composite image is generated. In 17-2 where α = 0.3, one composite image is not generated, but two split composite images are generated. In 17-3 where α = 0.5 and 17-4 where α = 0.8, one composite image is not generated, but three split composite images are generated. Furthermore, in 17-3 and 17-4, there is distortion and the proportion of padding area PD is high.

[0096] As a method for selecting the best composite image, the composite image results may be displayed, and the user may select the best composite image. Alternatively, a composite image that results in a single composite image and has the smallest proportion of padding area PD (least distortion) may be automatically selected. For example, the best composite image can be automatically selected by using a learning model based on an image recognition AI algorithm.

[0097] Next, feature points and corresponding points are extracted from the corresponding point extraction regions (step S13). Once the corresponding point extraction regions for each divided image have been obtained, a feature point extraction unit (not shown) of the CPU 330 extracts feature points from the corresponding point extraction regions. Feature points are extracted using known techniques such as SIFT (scale invariant feature transform), SURF (speeded up robust features), AKAZE (Accelerated-KAZE), or ORB (Oriented FAST and Rotated BRIEF).

[0098] A corresponding point extraction unit (not shown) of the CPU 330 matches feature points within corresponding point extraction regions between different divided images and extracts corresponding points between the different divided images. Note that the extraction of feature points and the matching of feature points (extraction of corresponding points) are performed using known techniques.

[0099] 18 , feature points P1 are extracted from corresponding point extraction regions 801B of segmented image 801 and corresponding point extraction regions 802B of segmented image 802. The extracted feature points P1 are matched as indicated by arrow S1. That is, matching of feature points P1 is performed between corresponding point extraction regions 801B and 802B, which are considered to be substantially planar. Therefore, in calculating synthesis parameters, which will be described later, it is possible to calculate synthesis parameters that do not cause splitting or distortion when generating a synthesized image.

[0100] Next, synthesis parameters are calculated based on the corresponding points in each corresponding point extraction region between each divided image (step S14). The synthesis parameter calculation unit 334 calculates synthesis parameters based on the relationship between the matched corresponding points. The synthesis parameters are parameters that represent the movement, rotation, and deformation of the images during synthesis, and include a projective transformation matrix and lens distortion parameters.

[0101] 18 shows an example of the synthesis parameters, which may include (1) pose parameters of the shape model, (2) pose parameters of the camera corresponding to each image, and (3) lens distortion parameters of the camera corresponding to each image.

[0102] (1) The pose parameters of the shape model can include a plane model and a cylindrical model (curved tunnel). The plane model is defined by a rotation matrix R s and the translation vector t s etc., and the cylindrical model is represented by a rotation matrix R s , translation vector t s , and the radius of the cylinder. (2) The camera corresponding to each image i The orientation parameters of i , translation vector t i (3) Cameras corresponding to each image i The lens distortion parameters (k1, k2) are included. The lens distortion parameters act as parameters that absorb not only lens distortion but also various nonlinear distortions such as wall distortion. i indicates all images. Note that the synthesis parameters in the case of planar synthesis may be the projective transformation matrix and lens distortion parameters of each image. Known techniques are used to calculate the synthesis parameters.

[0103] Next, a composite image is generated based on the synthesis parameters (step S15). The composite image generation unit 335 generates at least one composite image using images in each synthesis target region based on the synthesis parameters. FIG. 20 shows a panoramic synthesized composite image 2001. In this example, synthesis parameters calculated based on corresponding points in each corresponding point extraction region between each divided image are used, so a single composite image 2001 can be generated. A highly accurate composite image 2001 can be generated with no splitting and almost no padding regions. The composite image 2001 is displayed on the display device 320, allowing the user to confirm the displayed composite image 2001.

[0104] <Preferred Embodiment> Next, a preferred embodiment will be described.

[0105] The CPU 330 can extract damage based on a plurality of divided images or a composite image, and superimpose information about the damage on the composite image.

[0106] Specifically, the CPU 330 executes a damage detection process that detects damage based on a plurality of segmented images (so-called original images). For example, the damage detection process involves the CPU 330 using trained image recognition AI to determine whether a pixel (or group of pixels) on an image is damaged. The CPU 330 superimposes an image generated using polylines or the like on the segmented images based on the group of pixels extracted as "damage," allowing the user to recognize the damage status of the object 500.

[0107] An example will be described with reference to Fig. 21. 21-1 in Fig. 21 shows a composite image 2001. The composite image 2101 includes, for example, a continuous pixel group 2101A and a pixel group 2101B. However, in this state, it is unclear whether the pixel group 2101A and the pixel group 2101B are damaged or not.

[0108] The CPU 330 analyzes the composite image 2101 and determines whether the pixel group 2101A and the pixel group 2101B correspond to “damage.” In this example, the CPU 330 determines that the pixel group 2101A is damaged and that the pixel group 2101B is not damaged.

[0109] 21-2 in FIG. 21 shows a composite image 2101 after image analysis. In the composite image 2101, an image containing damage information (damage image 2101C) generated using polylines or the like is superimposed on a pixel group 2101A determined to be damaged. The image is displayed so as to be distinguishable from the composite image 2101 of 21-1 before image analysis. In the composite image 2101 of 21-2, for example, the color and / or thickness of the damage image 2101C is changed, or the damage image 2101C is made semi-transparent, allowing the pixel group 2101A to be seen through. In addition to the polylines that mimic the damage, it is also possible to assign unique numbers to the damage and display only those numbers.

[0110] On the other hand, pixel group 2101B determined not to be damaged is displayed as the same image as pixel group 2101B in 21-1. Note that although a composite image is exemplified in 21-1 as the image for analyzing damage, it may also be a divided image or a region to be combined within a divided image.

[0111] Next, another preferred embodiment will be described.

[0112] The CPU 330 can process the generated composite image by trimming or the like, and arrange them side by side to create an image that resembles a development.

[0113] 22-1 in Fig. 22 shows a composite image 2201 generated from divided images captured from direction D1 indicated by 22-4. The composite image 2201 is processed, such as by cropping, to create a development view, which will be described later. In this example, the composite image 2201 is cropped in the top, bottom, left, and right directions.

[0114] As shown in 22-3 of FIG. 22, composite images 2202, 2203, and 2204 generated from the divided images captured from the remaining directions D2, D3, and D4 are subjected to processing such as trimming in the same manner as composite image 2201. By arranging the trimmed composite images 2201, 2202, 2203, and 2204 side by side, an image that resembles a development can be created. CPU 330 creates the image that resembles a development and displays it on, for example, display device 320. 22-4 of FIG. 22 illustrates an example in which object 500 is captured from four directions, D1 to D4, but it may also be captured from six or eight directions.

[0115] <Application to Other Objects> Next, a case where the information processing device and method according to the embodiment are applied to a wall surface of a tunnel as an object other than a cylindrical object will be described.

[0116] 23-1 in FIG. 23 is a cross-sectional view of a tunnel, showing the state in which the wall surface 602 of the tunnel 600 is photographed by the camera 200. When photographing the wall surface 602, the camera 200 is rotated in the circumferential direction R while continuously photographing the wall surface 602. This results in the acquisition of multiple segmented images. However, the segmented images may include a bottom surface region whose shape deviates from the tunnel composite model (cylindrical model). Specifically, when photographed from the photographing direction SD indicated by the dashed line, as shown in 23-2 in FIG. 23, a segmented image 2301 includes a region of the wall surface 602 and a region of the bottom surface 604. While the wall surface 602 has a shape that matches the cylindrical model, the bottom surface 604 has a shape that deviates from the cylindrical model. On the other hand, the other segmented image (an image photographed from the photographing direction SD indicated by the solid line: not shown) includes only the region of the wall surface 602, and therefore has a shape that matches the cylindrical model. When this divided image 2301 is combined with another divided image to generate a composite image, distortion may occur in the composite image due to the influence of the lower surface 604 .

[0117] Therefore, in this example, as shown in 24-1 in FIG. 24, an area including the wall surface 602 and a lower surface 604P that is a portion of the lower surface 604 is acquired from the divided image 2301 as a synthesis target area 2301A. Meanwhile, as shown in 24-2 in FIG. 24, an area including the wall surface 602 is acquired from the divided image 2301 as a corresponding point extraction area 2301B. Areas including the wall surface 602 are also acquired from other divided images (not shown) as corresponding point extraction areas. In other words, corresponding point extraction areas are acquired according to the structure of the object. Feature points and corresponding points are extracted from the corresponding point extraction area 2301B, and synthesis parameters are calculated, so that a highly accurate synthesized image with little division or distortion in the circumferential direction can be generated.

[0118] The synthesis target region 2301A and the corresponding point extraction region 2301B may be determined by the user from the displayed screen, or may be determined by image recognition AI, etc. In determining the corresponding point extraction region 2301B, an area at a constant shooting distance from the camera 200 may be set as the corresponding point extraction region 2301B. The tunnel wall surface 602 can be considered to be approximately arc-shaped.

[0119] Figure 25 is a diagram for explaining the extraction of corresponding points between a composite unfolded image and adjacent images in the direction of travel. 25-1 in Figure 25 shows an unfolded image obtained by unfolding the composite image generated in this example onto a plane. From this result, it can be seen that no splitting or distortion has occurred and the composite image has been generated properly.

[0120] 25-2 in FIG. 25 is a diagram for explaining the extraction of corresponding points in the direction of travel of a tunnel. A composite image of a tunnel wall 602 requires a composite image in the circumferential direction and a composite image in the direction of travel, as shown in FIG. 24. In this example, corresponding points are extracted by matching feature points P1 of corresponding point extraction regions 2501A and 2502A in adjacent divided images 2501 and 2502, as indicated by arrow S1. Therefore, even when generating a composite image in the direction of travel, a highly accurate composite image with little splitting or distortion can be generated.

[0121] 26-1 in Figure 26 is a cross-sectional view of a tunnel different from that in Figure 23, showing the state in which the wall surface 602 of the tunnel 600 is photographed by the camera 200. The wall surface 602 of the tunnel 600 in 26-1 includes an arc-shaped top surface 602A and a flat side surface 602B. Therefore, when photographed from the photographing direction SD shown by the dashed line, as shown in 26-2 in Figure 26, a divided image 2601 includes the cylindrical top surface 602A and the side surface 602B, which has a shape that deviates from the cylindrical top surface 602A. If a composite image is generated based on multiple divided images including this divided image 2601, the composite image will be split or distorted.

[0122] Therefore, in this example, an area within divided image 2601 that includes top surface 602A and side surface 602B is acquired as synthesis target area 2601A, while an area within divided image 2601 that includes top surface 602A is acquired as corresponding point extraction area 2601B. In other words, the corresponding point extraction area is acquired according to the structure of the object. In this example as well, feature points and corresponding points are extracted from corresponding point extraction area 2601B that matches the circular model, and synthesis parameters are calculated, so that a highly accurate synthesized image with little splitting or distortion in the circular direction can be generated.

[0123] The synthesis target region 2601A and the corresponding point extraction region 2601B can be acquired by the acquisition method already described.

[0124] 27-1 in Fig. 27 is a photographed image of a tunnel different from those in Fig. 23 and Fig. 26. As shown in 27-1, a tunnel 600 has a wall 602. Meanwhile, accessories 606 (pipes, cables, lighting, reinforcing materials, fences, etc. attached to the wall 602) may be laid on the wall 602.

[0125] Therefore, when a wall surface 602 of a tunnel 600 is photographed, an accessory object 606 is reflected in a divided image 2701, as shown in 27-2. When a composite image is generated based on a plurality of divided images including this divided image 2701, the composite image will be split or distorted.

[0126] Therefore, in this example, an area in the divided image 2701 that includes the wall surface 602 and the accessory object 606 is acquired as a synthesis target area 2701A, and an area in the divided image 2701 that includes the wall surface 602 excluding the accessory object 606 is acquired as a corresponding point extraction area 2701B. In other words, the corresponding point extraction area is acquired according to the structure of the object. In this example, feature points and corresponding points are extracted from the corresponding point extraction area 2701B that does not include areas that are not relevant to the synthesized image, and synthesis parameters are calculated, so that a highly accurate synthesized image with little splitting or distortion in the circumferential direction can be generated.

[0127] The synthesis target region 2701A and the corresponding point extraction region 2701B can be obtained by the above-mentioned method.

[0128] Although the present invention has been described above, the present invention is not limited to the above examples, and various improvements and modifications can be made without departing from the gist of the present invention.

[0129] [Regarding the Processor Configuration] The CPU 330 is realized by various processors. The various processors include a CPU, which is a general-purpose processor that executes a program and functions as various processing units, a GPU (Graphic Processing Unit), a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically to execute specific processing. The term "program" is synonymous with "software."

[0130] A single processing unit may be composed of one of these various processors, or two or more processors of the same or different types. For example, a single processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA. Multiple processing units may also be composed of a single processor. Examples of multiple processing units composed of a single processor include: a first configuration, as typified by computers used as clients or servers, in which a single processor is composed of one or more CPUs and software, and this processor functions as multiple processing units; a second configuration, as typified by systems on chips (SoCs), in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip; and various processing units are thus composed of one or more of the above-mentioned various processors as a hardware structure. When the above-mentioned processor or electrical circuit executes software (programs), the processor-readable code of the software to be executed is stored in a non-transitory recording medium such as a ROM. The processor then references the software. The software stored in the non-transitory recording medium includes a program for executing the information processing method according to the present invention. Instead of ROM, the code may be recorded on a non-transitory recording medium such as various magneto-optical recording devices or semiconductor memory. When processing using the software, for example, RAM is used as a temporary storage area, and data stored in, for example, an EEPROM (Electronically Erasable and Programmable Read Only Memory) (not shown) may also be referenced.

[0131] 1 Photography system 100 Mobile body 102 Mobile body main body 104 Propulsion unit 110 Gimbal 120 Control device 200 Camera 210 Viewing angle range 250 Controller 300 Information processing device 310 Operation unit 320 Display device 330 CPU 331 Image acquisition unit 332 Synthesis target area acquisition unit 333 Corresponding point extraction area acquisition unit 334 Synthesis parameter calculation unit 335 Synthesized image generation unit 350 Input / output interface (input / output I / F) 360 Storage unit 370 RAM 380 ROM 401 Divided image 401A Overlapping area 403 Divided image 403A Overlapping area 500 Object 600 Tunnel 602 Wall surface 602A Ceiling surface 602B Side surface 604 Bottom surface 604P Lower surface 606 Attachment 801 Segmented image 801A Synthesis target area 801B Corresponding point extraction area 802 Segmented image 802A Synthesis target area 802B Corresponding point extraction area 901 Segmented image 901A Synthesis target area 902 Segmented image 902A Synthesis target area 1000A Synthesis target area 1001 Segmented image 1002 Segmented image 1003 Segmented image 1201 Segmented image 1201A Synthesis target area 1201B Corresponding point extraction area 1202 Segmented image 1202A Synthesis target area 1202B Corresponding point extraction area 1401 Segmented image 1401A Synthesis target area 1401B Corresponding point extraction area 1501 Segmented image 1501A Synthesis target area 1501B Corresponding point extraction area 1502 Segmented image 1502A Region to be synthesized 1502B Corresponding point extraction region 1601 Segmented image 1601A Corresponding point extraction region 1601B Corresponding point extraction region 1602 Segmented image 1602A Corresponding point extraction region 1602B Corresponding point extraction region 1603 Segmented image 1603A Corresponding point extraction region 1603B Corresponding point extraction region 2001 Synthesized image 2101 Synthesized image 2101A Pixel group 2101B Pixel group 2101C Damaged image 2201 Synthesized image 2202 Synthesized image 2203 Synthesized image 2204 Synthesized image 2301 Segmented image 2301A Region to be synthesized 2302B Corresponding point extraction region 2501 Segmented image 2501A Corresponding point extraction region 2502 Divided image 2502A Corresponding point extraction area 2601 Divided image 2601A Synthesis target area 2601B Corresponding point extraction area2701 Divided image 2701A Synthesis target area 2701B Corresponding point extraction area

Claims

1. An information processing device including a processor, wherein the processor acquires a plurality of divided images obtained by photographing an object in sections, acquires a synthesis target area for each of the divided images, acquires a corresponding point extraction area from each of the divided images according to the structure of the object, calculates synthesis parameters based on corresponding points in each of the corresponding point extraction areas between each of the divided images, and generates at least one synthetic image using images within each of the synthesis target areas based on the synthesis parameters.

2. The information processing device according to claim 1, wherein the processor determines an area located at a set distance from the image capturing device that captured the segmented image as the corresponding point extraction area.

3. The information processing device according to claim 1, wherein the processor determines an area having a ratio α, which is set in advance in the range of greater than 0 and less than 1, relative to the synthesis target area as the corresponding point extraction area.

4. The information processing device according to claim 1, wherein the processor determines the corresponding point extraction area based on the evaluation result of the generated composite image.

5. The information processing device according to claim 1, wherein said processor displays said synthesis target area and determines said corresponding point extraction area based on an instruction from a user.

6. The information processing device according to claim 1, wherein the processor determines a corresponding point extraction area for the divided image according to image quality distribution.

7. The information processing device according to claim 1, wherein the processor determines the synthesis target region from the divided images by image recognition.

8. The information processing device according to claim 1, wherein the processor displays the divided images and acquires the synthesis target region based on a command from a user.

9. The information processing device according to claim 1, wherein the at least one composite image is two or more composite images, and the processor arranges the two or more composite images side by side.

10. The information processing device according to claim 1, wherein the processor extracts damage based on the plurality of segmented images and superimposes information about the damage on the composite image.

11. The information processing device according to claim 1, wherein the object is a cylinder.

12. The information processing device according to claim 1, wherein the object is a wall of a tunnel.

13. An information processing method using an information processing device equipped with a processor, wherein the processor acquires a plurality of divided images obtained by photographing an object in sections, acquires a synthesis target area for each of the divided images, acquires a corresponding point extraction area from each of the divided images according to the structure of the object, calculates synthesis parameters based on corresponding points in each of the corresponding point extraction areas between each of the divided images, and generates at least one synthetic image using images within each of the synthesis target areas based on the synthesis parameters.

14. A program that causes a processor of an information processing device to perform information processing, the program causing the processor to acquire a plurality of divided images obtained by photographing an object in sections, acquire a synthesis target area for each of the divided images, acquire a corresponding point extraction area from each of the divided images according to the structure of the object, calculate synthesis parameters based on corresponding points in each of the corresponding point extraction areas between each of the divided images, and generate at least one synthetic image using images within each of the synthesis target areas based on the synthesis parameters.

15. A non-transitory computer-readable recording medium on which the program according to claim 14 is recorded.

Images