Information processing device, information processing method, and information processing program

Abstract

Provided are an information processing device, an information processing method, and an information processing program which are capable of reducing required resources.　A management device (11) comprises a CPU 6 (0A). The CPU 6 (0A): determines, in accordance with the shape of a designated imaging target region, an imaging method for repeatedly imaging the imaging target region; acquires image data for a plurality of detailed partial images (93n) (e.g. (93a)) obtained through the imaging using the imaging method; and generates a synthesis image (95) relating to the imaging target region by performing resizing processing in a contraction direction and synthesis processing. on the image data of the plurality of detailed partial images (93n).

Description

Information processing device, information processing method, and information processing program

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program.

[0002] Patent Literature 1 describes an image signal processing device that captures a shooting range by sequentially changing the shooting direction so that overlapping image areas are created, generates image signals, and stores unit images based on the generated image signals in a storage medium in association with other unit images that form overlapping image areas. Patent Literature 2 describes a shooting location setting device that acquires an image of a shooting target, displays it on a display unit, generates information indicating the location of a specified shooting location in the captured image, cuts out an image including the periphery of the shooting location from the image of the shooting target, and generates a reference image. Patent Literature 3 describes an image processing device that acquires multiple shot images from different angles, reduces the acquired multiple shot images, combines the reduced multiple shot images to generate a preview image, and displays the preview image on a part of a display unit that displays the shot images.

[0003] Japanese Patent Application Publication No. 2004-194113 International Publication No. 2020 / 162264 Japanese Patent Application Publication No. 2017-011687

[0004] One embodiment of the technique of the present disclosure provides an information processing device, an information processing method, and an information processing program that can reduce pressure on computational resources.

[0005] (1) An information processing device having a processor, wherein the processor determines a photographing method for photographing a specified photographing area multiple times according to the shape of the specified photographing area, acquires multiple first image data obtained by photographing using the photographing method, and generates second image data related to the photographing area by performing a resizing process in a reduction direction and a synthesis process on the multiple first image data.

[0006] (2) The information processing device according to (1), wherein the processor is capable of accepting a designation of a shape other than a rectangle as the shape of the subject area to be photographed.

[0007] (3) The information processing device according to (2), wherein the processor generates the second image data representing a rectangular image when a designation of a shape other than a rectangle is accepted as the shape of the subject area.

[0008] (4) The information processing device according to any one of (1) to (3), wherein the photographing method includes acquiring a photographing angle of view and a photographing position in the photographing.

[0009] (5) An information processing device according to any one of (1) to (4), wherein the processor generates first correspondence information that associates a positional relationship between a first image represented by the first image data and a second image represented by the second image data.

[0010] (6) The information processing device according to (5), wherein the processor controls a display device to display a first image represented by the first image data corresponding to the coordinates of the specified second image data based on the first correspondence information.

[0011] (7) The information processing device according to any one of (1) to (6), wherein the resizing process is a resizing process that includes a geometric change.

[0012] (8) The information processing device according to (7), wherein the resizing process including the geometric change is a process of generating the second image data by a resizing process in the reduction direction based on the first image data, a geometric process that applies a geometric change different from reduction and enlargement to the first image data, and the synthesis process.

[0013] (9) The information processing device according to (8), wherein the processor performs the geometric processing, the resizing processing in the reduction direction, and the synthesis processing in this order.

[0014] (10) The information processing device according to (8) or (9), wherein the geometric processing includes a rotation processing.

[0015] (11) The information processing device according to (10), wherein the rotation process includes a process based on an imaging condition under which the first image data was obtained.

[0016] (12) The information processing device according to (10), wherein the rotation process includes a process of calculating a parameter for correcting a tilt of the angle of view during telephoto shooting.

[0017] (13) The information processing device according to any one of (1) to (12), wherein the processor acquires a reduction ratio in the resizing process in the reduction direction based on the size of the second image represented by the second image data.

[0018] (14) An information processing device according to any one of (1) to (13), wherein the processor is capable of controlling a rotation mechanism that rotates the photographing device that performs the photographing, and generates second correspondence information that associates the first image data with a control value of the rotation mechanism at the time of photographing when the first image data was obtained.

[0019] (15) The information processing device according to (14), wherein the processor outputs a control value corresponding to a specified piece of first image data from among the plurality of pieces of first image data, based on the second correspondence information.

[0020] (16) The information processing device according to (14) or (15), wherein the processor extracts first image data from the plurality of first image data based on the degree of approximation of a specified control value based on the second correspondence information.

[0021] (17) An information processing device according to any one of (1) to (16), wherein the processor acquires distance measurement information for a plurality of positions in the area to be photographed, the number of times being less than the number of photographs to be taken using the photographing method, and then controls the photographing device to perform the photographing using the photographing method based on the distance measurement information.

[0022] (18) An information processing method by an information processing device, wherein a processor of the information processing device determines a photographing method for photographing a specified photographing area multiple times according to the shape of the specified photographing area, acquires multiple first image data obtained by photographing using the photographing method, and generates second image data related to the photographing area by performing resizing processing in a reduction direction and combining processing on the multiple first image data.

[0023] (19) An information processing program for an information processing device, which causes a processor of the information processing device to execute the following process: determine a photographing method for photographing a specified photographing area multiple times according to the shape of the specified photographing area; acquire multiple first image data obtained by photographing using the photographing method; and generate second image data related to the photographing area by performing resizing processing in a reduction direction and combining processing on the multiple first image data.

[0024] According to the present invention, it is possible to provide an information processing device, an information processing method, and an information processing program that are capable of suppressing pressure on computational resources.

[0025] 1 is a diagram showing an example of an imaging system 1 equipped with an information processing device (management device 11) of this embodiment. FIG. 1 is a diagram showing an example of rotation of the camera 10 in the pitch direction by the rotation mechanism 16. FIG. 2 is a diagram showing an example of rotation of the camera 10 in the yaw direction by the rotation mechanism 16. FIG. 3 is a block diagram showing an example of the configuration of the optical system and electrical system of the camera 10. FIG. 4 is a diagram showing an example of the configuration of the electrical system of the rotation mechanism 16 and the management device 11. FIG. 5 is a flowchart showing an example of processing for photographing a photographing target area and processing for compositing a composite image by a CPU 60A of the management device 11. FIG. 6 is a diagram showing an example of designating a photographing target area 92 in a wide-angle image 91. FIG. 7 is a diagram showing an example of setting multiple photographing areas rn photographed by the camera 10 with respect to the photographing target area 92. FIG. 8 is a diagram showing an example of telephoto photographing of multiple photographing areas rn by the camera 10. FIG. 9 is a diagram showing detailed partial images 93a to 93c and reduced images 94a to 94c generated based on the detailed partial images 93a to 93c. FIG. 10 is a diagram showing a composite image 95 generated from the reduced image 94n. 1 is a flowchart showing an example of a process for displaying a composite image and a detailed partial image by a CPU of a management device; FIG. 2 is a diagram showing an example of a process for displaying a composite image and a detailed partial image by a CPU of a management device; FIG. 3 is a diagram showing an example of a process for designating an inspection location in a composite image and a detailed partial image displayed based on the designated position; FIG. 4 is a diagram showing an example of a process for inspecting a power line of a transmission tower; FIG. 5 is a diagram showing an example of a process for designating a photographing target area by a point cloud; FIG. 6 is a diagram showing an example of a process for designating a photographing target area by a line; FIG. 7 is a flowchart showing an example of a process for photographing a photographing target area and a process for compositing a composite image when the photographing target area is designated by a point cloud; FIG. 8 is a diagram showing an example of a process for interpolating between photographing areas when the photographing target area is designated by a point cloud; FIG. 9 is a flowchart showing an example of a process for displaying a composite image and a detailed partial image when the photographing target area is designated by a point cloud; 1 is a diagram showing an example of a pseudo wide-angle image 150 captured by the camera 10 when the subject to be captured exceeds the capture range of the wide-angle end of the camera 10. FIG.1 is a diagram showing the designation of a photographing target area in a pseudo wide-angle image 150. FIG. 2 is a diagram showing a modified example of a composite image 95 displayed on the display 13a when an inspection location is designated. FIG. 3 is a diagram showing an example of performing geometric processing when generating a composite image. FIG. 4 is a diagram showing an example of how an information processing program for management control is installed in the control device 60 of the management device 11 from a storage medium on which the information processing program is stored.

[0026] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0027] <Photography system according to an embodiment> Fig. 1 is a diagram showing an example of a photography system 1 equipped with an information processing device according to an embodiment. As shown in Fig. 1 as an example, the photography system 1 includes a camera 10 and a management device 11. The camera 10 is an example of a photography device in the present invention. The management device 11 is an example of an information processing device in the present invention.

[0028] The camera 10 is a camera used to inspect facilities (infrastructure) that form the basis of daily life and industrial activity. The camera 10 inspects, for example, the walls of buildings, power lines, wind turbines, etc. A camera capable of telephoto shooting, an ultra-high resolution camera, etc. may be used as the camera 10. A wide-angle camera may also be used as the camera 10. The camera 10 is installed via a swivel mechanism 16, which will be described later, and captures an image of a subject. The camera 10 transmits the captured image and image information related to the image to the management device 11 via a communication line 12.

[0029] The management device 11 includes a display 13a, a keyboard 13b, a mouse 13c, and a secondary storage device 14. Examples of the display 13a include a liquid crystal display, a plasma display, an organic EL (Electro-Luminescence) display, and a CRT (Cathode Ray Tube) display. The display 13a is an example of a display device of the present invention.

[0030] An example of the secondary storage device 14 is a hard disk drive (HDD). The secondary storage device 14 is not limited to an HDD, and may be any non-volatile memory such as a flash memory, a solid state drive (SSD), or an electrically erasable and programmable read-only memory (EEPROM).

[0031] The management device 11 receives the captured images and captured information sent from the camera 10 , and displays the received captured images and captured information on the display 13 a and stores them in the secondary storage device 14 .

[0032] The management device 11 performs photography control, which controls photography by the camera 10. For example, the management device 11 performs photography control by communicating with the camera 10 via the communication line 12. Photography control is a control that sets photography parameters for the camera 10 to take a photograph in the camera 10 and causes the camera 10 to take a photograph. The photography parameters include parameters related to exposure and parameters for the zoom position.

[0033] The management device 11 also controls the rotation mechanism 16 to control (pan and tilt) the shooting direction of the camera 10. For example, the management device 11 sets the rotation direction, rotation amount, rotation speed, etc. of the camera 10 in response to operations of the keyboard 13b or the mouse 13c, or touch operations on the screen of the display 13a.

[0034] <Rotation of camera 10 by rotation mechanism 16> Figure 2 is a diagram showing an example of rotation of camera 10 in the pitch direction by rotation mechanism 16. Figure 3 is a diagram showing an example of rotation of camera 10 in the yaw direction by rotation mechanism 16. The camera 10 is attached to the rotation mechanism 16. The rotation mechanism 16 enables the camera 10 to rotate.

[0035] Specifically, as shown in Fig. 2 as an example, the rotation mechanism 16 is a two-axis rotation mechanism that can rotate the camera 10 in a rotation direction (pitch direction) that intersects with the yaw direction and has a pitch axis PA as the central axis, and in a rotation direction (yaw direction) that has a yaw axis YA as the central axis, as shown in Fig. 3 as an example. Note that, although the rotation mechanism 16 according to the present embodiment is an example of a two-axis rotation mechanism, the technology of the present disclosure is not limited to this, and may be a three-axis rotation mechanism or a single-axis rotation mechanism.

[0036] <Configuration of Optical System and Electrical System of Camera 10> FIG. 4 is a block diagram showing an example of the configuration of the optical system and electrical system of the camera 10. As shown in FIG. 4, the camera 10 includes an optical system 15 and an image sensor 25. The image sensor 25 is located downstream of the optical system 15. The optical system 15 includes an objective lens 15A and a lens group 15B. The objective lens 15A and the lens group 15B are arranged in this order along the optical axis OA of the optical system 15 from the target subject side (object side) to the light receiving surface 25A side (image side) of the image sensor 25. The lens group 15B includes an anti-vibration lens 15B1, a focus lens (not shown), a zoom lens 15B2, and the like. The zoom lens 15B2 is supported by a lens actuator 21 (described later) so as to be movable along the optical axis OA. The anti-vibration lens 15B1 is supported by a lens actuator 17 (described later) so as to be movable in a direction perpendicular to the optical axis OA.

[0037] By increasing the focal length with the zoom lens 15B2, the camera 10 becomes telephoto, thereby narrowing the angle of view (narrowing the shooting range). By decreasing the focal length with the zoom lens 15B2, the camera 10 becomes wide-angle, thereby widening the angle of view (widening the shooting range).

[0038] The optical system 15 may include various lenses (not shown) in addition to the objective lens 15A and the lens group 15B. The optical system 15 may also include an aperture. The positions of the lenses, lens group, and aperture included in the optical system 15 are not limited, and the technology of the present disclosure is valid even if they are located at positions different from those shown in FIG. 4 .

[0039] The vibration-proof lens 15B1 is movable in a direction perpendicular to the optical axis OA, and the zoom lens 15B2 is movable along the optical axis OA.

[0040] The optical system 15 includes lens actuators 17 and 21. The lens actuator 17 applies a force to the vibration-proof lens 15B1 that fluctuates in a direction perpendicular to the optical axis of the vibration-proof lens 15B1. The lens actuator 17 is controlled by an OIS (Optical Image Stabilizer) driver 23. When the lens actuator 17 is driven under the control of the OIS driver 23, the position of the vibration-proof lens 15B1 fluctuates in a direction perpendicular to the optical axis OA.

[0041] The lens actuator 21 applies a force to the zoom lens 15B2 to move it along the optical axis OA of the optical system 15. The lens actuator 21 is controlled by the lens driver 28. When the lens actuator 21 is driven under the control of the lens driver 28, the position of the zoom lens 15B2 moves along the optical axis OA. When the position of the zoom lens 15B2 moves along the optical axis OA, the focal length of the camera 10 changes.

[0042] In addition, if the outline of the captured image is, for example, a rectangle with a short side in the direction of the pitch axis PA and a long side in the direction of the yaw axis YA, the angle of view in the direction of the pitch axis PA is narrower than the angle of view in the direction of the yaw axis YA and is also narrower than the angle of view of the diagonal.

[0043] By the optical system 15 configured in this way, light representing the photographing target area is focused on the light receiving surface 25A of the photographing element 25, and the photographing target area is photographed by the photographing element 25.

[0044] Incidentally, vibrations imparted to the camera 10 include, if outdoors, vibrations caused by passing cars, vibrations caused by wind, vibrations caused by road construction, etc., and if indoors, vibrations caused by the operation of an air conditioner, vibrations caused by people entering and exiting, etc. Therefore, the camera 10 shakes due to the vibrations imparted to the camera 10 (hereinafter also simply referred to as "vibrations").

[0045] In this embodiment, "shake" refers to a phenomenon in which the image of a target subject on the light-receiving surface 25A of the imaging element 25 in the camera 10 fluctuates due to a change in the positional relationship between the optical axis OA and the light-receiving surface 25A. In other words, "shake" can also be said to be a phenomenon in which the optical image formed on the light-receiving surface 25A fluctuates due to the tilt of the optical axis OA caused by vibrations applied to the camera 10. The fluctuation of the optical axis OA means, for example, that the optical axis OA is tilted with respect to a reference axis (for example, the optical axis OA before the shake occurs). Hereinafter, shake caused by vibrations will also be simply referred to as "shake."

[0046] Shake is included in the captured image as a noise component and affects the image quality of the captured image. Therefore, in order to remove the noise component included in the captured image due to shake, the camera 10 is equipped with a lens-side shake correction mechanism 29, an image sensor-side shake correction mechanism 45, and an electronic shake correction unit 33, which are used to correct shake.

[0047] The lens-side shake correction mechanism 29 and the image sensor-side shake correction mechanism 45 are mechanical shake correction mechanisms that apply power generated by a drive source such as a motor (for example, a voice coil motor) to a shake correction element (for example, the vibration-proof lens 15B1 and / or the image sensor 25) to move the shake correction element in a direction perpendicular to the optical axis of the imaging optical system, thereby correcting shake.

[0048] Specifically, the lens-side image stabilization mechanism 29 applies power generated by a drive source such as a motor (e.g., a voice coil motor) to the image stabilization lens 15B1, thereby moving the image stabilization lens 15B1 in a direction perpendicular to the optical axis of the photographing optical system, thereby correcting shake. The image sensor-side image stabilization mechanism 45 applies power generated by a drive source such as a motor (e.g., a voice coil motor) to the image sensor 25, thereby moving the image sensor 25 in a direction perpendicular to the optical axis of the photographing optical system, thereby correcting shake. The electronic image stabilization unit 33 corrects shake by performing image processing on the captured image based on the amount of shake. In other words, the image stabilization unit (image stabilization component) performs shake correction mechanically or electronically using a hardware and / or software configuration. Here, mechanical shake correction refers to shake correction achieved by mechanically moving shake correction elements such as the vibration-proof lens 15B1 and / or the imaging element 25 using power generated by a driving source such as a motor (e.g., a voice coil motor), and electronic shake correction refers to shake correction achieved, for example, by image processing performed by a processor.

[0049] As an example, as shown in FIG. 4, the lens-side shake correction mechanism 29 includes an anti-vibration lens 15B1, a lens actuator 17, an OIS driver 23, and a position sensor 39.

[0050] Various well-known methods can be used to correct shake using the lens-side shake correction mechanism 29. In this embodiment, the method used is to correct shake by moving the vibration-proof lens 15B1 based on the amount of shake detected by a shake amount detection sensor 40 (described later). Specifically, shake correction is performed by moving the vibration-proof lens 15B1 in a direction that cancels out the shake and by an amount that cancels out the shake.

[0051] A lens actuator 17 is attached to the vibration-proof lens 15B1. The lens actuator 17 is a shift mechanism equipped with a voice coil motor, and by driving the voice coil motor, the vibration-proof lens 15B1 is moved in a direction perpendicular to the optical axis of the vibration-proof lens 15B1. Note that, although a shift mechanism equipped with a voice coil motor is used as the lens actuator 17 here, the technology of the present disclosure is not limited to this, and other power sources such as a stepping motor or a piezoelectric element may be used instead of the voice coil motor.

[0052] The lens actuator 17 is controlled by an OIS driver 23. When the lens actuator 17 is driven under the control of the OIS driver 23, the position of the vibration-proof lens 15B1 is mechanically changed within a two-dimensional plane perpendicular to the optical axis OA.

[0053] The position sensor 39 detects the current position of the vibration-proof lens 15B1 and outputs a position signal indicating the detected current position. Here, a device including a Hall element is used as an example of the position sensor 39. Here, the current position of the vibration-proof lens 15B1 refers to the current position within a two-dimensional plane of the vibration-proof lens. The vibration-proof lens two-dimensional plane refers to a two-dimensional plane perpendicular to the optical axis of the vibration-proof lens 15B1. Note that in this embodiment, a device including a Hall element is used as an example of the position sensor 39, but the technology of the present disclosure is not limited to this, and a magnetic sensor, a photosensor, or the like may be used instead of the Hall element.

[0054] The lens-side image stabilization mechanism 29 corrects shake by moving the vibration-proof lens 15B1 along at least one of the pitch axis PA direction and the yaw axis YA direction within the actual image capture range. In other words, the lens-side image stabilization mechanism 29 corrects shake by moving the vibration-proof lens 15B1 within the two-dimensional vibration-proof lens plane by an amount corresponding to the amount of shake.

[0055] The image sensor side shake correction mechanism 45 includes the image sensor 25 , a BIS (Body Image Stabilizer) driver 22 , an image sensor actuator 27 , and a position sensor 47 .

[0056] As with the shake correction method used by lens-side shake correction mechanism 29, various well-known methods can be used to correct shake using image sensor-side shake correction mechanism 45. In this embodiment, the shake correction method used is to correct shake by moving image sensor 25 based on the amount of shake detected by shake amount detection sensor 40. Specifically, shake correction is performed by moving image sensor 25 in a direction that cancels out the shake and by an amount that cancels out the shake.

[0057] An imaging element actuator 27 is attached to the imaging element 25. The imaging element actuator 27 is a shift mechanism equipped with a voice coil motor, and by driving the voice coil motor, the imaging element 25 is moved in a direction perpendicular to the optical axis of the vibration-proof lens 15B1. Note that, although a shift mechanism equipped with a voice coil motor is used as the imaging element actuator 27 here, the technology of the present disclosure is not limited to this, and other power sources such as a stepping motor or a piezoelectric element may be used instead of the voice coil motor.

[0058] The imaging element actuator 27 is controlled by the BIS driver 22. When the imaging element actuator 27 is driven under the control of the BIS driver 22, the position of the imaging element 25 is mechanically moved in a direction perpendicular to the optical axis OA.

[0059] The position sensor 47 detects the current position of the imaging element 25 and outputs a position signal indicating the detected current position. Here, a device including a Hall element is used as an example of the position sensor 47. Here, the current position of the imaging element 25 refers to the current position within a two-dimensional plane of the imaging element. The two-dimensional plane of the imaging element refers to a two-dimensional plane perpendicular to the optical axis of the vibration-proof lens 15B1. Note that in this embodiment, a device including a Hall element is used as an example of the position sensor 47, but the technology of the present disclosure is not limited to this, and a magnetic sensor, a photosensor, or the like may be used instead of the Hall element.

[0060] The camera 10 includes a computer 19, a DSP (Digital Signal Processor) 31, an image memory 32, an electronic shake correction unit 33, a communication I / F 34, a shake amount detection sensor 40, and a UI (User Interface) device 43. The computer 19 includes a memory 35, a storage 36, and a CPU (Central Processing Unit) 37.

[0061] The image sensor 25, DSP 31, image memory 32, electronic shake correction unit 33, communication I / F 34, memory 35, storage 36, CPU 37, shake amount detection sensor 40, and UI device 43 are connected to a bus 38. The OIS driver 23 is also connected to the bus 38. Note that, in the example shown in Fig. 4, for convenience of illustration, one bus is shown as the bus 38, but multiple buses may be used. The bus 38 may be a serial bus or a parallel bus such as a data bus, an address bus, and a control bus.

[0062] The memory 35 temporarily stores various information and is used as a work memory. An example of the memory 35 is a random access memory (RAM), but other types of storage devices may be used. The storage 36 stores various programs for the camera 10. The CPU 37 reads the various programs from the storage 36 and executes them on the memory 35 to control the entire camera 10. Examples of the storage 36 include a flash memory, an SSD, an EEPROM, and an HDD. Furthermore, various types of non-volatile memory, such as a magnetoresistive memory or a ferroelectric memory, may be used instead of or in combination with the flash memory.

[0063] The image sensor 25 is a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The image sensor 25 captures images of the target subject at a predetermined frame rate under the direction of the CPU 37. The "predetermined frame rate" here refers to, for example, several tens to several hundreds of frames per second. The image sensor 25 itself may have a built-in control device (image sensor control device), in which case the image sensor control device performs detailed control of the image sensor 25 in accordance with the image capture instructions output by the CPU 37. The image sensor 25 may also capture images of the target subject at a predetermined frame rate under the direction of the DSP 31, in which case the image sensor control device performs detailed control of the image sensor 25 in accordance with the image capture instructions output by the DSP 31. The DSP 31 is sometimes referred to as an ISP (Image Signal Processor).

[0064] The light-receiving surface 25A of the imaging element 25 is formed by a plurality of photosensitive pixels (not shown) arranged in a matrix. In the imaging element 25, each photosensitive pixel is exposed to light, and photoelectric conversion is performed for each photosensitive pixel. The charge obtained by photoelectric conversion for each photosensitive pixel is an analog imaging signal representing the target subject. Here, a plurality of photoelectric conversion elements (e.g., photoelectric conversion elements with color filters) sensitive to visible light are used as the photosensitive pixels. In the imaging element 25, the plurality of photoelectric conversion elements include photoelectric conversion elements sensitive to R (red) light (e.g., photoelectric conversion elements with R filters corresponding to R), photoelectric conversion elements sensitive to G (green) light (e.g., photoelectric conversion elements with G filters corresponding to G), and photoelectric conversion elements sensitive to B (blue) light (e.g., photoelectric conversion elements with B filters corresponding to B). The camera 10 uses these photosensitive pixels to capture images based on visible light (e.g., light on the short wavelength side of approximately 700 nanometers or less). However, this embodiment is not limited to this, and imaging may be performed based on infrared light (e.g., light with a wavelength longer than approximately 700 nanometers). In this case, a plurality of photoelectric conversion elements sensitive to infrared light may be used as the plurality of photosensitive pixels. In particular, for imaging of short-wavelength infrared (SWIR), for example, an InGaAs sensor and / or a type-II quantum well (T2SL; Simulation of Type-II Quantum Well) sensor may be used.

[0065] The image sensor 25 performs signal processing such as A / D (Analog / Digital) conversion on the analog image signal to generate a digital image, which is a digital image signal. The image sensor 25 is connected to the DSP 31 via the bus 38, and outputs the generated digital image to the DSP 31 via the bus 38 in frame units.

[0066] Note that, although a CMOS image sensor is described here as an example of the imaging element 25, the technology of the present disclosure is not limited to this, and a CCD (Charge Coupled Device) image sensor may also be used as the imaging element 25. In this case, the imaging element 25 is connected to the bus 38 via an AFE (Analog Front End) (not shown) that has a built-in CCD driver. The AFE generates a digital image by performing signal processing such as A / D conversion on the analog imaging signal obtained by the imaging element 25, and outputs the generated digital image to the DSP 31. The CCD image sensor is driven by a CCD driver built into the AFE. Of course, the CCD driver may also be provided separately.

[0067] The DSP 31 performs various types of digital signal processing on the digital image. The various types of digital signal processing refer to, for example, demosaic processing, noise removal processing, gradation correction processing, color correction processing, etc. The DSP 31 outputs the digital image after digital signal processing to the image memory 32 for each frame. The image memory 32 stores the digital image from the DSP 31.

[0068] The shake amount detection sensor 40 is a device that includes, for example, a gyro sensor, and detects the amount of shake of the camera 10. In other words, the shake amount detection sensor 40 detects the amount of shake in each of a pair of axial directions. The gyro sensor detects the amount of rotational shake around each of the pitch axis PA, the yaw axis YA, and the roll axis RA (an axis parallel to the optical axis OA) (see FIG. 1 ). The shake amount detection sensor 40 detects the amount of shake of the camera 10 by converting the amount of rotational shake around the pitch axis PA and the amount of rotational shake around the yaw axis YA detected by the gyro sensor into the amount of shake in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA.

[0069] Here, a gyro sensor is used as an example of the shake amount detection sensor 40, but this is merely an example, and the shake amount detection sensor 40 may also be an acceleration sensor. The acceleration sensor detects the amount of shake in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA. The shake amount detection sensor 40 outputs the detected amount of shake to the CPU 37.

[0070] Furthermore, although an example in which the amount of shake is detected by a physical sensor, namely, the shake amount detection sensor 40, is given here, the technology of the present disclosure is not limited to this. For example, a motion vector obtained by comparing successive captured images stored in the image memory 32 in chronological order may be used as the amount of shake. Furthermore, the amount of shake that is ultimately used may be derived based on the amount of shake detected by the physical sensor and the motion vector obtained by image processing.

[0071] The CPU 37 acquires the amount of shake detected by the shake amount detection sensor 40, and controls the lens-side shake correction mechanism 29, the image sensor-side shake correction mechanism 45, and the electronic shake correction unit 33 based on the acquired amount of shake. The amount of shake detected by the shake amount detection sensor 40 is used for shake correction by each of the lens-side shake correction mechanism 29 and the electronic shake correction unit 33.

[0072] The electronic shake correction unit 33 is a device including an ASIC (Application Specific Integrated Circuit). The electronic shake correction unit 33 corrects shake by performing image processing on the captured image in the image memory 32 based on the amount of shake detected by the shake amount detection sensor 40.

[0073] Note that, although a device including an ASIC is exemplified as the electronic shake correction unit 33 here, the technology of the present disclosure is not limited to this and may be, for example, a device including a field programmable gate array (FPGA) or a programmable logic device (PLD). Furthermore, for example, the electronic shake correction unit 33 may be a device including multiple ASICs, FPGAs, and PLDs. Furthermore, a computer including a CPU, storage, and memory may be employed as the electronic shake correction unit 33. There may be a single CPU or multiple CPUs. Furthermore, the electronic shake correction unit 33 may be realized by a combination of hardware and software configurations.

[0074] The communication I / F 34 is, for example, a network interface, and controls the transmission of various information between the camera 10 and the management device 11 via a network. This network is, for example, a wide area network (WAN) such as the Internet, or a local area network (LAN). The communication I / F 34 performs communication between the camera 10 and the management device 11.

[0075] The UI device 43 includes a reception device 43A and a display 43B. The reception device 43A is, for example, a hard key or a touch panel, and receives various instructions from the user. The CPU 37 acquires the various instructions received by the reception device 43A and operates in accordance with the acquired instructions.

[0076] The display 43B displays various information under the control of the CPU 37. Examples of the various information displayed on the display 43B include the contents of various instructions accepted by the accepting device 43A, captured images, and the like.

[0077] <Configuration of Electrical Systems of Swivel Mechanism 16 and Management Device 11> Fig. 5 is a diagram showing an example of the configuration of the electrical systems of the swing mechanism 16 and the management device 11. As shown in Fig. 5 as an example, the swing mechanism 16 includes a yaw axis swing mechanism 71, a pitch axis swing mechanism 72, a motor 73, a motor 74, a driver 75, a driver 76, and communication I / Fs 79 and 80.

[0078] The yaw axis rotation mechanism 71 rotates the camera 10 in the yaw direction. The motor 73 generates power by being driven under the control of a driver 75. The yaw axis rotation mechanism 71 rotates the camera 10 in the yaw direction by receiving the power generated by the motor 73. The pitch axis rotation mechanism 72 rotates the camera 10 in the pitch direction. The motor 74 generates power by being driven under the control of a driver 76. The pitch axis rotation mechanism 72 rotates the camera 10 in the pitch direction by receiving the power generated by the motor 74.

[0079] The communication I / Fs 79 and 80 are, for example, network interfaces, and control the transmission of various information between the management device 11 and the rotation mechanism 16 via a network. This network is, for example, a WAN such as the Internet or a LAN. The communication I / Fs 79 and 80 perform communication between the rotation mechanism 16 and the management device 11.

[0080] 5, the management device 11 includes a display 13a, a secondary storage device 14, a control device 60, a receiving device 62, and communication I / Fs 66, 67, and 68. The control device 60 includes a CPU 60A, a storage 60B, and a memory 60C. The CPU 60A is an example of a processor according to the present invention.

[0081] The accepting device 62, the display 13a, the secondary storage device 14, the CPU 60A, the storage 60B, the memory 60C, and the communication I / F 66 are each connected to a bus 70. In the example shown in Fig. 5, for convenience of illustration, one bus is shown as the bus 70, but multiple buses may be used. The bus 70 may be a serial bus or a parallel bus including a data bus, an address bus, a control bus, etc.

[0082] The memory 60C temporarily stores various information and is used as a work memory. An example of the memory 60C is a RAM, but other types of storage devices may be used. The storage 60B stores various programs for the management device 11 (hereinafter simply referred to as "management device programs").

[0083] The CPU 60A reads out the management device program from the storage 60B and executes the read management device program on the memory 60C to control the entire management device 11. The management device program includes the information processing program of the present invention.

[0084] The communication I / F 66 is, for example, a network interface. The communication I / F 66 is communicatively connected to the communication I / F 34 of the camera 10 via a network, and controls the transmission of various information between the camera 10 and the communication I / Fs 67 and 68. The communication I / F 67 is communicatively connected to the communication I / F 79 of the rotation mechanism 16 via a network, and controls the transmission of various information between the communication I / F 67 and the yaw axis rotation mechanism 71. The communication I / F 68 is communicatively connected to the communication I / F 80 of the rotation mechanism 16 via a network, and controls the transmission of various information between the communication I / F 67 and the pitch axis rotation mechanism 72.

[0085] The CPU 60A receives captured images, capture information, and the like from the camera 10 via the communication I / F 66 and the communication I / F 34. The CPU 60A controls the capture operation of the camera 10 to capture the target area via the communication I / F 66 and the communication I / F 34.

[0086] The CPU 60A controls the driver 75 and motor 73 of the turning mechanism 16 via the communication I / F 67 and communication I / F 79, thereby controlling the turning operation of the yaw axis turning mechanism 71. The CPU 60A also controls the driver 76 and motor 74 of the turning mechanism 16 via the communication I / F 68 and communication I / F 80, thereby controlling the turning operation of the pitch axis turning mechanism 72.

[0087] The CPU 60A receives a user-specified target area for the camera 10. Depending on the form of the target area for the camera 10 specified by the user, the CPU 60A determines a method for the camera 10 to photograph the specified target area multiple times.

[0088] The CPU 60A can accept the designation of a photographic target area having a shape other than a rectangle, for example. The shape of the photographic target area includes, for example, an area designated by a point cloud, an area designated by a line, an area designated by enclosing a predetermined area, and the like. When an area is designated by a point cloud, for example, only the points may be photographed, or a composition such as pan / tilt angles and angle of view may be determined so that nearby points are connected by lines or curves and the area between the points is interpolated, or a composition such as pan / tilt angles and angle of view may be determined so that specified points, rather than nearby points, are connected by lines or curves and the area between the points is interpolated. When an area is designated by a line, for example, photography may be performed along the specified line. When an area is designated by enclosing a region, for example, comprehensive photography of only the specified region may be performed.

[0089] Determining the imaging method of the camera 10 for imaging the imaging target area includes, for example, determining each imaging area for multiple telephoto imaging operations by the camera 10. The imaging areas for multiple telephoto imaging operations refer to multiple imaging areas (for example, each of the dashed line areas r1, r2, and r3 shown in FIG. 9 ) determined by the imaging angle of view and imaging position (pan / tilt values) of the camera 10 that have been set.

[0090] The CPU 60A also acquires data on multiple detailed partial images obtained by multiple telephoto shots using the camera 10, and performs, for example, resizing of the detailed partial images in a reduction direction. Resizing in a reduction direction is a process of reducing the size of an image, for example, by reducing the number of pixels. The CPU 60A also generates a composite image (hereinafter also referred to as a whole image) of the entire photographic target area by performing a synthesis process on the reduced images that have been resized in the reduction direction. The detailed partial images are an example of a first image represented by the first image data of the present invention. The synthetic image is an example of a second image represented by the second image data of the present invention.

[0091] In addition, the CPU 60A generates a rectangular composite image by performing resizing and synthesis processing in the reduction direction based on detailed partial images not only when it receives a specification of a rectangular shape as the shape of the area to be photographed, but also when it receives a specification of a shape other than a rectangle.

[0092] In addition to generating a composite image by resizing in the reduction direction based on the detailed partial images and combining processing, the CPU 60A may also generate a composite image by, for example, geometric processing that applies a geometric change different from reduction and enlargement to the detailed partial images. The geometric processing includes, for example, rotation processing of the detailed partial images. The rotation processing is a projective transformation based on the shooting conditions of the camera 10 under which the detailed partial images were obtained. The shooting conditions of the camera 10 include the set angle of view and pan / tilt values ​​of the camera 10. The rotation processing also includes processing to calculate parameters that correct the tilt (rotational distortion) of the angle of view of the camera 10 during telephoto shooting.

[0093] The resizing process for the detailed partial image is a resizing process that includes geometric changes. The resizing process that includes geometric changes is a process that generates a composite image by performing a reduction process based on the detailed partial image, a geometric process that applies a geometric change different from reduction and enlargement to the detailed partial image, and a compositing process. The CPU 60A generates the composite image by performing the geometric process, the reduction process, and the compositing process in this order.

[0094] The CPU 60A also generates coordinate correspondence information that associates the detailed partial image with the position of the detailed partial image in the composite image. Based on the coordinate correspondence information, the CPU 60A identifies the detailed partial image that corresponds to a specified position (coordinates) in the composite image, and displays the identified detailed partial image on the display 13a. The coordinate correspondence information is stored in the memory 60C or the secondary storage device 14. The coordinate correspondence information is an example of first correspondence information of the present invention.

[0095] Furthermore, when performing resizing in the reduction direction on detailed partial images, the CPU 60A sets the reduction ratio for the resizing in the reduction direction based on the size of the composite image. The size of the composite image refers to the number of vertical and horizontal pixels in the composite image. For example, the CPU 60A sets the reduction ratio so that the composite of the reduced images after reduction is inscribed in a rectangle of the size of the composite image. In other words, when arranging multiple reduced images to form a composite image, the reduction ratio is set so that the total number of pixels of the reduced images fits within the number of pixels of the composite image.

[0096] Furthermore, the CPU 60A generates rotation correspondence information that associates the detailed partial image with the control value of the rotation mechanism 16 at the time of shooting when the detailed partial image was obtained. The control value of the rotation mechanism 16 refers to the pan and tilt control values ​​of the camera 10 rotated by the rotation mechanism 16. The rotation correspondence information is stored in the memory 60C or the secondary storage device 14. Note that if the camera 10 is a camera with a zoom function, the rotation correspondence information may be generated by further associating the zoom position at the time of shooting with the control value of the rotation mechanism 16. The rotation correspondence information is an example of second correspondence information of the present invention.

[0097] Furthermore, based on the rotation support information, the CPU 60A outputs a control value corresponding to a specified detailed partial image among the multiple detailed partial images to, for example, the rotation mechanism 16. This makes it possible to easily re-photograph a specified detailed partial image in the composite image by referring to the rotation support information. For example, if a user wants to inspect the current state of a scratch on the wall of a building that was detected during an inspection several days ago, the user can specify a specified inspection position in the composite image displayed on the display 13a. The control value corresponding to the partial image at the specified position is set in the rotation mechanism 16 based on the rotation support information, and the partial image at the specified position can be re-photographed.

[0098] Furthermore, the CPU 60A extracts detailed partial images from multiple detailed partial images based on the degree of approximation of the specified control value based on the rotation support information. This allows, for example, multiple sets of "multiple detailed partial images" captured at different times to be extracted by specifying one detailed partial image from a set, and then extracting detailed partial images from other sets with control values ​​similar to that detailed partial image. For example, if the wall surface of a building is inspected every six months and the user wants to detect how previously detected damage to the wall surface has changed every six months, the user can specify a predetermined check position in the composite image displayed on the display 13a. Based on the rotation support information, detailed partial images from multiple previous inspections corresponding to the control values ​​at the specified check position can then be extracted, allowing the user to check the extracted detailed partial images.

[0099] Furthermore, the CPU 60A acquires distance measurement information for multiple positions in the photographing target area a number of times less than the number of photographs taken of the photographing target area, and photographs the photographing target area multiple times based on the distance measurement information acquired, to obtain multiple detailed partial images. Specifically, the CPU 60A calculates distance measurement information for positions that have not been measured based on distance measurement information for positions that have been measured, and photographs the photographing target area multiple times based on distance measurement information including the calculated distance measurement information, to obtain multiple detailed partial images. The distance measurement information refers to the photographing distance or the focus position where the image is in focus.

[0100] The reception device 62 is, for example, the keyboard 13b, the mouse 13c, or the touch panel of the display 13a, and receives various instructions from the user. The CPU 60A acquires the various instructions received by the reception device 62 and operates in accordance with the acquired instructions. For example, when the reception device 62 receives processing content for the camera 10 and / or the rotation mechanism 16, the CPU 60A operates the camera 10 and / or the rotation mechanism 16 in accordance with the instruction content received by the reception device 62.

[0101] The display 13a displays various information under the control of the CPU 60A. Examples of the various information displayed on the display 13a include the contents of various instructions accepted by the acceptance device 62, and captured images and capture information received by the communication I / F 66. The CPU 60A causes the contents of various instructions accepted by the acceptance device 62, and captured images and capture information received by the communication I / F 66 to be displayed on the display 13a.

[0102] The secondary storage device 14 is, for example, a non-volatile memory, and stores various types of information under the control of the CPU 60 A. Examples of the various types of information stored in the secondary storage device 14 include captured images and capture information received via the communication I / F 66. The CPU 60 A stores the captured images and capture information received via the communication I / F 66 in the secondary storage device 14.

[0103] <Photography Processing and Image Processing by CPU 60A of Management Device 11> FIG. 6 is a flowchart showing an example of the "photography processing of the photographic target area" and the "composite image synthesis processing" by the CPU 60A of the management device 11.

[0104] Assume that an inspection of infrastructure (such as the walls of buildings, power lines, and wind turbines) is being performed using camera 10. Camera 10 is installed facing the object to be photographed, and the zoom position of the zoom lens is set to the wide-angle end. Data on the wide-angle image captured by camera 10 is transmitted to management device 11 via communication line 12. Note that if the object cannot be fully captured even when camera 10 is set to the wide-angle end, a wide-angle camera attached to camera 10 may be used to capture the object.

[0105] An operator (user) stands in front of the management device 11 and looks at the image captured by the camera 10 displayed on the display 13a. The operator performs inspection work while operating the camera 10 via the communication line 12 by operating the keyboard 13b or mouse 13c of the management device 11 or by touching the display 13a.

[0106] While the wide-angle image is being displayed on the display 13a, the CPU 60A of the management device 11 starts the process shown in FIG. 6 in response to an operation by the worker to specify an inspection area (area to be photographed).

[0107] The CPU 60A accepts the designation of a subject area in the wide-angle image displayed on the display 13a (step S11). As described above, the designation of the subject area includes designating the subject area using a point cloud on the wide-angle image displayed on the display 13a, designating the subject area using a line, designating the subject area by enclosing a predetermined area, etc. The designation of the subject area will be described in detail later with reference to FIG. 7.

[0108] Next, the CPU 60A sets a plurality of shooting areas for the shooting target area specified in step S11 (step S12). Since the size of the shooting area that can be shot by telephoto shooting with the camera 10 is determined by the settings for the wide-angle image displayed on the display 13a, the plurality of shooting areas are set based on that size. The plurality of shooting areas are set in an arrangement such that the specified shooting target area is included in at least the plurality of shooting areas. The setting of the plurality of shooting areas will be described in detail later with reference to FIG. 8.

[0109] Next, the CPU 60A sets the angle of view of the camera 10 to the telephoto side to capture each of the multiple capture areas set in step S12 (step S13). For example, the CPU 60A sets the zoom position of the zoom lens of the camera 10 to the telephoto end. However, there may be cases where scratches or cracks on the wall surface of a building are so large that they cannot fit into a single image when the camera 10 is set to the telephoto end. Therefore, the zoom position may be specified by the operator.

[0110] Next, the CPU 60A derives pan / tilt values ​​corresponding to the next shooting area among the multiple shooting areas arranged so as to include the shooting target area (step S14). The coordinates of the multiple shooting areas arranged on the wide-angle image displayed on the display 13a and the pan / tilt values ​​thereof can be calculated based on the size and positional relationship between the wide-angle image and the shooting areas, so they are calculated in advance and stored as correspondence information in the memory 60C or the secondary storage device 14. The CPU 60A derives the pan / tilt values ​​corresponding to the next shooting area based on the correspondence information calculated in advance.

[0111] Next, the CPU 60A controls the rotation operation of the rotation mechanism 16 based on the pan / tilt values ​​derived in step S14 (step S15). Specifically, as described above, the CPU 60A controls the rotation operation of the yaw axis rotation mechanism 71 and the pitch axis rotation mechanism 72 in the rotation mechanism 16. Note that when controlling the rotation mechanism 16, the CPU 60A may perform distance measurement and focusing on the shooting area before capturing a detailed partial image of the shooting area.

[0112] Next, the CPU 60A acquires photographed image data (detailed partial images) of the photographing area identified by the rotation control of step S15 (step S16). When the photographing direction of the camera 10 is set by the rotation control of step S15, the CPU 60A outputs, for example, a photographing instruction signal to instruct the camera 10 to photograph the photographing area. The CPU 60A acquires photographed image data of the photographing area photographed by the camera 10. The acquisition of photographed image data will be described in detail later with reference to FIGS. 9 and 10.

[0113] Next, the CPU 60A reduces the photographic image data of the detailed partial images acquired in step S16, and then combines the reduced images to update the display of the composite image composed of the reduced images (step S17). In this example, a method is adopted in which the composite image is updated by adding reduced images each time photographic image data is acquired. However, for example, after photographic image data of multiple photographic areas has been acquired, a composite image may be created using the photographic image data. The reduction and combination of photographic image data will be described in detail later with reference to FIGS. 10 and 11.

[0114] Next, when the reduced image is added and the display of the composite image is updated in step S17, the CPU 60A updates the coordinate correspondence information by adding information indicating the correspondence between the coordinates of the added reduced image on the composite image and the captured image data of the detailed partial image acquired in step S16 (step S18). The coordinate correspondence information will be described in detail later with reference to FIG. 12. Note that in this example, the coordinate correspondence information is updated (step S18) after the display of the composite image is updated (step S17), but the order of these processes may be reversed. In other words, the coordinate correspondence information may be updated first, and then the display of the composite image may be updated.

[0115] Next, the CPU 60A determines whether or not photography of all of the plurality of photography areas set in step S12 has been completed (step S19).

[0116] In step S19, if the photographing of all the set photographing areas has not been completed (step S19: No), the CPU 60A returns to step S14 and repeats each process. In step S19, if the photographing of all the set photographing areas has been completed (step S19: Yes), the CPU 60A ends this process.

[0117] <Designation of a Target Area for Photography> FIG. 7 is a diagram showing an example of designation of a target area for photography 92 in a wide-angle image 91. As shown in FIG. 7, the infrastructure inspection in this example is a case where the presence or absence of scratches or cracks on the wall surface of building B is inspected. The wide-angle image 91 is an image captured by the camera 10 and is displayed on the display 13a of the management device 11. The worker designates the area to be inspected as the target area for photography 92 by touching the display 13a. The target area for photography 92 may be designated, for example, by freehand designation or by polygonal designation. In the example shown in the figure, the target area for photography 92 is designated freehand, and the target area for photography 92 is shown to surround building B, which is the inspection target. The range of the target area for photography is not limited to the entire building B, and can be set to a range of any size.

[0118] <Setting of Shooting Areas for Shooting Target Area 92> Figure 8 is a diagram showing an example in which multiple shooting areas rn are set for the shooting target area 92 specified in Figure 7 to be photographed by telephotography with the camera 10. The multiple shooting areas rn are arranged so that the shooting target area 92 is included inside the multiple shooting areas rn. The size of one shooting area rn is determined, for example, by the telephoto setting of the camera 10. The multiple shooting areas rn are arranged vertically and horizontally in a rectangular mesh pattern, and are arranged so that adjacent shooting areas rn slightly overlap each other.

[0119] <Acquisition of photographed image data of photographing area rn> Figure 9 is a diagram showing an example of how the multiple photographing areas rn set in Figure 8 are sequentially photographed with a telephoto lens by the camera 10. In this example, as indicated by the arrows in Figure 9, the multiple photographing areas rn shown in Figure 8 are photographed in the vertical direction from the right column of building B (the left column as seen from the camera 10) to the left column (the right column as seen from the camera 10), and photographed image data of each photographing area is acquired. For example, in the photographing area in the fourth column from the right, photographed image data is acquired in the order of photographing area r1, photographing area r2, and photographing area r3 while tilting from the top to the bottom of building B.

[0120] <Reduction Processing of Captured Images> FIG. 10 shows detailed partial images 93a, 93b, and 93c generated from the captured image data of the captured areas r1, r2, and r3 acquired in FIG. 9 , and reduced images 94a, 94b, and 94c generated based on the detailed partial images 93a, 93b, and 93c. As described above, the detailed partial images 93a, 93b, and 93c (hereinafter also referred to as detailed partial image 93n) are high-quality images obtained by telephoto-capturing the captured areas r1, r2, and r3 using the camera 10. As described above, the reduced images 94a, 94b, and 94c are generated by reducing the detailed partial images 93a, 93b, and 93c. The reduction processing includes a reduction process for reducing the size and a reduction process for reducing the number of pixels. Specifically, the reduced image 94a is generated by performing a reduction process on the detailed partial image 93a, reducing the number of pixels to reduce the size. The reduced image 94b is an image generated by performing a reduction process on the detailed partial image 93b to reduce the number of pixels and thereby reduce the size. The reduced image 94c is an image generated by performing a reduction process on the detailed partial image 93c to reduce the number of pixels and thereby reduce the size.

[0121] <Generation of Composite Image> FIG. 11 is a diagram illustrating a composite image 95 generated from the reduced images 94a, 94b, 94c, etc. (hereinafter, also referred to as reduced image 94n) generated in FIG. 10. The composite image 95 is generated by compositing the reduced images 94n. As described above, the compositing process of the composite image 95 is updated each time a reduced image 94n is generated. For example, when a reduced image 94a is generated based on the detailed partial image 93a, the composite process is performed by fitting the generated reduced image 94a into a position corresponding to the shooting area r1 from which the detailed partial image 93a was acquired. Next, when a reduced image 94b is generated based on the detailed partial image 93b, the composite process is performed by fitting the reduced image 94b into a position corresponding to the shooting area r2. Next, when a reduced image 94c is generated based on the detailed partial image 93c, the composite process is performed by fitting the reduced image 94c into a position corresponding to the shooting area r3. As shown in FIG. 11 , in the composite image 95, the area other than the area where the reduced image 94n is displayed is displayed as a background area 96. The background area 96 is displayed, for example, by being filled in with the same color as the edge of the composite image 95 .

[0122] <Generation of Coordinate Correspondence Information> Figure 12 shows an example of coordinate correspondence information 97 indicating the correspondence between the photographed image data of the photographing area rn acquired in Figure 9 and the coordinates on the composite image 95 of the detailed partial image 93n generated from the photographed image data. As described above, the coordinate correspondence information 97 is updated each time a detailed partial image 93n of each photographing area rn is photographed and the photographed image data is acquired. The coordinates of the composite image 95 indicate coordinates corresponding to the center position of the detailed partial image 93n. For example, the coordinates (X1, Y1) of the composite image corresponding to the photographed image data d1 are coordinates corresponding to the center position of the detailed partial image 93n generated based on the photographed image data d1.

[0123] <Image Display Processing by CPU 60A of Management Device 11> FIG. 13 is a flowchart showing an example of the display processing of the "composite image and detailed partial images" by the CPU 60A of the management device 11.

[0124] After the CPU 60A of the management device 11 has completed photographing all of the photographing areas in the process shown in Figure 6, it starts the process shown in Figure 13 in response to the operator's operation of the keyboard 13b or mouse 13c to check the photographed images.

[0125] First, the CPU 60A displays the composite image 95 on the display 13a (step S21).

[0126] Next, the CPU 60A determines whether or not a coordinate designation has been received from the operator for the composite image 95 displayed on the display 13a (step S22). The designation may be received, for example, by the operator clicking the mouse 13c. The designation for the composite image 95 will be described in detail later with reference to FIG. 14.

[0127] In step S22, if the designation of coordinates of composite image 95 has not been accepted (step S22: No), CPU 60A waits until the designation is accepted.

[0128] In step S22, if the designation of the coordinates of the composite image 95 is accepted (step S22: Yes), the CPU 60A searches for the photographic image data corresponding to the designated coordinates based on the coordinate correspondence information 97 (see FIG. 12) (step S23). The photographic image data corresponding to the designated coordinates is, for example, the photographic image data associated with the coordinate values ​​closest to the designated coordinate values.

[0129] Next, the CPU 60A displays on the display 13a a detailed partial image 93n generated based on the photographic image data retrieved in step S23 (step S24). The detailed partial image 93n may be displayed in a window separate from the composite image 95, or the composite image 95 and the detailed partial image 93n may be displayed by switching between them. Alternatively, the detailed partial image 93n may be displayed on a display separate from the display 13a on which the composite image 95 is displayed. The display of the detailed partial image 93n will be described in detail later with reference to FIG. 14. After displaying the detailed partial image 93n, the CPU 60A returns to step S22 and determines whether or not the next specification has been accepted.

[0130] <Specifying Coordinates in Composite Image 95 and Displaying Detailed Partial Image 93n> Figure 14 is a diagram showing an example of specifying an inspection location in the composite image 95 and displaying a detailed partial image 93n based on the specified location. As shown in Figure 14, the user moves the mouse cursor 98 to a desired inspection location in Building B on the composite image 95 and clicks to accept the specification of coordinates in the composite image 95 indicated by the mouse cursor 98. When the specification of coordinates in the composite image 95 is accepted, the captured image data dn (see Figure 12) stored as the coordinates (Xn, Yn) of the composite image is read out, which correspond to coordinate values ​​closest to the specified coordinate values. Then, for example, a detailed partial image 93a is generated based on the read captured image data. The generated detailed partial image 93a is displayed on the display 13a, for example, in a window separate from the composite image 95.

[0131] As described above, the CPU 60A of the management device 11 determines a photography method for photographing the specified photography target area 92 through multiple photography operations according to the shape of the specified photography target area 92, acquires multiple detailed partial images 93n obtained by photography using the determined photography method, and generates a composite image 95 of the photography target area 92 by resizing and compositing the acquired detailed partial images 93n in a reduced direction. This configuration allows the photography method to be determined according to the shape of the specified photography target area, and inspection work can be performed using the composite image 95 generated by resizing and compositing the captured detailed partial images 93n in a reduced direction. This reduces the resources required by the management device 11 during inspection work of the photography target area 92. Furthermore, by photographing only the specified photography target area 92, photography time can be reduced, thereby shortening the inspection work time.

[0132] Furthermore, the CPU 60A can accept designation of shapes other than a rectangle as the shape of the photographed area, so that only areas of any shape that require inspection can be designated as the photographed area, and only the areas that require inspection can be photographed, thereby shortening the inspection work time.

[0133] Furthermore, the CPU 60A stores coordinate correspondence information 97 that associates the positional relationship of each of the detailed partial images 93n in the composite image 95 with each of the partial images 93n. Therefore, the detailed partial image 93n at the position designated as the inspection area can be displayed based on the coordinate correspondence information 97, so that the work can be performed using the appropriate partial image 93n, thereby shortening the time required for the inspection work.

[0134] Furthermore, by designating any inspection location on the composite image 95 with, for example, a mouse cursor 98, image data of a detailed partial image 93n corresponding to the designated position can be read out and the detailed partial image 93n can be displayed on the display 13a together with the composite image 95. This makes it easy to recognize the positional relationship of the image of the inspection location relative to the overall image, thereby reducing work time.

[0135] <Another Example of Inspection Target> FIG. 15 is a diagram showing an example of inspecting an electric wire 102 of a transmission tower 101. As shown in FIG. 15 , a wide-angle image 91a of the electric wire 102 and the transmission tower 101 captured by the camera 10 is displayed on the display 13a of the management device 11. In the example shown in the figure, three electric wires 102a, 102b, and 102c are connected between the transmission tower 101 and an adjacent transmission tower (not shown). Furthermore, a photographing target area 92a indicating the range of photography is specified freehand by the operator through a touch operation on the display 13a as a horizontally elongated area surrounding the electric wire 102a to be inspected. When the photographing target area 92a is specified as a horizontally elongated area in this manner, multiple photographing areas rn are arranged horizontally along the photographing target area 92a. Then, photographed image data is acquired by sequentially photographing the horizontally elongated photographing areas rn with a telephoto lens.

[0136] FIG. 16 is a diagram showing an example of inspecting the blades 112 of a wind turbine 111. As shown in FIG. 16, a wide-angle image 91b of the wind turbine 111 captured by the camera 10 is displayed on the display 13a of the management device 11. In the example shown in the figure, the wind turbine 111 is provided with three blades 112. Furthermore, the three blades 112 are designated as inspection targets, and a target imaging area 92b indicating the range of imaging is designated by the operator through a freehand touch operation on the display 13a as a roughly triangular area surrounding the three blades 112 to be inspected. When the target imaging area 92b is designated as a triangular area in this manner, multiple imaging areas rn are arranged in a triangle such that the target imaging area 92b is included within each of the multiple imaging areas rn. Then, telephoto images of the multiple imaging areas rn arranged in a triangle are sequentially acquired.

[0137] <Other examples of designating the area to be photographed> In the examples of Figures 7, 15, and 16 described above, the areas to be photographed 92, 92a, and 92b are designated by surrounding the area with a line. However, the areas to be photographed may also be designated in the following manner, for example.

[0138] FIG. 17 is a diagram showing an example of specifying a photographing target area using a point cloud. When specifying a photographing target area using a point cloud, as shown in FIG. 17, the photographing target area is specified on the electric wire 102a to be inspected as, for example, photographing target points 121, 122, and 123. When specifying a photographing target area using a point cloud (photographing target points 121, 122, and 123), taking into consideration that the electric wire 102 of the transmission tower 101 forms a specific curve, position information of the electric wire 102a between the photographing target points 121, 122, and 123 is interpolated based on the position information of the photographing target points 121, 122, and 123. Then, based on the position information of the photographing target points 121, 122, and 123 and the calculated position information of the electric wire 102a between the photographing target points 121, 122, and 123, a horizontally elongated photographing target area surrounding the electric wire 102a similar to the photographing target area 92a shown in FIG. 15 is determined. In addition, as a method for generating a horizontally elongated photographic target area surrounding the power line 102a based on the photographic target points 121, 122, and 123, it is also possible to prepare information such as a power line designation mode, or to use image recognition using machine learning.

[0139] FIG. 18 is a diagram showing an example of specifying a photographing target area using a line. When specifying a photographing target area using a line, as shown in FIG. 18, the photographing target area is specified as a photographing target line 131, for example, along the electric wire 102a to be inspected. When specifying a photographing target area using a line (photographing target line 131), taking into consideration that the electric wire 102 of the transmission tower 101 forms a specific curve, a horizontally elongated photographing target area similar to the photographing target area 92a shown in FIG. 15 is determined along the photographing target line 131 so as to include the photographing target line 131. Note that, as a method for generating a horizontally elongated photographing target area surrounding the electric wire 102a based on the photographing target line 131, for example, information such as an electric wire specification mode may be prepared, or image recognition using machine learning may be used.

[0140] <Interpolation Processing When the Shooting Target Area is Designated by a Point Cloud> FIG. 19 is a flowchart showing an example of "Shooting Processing of the Shooting Target Area" and "Synthesis Processing of the Composite Image" when the shooting target area is designated by a point cloud.

[0141] Assume that a power line inspection is being performed using the camera 10. The camera 10 is set to face the object to be photographed, and the zoom position of the zoom lens is set to the wide-angle end. The wide-angle image captured by the camera 10 is displayed on the display 13a of the management device 11 (see, for example, FIG. 17 ).

[0142] 19 in response to an operation by an operator to specify an inspection area (area to be photographed) for the wide-angle image of the transmission tower 101 and the power lines 102 displayed on the display 13a. In this example, the area to be photographed is specified by a point cloud.

[0143] The processes from step S31 to step S33 in FIG. 19 are the same as the processes from step S11 to step S13 described with reference to FIG. 6, and therefore the description thereof will be omitted.

[0144] Next, the CPU 60A measures the distance to some of the shooting areas rn among the multiple shooting areas rn set in step S32 (step S34). The distance to some of the shooting areas rn is measured by the operator. The CPU 60A prompts the operator to measure the distance to some of the shooting areas rn using the autofocus button. The operator measures the distance to the shooting area rn including the shooting target points 121, 122, and 123 (see FIG. 17) designated as the shooting target area, for example.

[0145] Next, the CPU 60A derives pan / tilt and focus values ​​corresponding to the next imaging area among the multiple imaging areas arranged to include the imaging target area (step S35). The coordinates of the multiple imaging areas arranged on the wide-angle image displayed on the display 13a and the pan / tilt values ​​are calculated based on the size and positional relationship between the wide-angle image and the imaging areas. Furthermore, the focus value of the imaging area rn arranged between the imaging target points 121, 122, and 123 is calculated using the focus value of the imaging area rn including the imaging target points 121, 122, and 123 obtained by the distance measurement in step S34. The calculated pan / tilt and focus values ​​are stored as correspondence information in the memory 60C or the secondary storage device 14 in association with the coordinates of the imaging area. The CPU 60A derives pan / tilt and focus values ​​corresponding to the next imaging area based on the correspondence information calculated in advance.

[0146] Next, the CPU 60A controls the rotation operation of the rotation mechanism 16 based on the pan / tilt values ​​derived in step S35, and also controls the focus position of the camera 10 based on the derived focus value (step S36).

[0147] The processes from step S37 to step S40 in FIG. 19 are the same as the processes from step S16 to step S19 described with reference to FIG. 6, and therefore will not be described again.

[0148] FIG. 20 illustrates the interpolation process between imaging areas when the imaging target area is specified by a point cloud. As shown in FIG. 20, the imaging area rn including the specified imaging target points 121 and 122 in the imaging target area of ​​the electric wire 102a described in FIG. 19 is arranged horizontally, for example, as imaging areas r121, r122, r124, and r125. The imaging area r121 including the imaging target point 121 and the imaging area r122 including the imaging target point 122 are imaging areas for which focus values ​​are determined by distance measurement performed by the operator operating the autofocus button. In contrast, the imaging areas r124 and r125 arranged between the imaging areas r121 and r122 are imaging areas arranged based on the imaging areas r121 and r122, and are imaging areas for which focus values ​​are not determined. In this case, the focus values ​​of the imaging areas r124 and r125 are calculated using the focus values ​​of the imaging areas r121 and r122 determined by distance measurement.

[0149] In this way, by measuring the distance to a portion of the imaging area r n to obtain a focus value and then using that focus value to calculate the focus value for the remaining imaging area r n , the number of times the worker performs autofocus or manual focus operations can be reduced, which enables smooth imaging of the imaging target area and reduces the time required for inspection work.

[0150] In the case of a power line as shown in FIG. 20 or the wall of building B as shown in FIG. 7 , the target area is on a plane parallel to the direction of gravity (vertical direction). Therefore, to obtain focus information for a target area for which focus information has not been previously calculated, it is possible to calculate the focus information for another target area on the same parallel plane. In contrast, in the case of a blade 112 of a wind turbine 111 as shown in FIG. 16 , the target area (the surface of the blade 112) is often composed of a flat or curved surface oblique to the direction of gravity. This configuration is not limited to the blade 112 of the wind turbine 111, but is also applicable to target areas of bridges and tunnels, for example. Therefore, focus information measured in advance according to the characteristics of the target may be stored. When the characteristics of the target are detected during actual inspection, the focus information for the target may be calculated using the closest focus information from the stored focus information. This reduces the time required for inspections of various types of infrastructure.

[0151] <Image Display Processing When the Shooting Target Area is Designated by a Point Cloud> FIG. 21 is a flowchart showing an example of display processing of a "composite image and detailed partial images" when the shooting target area is designated by a point cloud.

[0152] After the CPU 60A of the management device 11 has completed photographing all of the photographing areas in the process shown in Fig. 19, the CPU 60A of the management device 11 starts the process shown in Fig. 21 in response to the operator's operation of the keyboard 13b or the mouse 13c to check the photographed images. Note that Fig. 19 shows a process for inspecting a power transmission line, and therefore in this example, a composite image of the power line 102a is displayed on the display 13a.

[0153] The processes of steps S41, S42, and S44 in FIG. 21 are similar to the processes of steps S21, S22, and S24 described with reference to FIG. 13, and therefore will not be described again.

[0154] When the CPU 60A receives the coordinate specification of the composite image 95 in step S42 (step S42: Yes), it searches for the captured image data of the subject (electric wire 102a) closest to the specified coordinates based on the coordinate correspondence information 97 (see FIG. 12 ) (step S43). However, as described above, the multiple image capturing areas rn are arranged so that adjacent image capturing areas rn slightly overlap each other. Therefore, the subject (electric wire 102a) closest to the specified coordinates may be located in the overlapping area of ​​adjacent image capturing areas rn and may be included in multiple image capturing areas rn. Therefore, the CPU 60A identifies the subject area in the composite image in advance by image recognition. Then, when the CPU 60A receives the coordinate specification of the composite image, it extracts the captured image data of the image capturing areas rn that overlap within a certain range centered on the specified coordinates as candidates. The number of image capturing areas rn that overlap within a certain range centered on the specified coordinates may be multiple. When a plurality of shooting regions rn are extracted, the photographic image data of the shooting region rn having the largest area occupied by the subject is searched for from the plurality of shooting regions rn based on the coordinate correspondence information 97. Note that it is also possible to search for the photographic image data of the shooting region rn having the highest contrast or the shooting region rn having the highest score in the results of flaw detection by AI (Artificial Intelligence) or the like from the plurality of shooting regions rn.

[0155] 22 is a diagram showing an example in which a subject close to a specified coordinate is included in multiple imaging regions rn. When multiple imaging regions rn are set for a specified imaging target region and the subject (electric wire 102a) is included in two or more imaging regions rn, as shown in FIG. 22 , when a composite image is generated, the subject (electric wire 102a) may be included in an overlapping area of, for example, reduced images 141 and 142 that make up the composite image. In this example, comparing the electric wire 102a included in reduced image 141 with the electric wire 102a included in reduced image 142, the reduced image 142 includes the entire electric wire 102a without any loss, whereas the reduced image 141 includes a portion of the electric wire 102a (the lower side of the figure) that is missing. In other words, the area occupied by the electric wire 102a in reduced image 142 is larger than the area occupied by the electric wire 102a in reduced image 141. In such a case, when coordinates on the reduced image 141 in the composite image are specified using the mouse cursor 98, as shown in Figure 22, the image data of the photographed area rn corresponding to the reduced image 142, which has the largest area occupied by the electric wire 102a in the reduced image, is searched for, rather than the image data of the photographed area rn corresponding to the reduced image 141.

[0156] <When a Captured Target Exceeds the Angle of View at the Wide-Angle End of Camera 10> Figure 23 is a diagram showing an example of capturing an image of a target captured by camera 10 that exceeds the capture range of the wide-angle end of camera 10. As shown in Figure 23, the range of the target to be inspected in the inspection work of this example is the range θ1 between transmission towers 101a and 101b to which power line 102 is connected. In contrast, the captureable range of camera 10 when set to the wide-angle end is the angle of view θ2, which is narrower than the range θ1 to be inspected. Therefore, in such a case, a pseudo-wide-angle image is generated by panoramic capturing at the wide-angle end of camera 10 so as to include the entire range of the target to be inspected.

[0157] Fig. 24 is a diagram showing an example of a pseudo wide-angle image captured by camera 10. As shown in Fig. 24, pseudo wide-angle image 150 is a wide-angle image generated by arranging wide-angle images 151 and 152 captured by camera 10 in a horizontal direction so that some areas overlap each other. Wide-angle image 151 is an image of the left side of the range of the object to be inspected. Wide-angle image 152 is an image of the right side of the range of the object to be inspected, opposite wide-angle image 151.

[0158] Figure 25 is a diagram showing the designation of a photographing target area in pseudo wide-angle image 150 generated as described in Figure 24. As shown in Figure 25, photographing target area 153 is designated freehand so as to surround electric wire 102a, which is the object to be inspected, and photographing process of photographing target area 153 and compositing process of a composite image are started. Note that in this example, a case where photographing is performed with telephoto camera 10 at the wide-angle end has been described, but similar photographing can also be applied when, for example, the object does not fit in a single image when photographed with a wide-angle camera.

[0159] This makes it possible to specify all areas to be inspected based on a single wide-angle image displayed on display 13a, even when the area to be photographed for the inspection target is large or the distance to the inspection target is short, thereby shortening the inspection work time.

[0160] <Modification of the Composite Image 95 Displayed When an Inspection Location is Specified> FIG. 26 is a diagram showing a modification of the composite image 95 displayed on the display 13a when an inspection location is specified in the composite image 95 of FIG. 14 described above. As described above, the multiple reduced images 94n constituting the composite image 95 are images generated by reducing the multiple detailed partial images 93n captured using telephoto photography. Therefore, when the detailed partial images 93n are captured, the object (the wall of Building B) captured in the partial images 93n is inspected for scratches or stains using, for example, AI image processing. If a scratch or stain is detected, the reduced image 94n corresponding to the detailed partial image 93n in which the scratch or stain was detected is marked and displayed on the composite image 95. For example, as shown in FIG. 26 , if a scratch 161 is detected on the wall of Building B using AI image processing, the reduced image 94n corresponding to the detailed partial image 93n in which the scratch 161 was detected is displayed on the composite image 95 with a diagonal line drawn. This makes it possible to easily detect areas that require checking during inspection work, thereby shortening the time required for inspection work.

[0161] <Generation of Composite Image by Geometric Processing> Fig. 27 is a diagram showing an example of performing geometric processing when generating a composite image. As described above in Fig. 15, when the inspection target is the power wire 102a of the power transmission tower 101, the shooting areas rn are arranged horizontally along the power wire 102a. When the shooting areas rn are arranged horizontally, the camera 10 is panned and tilted to sequentially capture telephoto images of the multiple shooting areas rn, thereby acquiring captured image data.

[0162] However, when telephotographing multiple imaging areas rn while panning and tilting the camera 10, the multiple imaging areas r171, r172, and r173 are rotated and tilted by a predetermined angle due to the panning and tilting, as shown in Figure 27. Therefore, when detailed partial images 193a, 193b, and 193c obtained by photographing imaging areas r171, r172, and r173 are arranged horizontally, the electric wires 102a included in each image are tilted by a predetermined angle. Therefore, when a composite image is generated based on these detailed partial images 193a, 193b, and 193c, simply performing resizing and composition in the reduction direction will result in a step between the electric wires 102a in each of the reduced images that make up the composite image, corresponding to the rotation of the predetermined angle.

[0163] Therefore, to eliminate the unevenness that occurs when generating the composite image, geometric processing different from the reduction and enlargement processing is applied to the detailed partial images 193a, 193b, and 193c. The rotation amount (tilt) of a predetermined angle that occurs when taking telephoto shots while panning and tilting the camera 10 is an amount of rotation that can be calculated from the angle of view and pan / tilt values, which are the shooting conditions of the camera 10. Therefore, when generating the composite image 195 from reduced images 194a, 194b, and 194c of the detailed partial images 193a, 193b, and 193c, the reduced images 194a, 194b, and 194c are each rotated (corrected) by a predetermined angle based on the rotation amount calculated from the shooting conditions so that unevenness does not occur in the electric wire 102a in the composite image 195. This makes it possible to correct distortion that may occur when generating the composite image 195, and to display an easy-to-view composite image 195 on the management device 11.

[0164] The order of the processes when generating the composite image 195 may be, for example, a geometric process for rotating the detailed partial images 193a, 193b, and 193c, a reduction process for reducing the detailed partial images 193a, 193b, and 193c that have been subjected to the geometric process, and a synthesis process for synthesizing the reduced images 194a, 194b, and 194c that have been reduced.

[0165] <Storage medium for information processing program> In each of the above management controls, an example has been given in which the information processing program of each embodiment is stored in the storage 60B of the management device 11, and the CPU 60A of the management device 11 executes the information processing program in the memory 60C, but the technology of the present disclosure is not limited to this.

[0166] 28 is a diagram showing an example of how an information processing program for management control is installed from a storage medium storing the information processing program into the control device 60 of the management device 11. As an example, as shown in FIG. 28, the information processing program 221 may be stored in a storage medium 220, which is a non-transitory storage medium. In the example shown in FIG. 28, the information processing program 221 stored in the storage medium 220 is installed into the control device 60, and the CPU 60A executes the above-described processes in accordance with the information processing program 221.

[0167] <Inspection Image Displayed on Management Device 11> In the above embodiment, when inspecting an image of a photographed object, an example has been described in which a composite image 95 composed of a plurality of reduced images 94n is displayed as the entire image on the management device 11. However, the technology of the present disclosure is not limited to this. For example, a wide-angle image (e.g., wide-angle image 91 in FIG. 7 ) captured by setting the camera 10 to the wide-angle end may be displayed as the entire image.

[0168] Although various embodiments have been described above, it goes without saying that the present invention is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention. Furthermore, the components of the above embodiments may be combined in any manner as long as they do not deviate from the spirit of the invention.

[0169] This application is based on a Japanese patent application (Patent Application No. 2022-102804) filed on June 27, 2022, the contents of which are incorporated herein by reference.

[0170] 1 Photography system 10 Camera 11 Management device 13a Display 16 Swivel mechanism 35, 60C Memory 37, 60A CPU 60 Control device 71 Yaw axis rotation mechanism 72 Pitch axis rotation mechanism 91 Wide-angle image 92 (92a, 92b) Photography target area 93n (93a to 93c, 193a to 193c) Detailed partial image 94n (94a to 94c), 141, 142, 194a to 194c Reduced image 95, 195 Composite image 96 Background area 97 Coordinate correspondence information 98 Mouse cursor 102 (102a to 102c) Power line 111 Wind turbine 112 Blade 121, 122, 123 Photography target point 131 Photography target line 150 Pseudo wide-angle image rn (r1, r2, r3, r121 to r125, r171 to r173) Shooting area

Claims

1. An information processing device including a processor, The processor, determining a photographing method including photographing the designated photographing target area a plurality of times in accordance with a form of the designated photographing target area and acquiring a photographing angle of view and a photographing position in the photographing; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; Information processing device.

2. 2. The information processing device according to claim 1, The processor, The shape of the imaging target area can be specified as a shape other than a rectangle. Information processing device.

3. 3. The information processing device according to claim 2, When the processor receives a designation of a shape other than a rectangle as the shape of the photographing target area, the processor generates the second image data representing a rectangular image. Information processing device.

4. An information processing device according to claim 1, The processor, generating first correspondence information that associates a positional relationship of a first image represented by the first image data with a second image represented by the second image data; Information processing device.

5. An information processing device according to claim 4, The processor, performing control to display, on a display device, a first image represented by the first image data corresponding to the coordinates of the specified second image data, based on the first correspondence information; Information processing device.

6. An information processing device according to claim 1, The resizing process is a resizing process including a geometric change. Information processing device.

7. An information processing device according to claim 6, The resizing process including the geometric transformation includes: a process of generating the second image data by a resizing process in the reduction direction based on the first image data, a geometric process of applying a geometric change different from reduction and enlargement to the first image data, and the synthesis process; Information processing device.

8. An information processing device according to claim 7, the processor performs the geometric processing, the resizing processing in the reduction direction, and the synthesis processing in this order. Information processing device.

9. An information processing device according to claim 7, The geometric processing includes a rotation processing. Information processing device.

10. An information processing device according to claim 9, The rotation process includes a process based on an imaging condition under which the first image data was obtained. Information processing device.

11. An information processing device according to claim 9, The rotation process includes a process of calculating a parameter for correcting the inclination of the angle of view during telephoto shooting. Information processing device.

12. An information processing device according to claim 1, The processor, A rotation mechanism for rotating an image capturing device that captures the image can be controlled, generating second correspondence information that associates the first image data with a control value of the rotation mechanism at the time of capturing the image when the first image data was obtained; Information processing device.

13. An information processing device according to claim 12, The processor, outputting a control value corresponding to a specified first image data item among the plurality of first image data items based on the second correspondence information; Information processing device.

14. An information processing device according to claim 12, The processor, extracting first image data from the plurality of first image data based on a degree of approximation of a specified control value based on the second correspondence information; Information processing device.

15. An information processing method by an information processing device, comprising: The processor of the information processing device determining a photographing method including photographing the designated photographing target area a plurality of times in accordance with a form of the designated photographing target area and acquiring a photographing angle of view and a photographing position in the photographing; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; Information processing methods.

16. An information processing program for an information processing device, comprising: A processor of the information processing device determining a photographing method including photographing the designated photographing target area a plurality of times in accordance with a form of the designated photographing target area and acquiring a photographing angle of view and a photographing position in the photographing; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; An information processing program for executing processing.

17. An information processing device having a processor, The processor, determining a photographing method for photographing the designated photographing target area a plurality of times according to the form of the designated photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; obtaining a reduction ratio in the resizing process in the reduction direction based on a size of the second image represented by the second image data; Information processing device.

18. An information processing method by an information processing device, comprising: The processor of the information processing device determining a photographing method for photographing the designated photographing target area a plurality of times according to the form of the designated photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; obtaining a reduction ratio in the resizing process in the reduction direction based on a size of the second image represented by the second image data; Information processing methods.

19. An information processing program for an information processing device, comprising: A processor of the information processing device determining a photographing method for photographing the designated photographing target area a plurality of times according to the form of the designated photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; obtaining a reduction ratio in the resizing process in the reduction direction based on a size of the second image represented by the second image data; An information processing program for executing processing.

20. An information processing device having a processor, The processor, determining a photographing method for photographing the designated photographing target area a plurality of times according to the form of the designated photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; and after acquiring distance measurement information of a plurality of positions in the photographing target area, the number of photographing operations being less than the number of photographing operations of the photographing method, controlling the photographing device to perform the photographing operation of the photographing method based on the distance measurement information. Information processing device.

21. An information processing method by an information processing device, comprising: The processor of the information processing device determining a photographing method for photographing the designated photographing target area a plurality of times according to the form of the designated photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; and after acquiring distance measurement information of a plurality of positions in the photographing target area, the number of photographing operations being less than the number of photographing operations of the photographing method, controlling the photographing device to perform the photographing operation of the photographing method based on the distance measurement information. Information processing methods.

22. An information processing program for an information processing device, comprising: A processor of the information processing device determining a photographing method for photographing the designated photographing target area a plurality of times according to the form of the designated photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; generating second image data relating to the subject area by performing a resizing process in a reducing direction and a synthesis process on the plurality of first image data; and after acquiring distance measurement information of a plurality of positions in the photographing target area, the number of photographing operations being less than the number of photographing operations of the photographing method, controlling the photographing device to perform the photographing operation of the photographing method based on the distance measurement information. An information processing program for executing processing.