Information processing device, information processing method, and information processing program

Abstract

The invention provides an information processing device, an information processing method, and an information processing program capable of reducing required resources. The management device (11) is provided with a CPU6 (0A). The CPU 6 (0A) performs: a process for specifying an imaging method for imaging the imaging target region a plurality of times on the basis of the form of the specified imaging target region; acquiring image data of a plurality of detailed partial images (93n) (e.g., 93a) obtained by imaging using an imaging method; and generating a composite image (95) relating to the imaging target region by performing resizing processing and composite processing in the reduction direction on the image data of the plurality of detailed partial images (93n).

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method and an information processing program. Background Art

[0002] Patent document 1 describes an image signal processing device that sequentially changes the photographic direction to shoot a photographic range in a manner that generates overlapping image areas, generates an image signal, and associates a unit image based on the generated image signal with other unit images that form overlapping image areas and records them in a storage medium. Patent document 2 describes a photographic part setting device that acquires an image of a photographic object and displays it on a display unit, generates information indicating the location of a designated photographic part in the photographic image, cuts out an image including the periphery of the photographic part from the image of the photographic object, and generates a reference image. Patent document 3 describes an image processing device that acquires a plurality of photographic images at different angles, reduces the acquired plurality of photographic images, generates a preview image by combining the reduced plurality of photographic images, and displays the preview image on a portion of the display unit that displays the photographic images.

[0003] Previous technical literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2004-194113

[0006] Patent Document 2: International Publication No. 2020 / 162264

[0007] Patent Document 3: Japanese Patent Application Publication No. 2017-011687 Summary of the invention

[0008] One embodiment of the technology according to the present invention provides an information processing device, an information processing method, and an information processing program that can suppress pressure on computing resources.

[0009] Means for solving technical problems

[0010] (1) An information processing device comprising a processor, wherein:

[0011] The above processor performs the following processing:

[0012] Determining a photographing method for photographing the designated photographing target area multiple times according to the shape of the designated photographing target area;

[0013] and obtaining a plurality of first image data obtained by photographing using the above-mentioned photographing method; and

[0014] The second image data related to the imaging target area is generated by performing resizing processing in the reduction direction and synthesizing processing on the plurality of first image data.

[0015] (2) The information processing device according to (1), wherein:

[0016] The processor may receive a designation of a shape other than a rectangle as the shape of the imaging target area.

[0017] (3) The information processing device according to (2), wherein:

[0018] The processor generates the second image data representing a rectangular image when receiving a designation of a form other than the rectangular form as the form of the imaging target area.

[0019] (4) The information processing device according to any one of (1) to (3), wherein:

[0020] The above-mentioned photography method includes obtaining the photography field angle and photography position under the above-mentioned shooting.

[0021] (5) The information processing device according to any one of (1) to (4), wherein:

[0022] The above processor performs the following processing:

[0023] First correspondence information is generated in which a positional relationship between a first image represented by the first image data and a second image represented by the second image data is associated with each other.

[0024] (6) The information processing device according to (5), wherein:

[0025] The above processor performs the following control:

[0026] Based on the first correspondence information, the first image represented by the first image data corresponding to the designated coordinates of the second image data is displayed on a display device.

[0027] (7) The information processing device according to any one of (1) to (6), wherein:

[0028] The above-mentioned resizing process is a resizing process including a geometric change.

[0029] (8) The information processing device according to (7), wherein:

[0030] The above resizing process including geometric changes is as follows:

[0031] The second image data is generated based on the first image data through the resizing process in the reduction direction, the geometric process of imparting a geometric change different from reduction and enlargement to the first image data, and the synthesis process.

[0032] (9) The information processing device according to (8), wherein:

[0033] The processor performs processing in the order of the geometric processing, the resizing processing in the reduction direction, and the synthesis processing.

[0034] (10) The information processing device according to (8) or (9), wherein:

[0035] The above-mentioned geometric processing includes rotation processing.

[0036] (11) The information processing device according to (10), wherein:

[0037] The rotation process includes a process based on the conditions of imaging in which the first image data is obtained.

[0038] (12) The information processing device according to (10), wherein:

[0039] The rotation process includes a process of calculating a parameter for correcting the inclination of the angle of view during telephoto photography.

[0040] (13) The information processing device according to any one of (1) to (12), wherein:

[0041] The above processor performs the following processing:

[0042] The reduction ratio in the resizing process in the reduction direction is acquired based on the size of the second image indicated by the second image data.

[0043] (14) The information processing device according to any one of (1) to (13), wherein:

[0044] The above processor performs the following processing:

[0045] A rotating mechanism capable of controlling the rotation of the photographing device for performing the above-mentioned photographing; and

[0046] Second correspondence information is generated in which the first image data is associated with a control value of the slewing mechanism at the time of imaging when the first image data is obtained.

[0047] (15) The information processing device according to (14), wherein:

[0048] The above processor performs the following processing:

[0049] Based on the second correspondence information, a control value corresponding to a designated first image data among the plurality of first image data is output.

[0050] (16) The information processing device according to (14) or (15), wherein:

[0051] The above processor performs the following processing:

[0052] The first image data is extracted based on the second correspondence information and the similarity of the control values ​​designated in the plurality of first image data.

[0053] (17) The information processing device according to any one of (1) to (16), wherein:

[0054] The above processor performs the following control:

[0055] After acquiring distance measurement information of a plurality of positions in the photographic target area less than the number of photographing times of the photographing method, the photographing device is caused to execute photographing of the photographing method according to the distance measurement information.

[0056] (18) An information processing method, performed by an information processing device, wherein:

[0057] The processor of the above information processing device performs the following processing:

[0058] Determining a photographing method for photographing the designated photographing target area multiple times according to the shape of the designated photographing target area;

[0059] Acquiring a plurality of first image data obtained by photographing using the above-mentioned photographing method; and

[0060] The second image data related to the imaging target area is generated by performing resizing processing in the reduction direction and synthesizing processing on the plurality of first image data.

[0061] (19) An information processing program, which is an information processing program for an information processing device, wherein:

[0062] The information processing program is used to cause the processor of the information processing device to perform the following processing:

[0063] Determining a photographing method for photographing the designated photographing target area multiple times according to the shape of the designated photographing target area;

[0064] Acquiring a plurality of first image data obtained by photographing using the above-mentioned photographing method; and

[0065] The second image data related to the imaging target area is generated by performing resizing processing in the reduction direction and synthesizing processing on the plurality of first image data.

[0066] Effects of the Invention

[0067] According to the present invention, it is possible to provide an information processing device, an information processing method, and an information processing program that can suppress the pressure on computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a diagram showing an example of a photographing system 1 equipped with an information processing device (management device 11) according to the present embodiment.

[0069] Figure 2 This is a diagram showing an example of rotation of the camera 10 in the pitch direction by the rotation mechanism 16 .

[0070] Figure 3 FIG. 1 is a diagram showing an example of rotation of the camera 10 in the yaw direction by the rotation mechanism 16.

[0071] Figure 4 It is a block diagram showing an example of the configuration of the optical system and the electrical system of the camera 10 .

[0072] Figure 5 It is a diagram showing an example of the configuration of the electric system of the turning mechanism 16 and the management device 11 .

[0073] Figure 6 This is a flowchart showing an example of a photographing process of a photographing target area and a synthesizing process of a synthesized image performed by the CPU 60A of the management device 11 .

[0074] Figure 7 1 is a diagram showing an example of designation of a photographing target area 92 in a wide-angle image 91 .

[0075] Figure 8 1 is a diagram showing an example of setting a plurality of photographing areas rn to be photographed by the camera 10 with respect to the photographing target area 92 .

[0076] Fig. 9 1 is a diagram showing an example of telephoto photography of a plurality of photography areas rn by the camera 10 .

[0077] Fig.10 Detailed partial images 93a to 93c and reduced images 94a to 94c generated based on the detailed partial images 93a to 93c are shown.

[0078] Fig.11 1 is a diagram showing a synthesized image 95 generated by reducing the image 94n.

[0079] Fig.12 The coordinate correspondence information 97 indicates the correspondence between the photographic image data of the detailed partial image 93 n and the coordinates on the synthesized image 95 of the detailed partial image 93 n .

[0080] Fig.13 This is a flowchart showing an example of display processing of a composite image and detailed partial images performed by the CPU 60A of the management device 11 .

[0081] Fig.14 1 and 2 are diagrams showing an example of designation of a maintenance location in a synthetic image 95 and a detailed partial image 93n displayed based on the designated position.

[0082] Fig.15 This is a diagram showing an example of maintenance of electric wires 102 of a transmission tower 101 .

[0083] Fig.16 This is a diagram showing an example of inspection of blades 112 of wind turbine 111 .

[0084] Fig.17 1 is a diagram showing an example of specifying a photographic target area using a point group.

[0085] Fig.18 FIG. 1 is a diagram showing an example of specifying a photographing target area using lines.

[0086] Fig.19 This is a flowchart showing an example of a photographing process of a photographing target region and a synthesis process of a synthetic image when the photographing target region is specified by a point group.

[0087] Fig. 20 This is a diagram showing interpolation processing between photographic areas when the photographic target area is specified using a point group.

[0088] Fig.21 This is a flowchart showing an example of display processing of a composite image and a detailed partial image when a photographing target area is specified using a point group.

[0089] Fig. 22 This is a diagram showing an example in which a subject close to designated coordinates is included in a plurality of photographic areas rn.

[0090] Fig.23 This is a diagram showing an example of photography in which the photographic object of the camera 10 exceeds the photographic range at the wide-angle end.

[0091] Fig.24 1 is a diagram showing an example of a pseudo wide-angle image 150 captured by the camera 10 .

[0092] Fig.251 is a diagram showing the designation of the imaging target region in the pseudo wide-angle image 150 .

[0093] Fig.26 1 is a diagram showing a modified example of the synthesized image 95 displayed on the display 13a when the maintenance location is designated.

[0094] Fig. 27 This is a diagram showing an example of geometric processing performed when generating a synthetic image.

[0095] Fig.28 This is a diagram showing an example of a method of installing the information processing program in the control device 60 of the management device 11 from a storage medium storing the information processing program for management control. DETAILED DESCRIPTION

[0096] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

[0097] <Photography System of Embodiment>

[0098] Figure 1 FIG. 1 is a diagram showing an example of a photographic system 1 equipped with an information processing device according to the present embodiment. Figure 1 As shown, as an example, the photographing system 1 includes a camera 10 and a management device 11. The camera 10 is an example of a photographing device in the present invention. The management device 11 is an example of an information processing device in the present invention.

[0099] The camera 10 is a camera used to inspect facilities (infrastructure) that are the basis of life or industrial activities. The camera 10, for example, inspects the walls of buildings, power lines, windmills, etc. The camera 10 uses a camera capable of telephoto photography, an ultra-high-resolution camera, etc. In addition, the camera 10 can also use a wide-angle camera. The camera 10 is set via the rotating mechanism 16 described later, and shoots a photographic object as a subject. The camera 10 sends the photographic image obtained by shooting and the photographic information related to the shooting to the management device 11 via the communication line 12.

[0100] The management device 11 includes a display 13a, a keyboard 13b, a mouse 13c, and a secondary storage device 14. Examples of the display 13a include a liquid crystal display, a plasma display, an organic EL (Electro-Luminescence) display, a CRT (Cathode Ray Tube) display, etc. The display 13a is an example of a display device of the present invention.

[0101] An example of the secondary storage device 14 is a hard disk drive (HDD). The secondary storage device 14 is not limited to a HDD, and may be any nonvolatile memory such as a flash memory, a solid state drive (SSD), or an electrically erasable and programmable read only memory (EEPROM).

[0102] The management device 11 receives the photographic image or photographic information transmitted from the camera 10 , and displays the received photographic image or photographic information on the display 13 a or stores it in the secondary storage device 14 .

[0103] The management device 11 performs photography control for controlling photography of the camera 10. For example, the management device 11 performs photography control by communicating with the camera 10 via the communication line 12. Photography control is to set photography parameters for photography by the camera 10 in the camera 10 and control to make the camera 10 perform photography. Photography parameters include parameters related to exposure and parameters of zoom position, etc.

[0104] Furthermore, the management device 11 controls the rotation mechanism 16 to control the photographing direction (panning or tilting) of the camera 10. For example, the management device 11 sets the rotation direction, rotation amount, rotation speed, etc. of the camera 10 according to the operation of the keyboard 13b or the mouse 13c or the touch operation on the screen of the display 13a.

[0105] <Rotation of Camera 10 by Rotation Mechanism 16>

[0106] Figure 2 This is a diagram showing an example of rotation of the camera 10 in the pitch direction by the rotation mechanism 16 . Figure 3 1 is a diagram showing an example of rotation of the camera 10 in the yaw direction by the rotation mechanism 16. The camera 10 is mounted on the rotation mechanism 16. The rotation mechanism 16 can rotate the camera 10.

[0107] Specifically, as an example, Figure 2 As shown, the rotation mechanism 16 is capable of rotating the camera 10 in a rotation direction (pitch direction) that intersects the yaw direction and has the pitch axis PA as the center axis, and as an example, Figure 3 The double-axis rotating mechanism shown rotates in the rotation direction (yaw direction) with the yaw axis YA as the central axis. In addition, in the rotating mechanism 16 involved in this embodiment, an example of a double-axis rotating mechanism is shown, but the technology of the present invention is not limited to this, and it can be a three-axis rotating mechanism or a single-axis rotating mechanism.

[0108] <Configuration of the Optical and Electrical Systems of Camera 10>

[0109] Figure 4 1 is a block diagram showing an example of the structure of the optical system and the electrical system of the camera 10. Figure 4 As shown, as an example, the camera 10 includes an optical system 15 and an imaging element 25. The imaging element 25 is located at the rear section of the optical system 15. The optical system 15 includes an objective lens 15A and a lens group 15B. The objective lens 15A and the lens group 15B are arranged in the order of the objective lens 15A and the lens group 15B along the optical axis OA of the optical system 15 from the object subject side (object side) to the light receiving surface 25A side (image side) of the imaging element 25. The lens group 15B includes an anti-vibration lens 15B1, a focusing lens (not shown) and a zoom lens 15B2. The zoom lens 15B2 is supported so as to be movable along the optical axis OA by a lens actuator 21 described later. The anti-vibration lens 15B1 is supported so as to be movable in a direction orthogonal to the optical axis OA by a lens actuator 17 described later.

[0110] By using the zoom lens 15B2 to extend the focal length, the camera 10 becomes telephoto, so the angle of view becomes smaller (the photographing range becomes narrower). By using the zoom lens 15B2 to shorten the focal length to become wide angle, the angle of view becomes larger (the photographing range becomes wider).

[0111] In addition, the optical system 15 may include various lenses not shown in the figure in addition to the objective lens 15A and the lens group 15B. In addition, the optical system 15 may include an aperture. The positions of the lenses, lens groups, and apertures included in the optical system 15 are not limited. For example, even if the positions of the lenses, lens groups, and apertures are not limited, Figure 4 The technology of the present invention is also applicable to positions different from the positions shown.

[0112] The anti-vibration lens 15B1 is movable in a direction perpendicular to the optical axis OA, and the zoom lens 15B2 is movable along the optical axis OA.

[0113] The optical system 15 includes lens actuators 17 and 21. The lens actuator 17 applies a force to the anti-vibration lens 15B1 to change the position of the anti-vibration lens 15B1 in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. The lens actuator 17 is controlled by an OIS (Optical Image Stabilizer) driver 23. By driving the lens actuator 17 under the control of the OIS driver 23, the position of the anti-vibration lens 15B1 changes in a direction perpendicular to the optical axis OA.

[0114] The lens actuator 21 applies a force to the zoom lens 15B2 for moving along the optical axis OA of the optical system 15. The lens actuator 21 is controlled by the lens driver 28. The position of the zoom lens 15B2 moves along the optical axis OA by driving the lens actuator 21 under the control of the lens driver 28. By the position of the zoom lens 15B2 moving along the optical axis OA, the focal length of the camera 10 changes.

[0115] In addition, when the outline of the photographic image is, for example, a rectangle having short sides in the pitch axis PA direction and long sides in the yaw axis YA direction, the field of view in the pitch axis PA direction is narrower than the field of view in the yaw axis YA direction, and narrower than the diagonal field of view.

[0116] By the optical system 15 configured in this way, light representing the imaging target area is imaged on the light receiving surface 25A of the imaging element 25 , and the imaging target area is imaged by the imaging element 25 .

[0117] The vibrations applied to the camera 10 include, for example, the vibrations caused by the passing of cars, the vibrations caused by wind, and the vibrations caused by road construction outdoors, and for example, the vibrations caused by the operation of an air conditioner and the vibrations caused by people coming in and out indoors. Therefore, the camera 10 is shaken due to the vibrations applied to the camera 10 (hereinafter, also simply referred to as "vibrations").

[0118] In addition, in the present embodiment, "jitter" refers to a phenomenon in which the image of the object subject on the light receiving surface 25A of the imaging element 25 in the camera 10 changes due to a change in the positional relationship between the optical axis OA and the light receiving surface 25A. In other words, "jitter" can also be said to be a phenomenon in which the optical image obtained by imaging on the light receiving surface 25A changes due to the tilt of the optical axis OA caused by the vibration applied to the camera 10. The change of the optical axis OA refers to, for example, the tilt of the optical axis OA relative to the reference axis (for example, the optical axis OA before the jitter occurs). Hereinafter, the jitter caused by vibration is also referred to as "jitter".

[0119] The shaking is included in the photographic image as a noise component and affects the image quality of the photographic image. Therefore, in order to remove the noise component included in the photographic image caused by the shaking, the camera 10 includes a lens-side shaking correction mechanism 29, an imaging element-side shaking correction mechanism 45, and an electronic shaking correction unit 33 for correcting the shaking.

[0120] The lens-side shake correction mechanism 29 and the imaging element-side shake correction mechanism 45 are mechanical shake correction mechanisms. The mechanical shake correction mechanism is a mechanism that corrects shake by applying power generated by a driving source such as a motor (for example, a voice coil motor) to a shake correction element (for example, the anti-vibration lens 15B1 and / or the imaging element 25) to move the shake correction element in a direction perpendicular to the optical axis of the imaging optical system.

[0121] Specifically, the lens-side shake correction mechanism 29 is a mechanism that corrects the shake by applying power generated by a driving source such as a motor (for example, a voice coil motor) to the anti-vibration lens 15B1, thereby moving the anti-vibration lens 15B1 in a direction perpendicular to the optical axis of the photographic optical system. The imaging element-side shake correction mechanism 45 is a mechanism that corrects the shake by applying power generated by a driving source such as a motor (for example, a voice coil motor) to the imaging element 25, thereby moving the imaging element 25 in a direction perpendicular to the optical axis of the photographic optical system. The electronic shake correction unit 33 corrects the shake by performing image processing on the photographic image according to the shake amount. That is, the shake correction unit (shake correction component) performs mechanical or electronic shake correction with a hardware structure and / or a software structure. Here, mechanical shake correction refers to shake correction achieved by mechanically moving shake correction elements such as the anti-vibration lens 15B1 and / or the photographic element 25 using power generated by a driving source such as a motor (e.g., a voice coil motor), and electronic shake correction refers to shake correction achieved, for example, by performing image processing using a processor.

[0122] like Figure 4 As shown, the lens-side image stabilization mechanism 29 includes, as an example, an anti-vibration lens 15B1 , a lens actuator 17 , an OIS driver 23 , and a position sensor 39 .

[0123] As the shake correction method performed by the lens-side shake correction mechanism 29, various well-known methods can be adopted. In the present embodiment, as the shake correction method, a method of correcting the shake by moving the anti-vibration lens 15B1 according to the shake amount detected by the shake amount detection sensor 40 (described later) is adopted. Specifically, the shake correction is performed by moving the anti-vibration lens 15B1 in the direction of eliminating the shake only by the amount of the shake to eliminate the shake.

[0124] The anti-vibration lens 15B1 is provided with a lens actuator 17. The lens actuator 17 is a displacement mechanism equipped with a voice coil motor, and drives the voice coil motor to move the anti-vibration lens 15B1 in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. In addition, here, as the lens actuator 17, a displacement mechanism equipped with a voice coil motor is used, but the technology of the present invention is not limited to this, and other power sources such as a stepping motor or a piezoelectric element can also be used instead of the voice coil motor.

[0125] The lens actuator 17 is controlled by the OIS driver 23. By driving the lens actuator 17 under the control of the OIS driver 23, the position of the anti-vibration lens 15B1 is mechanically changed within a two-dimensional plane perpendicular to the optical axis OA.

[0126] The position sensor 39 detects the current position of the anti-vibration lens 15B1, and outputs a position signal indicating the detected current position. Here, as an example of the position sensor 39, a device including a Hall element is used. Here, the current position of the anti-vibration lens 15B1 refers to the current position within the two-dimensional plane of the anti-vibration lens. The two-dimensional plane of the anti-vibration lens refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. In addition, in the present embodiment, as an example of the position sensor 39, a device including a Hall element is used, but the technology of the present invention is not limited to this, and a magnetic sensor or an optical sensor may also be used instead of the Hall element.

[0127] The lens-side shake correction mechanism 29 corrects the shake by moving the anti-vibration lens 15B1 in at least one of the pitch axis PA direction and the yaw axis YA direction within the actual shooting range. That is, the lens-side shake correction mechanism 29 corrects the shake by moving the anti-vibration lens 15B1 within the two-dimensional plane of the anti-vibration lens by an amount of movement corresponding to the amount of shake.

[0128] The image sensor side shake correction mechanism 45 includes the image sensor 25 , a BIS (Body Image Stabilizer) driver 22 , an image sensor actuator 27 , and a position sensor 47 .

[0129] As with the shake correction method performed by the lens-side shake correction mechanism 29, the shake correction method performed by the imaging element-side shake correction mechanism 45 can also adopt various well-known methods. In the present embodiment, as the shake correction method, a method of correcting the shake by moving the imaging element 25 according to the shake amount detected by the shake amount detection sensor 40 is adopted. Specifically, the shake correction is performed by moving the imaging element 25 in the direction to eliminate the shake only by the amount to eliminate the shake.

[0130] The imaging element actuator 27 is mounted on the imaging element 25. The imaging element actuator 27 is a displacement mechanism equipped with a voice coil motor, and drives the voice coil motor to move the imaging element 25 in a vertical direction relative to the optical axis of the anti-vibration lens 15B1. In addition, here, as the imaging element actuator 27, a displacement mechanism equipped with a voice coil motor is used, but the technology of the present invention is not limited to this, and other power sources such as a stepping motor or a piezoelectric element can also be used instead of the voice coil motor.

[0131] The imaging element actuator 27 is controlled by the BIS driver 22. By driving the imaging element actuator 27 under the control of the BIS driver 22, the position of the imaging element 25 is mechanically changed in the direction perpendicular to the optical axis OA.

[0132] The position sensor 47 detects the current position of the imaging element 25 and outputs a position signal indicating the detected current position. Here, as an example of the position sensor 47, a device including a Hall element is used. Here, the current position of the imaging element 25 refers to the current position in the two-dimensional plane of the imaging element. The two-dimensional plane of the imaging element refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. In addition, in this embodiment, as an example of the position sensor 47, a device including a Hall element is used, but the technology of the present invention is not limited to this, and a magnetic sensor or an optical sensor may also be used instead of the Hall element.

[0133] The camera 10 includes a computer 19, a DSP (Digital Signal Processor) 31, an image memory 32, an electronic shake correction unit 33, a communication I / F 34, a shake amount detection sensor 40, and a UI (User Interface) system device 43. The computer 19 includes a memory 35, a storage device 36, and a CPU (Central Processing Unit) 37.

[0134] The imaging element 25, DSP 31, image memory 32, electronic shake correction unit 33, communication I / F 34, memory 35, storage device 36, CPU 37, shake amount detection sensor 40, and UI system device 43 are connected to the bus 38. In addition, the OIS driver 23 is also connected to the bus 38. Figure 4 In the example shown, for convenience of illustration, one bus is shown as the bus 38, but a plurality of buses may be provided. The bus 38 may be a serial bus or a parallel bus such as a data bus, an address bus, and a control bus.

[0135] The memory 35 temporarily stores various information and is used as a working memory. As an example of the memory 35, a RAM (Random Access Memory) can be cited, but it is not limited to this and can be other types of storage devices. Various programs for the camera 10 are stored in the storage device 36. The CPU 37 controls the entire camera 10 by reading various programs from the storage device 36 and executing the read various programs on the memory 35. As the storage device 36, for example, a flash memory, an SSD, an EEPROM, or an HDD can be cited. In addition, for example, various non-volatile memories such as a magnetoresistive memory and a ferroelectric memory can be used instead of the flash memory or in combination with the flash memory.

[0136] The photographic element 25 is a CMOS (Complementary Metal Oxide Semiconductor) image sensor. Under the instruction of the CPU 37, the photographic element 25 captures the target subject at a predetermined frame rate. The "predetermined frame rate" mentioned here refers to, for example, tens of frames per second to hundreds of frames per second. In addition, the photographic element 25 itself may also have a built-in control device (photographic element control device). In this case, the photographic element control device performs detailed control inside the photographic element 25 according to the shooting instruction output by the CPU 37. Furthermore, the photographic element 25 can capture the target subject at a predetermined frame rate under the instruction of the DSP 31. In this case, the photographic element control device performs detailed control inside the photographic element 25 according to the shooting instruction output by the DSP 31. In addition, the DSP 31 is sometimes also referred to as an ISP (Image Signal Processor).

[0137] The light receiving surface 25A of the imaging element 25 is formed by a plurality of photosensitive pixels (not shown) arranged in a matrix. In the imaging element 25, each photosensitive pixel is exposed, and photoelectric conversion is performed for each photosensitive pixel. The charge obtained by photoelectric conversion for each photosensitive pixel is an analog imaging signal representing the object. Here, as the plurality of photosensitive pixels, a plurality of photoelectric conversion elements having sensitivity to visible light (for example, photoelectric conversion elements having color filters) are used. In the imaging element 25, as the plurality of photoelectric conversion elements, a photoelectric conversion element having sensitivity to R (red) light (for example, a photoelectric conversion element having R color filters corresponding to R), a photoelectric conversion element having sensitivity to G (green) light (for example, a photoelectric conversion element having G color filters corresponding to G), and a photoelectric conversion element having sensitivity to B (blue) light (for example, a photoelectric conversion element having B color filters corresponding to B) are used. In the camera 10, shooting based on visible light (for example, light on the short wavelength side of about 700 nanometers or less) is performed by using these photosensitive pixels. However, the present embodiment is not limited thereto, and photography based on infrared light (for example, light on the longer wavelength side than about 700 nanometers) can be performed. In this case, as a plurality of photosensitive pixels, a plurality of photoelectric conversion elements having sensitivity to infrared light can be used. In particular, for photography related to SWIR (Short-wavelength infrared), for example, an InGaAs sensor and / or a type II quantum well (T2SL; Simulation of Type-IIQuantum Well: simulation of type II quantum well) sensor can be used.

[0138] The imaging element 25 performs signal processing such as A / D (Analog / Digital) conversion on the analog imaging signal to generate a digital image as a digital imaging signal. The imaging element 25 is connected to the DSP 31 via the bus 38 and outputs the generated digital image to the DSP 31 via the bus 38 in units of frames.

[0139] In addition, here, as an example of the imaging element 25, a CMOS image sensor is cited for explanation, but the technology of the present invention is not limited to this, and a CCD (Charge Coupled Device) image sensor can be applied as the imaging element 25. In this case, the imaging element 25 is connected to the bus 38 via an AFE (Analog Front End) (not shown) with a built-in CCD driver. The AFE generates a digital image by performing signal processing such as A / D conversion on the analog imaging signal obtained by the imaging element 25, and outputs the generated digital image to the DSP 31. The CCD image sensor is driven by a CCD driver built into the AFE. Of course, a CCD driver can also be provided separately.

[0140] The DSP 31 performs various digital signal processing on the digital image. The various digital signal processing refers to, for example, demosaic processing, noise removal processing, grayscale correction processing, and color correction processing. The DSP 31 outputs the digital image after the digital signal processing to the image memory 32 for each frame. The image memory 32 stores the digital image from the DSP 31.

[0141] The shake amount detection sensor 40 is a device including a gyro sensor, for example, and detects the shake amount of the camera 10. In other words, the shake amount detection sensor 40 detects the shake amount about each of a pair of axial directions. The gyro sensor detects the respective axes (reference numerals 14 and 15 ) around the pitch axis PA, the yaw axis YA, and the roll axis RA (axis parallel to the optical axis OA). Figure 1 The shake amount detection sensor 40 detects the shake amount of the camera 10 by converting the rotational shake amount around the pitch axis PA and the rotational shake amount around the yaw axis YA detected by the gyro sensor into the shake amount in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA.

[0142] Here, a gyro sensor is cited as an example of the shake amount detection sensor 40, but this is only an example, and the shake amount detection sensor 40 may be an acceleration sensor. The acceleration sensor detects the shake amount in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA. The shake amount detection sensor 40 outputs the detected shake amount to the CPU 37.

[0143] Here, an example of a form in which the shake amount is detected by a physical sensor such as the shake amount detection sensor 40 is given, but the technology of the present invention is not limited to this. For example, a motion vector obtained by comparing the captured images stored in the image memory 32 in a time series before and after can be used as the shake amount. In addition, the shake amount to be finally used can be derived from the shake amount detected by the physical sensor and the motion vector obtained by image processing.

[0144] The CPU 37 acquires the amount of shake detected by the shake amount detection sensor 40, and controls the lens-side shake correction mechanism 29, the imaging element-side shake correction mechanism 45, and the electronic shake correction section 33 according to the acquired shake amount. The amount of shake detected by the shake amount detection sensor 40 is used for shake correction by each of the lens-side shake correction mechanism 29 and the electronic shake correction section 33.

[0145] The electronic image stabilization unit 33 is a device including an ASIC (Application Specific Integrated Circuit) and performs image processing on the captured image in the image memory 32 according to the image stabilization amount detected by the image stabilization amount detection sensor 40 to stabilize the image.

[0146] In addition, here, as the electronic jitter correction unit 33, a device including an ASIC is exemplified, but the technology of the present invention is not limited to this. For example, it can be a device including an FPGA (Field Programmable Gate Array) or a PLD (Programmable Logic Device). And, for example, the electronic jitter correction unit 33 can be a device including multiple devices among ASIC, FPGA and PLD. And, as the electronic jitter correction unit 33, a computer including a CPU, a storage device and a memory can be used. The CPU can be single or multiple. And the electronic jitter correction unit 33 can be implemented by a combination of hardware structure and software structure.

[0147] The communication I / F 34 is, for example, a network interface, and controls the transmission of various information between the camera 10 and the management device 11 via a network. The network is, for example, a WAN (Wide Area Network) such as the Internet or a LAN (Local Area Network). The communication I / F 34 performs communication between the camera 10 and the management device 11.

[0148] The UI system device 43 includes a receiving device 43A and a display 43B. The receiving device 43A is, for example, a hardware key or a touch panel, and receives various instructions from the user. The CPU 37 acquires various instructions received by the receiving device 43A and operates according to the acquired instructions.

[0149] The display 43B displays various information under the control of the CPU 37. Examples of the various information displayed on the display 43B include the contents of various instructions received by the receiving device 43A and photographed images.

[0150] <Structure of the electrical system of the rotary mechanism 16 and the management device 11>

[0151] Figure 5 1 is a diagram showing an example of the structure of the electrical system of the rotary mechanism 16 and the management device 11. Figure 5 As shown, the slewing mechanism 16 includes a yaw axis slewing mechanism 71 , a pitch axis slewing mechanism 72 , a motor 73 , a motor 74 , a driver 75 , a driver 76 , and communication I / Fs 79 and 80 , as an example.

[0152] The yaw axis rotation mechanism 71 rotates the camera 10 in the yaw direction. The motor 73 generates power by being driven under the control of the driver 75. The yaw axis rotation mechanism 71 rotates the camera 10 in the yaw direction by receiving the power generated by the motor 73. The pitch axis rotation mechanism 72 rotates the camera 10 in the pitch direction. The motor 74 generates power by being driven under the control of the driver 76. The pitch axis rotation mechanism 72 rotates the camera 10 in the pitch direction by receiving the power generated by the motor 74.

[0153] The communication I / Fs 79 and 80 are, for example, network interfaces, and control the transmission of various information between the management device 11 and the management device 11 via a network. The network is, for example, a WAN such as the Internet or a LAN. The communication I / Fs 79 and 80 perform communication between the rotation mechanism 16 and the management device 11.

[0154] like Figure 5 As shown, as an example, the management device 11 includes a display 13a, a secondary storage device 14, a control device 60, a receiving device 62, and communication I / Fs 66, 67, and 68. The control device 60 includes a CPU 60A, a storage device 60B, and a memory 60C. The CPU 60A is an example of a processor in the present invention.

[0155] Each of the receiving device 62, the display 13a, the secondary storage device 14, the CPU 60A, the storage device 60B, the memory 60C, and the communication I / F 66 is connected to the bus 70. Figure 5In the example shown, for convenience of illustration, one bus is shown as the bus 70, but a plurality of buses may be provided. The bus 70 may be a serial bus or a parallel bus including a data bus, an address bus, a control bus, and the like.

[0156] The memory 60C temporarily stores various information and is used as a working memory. As an example of the memory 60C, a RAM can be cited, but it is not limited to this and can be other types of storage devices. Various programs for the management device 11 (hereinafter referred to as "management device programs") are stored in the storage device 60B.

[0157] The CPU 60A reads out the management device program from the storage device 60B and executes the read management device program on the memory 60C to control the entire management device 11. The management device program includes the information processing program in the present invention.

[0158] The communication I / F 66 is, for example, a network interface. The communication I / F 66 is communicably connected to the communication I / F 34 of the camera 10 via a network, and performs transmission control of various information with the camera 10. The communication I / Fs 67 and 68 are, for example, network interfaces. The communication I / F 67 is communicably connected to the communication I / F 79 of the rotation mechanism 16 via a network, and performs transmission control of various information with the yaw axis rotation mechanism 71. The communication I / F 68 is communicably connected to the communication I / F 80 of the rotation mechanism 16 via a network, and performs transmission control of various information with the pitch axis rotation mechanism 72.

[0159] The CPU 60A receives the photographed image, photographic information, etc. from the camera 10 via the communication I / F 66 and the communication I / F 34. The CPU 60A controls the photographing operation of the photographic target area by the camera 10 via the communication I / F 66 and the communication I / F 34.

[0160] The CPU 60A controls the rotation of the yaw axis rotation mechanism 71 by controlling the driver 75 and the motor 73 of the rotation mechanism 16 via the communication I / F 67 and the communication I / F 79. In addition, the CPU 60A controls the rotation of the pitch axis rotation mechanism 72 by controlling the driver 76 and the motor 74 of the rotation mechanism 16 via the communication I / F 68 and the communication I / F 80.

[0161] The CPU 60A receives the user's designation of the photographing target area of ​​the camera 10. The CPU 60A determines the photographing method of the camera 10 for photographing the designated photographing target area a plurality of times, based on the form of the photographing target area designated by the user.

[0162] The CPU 60A can receive, for example, a designation of a photographic subject area having a shape other than a rectangle as the shape of the photographic subject area. The shape of the photographic subject area includes, for example, a shape of an area designated by a point group, a shape of an area designated by a line, a shape designated in a manner of surrounding a specified area, and the like. When the area is designated by a point group, for example, only the point portion may be photographed, and the composition such as the pan / tilt angle or the field angle may be determined and photographed by connecting nearby points with line segments or curves and interpolating the area between the points, or the composition such as the pan / tilt angle or the field angle may be determined and photographed by connecting areas between designated points other than nearby points and interpolating the areas between the points with line segments or curves. When the area is designated by a line, for example, photographing may be performed along the designated line segment. When the area is designated in a manner of surrounding the area, for example, only the designated area may be photographed.

[0163] The photographing method of the camera 10 that determines the photographing object area includes, for example, determining each photographing area in a plurality of telephoto photographings performed by the camera 10. Each photographing area in the plurality of telephoto photographings refers to a plurality of photographing areas (for example, Fig. 9 The dotted line areas r1, r2, r3 shown).

[0164] Furthermore, the CPU 60A acquires data of a plurality of detailed partial images obtained by a plurality of telephoto shots performed by the camera 10, and for example, performs resizing processing in a reduction direction on the detailed partial images. The resizing processing in a reduction direction is, for example, processing to reduce the size of an image by reducing the number of pixels. Furthermore, the CPU 60A generates a composite image (hereinafter also referred to as a whole image) related to the whole of the above-mentioned photographic object area by performing a synthesis processing on the reduced images that have undergone the resizing processing in the reduction direction. The detailed partial image is an example of a first image represented by the first image data of the present invention. The composite image is an example of a second image represented by the second image data of the present invention.

[0165] Furthermore, CPU 60A generates a rectangular composite image by resizing and synthesizing based on the reduction direction of the detailed partial image not only when receiving a rectangular shape as the shape of the photographic subject area but also when receiving a shape other than a rectangle.

[0166] In addition to generating a composite image by resizing processing and synthesizing processing based on the reduction direction of the detailed partial image, the CPU 60A can generate a composite image by, for example, applying geometric processing to the detailed partial image to give a geometric change different from reduction and enlargement. The geometric processing includes, for example, rotation processing for the detailed partial image. The rotation processing is a projective conversion based on the photographic conditions of the camera 10 that obtained the detailed partial image. The photographic conditions of the camera 10 refer to the set field of view angle and pan / tilt values ​​of the camera 10. In addition, the rotation processing includes processing to calculate parameters for correcting the inclination (rotational distortion) of the field of view angle of the camera 10 during telephoto shooting.

[0167] The resizing process for the detailed partial image is a resizing process including a geometric change. The resizing process including a geometric change is a process for generating a composite image by reducing the detailed partial image, applying a geometric change different from reduction and enlargement to the detailed partial image, and synthesizing. The CPU 60A generates a composite image by performing the processing in the order of the geometric processing, the reducing processing, and the synthesizing processing.

[0168] Furthermore, the CPU 60A generates coordinate correspondence information that establishes a correspondence relationship between the detailed partial image and the position of the detailed partial image in the synthesized image. The CPU 60A determines the detailed partial image corresponding to the position (coordinates) specified in the synthesized image based on the coordinate correspondence information, and displays the determined detailed partial image on the display 13a. The coordinate correspondence information is stored in the memory 60C or the secondary storage device 14. The coordinate correspondence information is an example of the first correspondence information of the present invention.

[0169] Furthermore, when resizing the detailed partial image in the reduction direction, the CPU 60A sets the reduction ratio in the resizing process in the reduction direction according to the size of the composite image. The size of the composite image refers to the number of vertical and horizontal pixels in the composite image. The CPU 60A sets the reduction ratio, for example, in such a way that the image synthesized by the reduced images after reduction is inscribed in a rectangle of the size of the composite image. That is, when a plurality of reduced images are arranged to form a composite image, the reduction ratio is set in such a way that the total number of pixels of the reduced images falls within the number of pixels of the composite image.

[0170] Furthermore, the CPU 60A generates rotation corresponding information that establishes a correspondence between the detailed partial image and the control value of the rotation mechanism 16 when the detailed partial image is captured. The control value of the rotation mechanism 16 refers to the pan and tilt control value of the camera 10 rotated by the rotation mechanism 16. The rotation corresponding information is stored in the memory 60C or the secondary storage device 14. In addition, when the camera 10 is a camera with a zoom function, the zoom position at the time of shooting can be further associated with the control value of the rotation mechanism 16 to generate the rotation corresponding information. The rotation corresponding information is an example of the second corresponding information of the present invention.

[0171] Furthermore, the CPU 60A outputs the control value corresponding to the specified detailed partial image among the plurality of detailed partial images, for example, to the rotating mechanism 16, etc., based on the rotation corresponding information. Thus, it is possible to simply re-shoot the specified detailed partial image in the composite image with reference to the rotation corresponding information. For example, when the user wants to inspect the current damage of the wall of a building detected in the inspection a few days ago, the user specifies the specified inspection position in the composite image displayed on the display 13a, and the control value corresponding to the partial image of the specified position is set to the rotating mechanism 16 based on the rotation corresponding information, so that the partial image of the specified position can be re-shot.

[0172] Furthermore, the CPU 60A extracts detailed partial images based on the rotation correspondence information and based on the approximation of the control values ​​specified in the plurality of detailed partial images. Thus, for example, there are a plurality of groups of "plural detailed partial images" with different shooting periods, and if one detailed partial image of a certain group is specified, detailed partial images with control values ​​similar to the detailed partial image can be extracted from other groups. For example, in the case where the wall of a building is inspected every six months, when it is desired to detect how the damage to the wall detected in the past changes every six months, the user specifies a specified confirmation position in the composite image displayed on the display 13a, thereby extracting a plurality of past detailed partial images corresponding to the control values ​​of the specified confirmation position based on the rotation correspondence information, and the extracted plurality of detailed partial images can be confirmed.

[0173] Furthermore, the CPU 60A acquires the distance measurement information of the plurality of positions in the photographing object area with a number of times less than the number of times of photographing the photographing object area, and photographs the photographing object area a plurality of times to acquire a plurality of detailed partial images based on the distance measurement information of the acquired number of times. Specifically, the CPU 60A calculates the distance measurement information of the unmeasured position based on the distance measurement information of the measured position, and photographs the photographing object area a plurality of times to acquire a plurality of detailed partial images based on the distance measurement information including the calculated distance measurement information. The distance measurement information refers to the photographing distance or the focus position at which the focus is in focus.

[0174] The receiving device 62 is, for example, a keyboard 13b, a mouse 13c, or a touch panel of the display 13a, and receives various instructions from the user. The CPU 60A obtains various instructions received by the receiving device 62 and operates according to the instructions obtained. For example, when the receiving device 62 receives processing content for the camera 10 and / or the rotating mechanism 16, the CPU 60A operates the camera 10 and / or the rotating mechanism 16 according to the instruction content received by the receiving device 62.

[0175] The display 13a displays various information under the control of the CPU 60A. Examples of the various information displayed on the display 13a include the contents of various instructions received by the receiving device 62 and the photographic images or photographic information received by the communication I / F 66. The CPU 60A displays the contents of various instructions received by the receiving device 62 and the photographic images or photographic information received by the communication I / F 66 on the display 13a.

[0176] The secondary storage device 14 is, for example, a nonvolatile memory, and stores various information under the control of the CPU 60A. Examples of the various information stored in the secondary storage device 14 include photographic images or photographic information received by the communication I / F 66. The CPU 60A stores the photographic images or photographic information received by the communication I / F 66 in the secondary storage device 14.

[0177] <Photographic Processing and Image Processing by CPU 60A of Management Device 11>

[0178] Figure 6 This is a flowchart showing an example of “photographing processing of a photographing target area” and “combining processing of a composite image” performed by the CPU 60A of the management device 11 .

[0179] It is assumed that the camera 10 is used to perform maintenance work on infrastructure (walls of buildings, power lines, windmills, etc.). The camera 10 is set toward the photographic object, and the zoom position of the zoom lens is set to the wide-angle end. The data of the wide-angle image captured by the camera 10 is sent to the management device 11 via the communication line 12. In addition, even if the camera 10 is set to the wide-angle end, if the photographic object cannot be fully photographed, a wide-angle camera installed in conjunction with the camera 10 can be used to photograph the photographic object.

[0180] An operator (user) is in front of the management device 11 and is viewing the photographic image of the camera 10 displayed on the display 13a. The operator performs maintenance work while operating the camera 10 via the communication line 12 by operating the keyboard 13b or mouse 13c of the management device 11 or by touching the display 13a.

[0181] The CPU 60A of the management device 11 starts to display the wide-angle image on the display 13a in response to the operator's designation of the inspection area (the area to be photographed). Figure 6 Processing shown.

[0182] The CPU 60A receives the designation of the photographic target area in the wide-angle image displayed on the display 13a (step S11). As described above, the designation of the photographic target area includes the following steps: designating the photographic target area by a point group, designating the photographic target area by a line, designating the photographic target area by surrounding a predetermined area, and the like. Figure 7 The following will be described later.

[0183] Next, the CPU 60A sets a plurality of photographic areas for the photographic target area specified in step S11 (step S12). For the wide-angle image displayed on the display 13a, the size of the photographic area that can be photographed by the telephoto shooting of the camera 10 is determined by setting, and therefore the plurality of photographic areas are set according to the size. The plurality of photographic areas are set so that the specified photographic target area is at least arranged within the plurality of photographic areas. Figure 8 The following will be described later.

[0184] Next, the CPU 60A sets the field of view of the camera 10 to the telephoto side in order to capture each of the plurality of photographic areas set in step S12 (step S13). For example, the CPU 60A sets the zoom position of the zoom lens of the camera 10 to the telephoto end. However, if the damage or cracks on the wall of a building are large and the camera 10 is at the telephoto end, it may sometimes be impossible to fit it in one image, so the operator may specify the zoom position.

[0185] Next, the CPU 60A derives the pan / tilt value corresponding to the next photographic area among the plurality of photographic areas arranged so as to include the photographic target area (step S14). The coordinates of the plurality of photographic areas arranged on the wide-angle image displayed on the display 13a and the pan / tilt values ​​thereof can be calculated based on the size and positional relationship between the wide-angle image and the photographic areas, and are therefore calculated in advance and stored as corresponding information in the memory 60C or the secondary storage device 14. The CPU 60A derives the pan / tilt value corresponding to the next photographic area based on the pre-calculated corresponding information.

[0186] Next, the CPU 60A controls the rotation of the rotation mechanism 16 based on the pan / tilt value derived in step S14 (step S15). Specifically, as described above, the CPU 60A controls the rotation of the yaw axis rotation mechanism 71 and the rotation of the pitch axis rotation mechanism 72 in the rotation mechanism 16. In addition, when the CPU 60A controls the rotation mechanism 16, it is possible to perform distance measurement and focusing of the photographic area before taking a detailed partial image of the photographic area.

[0187] Next, the CPU 60A acquires the photographic image data (detailed partial image) of the photographic area determined by the rotation control of step S15 (step S16). If the photographic direction of the camera 10 is set by the rotation control of step S15, the CPU 60A outputs, for example, a photographic instruction signal to the camera 10 to instruct the camera 10 to photograph the photographic area. The CPU 60A acquires the photographic image data of the photographic area photographed by the camera 10. Fig. 9 and Fig.10 The following will be described later.

[0188] Next, CPU 60A performs reduction processing on the photographic image data of the detailed partial image acquired in step S16, and performs synthesis processing on the reduced image that has been reduced, thereby updating the display of the synthetic image composed of the reduced images (step S17). In this example, a method of updating the synthesis of the synthetic image is adopted in which the reduced image is added each time the photographic image data is acquired. However, for example, after the photographic image data of multiple photographic areas are acquired, these multiple photographic image data can be used to create a synthetic image. Regarding the reduction / synthesis of the photographic image data, specifically in Fig.10 and Fig.11 The following will be described later.

[0189] Next, if the display of the composite image is updated by adding a reduced image in step S17, the CPU 60A updates the coordinate correspondence information by adding information indicating the correspondence between the coordinates of the composite image of the added reduced image and the photographic image data of the detailed partial image acquired in step S16 (step S18). Fig.12 In addition, in this example, after updating the display of the composite image (step S17), the coordinate correspondence information is updated (step S18), but the order of these processes can be interchanged. That is, the update process of the display of the composite image can be performed after the update process of the coordinate correspondence information is performed first.

[0190] Next, the CPU 60A determines whether or not all of the imaging areas set in step S12 have been captured (step S19 ).

[0191] In step S19, if all the shooting of the set multiple shooting areas is not completed (step S19: No), CPU60A returns to step S14 and repeats each process. In step S19, if all the shooting of the set multiple shooting areas is completed (step S19: Yes), CPU60A ends this process.

[0192] <Designation of Photographic Target Area>

[0193] Figure 7 91 is a diagram showing an example of specifying a photographic target area 92 in a wide-angle image 91. Figure 7 As shown, the maintenance of the infrastructure in this example is to inspect the wall of Building B for damage or cracks. The wide-angle image 91 is an image taken by the camera 10 and displayed on the display 13a of the management device 11. The operator designates the area to be inspected as the photographic object area 92 by touching the display 13a. The designation of the photographic object area 92 can be, for example, based on free-arm designation or based on polygonal designation. The example shown in the figure is based on free-arm designation, and the photographic object area 92 is shown in a manner that surrounds Building B, which is the inspection object. The range of the photographic object area is not limited to the entire building B, and can be set to a range of any size.

[0194] <Setting of Photographing Area for Photographing Target Area 92>

[0195] Figure 8 It means for Figure 7 92 designated in the image capturing area 92 is set to capture multiple image capturing areas rn using telephoto photography performed by the camera 10. The multiple image capturing areas rn are arranged in such a manner that the image capturing area 92 is included inside the multiple image capturing areas rn. The size of one image capturing area rn is determined, for example, by the telephoto setting degree of the camera 10. The multiple image capturing areas rn are arranged vertically and horizontally in a rectangular grid, and are arranged in such a manner that adjacent image capturing areas rn slightly overlap each other.

[0196] <Acquisition of Photographic Image Data of Photographic Area rn>

[0197] Fig. 9 It means that the camera 10 is used to sequentially Figure 8 FIG. 1 is a diagram showing an example of a situation where a plurality of photographic areas rn set in FIG. 1 are photographed with a telephoto lens. In this example, Fig. 9 As shown by the arrow, one side Figure 8The plurality of photographing areas rn shown are sequentially photographed vertically from the right column (left column when viewed from the camera 10) to the left column (right column when viewed from the camera 10) of the building B, and photographic image data of each photographing area is acquired. For example, in the photographing area of ​​the fourth column from the right, photographic image data of the photographing area r1, the photographing area r2, and the photographing area r3 are acquired in this order while tilting from the upper part to the lower part of the building B.

[0198] <Reduction Processing of Photographic Images>

[0199] Fig.10 Yes means through Fig. 9 Detailed partial images 93a, 93b, 93c generated by photographic image data of photographic areas r1, r2, r3 acquired in the image processing apparatus 10 and reduced images 94a, 94b, 94c generated based on the detailed partial images 93a, 93b, 93c. As described above, the detailed partial images 93a, 93b, 93c (hereinafter also referred to as detailed partial images 93n) are high-quality images obtained by telephoto shooting of the photographic areas r1, r2, r3 by the camera 10. As described above, the reduced images 94a, 94b, 94c are images generated by performing a reduction process on the detailed partial images 93a, 93b, 93c. The reduction process includes a reduction process of reducing the size and a reduction process of reducing the number of pixels. Specifically, the reduced image 94a is an image generated by performing a reduction process of reducing the number of pixels of the detailed partial image 93a to reduce the size. The reduced image 94b is an image generated by performing a reduction process of reducing the number of pixels of the detailed partial image 93b to reduce the size. The reduced image 94c is an image generated by performing a reduction process of reducing the number of pixels of the detailed partial image 93c to reduce the size.

[0200] <Generation of Synthetic Image>

[0201] Fig.11 It means by Fig.10FIG. 9 is a diagram of a composite image 95 generated by using the reduced images 94a, 94b, 94c, etc. (hereinafter, also referred to as reduced image 94n) generated in . The composite image 95 is generated by performing a composite process on the reduced image 94n. As described above, the composite process of the composite image 95 is updated each time each reduced image 94n is generated. For example, if a reduced image 94a is generated based on the detailed partial image 93a, the composite process is performed such that the generated reduced image 94a is embedded in a position corresponding to the photographic region r1 where the detailed partial image 93a is acquired. Next, if a reduced image 94b is generated based on the detailed partial image 93b, the composite process is performed such that the reduced image 94b is embedded in a position corresponding to the photographic region r2. Next, if a reduced image 94c is generated based on the detailed partial image 93c, the composite process is performed such that the reduced image 94c is embedded in a position corresponding to the photographic region r3. And, as Fig.11 As shown, in the synthesized image 95, except for the area where the reduced image 94n is displayed, the area is displayed as a background area 96. The background area 96 is displayed so as to be filled with the same color as the end of the synthesized image 95, for example.

[0202] <Generation of coordinate correspondence information>

[0203] Fig.12 Yes means Fig. 9 The coordinate correspondence information 97 is an example of the correspondence relationship between the photographic image data of the photographic region rn acquired in the image data and the coordinates on the synthetic image 95 of the detailed partial image 93n generated from the photographic image data. As described above, the coordinate correspondence information 97 is updated every time the detailed partial image 93n of each photographic region rn is captured and its photographic image data is acquired. The coordinates of the synthetic image 95 represent the coordinates corresponding to the center position of the detailed partial image 93n. For example, the coordinates (X1, Y1) of the synthetic image corresponding to the photographic image data d1 are the coordinates corresponding to the center position of the detailed partial image 93n generated based on the photographic image data d1.

[0204] <Image Display Processing by CPU 60A of Management Device 11>

[0205] Fig.13 This is a flowchart showing an example of the display processing of “the composite image and the detailed partial image” performed by the CPU 60A of the management device 11 .

[0206] The CPU 60A of the management device 11 Figure 6 After all the photographic areas are photographed in the process shown, the operator starts to operate the keyboard 13b or the mouse 13c to confirm the photographed images. Fig.13 Processing shown.

[0207] First, the CPU 60A displays the synthesized image 95 on the display 13 a (step S21 ).

[0208] Next, the CPU 60A determines whether the coordinates of the composite image 95 displayed on the display 13a have been designated by the operator (step S22). The designation is, for example, the receipt of a click operation by the operator's mouse 13c. Fig.14 The following will be described later.

[0209] In step S22 , when the designation of the coordinates of the composite image 95 is not received (step S22 : NO), the CPU 60A waits until the designation is received.

[0210] In step S22, when the coordinates of the composite image 95 are designated (step S22: YES), the CPU 60A generates the composite image 95 based on the coordinate correspondence information 97 (reference Fig.12 ) searches for photographic image data corresponding to the designated coordinates (step S23). The photographic image data corresponding to the designated coordinates refers to, for example, photographic image data that is associated with the coordinate value closest to the designated coordinate value.

[0211] Next, the CPU 60A displays the detailed partial image 93n generated based on the photographic image data searched in step S23 on the display 13a (step S24). The detailed partial image 93n may be displayed in a window different from the composite image 95, or the composite image 95 and the detailed partial image 93n may be displayed by switching between them. Alternatively, the detailed partial image 93n may be displayed on a display different from the display 13a that displays the composite image 95. Fig.14 After displaying the detailed partial image 93n, the CPU 60A returns to step S22 and determines whether to accept the next designation.

[0212] <Coordinate Designation of Synthesized Image 95 and Display of Detailed Partial Image 93n>

[0213] Fig.14 9 is a diagram showing an example of designation of a repair location in a composite image 95 and a detailed partial image 93n displayed based on the designated location. Fig.14 As shown in FIG. 1 , on the composite image 95, by moving the mouse cursor 98 to the part of the building B that is to be inspected and clicking, the coordinates in the composite image 95 indicated by the mouse cursor 98 are designated. When the coordinates in the composite image 95 are designated, the photographic image data dn (reference number dn) stored as the coordinates (Xn, Yn) of the composite image is retrieved. Fig.12), the photographic image data corresponding to the coordinate value closest to the designated coordinate value is read out. Then, based on the read photographic image data, for example, a detailed partial image 93a is generated. The generated detailed partial image 93a is displayed on the display 13a in a window different from the synthetic image 95, for example.

[0214] As described above, the CPU 60A of the management device 11 determines the photographing method for photographing the photography target area 92 by photographing multiple times according to the form of the designated photography target area 92, obtains multiple detailed partial images 93n obtained by photographing under the determined photographing method, and performs resizing processing and synthesis processing in the reduction direction on the acquired detailed partial images 93n, thereby generating a synthetic image 95 related to the photography target area 92. According to this structure, the photography method can be determined according to the form of the designated photography target area, and the synthetic image 95 generated by resizing processing and synthesis processing on the photographed detailed partial images 93n in the reduction direction can be used to perform the inspection work. Therefore, in the inspection work of the photography target area 92, the resources required for the management device 11 can be reduced. In addition, by photographing only the designated photography target area 92, the photography time can be reduced, so the time of the inspection work can be shortened.

[0215] Furthermore, the CPU 60A can accept the designation of a shape other than a rectangle as the shape of the photographing target area. Therefore, only the area of ​​any shape that needs maintenance can be designated as the photographing target area, and only the area that needs maintenance can be photographed, thereby shortening the maintenance work time.

[0216] Furthermore, the CPU 60A stores coordinate correspondence information 97 that associates the positional relationship of the detailed partial images 93n in the composite image 95 with each partial image 93n. Therefore, the detailed partial image 93n of the position designated as the maintenance area can be displayed based on the coordinate correspondence information 97, so that the operation can be performed using the appropriate partial image 93n, thereby shortening the time of the maintenance operation.

[0217] Furthermore, by, for example, designating an arbitrary inspection position on the composite image 95 with the mouse cursor 98, image data of a detailed partial image 93n corresponding to the designated position can be read out, and the detailed partial image 93n can be displayed on the display 13a together with the composite image 95. Therefore, the positional relationship of the image of the inspection position with respect to the overall image can be easily recognized, and the operation time can be shortened.

[0218] <Other examples of inspection targets>

[0219] Fig.15 1 is a diagram showing an example of a case where the electric wire 102 of the transmission tower 101 is being inspected. Fig.15 As shown, a wide-angle image 91a of the electric wire 102 and the transmission tower 101 photographed by the camera 10 is displayed on the display 13a of the management device 11. In the example shown in the figure, three electric wires 102a, 102b, and 102c are connected between the transmission towers (not shown) adjacent to the transmission tower 101. In addition, the photographic object area 92a representing the photographic range is designated as a horizontally long area by the operator through a touch operation of the display 13a and by a free arm to surround the inspection object, namely the electric wire 102a. In this way, when the photographic object area 92a is designated as a horizontally long area, a plurality of photographic areas rn are arranged in a horizontally long shape along the photographic object area 92a. Then, while the plurality of photographic areas rn arranged in a horizontally long shape are sequentially photographed with a telephoto focus, photographic image data is acquired.

[0220] Fig.16 1 is a diagram showing an example of a case where a blade 112 of a windmill 111 is inspected. Fig.16 As shown, a wide-angle image 91b of a windmill 111 captured by a camera 10 is displayed on a display 13a of a management device 11. In the example shown in the figure, three blades 112 are provided on the windmill 111. Furthermore, these three blades 112 are set as maintenance objects, and a photographic object area 92b indicating a photographic range is designated as a roughly triangular area by a touch operation of the display 13a performed by an operator and by a free arm in a manner of surrounding the maintenance object, i.e., the three blades 112. In this way, when the photographic object area 92b is designated as a triangular area, a plurality of photographic areas rn are arranged in a triangular shape in such a manner that the photographic object area 92b is included inside the plurality of photographic areas rn. Then, while telephoto photography is sequentially performed on the plurality of photographic areas rn arranged in a triangular shape, photographic image data is acquired.

[0221] <Other Designation Examples of Photographing Target Area>

[0222] In the above Figure 7 , Fig.15 and Fig.16 In the example of FIG. 1 , the form of specifying the photographing target areas 92 , 92 a , and 92 b as if the imaged area were surrounded by lines was described, but the specification may be performed in the following form, for example.

[0223] Fig.17 FIG. 1 is a diagram showing an example of specifying a photographic target area using a point group. Fig.17As shown, on the electric wire 102a to be inspected, for example, the photographic target area is designated as photographic target points 121, 122, and 123. When the photographic target area is designated by the point group (photographic target points 121, 122, and 123), the electric wire 102 of the transmission tower 101 draws a specific curve, and based on the position information of the photographic target points 121, 122, and 123, the position information of the electric wire 102a between the photographic target points 121, 122, and 123 is interpolated. Then, based on the position information of the photographic target points 121, 122, and 123 and the calculated position information of the electric wire 102a between the photographic target points 121, 122, and 123, the area surrounding and Fig.15 In addition, as a method of generating a horizontally long photographic target area surrounding the electric wire 102a based on the photographic target points 121, 122, and 123, for example, information such as a wire designation pattern may be prepared, or image recognition based on machine learning may be used.

[0224] Fig.18 FIG. 1 is a diagram showing an example of specifying a photographic target area by lines. Fig.18 As shown, the imaging target area is designated as, for example, the imaging target line 131 along the target electric wire 102a to be inspected. When the imaging target area is designated by a line (the imaging target line 131), considering that the electric wire 102 of the transmission tower 101 draws a specific curve, the imaging target area is determined along the imaging target line 131 in a manner including the imaging target line 131. Fig.15 In addition, as a method of generating a horizontally long imaging target area surrounding the wire 102a based on the imaging target line 131, for example, information such as a wire designation pattern can be prepared, or image recognition based on machine learning can be used.

[0225] <Interpolation Processing When Specifying the Image Capture Target Area Using a Point Group>

[0226] Fig.19 This is a flowchart showing an example of “photographing processing of a photographing target region” and “synthesizing processing of a synthesized image” when the photographing target region is specified by a point group.

[0227] Assume that the camera 10 is used to perform maintenance work on the power transmission line. The camera 10 is set toward the object to be photographed, and the zoom position of the zoom lens is set to the wide-angle end. The wide-angle image captured by the camera 10 is displayed on the display 13a of the management device 11 (for example, refer to Fig.17 ).

[0228] The CPU 60A of the management device 11 starts the inspection process based on the designation operation of the inspection area (photographing target area) of the wide-angle image of the transmission tower 101 and the power line 102 displayed on the display 13a by the operator. Fig.19 In this example, the imaging target area is specified by a point group.

[0229] Fig.19 Each process from step S31 to step S33 in Figure 6 The processes from step S11 to step S13 described in are the same, so the description is omitted.

[0230] Next, the CPU 60A measures the distance of a portion of the photographic area rn among the plurality of photographic areas rn set in step S32 (step S34). The operator measures the distance of the portion of the photographic area rn. The CPU 60A prompts the operator to measure the distance of the portion of the photographic area rn using the auto focus button. For example, the operator measures the distance of the photographic object points 121, 122, and 123 (see FIG. 1 ) designated as the photographic object area. Fig.17 )’s photographic area rn.

[0231] Next, the CPU 60A derives the pan / tilt value and focus value corresponding to the next photographic area among the plurality of photographic areas arranged so as to include the photographic object area (step S35). Based on the size and positional relationship between the wide-angle image and the photographic area, the coordinates of the plurality of photographic areas arranged on the wide-angle image displayed on the display 13a and their pan / tilt values ​​are calculated. Furthermore, the focus value of the photographic area rn arranged between the photographic object points 121, 122, and 123 is calculated in advance using the focus value of the photographic area rn including the photographic object points 121, 122, and 123 obtained by the distance measurement in step S34. The calculated pan / tilt value and focus value are associated with the coordinates of the photographic area and stored as corresponding information in the memory 60C or the secondary storage device 14. Based on the corresponding information calculated in advance, the CPU 60A derives the pan / tilt value and focus value corresponding to the next photographic area.

[0232] Next, the CPU 60A controls the swivel operation of the swivel mechanism 16 based on the pan / tilt values ​​derived in step S35 , and controls the focus position of the camera 10 based on the derived focus value (step S36 ).

[0233] Fig.19 Each process from step S37 to step S40 in Figure 6 The processes from step S16 to step S19 described in are the same, so the description is omitted.

[0234] Fig. 20is a diagram showing interpolation processing between photographic areas when the photographic target area is specified by a point group. Fig. 20 As shown in the above Fig.19 The photographic region rn including the designated photographic target points 121 and 122 in the photographic target region of the electric wire 102a described in the figure is arranged in a horizontally long shape, for example, like the photographic regions r121, r122, r124, and r125. The photographic region r121 including the photographic target point 121 and the photographic region r122 including the photographic target point 122 are photographic regions whose focus values ​​are obtained by distance measurement based on the operation of the automatic focus button by the operator. In contrast, the photographic regions r124 and r125 arranged between the photographic regions r121 and r122 are photographic regions arranged based on the photographic regions r121 and r122, and are photographic regions whose focus values ​​are not obtained. In this case, the focus values ​​of the photographic regions r124 and r125 are calculated using the focus values ​​of the photographic regions r121 and r122 obtained by distance measurement.

[0235] In this way, by measuring the distance of a part of the photographing area rn to obtain the focus value, and using the focus value to calculate the focus value of the photographing area rn other than the part, the number of times the operator performs the automatic focusing or manual focusing operation can be reduced. As a result, the photographing target area can be photographed smoothly, and the time for the maintenance operation can be shortened.

[0236] When photographing an object such as Fig. 20 The transmission line situation shown, or Figure 7 In the case of the wall of Building B shown in FIG. 1 , the photographic subject area is on a plane parallel to the gravity direction (vertical direction). Therefore, in order to obtain the focus information of the photographic area for which the focus information has not been obtained in advance, it can be calculated based on the focus information of other photographic areas obtained in advance on the same parallel plane. Fig.16 In the case of the blade 112 of the windmill 111 shown, the photographic object area (the surface of the blade 112) is generally composed of a plane or a curved surface inclined with respect to the gravity direction. This structure is not limited to the blade 112 of the windmill 111, and is also the same in the photographic object area of, for example, a bridge or a tunnel. Therefore, the measured focusing information can be saved in advance according to the characteristics of the photographic object, and when the characteristics of the photographic object are detected during shooting during actual maintenance, the focusing information of the photographic object can be calculated using approximate focusing information from the saved focusing information. As a result, the maintenance work time of various infrastructures can be shortened.

[0237] <Image Display Processing When Specifying the Photographic Target Area Using a Point Group>

[0238] Fig.21This is a flowchart showing an example of display processing of “synthesized image and detailed partial image” when the imaging target area is specified by a point group.

[0239] The CPU 60A of the management device 11 Fig.19 In the process shown, after the photographing of all photographic areas is completed, the operation is started by the operator using the keyboard 13b or the mouse 13c to confirm the photographed images. Fig.21 In addition, Fig.19 Since this is a process of performing maintenance work on the power transmission line, in the case of this example, a composite image of the electric wire 102a is displayed on the display 13a.

[0240] Fig.21 The processing of steps S41, S42 and S44 in Fig.13 The processes of step S21, step S22 and step S24 described in are the same, so the description is omitted.

[0241] In step S42, when the CPU 60A receives the designation of the coordinates of the composite image 95 (step S42: YES), the CPU 60A generates the composite image 95 based on the coordinate correspondence information 97 (reference Fig.12 ) searches for photographic image data relative to the subject (electric wire 102a) closest to the specified coordinates (step S43). However, as described above, a plurality of photographic areas rn are arranged in such a manner that adjacent photographic areas rn slightly overlap each other. Therefore, the subject (electric wire 102a) closest to the specified coordinates may sometimes result in an overlapping area of ​​adjacent photographic areas rn and be included in a plurality of photographic areas rn. Therefore, the CPU 60A predetermines the area of ​​the subject in the composite image by image recognition. Then, upon receiving the designation of the coordinates of the composite image, the CPU 60A extracts photographic image data of the photographic areas rn overlapping within a certain range centered on the specified coordinates as candidates. The number of photographic areas rn overlapping within a specified range centered on the specified coordinates may be multiple. In the case where the number of extracted photographic areas rn is multiple, based on the coordinate correspondence information 97, photographic image data of, for example, the photographic area rn that occupies the largest area of ​​the subject is searched from the multiple photographic areas rn. In addition, the photographic image data of the photographic region rn having the highest contrast or the highest score in the damage detection result based on AI (Artificial Intelligence) etc. can be searched from the plurality of photographic regions rn.

[0242] Fig. 221 is a diagram showing an example in which a subject close to the designated coordinates is included in a plurality of photographic regions rn. In the case in which a plurality of photographic regions rn are set for a designated photographic target region and a subject (electrical wire 102a) is included in two or more photographic regions rn, Fig. 22 As shown, when generating a composite image, the subject (electric wire 102a) is sometimes included in the overlapping area of, for example, reduced image 141 and reduced image 142 that constitute the composite image. In the case of this example, if the electric wire 102a included in the reduced image 141 is compared with the electric wire 102a included in the reduced image 142, the electric wire 102a is included in the reduced image 142 as a whole without any missing, whereas a part of the electric wire 102a (the lower side of the figure) is included in the reduced image 141 in a missing state. That is, the area occupied by the electric wire 102a in the reduced image 142 is larger than the area occupied by the electric wire 102a in the reduced image 141. In this case, when the electric wire 102a is moved by, for example, the mouse cursor 98 Fig. 22 When the coordinates on the reduced image 141 in the composite image are specified as shown, the photographic image data of the photographic area rn corresponding to the reduced image 141 is not searched, but the photographic image data of the photographic area rn corresponding to the reduced image 142 in which the electric wire 102a occupies the largest area in the reduced image is searched.

[0243] <When the photographic object exceeds the field angle of the camera 10 at the wide angle end>

[0244] Fig.23 1 is a diagram showing an example of photography when the photographic object of the camera 10 exceeds the photographic range of the camera 10 at the wide angle end. Fig.23 As shown, in the maintenance work of this example, the range of the object to be inspected is the range θ1 between the transmission towers 101a and 101b to which the electric wire 102 is connected. In contrast, when the camera 10 is set at the wide angle end, the photographable range of the camera 10 is the field angle θ2, which is narrower than the range θ1 to be inspected. Therefore, in this case, a pseudo wide-angle image is generated by panoramic shooting at the wide angle end of the camera 10 so that the range of the object to be inspected can be included.

[0245] Fig.24 is a diagram showing an example of a pseudo wide-angle image captured by the camera 10. Fig.24 As shown, the pseudo wide-angle image 150 is a wide-angle image generated by arranging the wide-angle image 151 and the wide-angle image 152 captured by the camera 10 in the horizontal direction so that a part of the area overlaps each other. The wide-angle image 151 is an image of the left area within the range of the object to be inspected. The wide-angle image 152 is an image of the right area on the opposite side of the wide-angle image 151 within the range of the object to be inspected.

[0246] Fig.25 It means Fig.24 A map of the designated photographic subject area in the pseudo wide-angle image 150 generated as described in FIG. Fig.25 As shown, by specifying the photographing target area 153 with the free arm in a manner of surrounding the electric wire 102a, which is the object to be inspected, the photographing process of the photographing target area 153 and the synthesis process of the synthetic image are started. In addition, in this example, the case where the camera 10 capable of telephoto photography is set at the wide-angle end for shooting is described, but the same shooting can also be applied when the object cannot be included in one image when shooting with a wide-angle camera.

[0247] Thus, even when the imaging target area of ​​the inspection object is large or the distance to the inspection object is short, all areas to be inspected can be specified based on one wide-angle image displayed on the display 13a, thereby shortening the time of the inspection work.

[0248] <Modification of the Synthetic Image 95 Displayed When the Inspection Part is Designated>

[0249] Fig.26 It means that when specifying the above Fig.14 FIG. 1 is a diagram of a modified example of the composite image 95 displayed on the display 13a when the inspection part in the composite image 95 is detected. As described above, the multiple reduced images 94n constituting the composite image 95 are images generated by resizing the multiple detailed local images 93n captured by telephoto shooting in the reduction direction. Therefore, when the detailed local image 93n is captured, for example, AI-based image processing is used to inspect whether there is damage or dirt on the object (the wall of the building B) captured in the local image 93n. Then, in the case where damage or dirt is detected, the reduced image 94n corresponding to the detailed local image 93n in which the damage or dirt was detected is marked and displayed on the composite image 95. For example, as Fig.26 As shown in FIG. 1 , when damage 161 is detected on the wall surface of building B by AI-based image processing, a reduced image 94n corresponding to a detailed partial image 93n where the damage 161 is detected is marked with oblique lines and displayed in a synthesized image 95. This makes it possible to easily detect a portion that needs to be checked during maintenance work, thereby shortening the time for maintenance work.

[0250] <Generation of synthetic images based on geometric processing>

[0251] Fig. 27 FIG. 1 is a diagram showing an example of geometric processing performed when generating a composite image. Fig.15As described above, when the inspection object is the electric wire 102a of the transmission tower 101, the photographing area rn is arranged in a horizontally long shape along the electric wire 102a. Then, when the photographing area rn is arranged in a horizontally long shape, the camera 10 is panned / tilted, thereby sequentially performing telephoto photography of a plurality of photographing areas rn and acquiring photographic image data.

[0252] However, if the camera 10 is panned / tilted while performing telephoto photography of a plurality of photographic areas rn, for example, Fig. 27 As shown, the plurality of photographic regions r171, r172, and r173 are tilted by rotating by a predetermined angle due to panning / tilting. Therefore, if the detailed partial images 193a, 193b, and 193c of the photographic regions r171, r172, and r173 are photographed in a horizontal arrangement, the electric wires 102a included in the respective images are tilted by a predetermined angle relative to each other. Therefore, when a composite image is generated based on these detailed partial images 193a, 193b, and 193c, when only a simple resizing process and a synthesis process in the reduction direction are performed, a step difference of a rotation amount of a predetermined angle is generated between the electric wires 102a of the respective reduced images constituting the composite image.

[0253] Therefore, in order to eliminate the step difference generated when generating the synthetic image, the detailed partial images 193a, 193b, 193c are subjected to geometric processing different from the reduction processing and the enlargement processing. The rotation amount (inclination) of the predetermined angle generated by performing telephoto shooting while panning / tilting the camera 10 is a rotation amount that can be calculated from the shooting conditions of the camera 10, that is, the field angle and the panning / tilting value. Therefore, when generating the synthetic image 195 from the reduced images 194a, 194b, 194c of the detailed partial images 193a, 193b, 193c, the reduced images 194a, 194b, 194c are respectively subjected to rotation processing (correction) of the predetermined angle according to the rotation amount calculated from the shooting conditions so that the electric wire 102a of the synthetic image 195 does not generate a step difference. Thus, the distortion that may be generated when generating the synthetic image 195 can be corrected, and the synthetic image 195 that is easy to see can be displayed on the management device 11.

[0254] In addition, the order of each process when generating the composite image 195 can be, for example, the order of geometric processing for rotating the detailed local images 193a, 193b, 193c, the reduction processing for reducing the detailed local images 193a, 193b, 193c that have undergone the geometric processing, and the synthesis processing for synthesizing the reduced images 194a, 194b, 194c that have undergone the reduction processing.

[0255] <Storage Medium of Information Processing Program>

[0256] In the above management controls, an example is given in which the information processing program of each embodiment is stored in the storage device 60B of the management device 11, and the CPU 60A of the management device 11 executes the information processing program through the memory 60C. However, the technology of the present invention is not limited to this.

[0257] Fig.28 1 is a diagram showing an example of a method of installing an information processing program in the control device 60 of the management device 11 from a storage medium storing the information processing program for management control. Fig.28 As shown, as an example, the information processing program 221 can be stored in a non-temporary storage medium, that is, a storage medium 220. Fig.28 In the illustrated example, the information processing program 221 stored in the storage medium 220 is installed in the control device 60 , and the CPU 60A executes the above-described processes according to the information processing program 221 .

[0258] <Inspection Image Displayed on Management Device 11>

[0259] In the above embodiment, an example is given in which a composite image 95 composed of a plurality of reduced images 94n is displayed as a whole image displayed on the management device 11 when inspecting the captured object image, but the technology of the present invention is not limited to this. For example, a wide-angle image (for example, Figure 7 The wide-angle image 91) is displayed as a whole image.

[0260] Various embodiments have been described above, but the present invention is certainly not limited to these examples. Obviously, those skilled in the art can think of various variations or modifications within the scope described in the technical solution, and it should be understood that these variations or modifications naturally also fall within the technical scope of the present invention. Furthermore, the constituent elements in the above embodiments can be arbitrarily combined within the scope of the purpose of the invention.

[0261] In addition, this application is based on the Japanese patent application (Japanese Patent Application No. 2022-102804) filed on June 27, 2022, and the contents thereof are incorporated herein by reference.

[0262] Explanation of symbols

[0263] 1-photographic system, 10-camera, 11-management device, 13a-display, 16-rotation mechanism, 35, 60C-memory, 37, 60A-CPU, 60-control device, 71-yaw axis rotation mechanism, 72-pitch axis rotation mechanism, 91-wide-angle image, 92 (92a, 92b)-photographic object area, 93n (93a~93c, 193a~193c)-detailed local image, 94n (94a~94c), 141, 1 42, 194a~194c-reduced image, 95, 195-synthesized image, 96-background area, 97-coordinate correspondence information, 98-mouse cursor, 102 (102a~102c)-electric wires, 111-windmill, 112-blades, 121, 122, 123-photographic object points, 131-photographic object lines, 150-suspected wide-angle image, rn (r1, r2, r3, r121~r125, r171~r173)-photographic area.

Claims

1. An information processing device comprising a processor, wherein The processor performs the following processing: Determining a photographing method for photographing the designated photographing target area multiple times according to the morphology of the photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; and Second image data related to the imaging target area is generated by performing resizing processing in a reduction direction and synthesizing processing on the plurality of first image data.

2. The information processing device according to claim 1, wherein: The processor is capable of receiving a designation of a form other than a rectangle as the form of the imaging target area.

3. The information processing device according to claim 2, wherein: The processor generates the second image data representing a rectangular image when receiving designation of a form other than the rectangle as the form of the imaging target area.

4. The information processing device according to claim 1, wherein: The photography method includes obtaining the photography field angle and the photography position during the shooting.

5. The information processing device according to claim 1, wherein: The processor performs the following processing: First correspondence information is generated in which a positional relationship of a first image represented by the first image data in a second image represented by the second image data is associated with each other.

6. The information processing device according to claim 5, wherein: The processor performs the following control: Based on the first correspondence information, the first image represented by the first image data corresponding to the designated coordinates of the second image data is displayed on a display device.

7. The information processing device according to claim 1, wherein: The resizing process is a resizing process including a geometric change.

8. The information processing device according to claim 7, wherein: The resizing process including geometric changes is as follows: The second image data is generated based on the first image data through the resizing process in the reduction direction, the geometric process of applying a geometric change different from reduction and enlargement to the first image data, and the synthesis process.

9. The information processing device according to claim 8, wherein: The processor performs processing in the order of the geometric processing, the resizing processing in the reduction direction, and the synthesis processing.

10. The information processing device according to claim 8, wherein: The geometric processing includes rotation processing.

11. The information processing device according to claim 10, wherein: The rotation process includes a process based on a condition of capturing the first image data.

12. The information processing device according to claim 10, wherein: The rotation process includes a process of calculating a parameter for correcting the inclination of the angle of view during telephoto photography.

13. The information processing device according to claim 1, wherein: The processor performs the following processing: A reduction ratio in the resizing process in the reduction direction is acquired based on the size of the second image represented by the second image data.

14. The information processing device according to claim 1, wherein: The processor performs the following processing: controlling a rotating mechanism that rotates the photographing device that performs the photographing; and Second correspondence information is generated in which the first image data is associated with a control value of the slewing mechanism at the time of capturing the first image data.

15. The information processing device according to claim 14, wherein: The processor performs the following processing: Based on the second correspondence information, a control value corresponding to a designated first image data among the plurality of first image data is output.

16. The information processing device according to claim 14, wherein: The processor performs the following processing: The first image data is extracted based on the second correspondence information and the similarity of the control values ​​designated in the plurality of first image data.

17. The information processing device according to claim 1, wherein: The processor performs the following control: After acquiring distance measurement information of a plurality of positions in the photographic target area less than the number of photographing times of the photographing method, the photographing device is caused to execute photographing of the photographing method based on the distance measurement information.

18. An information processing method, performed by an information processing device, wherein: The processor of the information processing device performs the following processing: Determining a photographing method for photographing the designated photographing target area multiple times according to the morphology of the photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; and Second image data related to the imaging target area is generated by performing resizing processing in a reduction direction and synthesizing processing on the plurality of first image data.

19. An information processing program, which is an information processing program for an information processing device, wherein: The information processing program is used to cause the processor of the information processing device to perform the following processing: Determining a photographing method for photographing the designated photographing target area multiple times according to the morphology of the photographing target area; acquiring a plurality of first image data obtained by photographing using the photographing method; and Second image data related to the imaging target area is generated by performing resizing processing in a reduction direction and synthesizing processing on the plurality of first image data.

Images