Control device, control method, and program

Abstract

This control device is applied to a mobile body to which an imaging device and a light source are mounted. The imaging device images a first imaging target region and a second imaging target region which are within a subject in accordance with the movement positions of the mobile body. A first region that is a portion of the first imaging target region overlaps a second region that is a portion of the second imaging target region. The control device comprises a processor. The processor controls the light source so that the first imaging target region is irradiated with light. The intensity of the light irradiating the first region is higher than the intensity of the light irradiating the surroundings of the first region.

Description

Control device, control method, and program

[0001] The technology of the present disclosure relates to a control device, a control method, and a program.

[0002] International Publication No. 2019 / 150872 discloses an image processing device including an image input unit, a damage detection unit, an image determination unit, a display control unit, and a detection result correction unit. The image input unit inputs multiple images of a subject captured in separate images. The damage detection unit detects damage to the subject from individual images that constitute the multiple images. The image determination unit determines whether the individual image is to be used as a confirmation target image for prompting a user to confirm the detection results for the individual image. The display control unit displays, on the display device, a partial image obtained by cropping the confirmation target image or a portion of the confirmation target image to fit the display area of ​​the display device, in association with the detection results for the confirmation target image or the partial image. The detection result correction unit corrects the detection results based on user input instructions.

[0003] Japanese Patent Application Laid-Open Publication No. 2020-155902 discloses an imaging device that is mounted on a moving body and captures an image of a subject. The imaging device has multiple illumination and imaging units that each capture a different area on the subject in a direction intersecting the direction of movement of the moving body. Each of the multiple illumination and imaging units includes an illumination unit that illuminates at least a portion of an illumination range on the subject, and an imaging unit that captures an area included in the illumination range. The imaging device captures non-overlapping areas on the subject where the illumination ranges of the multiple illumination and imaging units do not overlap.

[0004] Japanese Patent Application Laid-Open Publication No. 2022-007039 discloses an imaging device including two light sources, an image sensor, a control unit, an identification unit, and a processing unit. The image sensor captures an image of a subject using light emitted from the two light sources and reflected by the subject. The control unit controls the emission timing of each of the two light sources and the exposure timing of the image sensor to obtain a first image of the subject captured by the image sensor using only the first light source of the two light sources, and a second image of the subject captured by the image sensor using only the second light source of the two light sources. The identification unit identifies an area in the first image where blown-out highlights have occurred as a blown-out highlight area. The processing unit generates a composite image by combining an image of the first image other than the blown-out highlight area with an image of an area in the second image corresponding to the blown-out highlight area, and outputs the generated composite image.

[0005] One embodiment of the technology disclosed herein provides, for example, a control device, a control method, and a program that can obtain an image used to generate a composite image that contributes to ensuring appropriate overall brightness of the composite image.

[0006] A first aspect of the technology of the present disclosure is a control device applied to a moving body equipped with an imaging device and a light source, wherein the imaging device images a first imaging target area and a second imaging target area of ​​a subject according to the moving position of the moving body, wherein a first area that is a part of the first imaging target area overlaps with a second area that is a part of the second imaging target area, and the control device is equipped with a processor, which controls the light source to irradiate light onto the first imaging target area, and wherein the intensity of the light irradiated onto the first area is higher than the intensity of the light irradiated onto the surrounding area of ​​the first area.

[0007] A second aspect of the technology of the present disclosure is a control device according to the first aspect, wherein the processor generates a composite image by combining a first captured image obtained by capturing an image of a first captured target area and a second captured image obtained by capturing an image of a second captured target area, and when generating the composite image, uses pixel values ​​of an image area corresponding to the second area in the second captured image as pixel values ​​of overlapping image areas corresponding to the first and second areas in the composite image.

[0008] A third aspect of the technology of the present disclosure is a control device according to the first or second aspect, wherein the control includes control of fitting an image of a light source into an image area corresponding to a first area in a first captured image obtained by capturing an image of a first capture target area.

[0009] A fourth aspect of the technology of the present disclosure is a control device relating to any one of the first to third aspects, wherein the imaging device includes an image sensor having a light receiving surface, and the processor is a control device that adjusts the exposure amount of the light receiving surface based on the brightness of a third area other than the first area in the first imaging target area when the first imaging target area is imaged by the imaging device.

[0010] A fifth aspect of the technique of the present disclosure is the control device according to the fourth aspect, wherein the exposure amount is adjusted by changing the irradiation position and / or intensity of light.

[0011] A sixth aspect of the technique of the present disclosure is the control device according to the fourth aspect, wherein the amount of exposure light is adjusted by changing the exposure of the light receiving surface.

[0012] A seventh aspect of the technology of the present disclosure is a control device relating to any one of the fourth to sixth aspects, in which the exposure amount is set to an exposure amount such that the pixel value of an image area corresponding to a first area in a first captured image obtained by capturing a first imaging target area is equal to or greater than a first default pixel value.

[0013] An eighth aspect of the technology of the present disclosure is a control device according to the seventh aspect, wherein the first default pixel value is set to a pixel value at which the pixel values ​​of at least a portion of the image area are saturated.

[0014] A ninth aspect of the technology of the present disclosure is a control device according to the seventh or eighth aspect, wherein the first imaging target area and the second imaging target area are areas imaged in the order of the first imaging target area and the second imaging target area, and the processor obtains the length of the first area along the direction of movement of the moving body based on the pixel values ​​of the image area, and moves the moving body to a position where the first area and the second area overlap based on the length.

[0015] A tenth aspect of the technology of the present disclosure is a control device relating to any one of the seventh to ninth aspects, in which the exposure amount is set to an exposure amount such that the pixel value of an image area corresponding to a third area in the first captured image is greater than or equal to a second default pixel value and less than a first default pixel value.

[0016] An eleventh aspect of the technique of the present disclosure is the control device according to the tenth aspect, wherein the amount of overlap between the first region and the second region is a predetermined amount of overlap.

[0017] A twelfth aspect of the technology of the present disclosure is a control device relating to any one of the first to eleventh aspects, in which the light source is arranged in front of the imaging device in the direction of movement of the moving body, and the first imaging target area and the second imaging target area are areas imaged in the order of the first imaging target area and the second imaging target area.

[0018] A thirteenth aspect of the technology of the present disclosure is a control device relating to any one of the first to eleventh aspects, in which the light source is arranged behind the imaging device in the direction of movement of the moving body, and the first imaging target area and the second imaging target area are areas imaged in the order of the second imaging target area and the first imaging target area.

[0019] A fourteenth aspect of the technology of the present disclosure is a control device relating to any one of the first to thirteenth aspects, wherein when a second imaging target area is imaged by the imaging device, the processor controls the light source to irradiate light onto the second imaging target area, and the intensity of the light irradiated onto a fourth area of ​​the second imaging target area opposite the second area is higher than the intensity of the light irradiated onto the periphery of the fourth area.

[0020] A fifteenth aspect of the technology of the present disclosure is a control device according to any one of the first to fourteenth aspects, in which light is irradiated along the normal direction of the first region.

[0021] A sixteenth aspect of the technology of the present disclosure is a control device relating to any one of the first to fifteenth aspects, wherein the control is such that when a moving object moves to a position where a first imaging target area is imaged by an imaging device, the light source is caused to emit light.

[0022] A seventeenth aspect of the technology of the present disclosure is a control device according to any one of the first to fifteenth aspects, wherein the control is a control device that causes a light source to irradiate light when a moving object moves to a position where a first imaging target area is imaged by an imaging device.

[0023] An 18th aspect of the technology of the present disclosure is a control method applied to a moving body equipped with an imaging device and a light source, wherein the imaging device images a first imaging target area and a second imaging target area of ​​a subject according to the moving position of the moving body, a first area that is a part of the first imaging target area overlaps with a second area that is a part of the second imaging target area, and the control method includes controlling the light source to irradiate light onto the first imaging target area, wherein the intensity of the light irradiated onto the first area is higher than the intensity of light irradiated onto the surrounding area of ​​the first area.

[0024] A 19th aspect of the technology of the present disclosure is a program for causing a computer to execute processing, which is applied to a moving body equipped with an imaging device and a light source, wherein the imaging device images a first imaging target area and a second imaging target area of ​​a subject according to the moving position of the moving body, a first area that is a part of the first imaging target area overlaps with a second area that is a part of the second imaging target area, and the processing includes controlling the light source to irradiate light onto the first imaging target area, and the intensity of the light irradiated onto the first area is higher than the intensity of light irradiated onto the surrounding area of ​​the first area.

[0025] 1 is a front view showing an example of a mode in which a plurality of imaging target areas are sequentially imaged by the flying imaging device according to the first embodiment. FIG. 2 is a block diagram showing an example of a hardware configuration of the flying imaging device according to the first embodiment. FIG. 3 is a block diagram showing an example of a hardware configuration of the imaging device according to the first embodiment. FIG. 4 is a two-sided view showing an example of a mode in which the Nth imaging target area and the N+1th imaging target area are imaged depending on the flight position of the flying imaging device according to the first embodiment. FIG. 5 is a block diagram showing an example of a functional configuration for realizing the flight imaging process according to the first embodiment. FIG. 6 is a block diagram showing an example of the operation of the lighting control unit and the first arrival determination unit according to the first embodiment. FIG. 7 is a block diagram showing an example of the operation of the first arrival determination unit, the first imaging control unit, the first brightness acquisition unit, and the first exposure amount derivation unit according to the first embodiment. FIG. 8 is a block diagram showing an example of the operation of the first exposure amount derivation unit, the second imaging control unit, and the width acquisition unit according to the first embodiment. FIG. 9 is a block diagram showing an example of the operation of the width acquisition unit, the imaging position correction unit, and the second arrival determination unit according to the first embodiment. FIG. 10 is a block diagram showing an example of the operation of the second arrival determination unit, the third imaging control unit, the second brightness acquisition unit, and the second exposure amount derivation unit according to the first embodiment. FIG. 11 is a block diagram showing an example of the operation of the second exposure amount derivation unit and the fourth imaging control unit according to the first embodiment. FIG. 12 is a block diagram showing an example of the operation of the termination determination unit and the extinguishing control unit according to the first embodiment. 10 is a block diagram showing an example of a functional configuration for realizing a composite image generation process according to the first embodiment. FIG. 11 is a block diagram showing an example of the operation of an image acquisition unit and an image synthesis unit according to the first embodiment. FIG. 12 is a flowchart showing an example of the flow of a flight imaging process according to the first embodiment. FIG. 13 is a block diagram showing an example of the operation of a first exposure amount derivation unit, a light source control unit, and a second imaging control unit according to the second embodiment. FIG. 14 is a two-sided view showing an example of a mode in which an Nth imaging target area and an N+1th imaging target area are imaged depending on the flight position of a flight imaging device according to the third embodiment. FIG. 15 is a block diagram showing an example of the operation of a second exposure amount derivation unit, a light source control unit, and a fourth imaging control unit according to the third embodiment. FIG. 16 is a block diagram showing an example of the operation of an image acquisition unit and an image synthesis unit according to the third embodiment.10 is a front view showing an example of a mode in which a plurality of imaging target areas are sequentially imaged by a flight imaging device according to a fourth embodiment. FIG. 11 is a two-sided view showing an example of a mode in which the Nth imaging target area and the N+1th imaging target area are imaged when the flight imaging device according to the fourth embodiment flies to the first side. FIG. 12 is a two-sided view showing an example of a mode in which the Nth imaging target area and the N+1th imaging target area are imaged when the flight imaging device according to the fourth embodiment flies to the second side. FIG. 13 is a front view showing an example of a mode in which the flight imaging device according to the fifth embodiment sequentially images a plurality of imaging target areas while flying the Nth flight route. FIG. 14 is a front view showing an example of a mode in which the flight imaging device according to the fifth embodiment sequentially images a plurality of imaging target areas while flying the Nth flight route. FIG. 15 is a front view showing an example of a mode in which the flight imaging device according to the sixth embodiment sequentially images a plurality of imaging target areas while flying the Nth flight route. FIG. 16 is a front view showing an example of a mode in which the flight imaging device according to the sixth embodiment sequentially images a plurality of imaging target areas while flying the N+1th flight route. FIG. 17 is a block diagram showing an example of the operation of the image acquisition unit and the image synthesis unit according to the sixth embodiment.

[0026] Hereinafter, an example of an embodiment of a control device, a control method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] I / F is an abbreviation for "Interface". RAM is an abbreviation for "Random Access Memory". CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". HDD is an abbreviation for "Hard Disk Drive". SSD is an abbreviation for "Solid State Drive". DRAM is an abbreviation for "Dynamic Random Access Memory". SRAM is an abbreviation for "Static Random Access Memory". GNSS is an abbreviation for "Global Navigation Satellite System". GPS is an abbreviation for "Global Positioning System". LiDAR is an abbreviation for "light detection and ranging". NVM is an abbreviation for "Non-Volatile Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". FPGA is an abbreviation for "Field-Programmable Gate Array". PLD is an abbreviation for "Programmable Logic Device". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". RGB is an abbreviation for "Red Green Blue". CIE is an abbreviation for "Commission Internationale de l'Eclairage". TPU is an abbreviation for "Tensor Processing Unit". USB is an abbreviation for "Universal Serial Bus". SoC is an abbreviation for "System-on-a-Chip". IC is an abbreviation for "Integrated Circuit".

[0029] In the description of this specification, the term "vertical direction" refers to a perfectly vertical direction as well as a vertical direction generally accepted in the technical field to which the technology of the present disclosure belongs, with a degree of tolerance that does not deviate from the spirit of the technology of the present disclosure. In the description of this specification, the term "horizontal direction" refers to a perfectly horizontal direction as well as a horizontal direction generally accepted in the technical field to which the technology of the present disclosure belongs, with a degree of tolerance that does not deviate from the spirit of the technology of the present disclosure. In the description of this specification, the term "rectangle" refers to a perfectly horizontal direction as well as a rectangle generally accepted in the technical field to which the technology of the present disclosure belongs, with a degree of tolerance that does not deviate from the spirit of the technology of the present disclosure. In the description of this specification, the term "vertical" refers to a perfectly vertical direction as well as a vertical direction generally accepted in the technical field to which the technology of the present disclosure belongs, with a degree of tolerance that does not deviate from the spirit of the technology of the present disclosure. In the description of this specification, "constant" refers to a constant that is generally accepted in the technical field to which the technology of the present disclosure belongs, as well as a complete constant, and includes a degree of error that does not violate the spirit of the technology of the present disclosure. In the description of this specification, "overlap" refers to a constant that is generally accepted in the technical field to which the technology of the present disclosure belongs, as well as a complete overlap, and includes a degree of error that does not violate the spirit of the technology of the present disclosure. In the description of this specification, "maximum value" refers to a maximum value that is generally accepted in the technical field to which the technology of the present disclosure belongs, as well as a complete maximum value, and includes a degree of error that is generally accepted in the technical field to which the technology of the present disclosure belongs, and does not violate the spirit of the technology of the present disclosure. In the description of this specification, "minimum value" refers to a minimum value that is generally accepted in the technical field to which the technology of the present disclosure belongs, as well as a complete minimum value, and includes a degree of error that is generally accepted in the technical field to which the technology of the present disclosure belongs, and does not violate the spirit of the technology of the present disclosure. In the description of this specification, the term "median value" refers to a median value that includes not only the exact median value, but also an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and that does not contradict the spirit of the technology of the present disclosure.In the description of this specification, the term "average value" refers to an average value that includes not only the exact average value, but also an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and that does not contradict the spirit of the technology of the present disclosure.

[0030] First Embodiment First, the first embodiment will be described.

[0031] 1, a flying imaging device 10 has a flying function and an imaging function, and captures an image of a wall surface 2A of an object 2 while flying. In the description of this specification, the concept of "flying" includes not only the meaning of the flying imaging device 10 moving through the air, but also the meaning of the flying imaging device 10 standing still in the air. The flying imaging device is an example of a "mobile imaging device" according to the technology of the present disclosure.

[0032] As an example, the object 2 having the wall surface 2A is a bridge pier provided on a bridge. The bridge pier is made of, for example, reinforced concrete. Here, a bridge pier is given as an example of the object 2, but the object 2 may be an object other than a bridge pier (for example, a tunnel or a dam).

[0033] The flight imaging device 10 includes an air vehicle 20 and an imaging device 60. The air vehicle 20 is, for example, an unmanned aerial vehicle such as a drone. The flight function of the flight imaging device 10 is realized by the air vehicle 20. The air vehicle 20 has multiple propellers 42, and flies when the multiple propellers 42 rotate. The flight of the air vehicle 20 is synonymous with the flight of the flight imaging device 10. The air vehicle 20 is an example of a "moving body" according to the technology of the present disclosure.

[0034] The imaging device 60 is, for example, a digital camera or a video camera. The imaging function of the flying imaging device 10 is realized by the imaging device 60. The imaging device 60 is mounted on the flying object 20. As an example, the imaging device 60 is provided on the bottom of the flying object 20. The imaging device 60 is an example of an "imaging device" according to the technology of the present disclosure.

[0035] A plurality of flight routes 4 are set for the target object 2. In the example shown in FIG. 1 , the plurality of flight routes 4 are set on a virtual surface opposite the wall surface 2A. The plurality of flight routes 4 are aligned vertically, and each flight route 4 extends horizontally. As will be described later, the flight imaging device 10 stores flight route information 122 (see FIG. 6 ) indicating each flight route 4, and is capable of autonomously flying each flight route 4 without relying on a flight instruction signal from a transmitter (not shown) or a base station (not shown), etc.

[0036] The flying imaging device 10 flies sequentially along a plurality of flight routes 4. The flying imaging device 10 moves horizontally by flying along each flight route 4. The plurality of flight routes 4 are flight routes that cause the flying imaging device 10 to fly in the same direction. In the example shown in FIG. 1 , each flight route 4 is a flight route that causes the flying imaging device 10 to fly from left to right toward the wall surface 2A. A plurality of imaging positions 5 are set on each flight route 4. At each imaging position 5, the flying imaging device 10 captures an image of the wall surface 2A.

[0037] Note that, although an example is given here in which the flight imaging device 10 flies each flight route 4 independently, the flight imaging device 10 may also fly each flight route 4 based on a flight instruction signal from a transmitter or base station, etc.

[0038] The flying imaging device 10 captures images of the wall surface 2A at each imaging position 5, thereby sequentially capturing images of multiple imaging target areas 3 on the wall surface 2A. Each imaging target area 3 corresponds to a corresponding imaging position 5. The imaging target area 3 is an area determined by the angle of view of the flying imaging device 10. In the example shown in FIG. 1, a rectangular area is shown as an example of the imaging target area 3. The flying imaging device 10 captures images of the wall surface 2A at each imaging position 5 while flying sequentially along multiple flight routes 4, thereby capturing images of multiple imaging target areas 3 lined up in the horizontal and vertical directions. The multiple imaging target areas 3 are captured in a manner such that adjacent imaging target areas 3 in the horizontal or vertical direction partially overlap each other.

[0039] 1 shows an example in which each imaging target area 3 is imaged by the imaging device 60 with the optical axis OA (see FIG. 3) of the imaging device 60 perpendicular to the wall surface 2A. The following description will be given on the assumption that each imaging target area 3 is imaged by the imaging device 60 with the optical axis OA of the imaging device 60 perpendicular to the wall surface 2A. For convenience, the following description will be given on the assumption that the distance between the wall surface 2A and the imaging device 60 is constant.

[0040] A plurality of images to be combined 142 are obtained by sequentially capturing images of a plurality of imaging target regions 3 using the imaging device 60. A composite image 140 is generated by combining the plurality of images to be combined 142. The plurality of images to be combined 142 are combined such that adjacent images to be combined 142 in the horizontal or vertical direction partially overlap with each other. Hereinafter, the partial overlap of adjacent imaging target regions 3 and the partial overlap of adjacent images to be combined 142 may each be referred to as "overlap."

[0041] A two-dimensional panoramic image is an example of the composite image 140. The two-dimensional panoramic image is merely an example, and a three-dimensional image (e.g., a three-dimensional panoramic image) may be generated as the composite image 140 in the same manner as a two-dimensional panoramic image is generated as the composite image 140. The composite image 140 is used, for example, for inspecting and / or surveying the wall surface 2A of the object 2.

[0042] 2, the flying object 20 includes a flight device 22, an input / output I / F 24, a computer 26, a positioning unit 28, and an acceleration sensor 30. The computer 26 is an example of a "control device" and a "computer" according to the techniques of the present disclosure.

[0043] The computer 26 includes a processor 34, a storage 36, and a RAM 38. The processor 34, the storage 36, and the RAM 38 are connected to one another via a bus 40, and the bus 40 is connected to the input / output I / F 24.

[0044] The processor 34 has, for example, a CPU and controls the entire flight imaging device 10. Here, an example in which the processor 34 has a CPU is given, but this is merely one example. For example, the processor 34 may have a CPU and a GPU. In this case, for example, the GPU operates under the control of the CPU and is responsible for executing image processing. The processor 34 is an example of a "processor" according to the technology of the present disclosure.

[0045] The storage 36 is a non-volatile storage device that stores various programs, various parameters, etc. Examples of the storage 36 include an HDD and an SSD. Note that the HDD and SSD are merely examples, and a flash memory, a magnetoresistive memory, and / or a ferroelectric memory may be used instead of the HDD and / or SSD, or together with the HDD and / or SSD.

[0046] The RAM 38 is a memory that temporarily stores information and is used as a work memory by the processor 34. The RAM 38 may be, for example, a DRAM and / or an SRAM.

[0047] The flight device 22 has multiple propellers 42, multiple motors 44, and a motor driver 46. The motor driver 46 is connected to the processor 34 via the input / output I / F 24 and the bus 40. The motor driver 46 individually controls the multiple motors 44 in accordance with instructions from the processor 34. The number of the multiple motors 44 is the same as the number of the multiple propellers 42.

[0048] A propeller 42 is fixed to the rotation shaft of each motor 44. Each motor 44 rotates the propeller 42. The rotation of the multiple propellers 42 causes the flying object 20 to fly. Note that, while the number of multiple propellers 42 (in other words, the number of multiple motors 44) provided to the flying object 20 is four, this is merely an example, and the number of multiple propellers 42 may be, for example, three, or five or more.

[0049] The positioning unit 28 is a device that detects the position of the aircraft 20. The position of the aircraft 20 is detected, for example, using GNSS (e.g., GPS). The positioning unit 28 has a GNSS receiver (not shown). The GNSS receiver receives, for example, radio waves transmitted from multiple satellites. The positioning unit 28 detects the position of the aircraft 20 based on the radio waves received by the GNSS receiver and outputs positioning data (e.g., data indicating latitude, longitude, and altitude) corresponding to the detected position.

[0050] The acceleration sensor 30 detects acceleration in each of the axial directions of the pitch axis, yaw axis, and roll axis of the flying object 20. The acceleration sensor 30 outputs acceleration data corresponding to the acceleration in each axial direction of the flying object 20. The processor 34 acquires the position of the flying object 20 based on the positioning data and / or the acceleration data.

[0051] If the processor 34 acquires the position of the flying object 20 based only on the positioning data, the acceleration sensor 30 may be omitted. On the other hand, if the processor 34 acquires the position of the flying object 20 based only on the acceleration data, the positioning unit 28 may be omitted.

[0052] When the processor 34 acquires the position of the flying object 20 based on the positioning data, the position in the absolute coordinate system is derived based on the positioning data. On the other hand, when the processor 34 acquires the image capture position 5 based on the acceleration data, the amount of change in the position relative to a reference position defined in the relative coordinate system is derived based on the acceleration data.

[0053] Furthermore, the flying object 20 may be equipped with other devices for detecting the position of the flying object 20 instead of or in addition to the positioning unit 28 and / or the acceleration sensor 30. Examples of such other devices include a LiDAR scanner, a stereo camera, a magnetic compass, a barometric altimeter, or an ultrasonic sensor.

[0054] The flight imaging device 10 also has an illumination function. The illumination function of the flight imaging device 10 is a function of irradiating light onto the wall surface 2A (see FIG. 1). The flight imaging device 10 has a light source 48, and the illumination function of the flight imaging device 10 is realized by the light source 48. The light source 48 is, for example, an LED light source, a laser light source, a strobe light source, or an incandescent light bulb. The light source 48 is an example of a "light source" according to the technology of the present disclosure.

[0055] As an example, as shown in FIG. 3, the imaging device 60 includes an imaging lens 62, an aperture 64, an aperture actuator 66, a shutter 68, a shutter actuator 70, a controller 72, an image sensor 74, and an image sensor driver 76.

[0056] The controller 72 and the image sensor driver 76 are connected to the processor 34 via the input / output I / F 24 and the bus 40. The imaging lens 62 includes, for example, an objective lens (not shown) and a focus lens (not shown). The imaging device 60 also includes a zoom lens (not shown). The imaging lens 62 is disposed on the object side of the aperture 64, and the zoom lens is disposed between the aperture 64 and the shutter 68.

[0057] The aperture actuator 66 has a power transmission mechanism (not shown) and an aperture motor (not shown). The aperture 64 has an opening 64A, and the size of the opening 64A is variable. The opening 64A is formed by multiple blades (not shown). The multiple blades are connected to the power transmission mechanism. The power transmission mechanism is connected to an aperture motor, and the power transmission mechanism transmits the power of the aperture motor to the multiple blades. The multiple blades are actuated by receiving power transmitted from the power transmission mechanism, thereby changing the size of the aperture 64A. The aperture 64 adjusts the exposure by changing the size of the opening 64A. The aperture actuator 66 is connected to a controller 72.

[0058] The controller 72 is a device having a computer including, for example, a CPU, NVM, RAM, etc. (All of these are omitted from the drawing.) Note that although a computer is used here as an example, this is merely one example, and devices including an ASIC, FPGA, and / or PLD may also be used. Furthermore, the controller 72 may be implemented by combining hardware and software, for example. The controller 72 controls the aperture actuator 66 in accordance with instructions from the processor 34.

[0059] The image sensor 74 includes a photoelectric conversion element 78 and a signal processing circuit 80. The image sensor 74 is, for example, a CMOS image sensor. Here, a CMOS image sensor is exemplified as the image sensor 74, but the technology of the present disclosure is not limited to this. For example, the technology of the present disclosure can also be applied when the image sensor 74 is another type of image sensor, such as a CCD image sensor. The photoelectric conversion element 78 is connected to an image sensor driver 76. The image sensor driver 76 controls the photoelectric conversion element 78 in accordance with instructions from the processor 34.

[0060] The photoelectric conversion element 78 has a light receiving surface 78A on which a plurality of pixels (not shown) are provided. The photoelectric conversion element 78 outputs electrical signals output from the plurality of pixels as imaging data to a signal processing circuit 80. The signal processing circuit 80 digitizes the analog imaging data input from the photoelectric conversion element 78. The signal processing circuit 80 is connected to the input / output I / F 24. The processor 34 performs various processes on the digitized imaging data.

[0061] The shutter 68 is, for example, a focal plane shutter, and is disposed between the diaphragm 64 and the light receiving surface 78A. The shutter 68 includes a first curtain 68A and a second curtain 68B. For example, the first curtain 68A and the second curtain 68B each include a plurality of blades (not shown). The first curtain 68A is disposed closer to the subject than the second curtain 68B.

[0062] The shutter actuator 70 has a link mechanism (not shown), a leading curtain solenoid (not shown), and a trailing curtain solenoid (not shown). The leading curtain solenoid is the drive source for the leading curtain 68A and is mechanically linked to the leading curtain 68A via the link mechanism. The trailing curtain solenoid is the drive source for the trailing curtain 68B and is mechanically linked to the trailing curtain 68B via the link mechanism. The shutter actuator 70 is connected to a controller 72. The controller 72 controls the shutter actuator 70 in accordance with instructions from the processor 34.

[0063] The first curtain solenoid generates power under the control of the controller 72 and applies the generated power to the first curtain 68A to selectively wind up or down the first curtain 68A. The second curtain solenoid generates power under the control of the controller 72 and applies the generated power to the second curtain 68B to selectively wind up or down the second curtain 68B. In the imaging device 60, the opening and closing of the first curtain 68A and the second curtain 68B are controlled by the processor 34, thereby adjusting the amount of exposure to the image sensor 74. The shutter speed of the shutter 68 is determined by the time the first curtain 68A and the second curtain 68B are open.

[0064] Although a focal plane shutter has been described as an example of the shutter 68, this is merely an example, and the shutter 68 may also be a lens shutter. Also, although an example in which the shutter speed of the shutter 68 is specified has been described, this is merely an example. For example, the shutter speed of an electronic shutter (e.g., an electronic front-curtain shutter or a fully electronic shutter) may also be specified.

[0065] 2 and 3, the computer 26 is common to the aircraft 20 and the imaging device 60, but the computer 26 may be composed of a first computer provided in the aircraft 20 and a second computer provided in the imaging device 60. The computer 26 is mounted on the aircraft 20, but may also be mounted on the imaging device 60, or on a transmitter or base station.

[0066] 4 shows an example of an aspect in which the Nth imaging target area 3 and the N+1th imaging target area 3 are imaged according to the flight position of the flight imaging device 10. N is a natural number indicating the order in which the images were captured. The Nth imaging target area 3 and the N+1th imaging target area 3 are areas imaged in the order of the Nth imaging target area 3 and the N+1th imaging target area 3. The Nth imaging target area 3 and the N+1th imaging target area 3 are areas aligned horizontally.

[0067] Each imaging target area 3 has a first area 3A and a second area 3B. The first area 3A and the second area 3B are parts of the imaging target area 3. The first area 3A is an area of ​​the imaging target area 3 that is located in front of the flight direction of the aircraft 20, and the second area 3B is an area of ​​the imaging target area 3 that is located behind the flight direction of the aircraft 20. The first area 3A of the Nth imaging target area 3 overlaps with the second area 3B of the N+1th imaging target area 3.

[0068] In the first embodiment, the Nth imaging target area 3 is an example of a "first imaging target area" according to the technology of the present disclosure. The N+1th imaging target area 3 is an example of a "second imaging target area" according to the technology of the present disclosure. The first area 3A of the Nth imaging target area 3 is an example of a "first area" according to the technology of the present disclosure. The second area 3B of the N+1th imaging target area 3 is an example of a "second area" according to the technology of the present disclosure.

[0069] The light source 48 is disposed forward of the imaging device 60 in the flight direction of the flying object 20. The light source 48 irradiates each imaging target area 3 with light. The light source 48 is disposed at a position where the optical axis passes through each first area 3A (for example, the center of each first area 3A) when each imaging target area 3 is imaged by the imaging device 60. In other words, the light source 48 is disposed at a position where it irradiates light along the normal direction of each first area 3A when each imaging target area 3 is imaged by the imaging device 60. The normal direction may be the average normal direction of the first areas 3A.

[0070] As an example, Fig. 4 shows the orientation characteristics of the light source 48. (A) shows the light intensity distribution, and (B) shows the relationship between the light intensity and the distance from the center of the light. As shown by the orientation characteristics, the light emitted by the light source 48 has a characteristic in which the intensity decreases with increasing distance from the center. Because the light source 48 has such orientation characteristics, the intensity of the light irradiated onto the first region 3A is higher than the intensity of the light irradiated onto the periphery of the first region 3A.

[0071] Furthermore, the size of the light source 48 is set so that, for example, when the Nth image to be combined 142 is obtained by capturing an image of the Nth imaging target region 3, an image 50 of the light source 48 (i.e., an image of reflected light) can fit in an image region 142A corresponding to the first region 3A in the Nth image to be combined 142. Hereinafter, the image 50 of the light source 48 will be referred to as the "light source image 50."

[0072] 5 , a flight imaging program 90 is stored in the storage 36. The flight imaging program 90 is an example of a "program" according to the technology of the present disclosure. The processor 34 reads the flight imaging program 90 from the storage 36 and executes the read flight imaging program 90 on the RAM 38. The processor 34 performs flight imaging processing to capture images of each imaging target area 3 while flying along each flight route 4 in accordance with the flight imaging program 90 executed on the RAM 38.

[0073] The flight imaging process is realized by the processor 34 operating in accordance with the flight imaging program 90 as a lighting control unit 92, a first arrival determination unit 94, a first imaging control unit 96, a first brightness acquisition unit 98, a first exposure amount derivation unit 100, a second imaging control unit 102, a width acquisition unit 104, an imaging position correction unit 106, a second arrival determination unit 108, a third imaging control unit 110, a second brightness acquisition unit 112, a second exposure amount derivation unit 114, a fourth imaging control unit 116, an end determination unit 118, and a lights-off control unit 120. The flight imaging process is executed when the flight imaging device 10 starts flying from the start of each flight route 4.

[0074] As an example, as shown in FIG. 6 , the lighting control unit 92 outputs a lighting instruction signal to the light source 48, thereby turning on the light source 48. As a result, light is irradiated onto the wall surface 2A. The lighting control unit 92 causes the light source 48 to irradiate light by the time the flying object 20 reaches the Nth imaging position 5. The Nth imaging position 5 shown in FIG. 6 is, as an example, the first imaging position 5. In the first embodiment, the intensity of light irradiated from the light source 48 is constant. The control by the lighting control unit 92 is an example of "control for causing the light source to irradiate light onto the first imaging target area" and "control for causing the light source to irradiate light when the moving object moves to a position where the first imaging target area is imaged by the imaging device" according to the technology of the present disclosure.

[0075] The storage 36 stores flight route information 122 indicating the flight route 4. The flight route information 122 includes imaging position information indicating the position of each imaging position 5 set on the flight route 4.

[0076] The first arrival determination unit 94 acquires the position of the flying object 20 based on the positioning data input from the positioning unit 28 and / or the acceleration data input from the acceleration sensor 30. Then, the first arrival determination unit 94 determines whether the flying object 20 has reached the Nth image capturing position 5 based on the acquired position of the flying object 20 and the Nth image capturing position 5 indicated by the flight route information 122.

[0077] 7 shows an example of a state in which the flying object 20 has reached the Nth imaging position 5. When the flying object 20 has reached the Nth imaging position 5, it temporarily comes to a standstill. When the flying object 20 has reached the Nth imaging position 5, light is irradiated from the light source 48 onto the Nth imaging target area 3. In this case, the intensity of the light irradiated onto the first area 3A of the Nth imaging target area 3 is higher than the intensity of the light irradiated onto the periphery of the first area 3A of the Nth imaging target area 3.

[0078] When the first arrival determination unit 94 determines that the flying object 20 has reached the Nth imaging position 5, the first imaging control unit 96 outputs an imaging instruction signal to the image sensor 74, causing the image sensor 74 to image the Nth imaging target area 3. The image sensor 74 images the Nth imaging target area 3 under the control of the first imaging control unit 96, thereby obtaining captured image data. The captured image data is image data that represents the captured image. The first imaging control unit 96 acquires the captured image corresponding to the Nth imaging target area 3 based on the captured image data. The captured image may be a brightness detection image for detecting brightness, or may be a display image such as a live view image displayed on a display device (not shown).

[0079] The first brightness acquisition unit 98 acquires the brightness of a third region 3C other than the first region 3A in the Nth imaging target region 3 based on the captured image acquired by the first imaging control unit 96. As the brightness of the third region 3C, the first brightness acquisition unit 98 may acquire the brightness of a central region of the Nth imaging target region 3, may acquire the brightness of a central region of the third region 3C, or may acquire the brightness of the entire third region 3C. Furthermore, the brightness may be a representative value (e.g., a maximum value, a minimum value, or a median value) or an average value. The third region 3C is an example of a "third region" according to the technology of the present disclosure.

[0080] The first exposure amount derivation unit 100 derives the exposure amount based on the brightness acquired by the first brightness acquisition unit 98. The exposure amount may be derived based on a table stored in the storage 36, or may be derived based on a relational expression stored in the storage 36.

[0081] 7 shows, as an example, a graph indicating pixel values ​​of an Nth compositing image 142 (see also FIG. 8 ), which will be described later, and the correspondence between the compositing image 142. The graph indicating pixel values ​​of the Nth compositing image 142 has a shape corresponding to the graph indicating the orientation characteristics of the light source 48 (see FIG. 4 ).

[0082] The exposure amount is set, for example, as follows: That is, as will be described later, when the Nth compositing image 142 is obtained, the exposure amount is set so that the pixel value of an image region 142A corresponding to the first region 3A in the Nth compositing image 142 is equal to or greater than a first default pixel value. The first default pixel value is set, for example, to a pixel value at which the pixel values ​​of the image region 142A are saturated. Note that the first default pixel value may also be set to a pixel value at which the pixel values ​​of at least a portion of the image region 142A are saturated.

[0083] Furthermore, the exposure amount is set to an exposure amount such that the pixel value of an image region 142C corresponding to the third region 3C in the Nth compositing image 142 is equal to or greater than the second predetermined pixel value and less than the first predetermined pixel value. The second predetermined pixel value is set to a pixel value corresponding to brightness at which inspection and / or surveying can be performed (i.e., brightness suitable for inspection and / or surveying), for example, when inspection and / or surveying is performed based on the image region 142C corresponding to the third region 3C. By irradiating the imaging target region 3 with an amount of light sufficient to saturate the pixel values ​​of the image region 142A corresponding to the first region 3A, the amount of light irradiated onto the third region 3C adjacent to the first region 3A is ensured.

[0084] 8 shows an example of a state in which the flying object 20 is temporarily stationary at the Nth imaging position 5. The second imaging control unit 102 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to image the Nth imaging target area 3. Furthermore, when causing the image sensor 74 to image the Nth imaging target area 3, the second imaging control unit 102 adjusts the exposure amount of the light receiving surface 78A (see FIG. 3 ) of the image sensor 74 to the exposure amount derived by the first exposure amount derivation unit 100.

[0085] The exposure amount is adjusted, for example, by changing the exposure of the light receiving surface 78 A. Specifically, the second imaging control unit 102 controls the aperture actuator 66 to change the size of the aperture 64, and / or controls the shutter actuator 70 to change the shutter speed of the shutter 68, thereby changing the exposure of the light receiving surface 78 A.

[0086] Here, an example is given in which the second imaging control unit 102 changes the size of the aperture 64, and the second imaging control unit 102 changes the shutter speed of the shutter 68 (hereinafter referred to as "two types of control"), but either one of the two types of control may be omitted.

[0087] The Nth compositing image data is obtained by the image sensor 74 capturing an image of the Nth imaging target area 3 under the control of the second imaging control unit 102. The compositing image data is image data representing the compositing image 142. The second imaging control unit 102 acquires the Nth compositing image 142 corresponding to the Nth imaging target area 3 based on the compositing image data.

[0088] An image region 142A of the Nth compositing image 142, which corresponds to the first region 3A, includes the light source image 50. The Nth compositing image 142 is stored in the storage 36. When the Nth compositing image 142 is acquired, the flying object 20 resumes moving in the horizontal direction.

[0089] Note that, for example, the pixel value of image area 142A may fluctuate as indicated by arrow A due to variations in at least one of the distance between wall surface 2A and flying imaging device 10, the reflectance of light irradiated onto Nth imaging target area 3, and the exposure amount. When the pixel value of image area 142A fluctuates, the width Wa of image area 142A (in other words, the position of the boundary between image area 142A and image area 142C) fluctuates as indicated by arrow B.

[0090] The width acquisition unit 104 identifies the width Wa of the image region 142A (i.e., the width of the region whose pixel values ​​are equal to or greater than the first predetermined pixel value) based on the pixel values ​​of the image region 142A in the Nth synthesis image 142 acquired by the second imaging control unit 102. Then, the width acquisition unit 104 acquires the width W of the first region 3A by deriving the width W of the first region 3A corresponding to the width Wa of the image region 142A based on the width Wa of the image region 142A. The width W of the first region 3A corresponds to the length of the first region 3A along the flight direction of the aircraft 20. The flight direction is an example of the "direction of movement" according to the technology of the present disclosure, and the width W of the first region 3A is an example of the "length of the first region" according to the technology of the present disclosure.

[0091] 9 shows an example of the state before the flying object 20 reaches the N+1th imaging position 5. The N+1th imaging position 5 shown in FIG. 9 is, by way of example, the second imaging position 5. The imaging position correction unit 106 corrects the position of the N+1th imaging position 5 corresponding to the N+1th imaging target area 3, as indicated by arrow C, based on the width W of the first area 3A acquired by the width acquisition unit 104. When the position of the imaging position 5 is corrected, the position of the N+1th imaging target area 3 is corrected, as indicated by arrow D.

[0092] Specifically, the imaging position correction unit 106 corrects the position of the (N+1)th imaging position 5 to a position where the first region 3A (i.e., the first region 3A having the width acquired by the width acquisition unit 104) of the Nth imaging target region 3 overlaps with the second region 3B of the (N+1)th imaging target region 3. In a state where the first region 3A and the second region 3B overlap, the second region 3B has the same width as the first region 3A. By changing the widths of the first region 3A and the second region 3B in this way, the overlap amount OL of the first region 3A and the second region 3B is changed.

[0093] The second arrival determination unit 108 acquires the position of the flying object 20 based on the positioning data input from the positioning unit 28 and / or the acceleration data input from the acceleration sensor 30. Then, the second arrival determination unit 108 determines whether the flying object 20 has reached the (N+1)th image capturing position 5 based on the acquired position of the flying object 20 and the (N+1)th image capturing position 5 corrected by the image capturing position correction unit 106.

[0094] 10 shows an example of a state in which the flying object 20 has reached the N+1th imaging position 5. When the flying object 20 reaches the N+1th imaging position 5, it temporarily comes to a standstill. When the flying object 20 reaches the N+1th imaging position 5, light is irradiated from the light source 48 onto the N+1th imaging target area 3. In this case, the intensity of the light irradiated onto the first area 3A of the N+1th imaging target area 3 is higher than the intensity of light irradiated onto the periphery of the first area 3A of the N+1th imaging target area 3. The first area 3A of the N+1th imaging target area 3 is an example of a "fourth area" according to the technology of the present disclosure.

[0095] When the arrival determination unit determines that the flying object 20 has reached the (N+1)th imaging position 5, the third imaging control unit 110 outputs an imaging instruction signal to the image sensor 74, causing the image sensor 74 to image the (N+1)th imaging target area 3. The image sensor 74 images the (N+1)th imaging target area 3 under the control of the third imaging control unit 110, thereby obtaining captured image data. The third imaging control unit 110 acquires a captured image corresponding to the (N+1)th imaging target area 3 based on the captured image data.

[0096] The second brightness acquisition unit 112 acquires the brightness of a third region 3C other than the first region 3A in the N+1-th imaging target region 3 based on the captured image acquired by the third imaging control unit 110. As the brightness of the third region 3C, the second brightness acquisition unit 112 may acquire the brightness of a central region in the N+1-th imaging target region 3, may acquire the brightness of a central region in the third region 3C, or may acquire the brightness of the entire third region 3C. Furthermore, the brightness may be a representative value (e.g., a maximum value, a minimum value, or a median value) or an average value.

[0097] The second exposure amount derivation unit 114 derives the exposure amount based on the brightness acquired by the second brightness acquisition unit 112. The exposure amount may be derived based on a table stored in the storage 36, or may be derived based on a relational expression stored in the storage 36.

[0098] 10 shows, as an example, a graph showing pixel values ​​of an N+1th compositing image 142 (see also FIG. 11 ), which will be described later, and the correspondence between the compositing image 142. The graph showing pixel values ​​of the N+1th compositing image 142 has a shape corresponding to the graph showing the orientation characteristics of the light source 48 (see FIG. 4 ).

[0099] The exposure amount is set, for example, to the following exposure amount. That is, as will be described later, the exposure amount is set to such that when the N+1th compositing image 142 is obtained, the pixel value of an image region 142A in the N+1th compositing image 142 that corresponds to the first region 3A will be equal to or greater than a first default pixel value. The first default pixel value is set, for example, to a pixel value at which the pixel values ​​of the image region 142A are saturated. Note that the first default pixel value may also be set to a pixel value at which the pixel values ​​of at least a portion of the image region 142A are saturated.

[0100] Furthermore, the exposure amount is set to an exposure amount such that the pixel value of image region 142C corresponding to third region 3C in N+1-th compositing image 142 is equal to or greater than the second predetermined pixel value and less than the first predetermined pixel value. For example, when inspection and / or surveying is performed based on image region 142C corresponding to third region 3C, the second predetermined pixel value is set to a pixel value corresponding to brightness at which inspection and / or surveying can be performed (i.e., brightness suitable for inspection and / or surveying). By irradiating imaging target region 3 with an amount of light sufficient to saturate the pixel values ​​of image region 142A corresponding to first region 3A, the amount of light irradiated onto third region 3C adjacent to first region 3A is ensured.

[0101] 11 shows an example of an aspect in which the flying object 20 is temporarily stationary at the (N+1)th imaging position 5. The fourth imaging control unit 116 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to image the (N+1)th imaging target area 3. Furthermore, when causing the image sensor 74 to image the (N+1)th imaging target area 3, the fourth imaging control unit 116 adjusts the exposure amount of the light receiving surface 78A (see FIG. 3 ) to the exposure amount derived by the second exposure amount derivation unit 114.

[0102] The exposure amount is adjusted, for example, by changing the exposure of the light receiving surface 78 A. Specifically, the fourth imaging control unit 116 controls the aperture actuator 66 to change the size of the aperture 64, and / or controls the shutter actuator 70 to change the shutter speed of the shutter 68, thereby changing the exposure of the light receiving surface 78 A.

[0103] Here, an example is given in which the fourth imaging control unit 116 changes the size of the aperture 64, and the fourth imaging control unit 116 changes the shutter speed of the shutter 68 (hereinafter referred to as "two types of control"), but either one of the two types of control may be omitted.

[0104] The N+1th image data to be combined is obtained by the image sensor 74 capturing an image of the N+1th imaging target area 3 under the control of the fourth imaging control unit 116. The fourth imaging control unit 116 acquires the N+1th image to be combined 142 corresponding to the N+1th imaging target area 3 based on the image data to be combined.

[0105] An image area 142A corresponding to the first area 3A in the N+1th compositing image 142 includes the light source image 50, while an image area 142C corresponding to a third area 3C other than the first area 3A in the N+1th compositing image 142 does not include the light source image 50. The third area 3C other than the first area 3A includes the second area 3B, and the image area 142C other than the image area 142A includes the image area 142B corresponding to the second area 3B. The N+1th compositing image 142 is stored in the storage 36. When the N+1th compositing image 142 is acquired, the flying object 20 resumes moving horizontally.

[0106] 12, the termination determination unit 118 determines whether a condition for terminating the flight imaging process (hereinafter referred to as the "termination condition") is met. Examples of the termination condition include a condition that the number of multiple compositing images 142 stored in the storage 36 reaches the number of multiple imaging positions 5 set on the flight route 4, or a condition that a user or the like issues an instruction to terminate the flight imaging process to the flight imaging device 10.

[0107] If the termination determination unit 118 determines that the termination condition is not met, the (N+1)th compositing image 142 is treated as the Nth compositing image 142. Then, in order to obtain a new (N+1)th compositing image 142, the processes from the width acquisition unit 104 to the fourth imaging control unit 116 are executed again.

[0108] When the termination determination unit 118 determines that the termination condition is met, the light-off control unit 120 outputs a light-off instruction signal to the light source 48 to turn it off.

[0109] Note that although an example is given here in which the flight imaging process is performed by the flight imaging device 10, the flight imaging process may also be performed by a transmitter (not shown) or a base station (not shown) communicatively connected to the flight imaging device 10.

[0110] 13 , a composite image generation program 130 is stored in the storage 36. The composite image generation program 130 is an example of a "program" according to the technology of the present disclosure. The processor 34 reads the composite image generation program 130 from the storage 36 and executes the read composite image generation program 130 on the RAM 38. The processor 34 performs a composite image generation process for capturing an image of each imaging target area 3 while flying along each flight route 4 in accordance with the composite image generation program 130 executed on the RAM 38.

[0111] The composite image generation process is realized by the processor 34 operating as the image acquisition unit 132 and the image synthesis unit 134 in accordance with the flight imaging program 90. The composite image generation process may be executed each time each synthesis image 142 from the second frame onwards is obtained, or may be executed after a plurality of synthesis images 142 are obtained for the wall surface 2A. Below, the composite image generation process will be described using as an example a case where the Nth synthesis image 142 and the N+1th synthesis image 142 are synthesized to generate the composite image 140.

[0112] 14 , the image acquisition unit 132 acquires the Nth compositing image 142 and the N+1th compositing image 142 stored in the storage 36. An image region 142A of the Nth compositing image 142 includes a light source image 50, while an image region 142B of the N+1th compositing image 142 does not include the light source image 50.

[0113] The image synthesis unit 134 generates a synthesized image 140 by synthesizing the Nth synthesis image 142 and the N+1th synthesis image 142. When generating the synthesized image 140, the image synthesis unit 134 uses pixel values ​​of the image region 142B of the N+1th synthesis image 142 as pixel values ​​of the overlapping image region 144 of the synthesized image 140. This results in a synthesized image 140 in which the light source image 50 is not included in the overlapping image region 144. The overlapping image region 144 is a region corresponding to the first region 3A and the second region 3B (see FIG. 12 ) that overlap each other.

[0114] Note that although an example is given here in which the composite image generation process is performed by the flight imaging device 10, the composite image generation process may also be performed by an external device (not shown) communicatively connected to the flight imaging device 10.

[0115] Furthermore, when the composite image 140 is generated based on a plurality of compositing images 142, the last compositing image 142 included in the composite image 140 includes the light source image 50. Therefore, for example, when the composite image 140 is generated based on three or more frames of compositing images 142, the image region 142A of the last compositing image 142 included in the composite image 140 may be deleted from the composite image 140. Furthermore, the first region 3A of the last imaging target region 3 may be a region other than the inspection target region.

[0116] The Nth compositing image 142 is an example of a "first captured image" according to the technology of the present disclosure. The N+1th compositing image 142 is an example of a "second captured image" according to the technology of the present disclosure. The composite image 140 is an example of a "composited image" according to the technology of the present disclosure.

[0117] Next, the operation of the flight imaging device 10 according to the first embodiment will be described. First, the flight imaging process will be described. Figure 15 shows an example of the flow of the flight imaging process according to the first embodiment.

[0118] In the flight imaging process shown in Figure 15, first, in step ST10, the lighting control unit 92 turns on the light source 48 (see Figure 6). This causes light to be irradiated onto the wall surface 2A. After the process of step ST10 is executed, the flight imaging process proceeds to step ST12.

[0119] In step ST12, the first arrival determination unit 94 acquires the position of the flying object 20 based on the positioning data input from the positioning unit 28 and / or the acceleration data input from the acceleration sensor 30. Then, the first arrival determination unit 94 determines whether the flying object 20 has reached the Nth image capture position 5 based on the acquired position of the flying object 20 and the Nth image capture position 5 indicated by the flight route information 122 (see FIG. 6 ). In step ST12, if the flying object 20 has reached the Nth image capture position 5, the determination is affirmative, and the flight image capture processing proceeds to step ST14. In step ST12, if the flying object 20 has not reached the Nth image capture position 5, the determination is negative, and the flight image capture processing executes the processing of step ST12 again.

[0120] In step ST14, the first imaging control unit 96 causes the image sensor 74 to capture an image of the Nth imaging target area 3 (see FIG. 7 ). This results in a captured image corresponding to the Nth imaging target area 3. After the processing of step ST14 is executed, the flight imaging processing proceeds to step ST16.

[0121] In step ST16, the first brightness acquisition unit 98 acquires the brightness of a third region 3C other than the first region 3A in the N-th imaging target region 3 based on the captured image acquired in step ST14 (see FIG. 7 ). After the processing of step ST16 is executed, the flight imaging processing proceeds to step ST18.

[0122] In step ST18, the first exposure amount derivation unit 100 derives the exposure amount based on the brightness acquired in step ST16 (see FIG. 7). After the processing of step ST18 is executed, the flight imaging processing proceeds to step ST20.

[0123] In step ST20, the second imaging control unit 102 causes the image sensor 74 to capture an image of the Nth imaging target area 3 (see FIG. 8 ). When the second imaging control unit 102 causes the image sensor 74 to capture an image of the Nth imaging target area 3, the second imaging control unit 102 controls the aperture actuator 66 and / or the shutter actuator 70 to adjust the exposure amount of the light receiving surface 78A to the exposure amount derived in step ST18. The Nth imaging target area 3 is captured with the exposure amount adjusted, thereby obtaining a synthesis image 142 corresponding to the Nth imaging target area 3. After the processing of step ST20 is executed, the flight imaging processing proceeds to step ST22.

[0124] In step ST22, the width acquisition unit 104 identifies the width Wa of the image region 142A based on the pixel values ​​of the image region 142A corresponding to the first region 3A in the Nth synthesis image 142 acquired in step ST20. Then, the width acquisition unit 104 acquires the width W of the first region 3A by deriving the width W of the first region 3A corresponding to the width Wa of the image region 142A based on the width Wa of the image region 142A (see FIG. 8 ). After the processing of step ST22 is executed, the flight imaging processing proceeds to step ST24.

[0125] In step ST24, the imaging position correction unit 106 corrects the position of the imaging position 5 corresponding to the (N+1)th imaging target area 3 based on the width W of the first area 3A acquired by the width acquisition unit 104 (see FIG. 9 ). After the processing of step ST24 is executed, the flight imaging processing proceeds to step ST26.

[0126] In step ST26, the second arrival determination unit 108 acquires the position of the aircraft 20 based on the positioning data input from the positioning unit 28 and / or the acceleration data input from the acceleration sensor 30. Then, the second arrival determination unit 108 determines whether the aircraft 20 has reached the (N+1)th image capture position 5 based on the acquired position of the aircraft 20 and the (N+1)th image capture position 5 corrected in step ST24 (see FIG. 9 ). In step ST26, if the aircraft 20 has reached the (N+1)th image capture position 5, the determination is affirmative, and the flight imaging process proceeds to step ST28. In step ST26, if the aircraft 20 has not reached the (N+1)th image capture position 5, the determination is negative, and the flight imaging process executes the process of step ST26 again.

[0127] In step ST28, the third imaging control unit 110 causes the image sensor 74 to capture an image of the (N+1)th imaging target area 3 (see FIG. 10 ). This results in the acquisition of a captured image corresponding to the (N+1)th imaging target area 3. After the processing of step ST28 is executed, the flight imaging processing proceeds to step ST30.

[0128] In step ST30, the second brightness acquisition unit 112 acquires the brightness of a third region 3C other than the first region 3A in the (N+1)th imaging target region 3 based on the captured image acquired in step ST28 (see FIG. 10 ). After the processing of step ST30 is executed, the flight imaging processing proceeds to step ST32.

[0129] In step ST32, the second exposure amount derivation unit 114 derives the exposure amount based on the brightness acquired in step ST30 (see FIG. 10 ). After the processing of step ST32 is executed, the flight imaging processing proceeds to step ST34.

[0130] In step ST34, the fourth imaging control unit 116 causes the image sensor 74 to capture an image of the (N+1)th imaging target area 3 (see FIG. 11 ). Furthermore, when the fourth imaging control unit 116 causes the image sensor 74 to capture an image of the (N+1)th imaging target area 3, it controls the aperture actuator 66 and / or the shutter actuator 70 to adjust the exposure amount of the light receiving surface 78A to the exposure amount derived in step ST32. The (N+1)th imaging target area 3 is captured with the exposure amount adjusted, thereby obtaining a compositing image 142 corresponding to the (N+1)th imaging target area 3. After the processing of step ST34 is executed, the flight imaging processing proceeds to step ST36.

[0131] In step ST36, the termination determination unit 118 determines whether or not a termination condition for terminating the flight imaging process is met (see FIG. 12 ). If the termination condition is not met in step ST36, the determination is negative, and the flight imaging process proceeds to step ST38. If the termination condition is met in step ST36, the determination is positive, and the flight imaging process proceeds to step ST40.

[0132] In step ST38, the processor 34 treats the (N+1)th compositing image 142 as the Nth compositing image 142. After the processing of step ST38 is executed, the flight imaging processing proceeds to step ST22.

[0133] In step ST40, the lights-off control unit 120 turns off the light source 48 (see FIG. 12). After the processing of step ST40 is executed, the flight imaging processing ends.

[0134] Next, the composite image generation process will be described. An example of the flow of the composite image generation process according to the first embodiment is shown in FIG.

[0135] 16 , first, in step ST50, the image acquisition unit 132 acquires the Nth and N+1th compositing images 142 stored in the storage 36 (see FIG. 14 ). After the processing of step ST50 is executed, the composite image generation processing proceeds to step ST52.

[0136] In step ST52, the image synthesis unit 134 generates a synthesized image 140 by synthesizing the Nth synthesis image 142 and the N+1th synthesis image 142 (see FIG. 14 ). When generating the synthesized image 140, the image synthesis unit 134 uses the pixel values ​​of the image region 142B of the N+1th synthesis image 142 as the pixel values ​​of the overlap image region 144 of the synthesized image 140. This results in a synthesized image 140 in which the light source image 50 is not included in the overlap image region 144. After the processing of step ST52 is executed, the synthesized image generation processing proceeds to step ST54.

[0137] In step ST54, the processor 34 determines whether a condition for terminating the composite image generation process (hereinafter referred to as the "termination condition") is met. Examples of the termination condition include a condition in which all of the multiple compositing images 142 stored in the storage 36 have been composited, or a condition in which a user or the like has issued an instruction to the flight imaging device 10 to terminate the composite image generation process. If the termination condition is not met in step ST54, the determination is negative, and the composite image generation process proceeds to step ST50. If the termination condition is met in step ST54, the determination is positive, and the composite image generation process terminates. The control method described above as the operation of the flight imaging device 10 is an example of a "control method" according to the technology of the present disclosure.

[0138] Next, the effects of the flying imaging device 10 according to the first embodiment will be described.

[0139] In the first embodiment, the Nth compositing image 142 and the N+1th compositing image 142 are combined to generate the composite image 140 (see FIG. 14 ). Here, when the composite image 140 is generated, the pixel values ​​of the image region 142B of the N+1th compositing image 142 are used as the pixel values ​​of the overlapping image region 144 of the composite image 140. This makes it possible to avoid using the pixel values ​​of the image region 142A of the Nth compositing image 142 (i.e., the image region 142A including the light source image 50) as the pixel values ​​of the overlapping image region 144.

[0140] Furthermore, the light source image 50 fits within the image region 142A of the Nth compositing image 142 (see FIG. 4 ). Therefore, by avoiding the use of pixel values ​​of the image region 142A of the Nth compositing image 142 as pixel values ​​of the overlapping image region 144, a composite image 140 that does not include the light source image 50 can be obtained.

[0141] Furthermore, the intensity of light irradiated onto the first region 3A of the Nth imaging target region 3 is higher than the intensity of light irradiated onto the periphery of the first region 3A (see FIG. 4 ). Therefore, it is possible to increase the light intensity of the first region 3A corresponding to the image region 142A of the Nth compositing image 142 while ensuring the intensity of light around the first region 3A (i.e., the light intensity of the third region 3C). This ensures the brightness of the third region 3C of the Nth imaging target region 3, and therefore it is possible to obtain, as the Nth compositing image 142, an image that contributes to ensuring appropriate brightness of the entire composite image 140.

[0142] Furthermore, when the Nth imaging target area 3 is imaged by the imaging device 60, the exposure amount of the light receiving surface 78A is adjusted based on the brightness of the third area 3C of the Nth imaging target area 3 (see FIGS. 7 and 8 ). Therefore, by adjusting the exposure amount to correspond to the brightness of the third area 3C, it is possible to ensure the brightness of the image area 142C of the Nth compositing image 142 that corresponds to the third area 3C.

[0143] Furthermore, for example, by performing control to change the size of the aperture 64 and / or control to change the shutter speed, the exposure to the light receiving surface 78A is changed, and the amount of exposure is thereby adjusted (see FIG. 8). Therefore, compared to the case where special control other than control to change the size of the aperture 64 and / or control to change the shutter speed is performed, the amount of exposure can be adjusted by simple control.

[0144] Furthermore, the exposure amount is set to an exposure amount that makes the pixel values ​​of image area 142A in N-th compositing image 142 equal to or greater than the first predetermined pixel value (see FIG. 7 ). Therefore, by avoiding the use of pixel values ​​of image area 142A in N-th compositing image 142 as pixel values ​​of overlapping image area 144, it is possible to prevent the pixel values ​​of overlapping image area 144 from becoming equal to or greater than the first predetermined pixel value.

[0145] The first default pixel value is set to a pixel value at which the pixel values ​​of at least a portion of image region 142A are saturated (see FIG. 7 ). Therefore, by avoiding the use of pixel values ​​of image region 142A of Nth compositing image 142 as pixel values ​​of overlapping image region 144, it is possible to avoid generating composite image 140 in which pixel values ​​of at least a portion of the region are saturated.

[0146] Furthermore, light is used that has an intensity sufficient to saturate pixel values ​​in at least a portion of image region 142A, which allows the exposure time to be shorter than when light with an intensity sufficient to prevent pixel values ​​from saturating is used, thereby suppressing image blur.

[0147] The exposure amount is set to an amount that makes the pixel value of image region 142C in N-th compositing image 142 equal to or greater than the second predetermined pixel value and less than the first predetermined pixel value (see FIG. 7 ). Therefore, the brightness of image region 142C in N-th compositing image 142 can be ensured.

[0148] Furthermore, the width Wa of the image area 142A is determined based on the pixel values ​​of the image area 142A in the Nth compositing image 142, and the width W of the first area 3A corresponding to the width Wa of the image area 142A is derived based on the width Wa of the image area 142A (see FIG. 8 ). Then, the position of the imaging position 5 corresponding to the (N+1)th imaging target area 3 is corrected based on the width W of the first area 3A (see FIG. 9 ). Therefore, even if the width W of the first area 3A corresponding to the width Wa of the image area 142A changes as the pixel values ​​of the image area 142A change, the amount of overlap between the first area 3A and the second area 3B can be changed.

[0149] Furthermore, the light source 48 is disposed ahead of the imaging device 60 in the flight direction of the flight imaging device 10 (see FIG. 4 ). Therefore, when the Nth compositing image 142 and the N+1th compositing image 142 are combined to generate the composite image 140, it is possible to prevent the light source image 50 from being included in the Nth compositing image 142.

[0150] Furthermore, the light is irradiated along the normal direction of the first region 3A (see FIG. 4 ). This allows the intensity of the light irradiated onto the first region 3A of the Nth imaging target region 3 to be higher than the intensity of the light irradiated onto the periphery of the first region 3A.

[0151] Furthermore, the intensity of light irradiated onto the first region 3A of the N+1-th imaging target region 3 is higher than the intensity of light irradiated onto the periphery of the first region 3A (see FIG. 10 ). This makes it possible to increase the light intensity of the first region 3A corresponding to the image region 142A of the N+1-th compositing image 142 while ensuring the intensity of light around the first region 3A (i.e., the light intensity of the third region 3C). This ensures the brightness of the third region 3C of the N+1-th imaging target region 3, and therefore makes it possible to obtain, as the N+1-th compositing image 142, an image that contributes to ensuring appropriate brightness of the entire composite image 140.

[0152] Furthermore, the flying imaging device 10 moves to each imaging position 5 with the light source 48 irradiating light. Therefore, it is not necessary to control the light source 48 to turn on and off each time the flying imaging device 10 reaches an imaging position 5.

[0153] Second Embodiment Next, a second embodiment will be described.

[0154] 17 shows an example of an aspect in which the flying object 20 has reached the Nth imaging position 5. In the second embodiment, the flight imaging process is modified as follows compared to the first embodiment. That is, in the second embodiment, the process by the width acquisition unit 104 (see FIG. 8) and the process by the imaging position correction unit 106 (see FIG. 9) are omitted. Also, in the second embodiment, the overlap amount OL between the first region 3A and the second region 3B is fixed to a predetermined overlap amount.

[0155] The exposure amount derived by the first exposure amount derivation unit 100 is set to an exposure amount in which the width W of the first region 3A corresponding to the width Wa of the image region 142A of the Nth synthesis image 142 described later (i.e., the width of the region whose pixel value is equal to or greater than the first predetermined pixel value) matches the overlap amount OL.

[0156] In the second embodiment, the processor 34 (see FIG. 5 ) operates as a light source control unit 124. The light source control unit 124 controls the light source 48 to change the intensity of light emitted from the light source 48. Specifically, the light source control unit 124 changes the intensity of light based on the exposure amount derived by the first exposure amount derivation unit 100.

[0157] The second imaging control unit 102 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to image the Nth imaging target area 3. Furthermore, when causing the image sensor 74 to image the Nth imaging target area 3, the second imaging control unit 102 adjusts the exposure amount of the light receiving surface 78A (see FIG. 3 ) based on the exposure amount derived by the first exposure amount derivation unit 100.

[0158] The exposure amount is adjusted, for example, by changing the exposure of the light receiving surface 78 A. Specifically, the second imaging control unit 102 controls the aperture actuator 66 to change the size of the aperture 64, and / or controls the shutter actuator 70 to change the shutter speed of the shutter 68, thereby changing the exposure of the light receiving surface 78 A.

[0159] Then, by performing control to change the light intensity by the light source control unit 124, control to change the size of the aperture 64 by the second imaging control unit 102, and control to change the shutter speed of the shutter 68 by the second imaging control unit 102 (hereinafter referred to as "three types of control"), the exposure amount of the light receiving surface 78A is set to the exposure amount derived by the first exposure amount derivation unit 100.

[0160] According to the second embodiment, three types of control are performed, and the exposure amount of the light-receiving surface 78A is set to the exposure amount derived by the first exposure amount derivation unit 100. As a result, the exposure amount of the light-receiving surface 78A is set to the exposure amount that matches the overlap amount OL of the width W of the first region 3A corresponding to the width Wa of the image region 142A of the Nth compositing image 142 (i.e., the width of the region whose pixel value is equal to or greater than the first predetermined pixel value), and therefore the overlap amount OL can be fixed to the predetermined overlap amount.

[0161] Furthermore, since the overlap amount OL is fixed to a default overlap amount, the efficiency of imaging multiple imaging target areas 3 can be improved compared to when the overlap amount OL is changed to an overlap amount greater than the default overlap amount.

[0162] Note that the exposure amount of the light receiving surface 78A may be set to the exposure amount derived by the first exposure amount derivation unit 100 by performing any one or two of the three types of control.

[0163] Alternatively, the light source 48 may be configured to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position. Alternatively, the light source 48 may be configured to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position and the light intensity. In this manner, the exposure amount of the light receiving surface 78A can be set to the exposure amount derived by the first exposure amount derivation unit 100.

[0164] Third Embodiment Next, a third embodiment will be described.

[0165] As shown in FIG. 18 as an example, in the third embodiment, the flying imaging device 10 is modified as follows compared to the second embodiment. That is, the light source 48 is disposed behind the imaging device 60 in the flight direction of the flying object 20. The light source 48 irradiates each imaging target area 3 with light. The light source 48 is disposed at a position where the optical axis passes through each second area 3B (for example, the center of each second area 3B) when each imaging target area 3 is imaged by the imaging device 60. In other words, the light source 48 is disposed at a position where it irradiates light along the normal direction of each second area 3B when each imaging target area 3 is imaged by the imaging device 60. The normal direction may be the average normal direction of the second area 3B.

[0166] Furthermore, the size of the light source 48 is set to a size such that, for example, when the Nth synthesis image 142 is obtained by imaging the Nth imaging target area 3, the light source image 50 (i.e., the image of reflected light) fits into the image area 142B corresponding to the second area 3B of the Nth synthesis image 142.

[0167] In the third embodiment, the N+1th imaging target area 3 is an example of a "first imaging target area" according to the technology of the present disclosure. The Nth imaging target area 3 is an example of a "second imaging target area" according to the technology of the present disclosure. The second area 3B of the N+1th imaging target area 3 is an example of a "first area" according to the technology of the present disclosure. The first area 3A of the Nth imaging target area 3 is an example of a "second area" according to the technology of the present disclosure.

[0168] FIG. 19 shows an example of an aspect in which the flying object 20 has reached the (N+1)th imaging position 5. In the third embodiment, the flight imaging process is modified as follows compared to the first embodiment. That is, in the third embodiment, the processing by the width acquisition unit 104 (see FIG. 8) and the processing by the imaging position correction unit 106 (see FIG. 9) are omitted. Also, in the third embodiment, the overlap amount OL between the first region 3A and the second region 3B is fixed to a predetermined overlap amount. The exposure amount is derived based on the brightness of a fourth region 3D other than the second region 3B in the (N+1)th imaging target region 3. The fourth region 3D is an example of a "third region" according to the technology of the present disclosure.

[0169] The exposure amount derived by the second exposure amount derivation unit 114 is set to an exposure amount in which the width of the second region 3B corresponding to the width Wb of the image region 142B of the N+1th synthesis image 142 (i.e., the width of the region whose pixel value is equal to or greater than the first predetermined pixel value) matches the overlap amount OL.

[0170] In the third embodiment, the processor 34 (see FIG. 5 ) operates as a light source control unit 124. The light source control unit 124 controls the light source 48 to change the intensity of light emitted from the light source 48. Specifically, the light source control unit 124 changes the intensity of light based on the exposure amount derived by the second exposure amount derivation unit 114.

[0171] The fourth imaging control unit 116 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to capture an image of the (N+1)th imaging target area 3. Furthermore, when causing the image sensor 74 to capture an image of the (N+1)th imaging target area 3, the fourth imaging control unit 116 adjusts the exposure amount of the light receiving surface 78A (see FIG. 3 ) based on the exposure amount derived by the second exposure amount derivation unit 114.

[0172] The exposure amount is adjusted, for example, by changing the exposure of the light receiving surface 78 A. Specifically, the fourth imaging control unit 116 controls the aperture actuator 66 to change the size of the aperture 64, and / or controls the shutter actuator 70 to change the shutter speed of the shutter 68, thereby changing the exposure of the light receiving surface 78 A.

[0173] Then, by performing control to change the light intensity by the light source control unit 124, control to change the size of the aperture 64 by the second imaging control unit 102, and control to change the shutter speed of the shutter 68 by the second imaging control unit 102 (hereinafter referred to as "three types of control"), the exposure amount of the light receiving surface 78A is set to the exposure amount derived by the second exposure amount derivation unit 114.

[0174] 20 as an example, in the third embodiment, the composite image generation process is modified as follows from that in the first embodiment. That is, in the third embodiment, the image acquisition unit 132 acquires the Nth composite image 142 and the N+1th composite image 142 stored in the storage 36. An image region 142B of the N+1th composite image 142 includes a light source image 50, and an image region 142A of the Nth composite image 142 does not include the light source image 50.

[0175] The image synthesis unit 134 generates a synthesized image 140 by synthesizing the Nth synthesis image 142 and the N+1th synthesis image 142. When generating the synthesized image 140, the image synthesis unit 134 uses the pixel values ​​of the image region 142A of the Nth synthesis image 142 as the pixel values ​​of the overlap image region 144 of the synthesized image 140. This results in a synthesized image 140 in which the light source image 50 is not included in the overlap image region 144.

[0176] According to the third embodiment, three types of control are performed, and the exposure amount of the light-receiving surface 78A is set to the exposure amount derived by the second exposure amount derivation unit 114. As a result, the exposure amount of the light-receiving surface 78A is set to the exposure amount that matches the overlap amount OL of the width W of the second region 3B corresponding to the width Wb of the image region 142B in the N+1th compositing image 142 (i.e., the width of the region whose pixel value is equal to or greater than the first predetermined pixel value), and therefore the overlap amount OL can be fixed to the predetermined overlap amount.

[0177] Furthermore, since the overlap amount OL is fixed to a default overlap amount, the efficiency of imaging multiple imaging target areas 3 can be improved compared to when the overlap amount OL is changed to an overlap amount greater than the default overlap amount.

[0178] Furthermore, the light source 48 is disposed behind the imaging device 60 in the flight direction of the flight imaging device 10. Therefore, when the Nth compositing image 142 and the N+1th compositing image 142 are combined to generate the composite image 140, it is possible to prevent the light source image 50 from being included in the N+1th compositing image 142.

[0179] Note that the exposure amount of the light receiving surface 78A may be set to the exposure amount derived by the second exposure amount derivation unit 114 by performing any one or two of the three types of control.

[0180] Alternatively, the light source 48 may be configured to be able to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position. Alternatively, the light source 48 may be configured to be able to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position and the light intensity. In this case, too, the exposure amount of the light receiving surface 78A can be set to the exposure amount derived by the second exposure amount derivation unit 114.

[0181] Furthermore, when composite image 140 is generated based on a plurality of compositing images 142, the first compositing image 142 included in composite image 140 includes light source image 50. Therefore, for example, when composite image 140 is generated based on three or more frames of compositing images 142, image region 142B of the first compositing image 142 included in composite image 140 may be deleted from composite image 140. Furthermore, second region 3B of first imaging target region 3 may be a region other than the inspection target region.

[0182] The (N+1)th compositing image 142 is an example of a "first captured image" according to the technology of the present disclosure. The Nth compositing image 142 is an example of a "second captured image" according to the technology of the present disclosure.

[0183] Fourth Embodiment Next, a fourth embodiment will be described.

[0184] 21 , in the fourth embodiment, the multiple flight routes 4 are modified as follows compared to the first embodiment: That is, the multiple flight routes 4 include two types of flight routes 4 with different directions.

[0185] 22 and 23 , in the fourth embodiment, the flight imaging device 10 is modified as follows compared to the first embodiment. That is, the flight imaging device 10 includes a first light source 48A and a second light source 48B. The first light source 48A and the second light source 48B are examples of the "light source" according to the technology of the present disclosure. The first light source 48A is disposed on a first side (for example, the right side) of the imaging device 60 in the flight direction of the flying object 20, and the second light source 48B is disposed on a second side (for example, the left side) of the imaging device 60 in the flight direction of the flying object 20.

[0186] The first light source 48A and the second light source 48B have the same configuration as the light source 48 according to the first embodiment. The second light source 48B is disposed symmetrically to the first light source 48A in the flight direction of the flying object 20.

[0187] The processor 34 (see FIG. 5 ) turns on the first light source 48A disposed on the first side relative to the imaging device 60 when the flying object 20 flies to the first side, and turns on the second light source 48B disposed on the second side relative to the imaging device 60 when the flying object 20 flies to the second side. The flight imaging process is executed when the flight imaging device 10 starts flying from the start of each flight route 4.

[0188] According to the fourth embodiment, even when the multiple flight routes 4 include two types of flight routes 4 with different directions, multiple images to be combined 142 can be obtained by sequentially capturing images of multiple imaging target areas 3 using the imaging device 60. Therefore, the multiple images to be combined 142 can be combined to generate the composite image 140.

[0189] The fourth embodiment may be combined with the third embodiment. The processor 34 may turn on the second light source 48B disposed on the second side relative to the imaging device 60 when the flying object 20 flies toward the first side, and may turn on the second light source 48B disposed on the first side relative to the imaging device 60 when the flying object 20 flies toward the second side. Even in this case, multiple images to be combined 142 are obtained by sequentially capturing images of multiple imaging target areas 3 using the imaging device 60. Therefore, the multiple images to be combined 142 can be combined to generate the composite image 140.

[0190] Fifth Embodiment Next, a fifth embodiment will be described.

[0191] As an example, as shown in Figures 24 and 25 , in the fifth embodiment, the position of the light source 48 is changed as follows compared to the first embodiment. That is, the light source 48 is disposed above the imaging device 60. The light source 48 irradiates light onto each imaging target area 3. Each imaging target area 3 has a first area 3A and a second area 3B. The first area 3A and the second area 3B are parts of the imaging target area 3. The first area 3A is an upper area of ​​the imaging target area 3, and the second area 3B is a lower area of ​​the imaging target area 3.

[0192] The light source 48 is disposed at a position where the optical axis passes through each first region 3A (for example, the center of the first region 3A) when each imaging target region 3 is imaged by the imaging device 60. In other words, the light source 48 is disposed at a position where it irradiates light along the normal direction of each first region 3A when each imaging target region 3 is imaged by the imaging device 60. The normal direction may be the average normal direction of the first region 3A. The intensity of the light irradiated onto the first region 3A is higher than the intensity of the light irradiated onto the periphery of the first region 3A.

[0193] In addition, the size of the light source 48 is set to a size such that when the composite image 142 is obtained by capturing an image of the imaging target area 3, the light source image 50 (i.e., the image of reflected light) will fit within the image area 142A corresponding to the first area 3A of the composite image 142.

[0194] 24 as an example, when the flight imaging device 10 captures images while flying along the Nth flight route 4, multiple images to be synthesized 142 are obtained for the Nth flight route 4. Similarly, when the flight imaging device 10 captures images while flying along the N+1th flight route 4, multiple images to be synthesized 142 are obtained for the N+1th flight route 4 as shown in FIG.

[0195] 26 as an example, in the fifth embodiment, the composite image generation process is modified from that in the first embodiment as follows: That is, when the flight imaging device 10 captures images while flying along the Nth flight route 4, the image synthesis unit 134 synthesizes a plurality of synthesis images 142 obtained along the Nth flight route 4 to generate the Nth composite image 150.

[0196] When the flight imaging device 10 captures images while flying the Nth flight route 4, it moves to the N+1th flight route 4, which is one above the Nth flight route 4, and captures images while flying the N+1th flight route 4. When the flight imaging device 10 captures images while flying the N+1th flight route 4, the image synthesis unit 134 synthesizes the multiple synthesis images 142 obtained for the N+1th flight route 4 to generate the N+1th synthesized image 150.

[0197] Then, the image synthesis unit 134 generates the synthetic image 140 by synthesizing the Nth synthetic image 150 and the N+1th synthetic image 150. When generating the synthetic image 140, the image synthesis unit 134 uses the pixel values ​​of the image region 142B of the N+1th synthetic image 150 as the pixel values ​​of the overlapping image region 154 of the synthetic image 140. In this way, it is possible to obtain the synthetic image 140 in which the light source image 50 is not included in the overlapping image region 154.

[0198] In the fifth embodiment, the Nth imaging target area 3 is an example of a "first imaging target area" according to the technology of the present disclosure. The N+1th imaging target area 3 is an example of a "second imaging target area" according to the technology of the present disclosure. The first area 3A is an example of a "first area" according to the technology of the present disclosure. The second area 3B is an example of a "second area" according to the technology of the present disclosure. The Nth composite image 150 is an example of a "first captured image" according to the technology of the present disclosure. The N+1th composite image 150 is an example of a "second captured image" according to the technology of the present disclosure.

[0199] Sixth Embodiment Next, a sixth embodiment will be described.

[0200] As an example, as shown in Figures 27 and 28, in the sixth embodiment, the position of the light source 48 is changed as follows compared to the third embodiment. That is, the light source 48 is disposed below the imaging device 60. The light source 48 irradiates light onto each imaging target area 3. Each imaging target area 3 has a first area 3A and a second area 3B. The first area 3A and the second area 3B are parts of the imaging target area 3. The first area 3A is an upper area of ​​the imaging target area 3, and the second area 3B is a lower area of ​​the imaging target area 3.

[0201] The light source 48 is disposed at a position where the optical axis passes through each second region 3B (for example, the center of the second region 3B) when each imaging target region 3 is imaged by the imaging device 60. In other words, the light source 48 is disposed at a position where it irradiates light along the normal direction of each second region 3B when each imaging target region 3 is imaged by the imaging device 60. The normal direction may be the average normal direction of the second regions 3B. The intensity of light irradiated onto the second region 3B is higher than the intensity of light irradiated onto the periphery of the second region 3B.

[0202] In addition, the size of the light source 48 is set to a size such that when the composite image 142 is obtained by capturing an image of the imaging target area 3, the light source image 50 (i.e., the image of reflected light) will fit in the image area 142B corresponding to the second area 3B of the composite image 142.

[0203] 27 as an example, when the flight imaging device 10 captures images while flying along the Nth flight route 4, multiple images to be synthesized 142 are obtained for the Nth flight route 4. Similarly, when the flight imaging device 10 captures images while flying along the N+1th flight route 4, multiple images to be synthesized 142 are obtained for the N+1th flight route 4 as shown in FIG.

[0204] 29 as an example, in the sixth embodiment, the composite image generation process is modified from that in the third embodiment as follows: That is, when the flight imaging device 10 captures images while flying along the Nth flight route 4, the image synthesis unit 134 synthesizes a plurality of synthesis images 142 obtained along the Nth flight route 4 to generate the Nth composite image 150.

[0205] When the flight imaging device 10 captures images while flying the Nth flight route 4, it moves to the N+1th flight route 4, which is one above the Nth flight route 4, and captures images while flying the N+1th flight route 4. When the flight imaging device 10 captures images while flying the N+1th flight route 4, the image synthesis unit 134 synthesizes the multiple synthesis images 142 obtained for the N+1th flight route 4 to generate the N+1th synthesized image 150.

[0206] Then, the image synthesis unit 134 generates the synthetic image 140 by synthesizing the Nth synthetic image 150 and the (N+1)th synthetic image 150. When generating the synthetic image 140, the image synthesis unit 134 uses the pixel values ​​of the image region 142A of the Nth synthetic image 150 as the pixel values ​​of the overlapping image region 154 of the synthetic image 140. In this way, it is possible to obtain the synthetic image 140 in which the light source image 50 is not included in the overlapping image region 154.

[0207] In the sixth embodiment, the N+1th imaging target area 3 is an example of a "first imaging target area" according to the technology of the present disclosure. The Nth imaging target area 3 is an example of a "second imaging target area" according to the technology of the present disclosure. The second area 3B of the N+1th imaging target area 3 is an example of a "first area" according to the technology of the present disclosure. The first area 3A of the Nth imaging target area 3 is an example of a "second area" according to the technology of the present disclosure. The N+1th composite image 150 is an example of a "first captured image" according to the technology of the present disclosure. The Nth composite image 150 is an example of a "second captured image" according to the technology of the present disclosure.

[0208] Next, a modified example common to the first to sixth embodiments will be described.

[0209] In the above embodiment, the lighting control unit 92 turns on the light source 48 when the flight imaging device 10 starts flying along the flight route 4, and the lighting-off control unit 120 turns off the light source 48 when the flight imaging device 10 finishes flying along the flight route 4. However, the lighting control unit 92 may turn on the light source 48 when the flight imaging device 10 reaches each imaging position 5 on the flight route 4, and the lighting-off control unit 120 may turn off the light source 48 when the flight imaging device 10 acquires the composite image 142 at each imaging position 5.

[0210] In this way, it is possible to reduce the power supplied to the light source 48 compared to when the light source 48 is turned on when the flight imaging device 10 starts flying along the flight route 4 and when the light source 48 is turned off when the flight imaging device 10 finishes flying along the flight route 4. The control by the lighting control unit 92 in this case is an example of "control that causes the light source to irradiate light when the moving object moves to a position where the first imaging target area is imaged by the imaging device" according to the technology of the present disclosure.

[0211] In addition, in the above embodiment, the size of the light source 48 is set to a size such that, for example, when the Nth synthesis image 142 is obtained by imaging the Nth imaging target area 3, the light source image 50 (i.e., the image of reflected light) fits within the image area 142A of the Nth synthesis image 142.

[0212] However, the processor 34 may perform control to fit the light source image 50 into the image region 142A of the compositing image 142. Examples of control to fit the light source image 50 into the image region 142A include control to increase or decrease the intensity of light emitted from the light source 48, control to change the position of the light source 48, control to change the angle of the light source 48, or control to change the distance between the wall surface 2A and the flying imaging device 10. Even in this case, the pixel values ​​of the image region 142A of the Nth compositing image 142 are not used as the pixel values ​​of the overlap image region 144, thereby making it possible to obtain a composite image 140 that does not include the light source image 50.

[0213] In the above embodiment, the brightness of the imaging target area 3 is acquired based on an image obtained by imaging using the imaging device 60. However, the brightness of the imaging target area 3 may also be acquired based on an image obtained by imaging using an overhead camera (not shown) that images the wall surface 2A. Furthermore, the flying imaging device 10 may be provided with a sensor (not shown) that detects the brightness of the imaging target area 3, separate from the imaging device 60, and the brightness of the imaging target area 3 may be acquired based on the detection result of the sensor.

[0214] In the above embodiment, the composite image 142 is acquired while the flying imaging device 10 is temporarily stopped at each imaging position 5. However, for example, when the brightness of the imaging target area 3 is acquired using an overhead camera or a sensor, the flying imaging device 10 may acquire the composite image 142 while passing each imaging device 60.

[0215] In the above embodiment, the flying object 20 is used as an example of the moving object, but any moving object that moves along a moving route may be used, such as a car, a motorcycle, a bicycle, a dolly, a gondola, an airplane, a flying object, or a ship.

[0216] In the above embodiment, the flight route 4 extends horizontally, but may extend in a direction other than horizontal. The orientation of the flight route 4 may also be changed relative to the orientation shown in each drawing.

[0217] In addition, in the above embodiment, processor 34 is exemplified, but at least one other CPU, at least one GPU, and / or at least one TPU may be used instead of processor 34 or together with processor 34.

[0218] Furthermore, in the above embodiment, an example has been described in which the flight imaging program 90 and the composite image generation program 130 are stored in the storage 36, but the technology of the present disclosure is not limited to this. For example, the flight imaging program 90 and / or the composite image generation program 130 may be stored in a portable, non-transitory, computer-readable storage medium (hereinafter simply referred to as a "non-transitory storage medium") such as an SSD or USB memory. The flight imaging program 90 and / or the composite image generation program 130 stored in the non-transitory storage medium may be installed in the computer 26 of the flight imaging device 10.

[0219] In addition, the flight imaging program 90 and / or the composite image generation program 130 may be stored in a storage device such as another computer or server device connected to the flight imaging device 10 via a network, and the flight imaging program 90 and / or the composite image generation program 130 may be downloaded and installed on the computer 26 in response to a request from the flight imaging device 10.

[0220] Furthermore, it is not necessary to store all of the flight imaging program 90 and / or composite image generation program 130 in a storage device such as another computer or server device connected to the flight imaging device 10, or in the storage 36; only part of the flight imaging program 90 and / or composite image generation program 130 may be stored.

[0221] Furthermore, although the flying imaging device 10 has a built-in computer 26, the technology of the present disclosure is not limited to this, and for example, the computer 26 may be provided outside the flying imaging device 10.

[0222] In addition, in the above embodiment, the computer 26 includes the processor 34, the storage 36, and the RAM 38, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 26. Furthermore, a combination of a hardware configuration and a software configuration may be used instead of the computer 26.

[0223] Furthermore, the following various processors can be used as hardware resources for executing the various processes described in the above embodiments. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource for executing various processes by executing software, i.e., a program. Examples of processors include dedicated electronic circuits, such as FPGAs, PLDs, and ASICs, which are processors with a circuit configuration designed specifically for executing specific processes. Each processor has built-in or connected memory, and each processor uses the memory to execute various processes.

[0224] The hardware resources that execute various processes may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resources that execute various processes may be a single processor.

[0225] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes various processes. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute various processes, on a single IC chip, as typified by SoC. In this way, various processes are realized using one or more of the above-mentioned various processors as hardware resources.

[0226] Furthermore, the hardware structure of these various processors can be, more specifically, electronic circuits that combine circuit elements such as semiconductor devices. The above-described gaze detection process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0227] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0228] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0229] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0230] The following additional notes are provided regarding the above-described embodiments.

[0231] (Supplementary Note 1) An image processing device comprising: a processor, wherein when a first imaging target area and a second imaging target area of ​​a subject are imaged by the imaging device in accordance with a movement position of a moving body on which an imaging device is mounted, the processor generates a composite image by combining a first imaged image obtained by imaging the first imaging target area and a second imaged image obtained by imaging the second imaging target area; a first area that is a part of the first imaging target area overlaps with a second area that is a part of the second imaging target area; the first imaged image is an image obtained by imaging the first imaging target area with the imaging device in a state in which light is irradiated onto the first imaging target area by a light source mounted on the moving body; the intensity of the light irradiated onto the first area is higher than the intensity of the light irradiated around the first area; and the composite image is an image in which pixel values ​​of an image area in the second imaging image corresponding to the second area are used as pixel values ​​of overlapping image areas corresponding to the first area and the second area in the composite image. (Supplementary Note 2) An image processing method comprising: when a first imaging target area and a second imaging target area of ​​a subject are imaged by the imaging device in accordance with a movement position of the moving body on which the imaging device is mounted, generating a composite image by combining a first image obtained by imaging the first imaging target area and a second image obtained by imaging the second imaging target area; a first area that is a part of the first imaging target area overlaps with a second area that is a part of the second imaging target area; the first image is an image obtained by imaging the first imaging target area with the imaging device in a state in which light is irradiated onto the first imaging target area by a light source mounted on the moving body; the intensity of the light irradiated onto the first area is higher than the intensity of the light irradiated around the first area; and the composite image is an image in which pixel values ​​of an image area in the second imaging image corresponding to the second area are used as pixel values ​​of overlapping image areas corresponding to the first area and the second area in the composite image.(Supplementary Note 3) A program for causing a computer to execute processing, the processing including, when a first imaging target area and a second imaging target area of ​​a subject are imaged by the imaging device in accordance with a moving position of a moving body on which an imaging device is mounted, generating a composite image by combining a first image obtained by imaging the first imaging target area and a second image obtained by imaging the second imaging target area, wherein a first area that is a part of the first imaging target area overlaps with a second area that is a part of the second imaging target area, the first image is an image obtained by imaging the first imaging target area with the imaging device in a state in which light is irradiated onto the first imaging target area by a light source mounted on the moving body, the intensity of the light irradiated onto the first area is higher than the intensity of the light irradiated around the first area, and the composite image is an image in which pixel values ​​of an image area in the second imaging image corresponding to the second area are used as pixel values ​​of overlapping image areas corresponding to the first area and the second area in the composite image. (Supplementary Note 4) A mobile imaging device comprising: an imaging device, a light source, and a mobile body equipped with the imaging device and the light source, wherein the imaging device images a first imaging target area and a second imaging target area of ​​the subject according to a movement position of the mobile body, a first area that is a part of the first imaging target area overlaps with a second area that is a part of the second imaging target area, the light source irradiates the first imaging target area with light, and an intensity of the light irradiated onto the first area is higher than an intensity of the light irradiated onto a periphery of the first area. (Supplementary Note 5) The mobile imaging device according to Supplementary Note 4, comprising a control device that controls the imaging device and the light source.

Claims

1. A control device applied to a moving body equipped with an imaging device and a light source, the imaging device images a first imaging target area and a second imaging target area of ​​a subject in accordance with a moving position of the moving object; a first region that is a part of the first imaging target region overlaps with a second region that is a part of the second imaging target region; The control device includes a processor. The processor controls the light source to irradiate the first imaging target area with light; The intensity of the light irradiated to the first region is higher than the intensity of the light irradiated to the periphery of the first region. Control device.

2. The processor, generating a composite image by combining a first captured image obtained by capturing an image of the first imaging target area and a second captured image obtained by capturing an image of the second imaging target area; When generating the composite image, pixel values ​​of an image area in the second captured image corresponding to the second area are used as pixel values ​​of an overlapping image area in the composite image corresponding to the first area and the second area. The control device according to claim 1 .

3. The control includes control of fitting an image of the light source in an image region corresponding to the first region in a first captured image obtained by capturing an image of the first imaging target region. The control device according to claim 1 .

4. The imaging device includes an image sensor having a light receiving surface, The processor adjusts the amount of exposure of the light receiving surface based on brightness of a third area other than the first area in the first imaging target area when the first imaging target area is imaged by the imaging device. The control device according to claim 1 .

5. The amount of exposure is adjusted by changing the irradiation position and / or intensity of the light. The control device according to claim 4.

6. The amount of light exposure is adjusted by varying the exposure of the light receiving surface. The control device according to claim 4.

7. The exposure amount is set to an exposure amount at which a pixel value of an image area corresponding to the first area in a first captured image obtained by capturing an image of the first imaging target area is equal to or greater than a first default pixel value. The control device according to claim 4.

8. The first default pixel value is set to a pixel value at which pixel values ​​in at least a part of the image region are saturated. The control device according to claim 7.

9. the first imaging target area and the second imaging target area are areas imaged in this order, The processor, obtaining a length of the first region along a moving direction of the moving object based on pixel values ​​of the image region; The moving body is moved to a position where the first area and the second area overlap based on the length. The control device according to claim 7.

10. The exposure amount is set to an exposure amount such that a pixel value of an image area in the first captured image corresponding to the third area is equal to or greater than a second default pixel value and is less than the first default pixel value. The control device according to claim 7.

11. The amount of overlap between the first region and the second region is a predetermined amount of overlap. The control device according to claim 10.

12. the light source is disposed on the front side of the imaging device in a direction of movement of the moving body, The first imaging target area and the second imaging target area are areas imaged in this order. The control device according to claim 1 .

13. the light source is disposed on the rear side of the imaging device in a moving direction of the moving body, The first imaging target area and the second imaging target area are areas imaged in the order of the second imaging target area and the first imaging target area. The control device according to claim 1 .

14. The processor controls the light source to irradiate the second imaging target area with the light when the second imaging target area is imaged by the imaging device; The intensity of the light irradiated to a fourth region on the opposite side of the second region in the second imaging target region is higher than the intensity of the light irradiated to a periphery of the fourth region. The control device according to claim 1 .

15. The light is irradiated along a normal direction of the first region. The control device according to claim 1 .

16. The control is a control for causing the light source to emit the light when the moving object moves to a position where the first imaging target area is imaged by the imaging device. The control device according to claim 1 .

17. The control is a control for causing the light source to irradiate the light when the moving object moves to a position where the first imaging target area is imaged by the imaging device. The control device according to claim 1 .

18. A control method applied to a moving body equipped with an imaging device and a light source, comprising: the imaging device images a first imaging target area and a second imaging target area of ​​a subject in accordance with a moving position of the moving object; a first region that is a part of the first imaging target region overlaps with a second region that is a part of the second imaging target region; The control method includes controlling the light source to irradiate the first imaging target area with light, The intensity of the light irradiated to the first region is higher than the intensity of the light irradiated to the periphery of the first region. Control methods.

19. A program for causing a computer to execute processing, the program being applied to a moving body equipped with an imaging device and a light source, the imaging device images a first imaging target area and a second imaging target area of ​​a subject in accordance with a moving position of the moving object; a first region that is a part of the first imaging target region overlaps with a second region that is a part of the second imaging target region; The process includes controlling the light source to irradiate the first imaging target area with light, The intensity of the light irradiated to the first region is higher than the intensity of the light irradiated to the periphery of the first region. program.

Images