Imaging control device, imaging control method, and program

Abstract

This imaging control device is provided with a processor. The processor causes an imaging device to image a first region to be imaged, and, in a process in which a mobile body with the imaging device mounted thereon moves, performs an overlap imaging process in which the imaging device is caused to image a second region to be imaged if a part of the second region to be imaged overlaps a part of the first region to be imaged. The processor, if the overlap imaging process is unsuccessful, performs an interval imaging process in which the imaging device is caused to image a third region to be imaged, on condition that the movement distance by which the mobile body has moved from a first position at which the first region to be imaged was imaged by the imaging device has reached a first predetermined movement distance.

Description

Imaging control device, imaging control method, and program

[0001] The technology of the present disclosure relates to an imaging control device, an imaging control method, and a program.

[0002] Japanese Patent Laid-Open Publication No. 2018-151775 discloses a method for creating a distribution map of physical quantities that differ for each location within a target range. The disclosed method includes a mobile measurement step, a physical quantity setting step, an orthoimage creation step, and a distribution map creation step. The mobile measurement step is a step of acquiring multiple ground images by photographing the ground so that adjacent images overlap while moving within the target range, and measuring physical quantities. The physical quantity setting step is a step of assigning a representative physical quantity to each ground image based on the physical quantity obtained in the mobile measurement step. The orthoimage creation step is a step of creating an orthoimage of the target range based on multiple ground images. The distribution map creation step is a step of creating a physical quantity distribution map by displaying the representative physical quantity on the orthoimage.

[0003] Japanese Patent Application Laid-Open Publication No. 2020-113843 discloses an image capture support device that supports capturing multi-viewpoint images used to restore a three-dimensional shape model of a target object. The image capture support device includes a feature point extraction unit that extracts feature points, a matching processing unit, and a support information notification unit. The feature point extraction unit extracts feature points from captured image data, which is image data of a target object captured immediately before, and from preview image data. The matching processing unit detects first corresponding points for each of the feature points in the captured image data and the preview image data. The support information notification unit displays a preview image of the preview image data on which the first corresponding points are superimposed, and notifies the preview image of support information corresponding to the capture.

[0004] Japanese Patent Application Laid-Open Publication No. 2010-045587 discloses a camera device. The camera device includes an image capturing unit, an image display unit, a shake detection unit, an image recording unit, a relative relationship calculation unit, a display control unit, an overlap calculation unit, a notification unit, and a capture control unit. The image capturing unit captures an image. The image display unit has at least a screen for displaying the image. The shake detection unit detects device shake when the image capturing unit captures an image. The image recording unit records information about the image captured by the image capturing unit. The relative relationship calculation unit calculates a relative relationship parameter that represents at least the relative positional relationship between the capture range of a first image captured immediately before by the image capturing unit and recorded in the image recording unit, and the capture range of a second image captured by the image capturing unit after the first image. The display control unit generates an image for clearly indicating the relative positional relationship between the capture ranges from the relative relationship parameter calculated by the relative relationship calculation unit, and displays the image together with the second image on the screen of the image display unit. The overlap calculation unit calculates an overlap parameter that indicates the degree of overlap between the shooting range of the first image and the shooting range of the second image. The notification unit issues a predetermined notification to the photographer according to the overlap parameter calculated by the overlap calculation unit. The shooting control unit causes the image shooting unit to shoot an image when the overlap parameter calculated by the overlap calculation unit falls within a predetermined threshold range and it can be determined from the detection output of the shake detection unit that there is almost no device shake during image shooting by the image shooting unit.

[0005] International Publication WO 2018 / 168406 discloses an imaging control device that controls imaging of a moving object equipped with a camera. The imaging control device includes a wide-angle image acquisition unit, an imaging information acquisition unit, a margin information acquisition unit, a region information acquisition unit, an imaging region calculation unit, and a control unit. The wide-angle image acquisition unit acquires a wide-angle image in which an entire image of an object is captured at a wide angle. The imaging information acquisition unit acquires imaging information related to the number of captured split images or the imaging angle of view, which are acquired by capturing portions of the entire image of the object in close-up with a camera on the moving object. The margin information acquisition unit acquires margin information related to margins used when generating a composite image of the object by combining the split images. The region information acquisition unit acquires imaging object region information related to the region of the entire image of the object. The imaging region calculation unit calculates, based on the imaging information, margin information, and imaging object region information, each imaging region in the wide-angle image in which a margin is ensured, which is the imaging region of each split image that constitutes the composite image. The control unit moves the moving body, causes the camera to take close-up images of each calculated shooting area, and acquires the captured close-up images as divided images. The control unit compares an image corresponding to each shooting area of ​​the acquired wide-angle image with an image taken close-up by the camera, and controls the position of the moving body to cause the camera to take close-up images of each shooting area.

[0006] JP 2014-519739 A discloses an image registration method, which includes the steps of obtaining position information from a device, obtaining first and second images from the device, aligning a plurality of regions in the first image with a corresponding plurality of regions in the second image to identify a plurality of corresponding regions, determining a search vector for each of the plurality of corresponding regions, selecting only corresponding regions having search vectors that match the position information from the plurality of corresponding regions to identify a plurality of matching regions, and aligning the first and second images using the plurality of matching regions.

[0007] One embodiment of the technique of the present disclosure provides an imaging control device, an imaging control method, and a program that are capable of capturing an image of a third imaging target area even if overlap imaging processing has failed.

[0008] A first aspect of the technology of the present disclosure is an imaging control device that includes a processor, which causes an imaging device to capture an image of a first imaging target area, and if a part of a second imaging target area overlaps with a part of the first imaging target area while a moving body equipped with the imaging device is moving, performs overlapping imaging processing that causes the imaging device to capture the second imaging target area, and if the overlapping imaging processing fails, performs interval imaging processing that causes the imaging device to capture an image of a third imaging target area, on the condition that the distance traveled by the moving body from a first position where the first imaging target area was captured by the imaging device has reached a first predetermined travel distance.

[0009] A second aspect of the technology of the present disclosure is an imaging control device according to the first aspect, in which a case where the overlap imaging process fails is a case where the second imaging target area is not imaged by the imaging device, and includes a case where the travel distance exceeds the distance from the first position to the second position where the second imaging target area is imaged by the imaging device.

[0010] A third aspect of the technology of the present disclosure is an imaging control device in which, in the imaging control device of the first or second aspect, a case in which the overlap imaging process fails is a case in which a second imaging target area is imaged by the imaging device, and a first overlap amount between a portion of the first image obtained by imaging the first imaging target area and a portion of the second image obtained by imaging the second imaging target area is outside a first predetermined range.

[0011] A fourth aspect of the technology of the present disclosure is an imaging control device in which, in an imaging control device relating to any one of the first to third aspects, a case in which the overlap imaging process fails includes a case in which the third image obtained by imaging the second imaging target area by the imaging device does not meet the specified image quality.

[0012] A fifth aspect of the technology disclosed herein is an imaging control device according to any one of the first to fourth aspects, in which a portion of the third imaging target area overlaps with a portion of the second imaging target area.

[0013] A sixth aspect of the technology of the present disclosure is an imaging control device relating to any one of the first to fifth aspects, wherein the first predetermined movement distance is the distance from the first position to the third position where a portion of the third imaging target area overlaps with a portion of the second imaging target area.

[0014] A seventh aspect of the technology of the present disclosure is an imaging control device according to any one of the first to sixth aspects, wherein the first predetermined movement distance is a natural number multiple of two or more of the distance from the first position to the fourth position where the second imaging target area is imaged by the imaging device.

[0015] An eighth aspect of the technology of the present disclosure is an imaging control device according to any one of the first to seventh aspects, wherein the overlap imaging process is performed on the condition that a second overlap amount between a portion of a fourth image obtained by imaging the first imaging target area with the imaging device and a portion of a fifth image obtained by imaging the second imaging target area is within a second predetermined range.

[0016] A ninth aspect of the technology of the present disclosure is an imaging control device according to any one of the first to eighth aspects, in which the travel distance is derived based on acceleration measured by an acceleration sensor mounted on the imaging device and / or the moving body.

[0017] A tenth aspect of the technology of the present disclosure is an imaging control device relating to any one of the first to ninth aspects, in which the condition that the movement distance reaches a first predetermined movement distance is determined when, when the moving body moves at a constant speed, the time elapsed from the first timing at which the first imaging target area is imaged by the imaging device reaches a first predetermined time.

[0018] An eleventh aspect of the technology of the present disclosure is an imaging control device according to any one of the first to tenth aspects, in which the moving distance is derived based on the moving speed of the moving object derived based on a plurality of sixth images obtained by capturing images by the imaging device and the time interval between when the plurality of sixth images are obtained.

[0019] A twelfth aspect of the technology of the present disclosure is an imaging control device according to any one of the first to eleventh aspects, wherein, when the overlapping imaging process fails, the processor acquires position information regarding the position of the second imaging target area, first image information regarding a seventh image obtained by imaging the first imaging target area by the imaging device, and second image information regarding an eighth image obtained by imaging the third imaging target area by the imaging device, and the position information regarding the position of the second imaging target area is stored in memory in association with at least one of the first image information and the second image information.

[0020] A thirteenth aspect of the technology of the present disclosure is an imaging control device according to any one of the first to twelfth aspects, wherein a processor acquires a movement speed of a moving body and outputs movement speed data indicating the movement speed, and the movement speed is derived based on a plurality of ninth images obtained by capturing images by the imaging device.

[0021] A fourteenth aspect of the technology of the present disclosure is an imaging control method comprising: causing an imaging device to capture an image of a first imaging target area; performing overlapping imaging processing to cause the imaging device to capture the second imaging target area when a part of the second imaging target area overlaps with a part of the first imaging target area while a moving body equipped with the imaging device is moving; and performing interval imaging processing to cause the imaging device to capture a third imaging target area when the overlapping imaging processing fails, on the condition that the distance traveled by the moving body from a first position where the first imaging target area was captured by the imaging device has reached a first predetermined travel distance.

[0022] A fifteenth aspect of the technology of the present disclosure is a program for causing a computer to execute processing including causing an imaging device to capture an image of a first imaging target area, performing overlapping imaging processing for causing the imaging device to capture an image of the second imaging target area when a part of the second imaging target area overlaps with a part of the first imaging target area while a moving body equipped with the imaging device is moving, and performing interval imaging processing for causing the imaging device to capture an image of a third imaging target area when the overlapping imaging processing fails, on the condition that the distance traveled by the moving body from a first position where the first imaging target area was captured by the imaging device has reached a first predetermined travel distance.

[0023] 1 is a perspective view showing an example of a flying imaging device and an object; FIG. 2 is a block diagram showing an example of a hardware configuration of an imaging device; FIG. 3 is a perspective view showing an example of a flying imaging device, a first imaging target area, a second imaging target area, and a third imaging target area; FIG. 4 is a block diagram showing an example of a functional configuration of an imaging device for executing imaging processing; FIG. 5 is an explanatory diagram illustrating an example of a first operation of a processor in imaging processing; FIG. 6 is an explanatory diagram illustrating an example of a second operation of a processor in imaging processing; FIG. 7 is an explanatory diagram illustrating an example of a third operation of a processor in imaging processing; FIG. 8 is an explanatory diagram illustrating an example of a fourth operation of a processor in imaging processing; FIG. 9 is an explanatory diagram illustrating an example of a fifth operation of a processor in imaging processing; FIG. 10 is an explanatory diagram illustrating an example of a sixth operation of a processor in imaging processing; FIG. 11 is an explanatory diagram illustrating an example of a seventh operation of a processor in imaging processing; FIG. 12 is an explanatory diagram illustrating an example of an eighth operation of a processor in imaging processing; FIG. 13 is an explanatory diagram illustrating an example of a ninth operation of a processor in imaging processing; FIG. 14 is a block diagram showing an example of a functional configuration of an imaging device for executing re-imaging processing; FIG. 15 is an explanatory diagram illustrating an example of a first operation of a processor in re-imaging processing; FIG. 16 is an explanatory diagram illustrating an example of a second operation of a processor in re-imaging processing; FIG. 17 is an explanatory diagram illustrating an example of a third operation of a processor in re-imaging processing; FIG. 10 is an explanatory diagram illustrating an example of a fifth operation of the processor in the re-imaging process. FIG. 11 is a flowchart illustrating an example of the flow of the imaging process. FIG. 12 is a flowchart illustrating an example of the flow of the re-imaging process. FIG. 13 is an explanatory diagram illustrating a first modified example of the fifth operation of the processor in the imaging process. FIG. 14 is an explanatory diagram illustrating a first modified example of the seventh operation of the processor in the imaging process. FIG. 15 is an explanatory diagram illustrating a second modified example of the fifth operation of the processor in the imaging process.

[0024] Hereinafter, an example of an embodiment of an imaging control device, an imaging control method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] I / F is an abbreviation for "Interface." RAM is an abbreviation for "Random Access Memory." EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory." CPU is an abbreviation for "Central Processing Unit." HDD is an abbreviation for "Hard Disk Drive." SSD is an abbreviation for "Solid State Drive." DRAM is an abbreviation for "Dynamic Random Access Memory." SRAM is an abbreviation for "Static Random Access Memory." CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor." GPU is an abbreviation for "Graphics Processing Unit." TPU is an abbreviation for "Tensor Processing Unit." USB is an abbreviation for "Universal Serial Bus." ASIC is an abbreviation for "Application Specific Integrated Circuit." FPGA is an abbreviation for "Field-Programmable Gate Array." PLD is an abbreviation for "Programmable Logic Device." SoC is an abbreviation for "System-on-a-chip." IC is an abbreviation for "Integrated Circuit."

[0027] In the description of this specification, "constant" refers to a constant in the sense of being completely constant, as well as a constant in the sense of being within a tolerance generally acceptable in the technical field to which the technology of the present disclosure belongs, and which does not violate the spirit of the technology of the present disclosure. "Vertical" refers to a vertical in the sense of being completely vertical, as well as a vertical in the sense of being within a tolerance generally acceptable in the technical field to which the technology of the present disclosure belongs, and which does not violate the spirit of the technology of the present disclosure. In the description of this specification, "horizontal direction" refers to a horizontal direction in the sense of being completely horizontal, as well as a horizontal in the sense of being within a tolerance generally acceptable in the technical field to which the technology of the present disclosure belongs, and which does not violate the spirit of the technology of the present disclosure. In the description of this specification, "vertical direction" refers to a vertical direction in the sense of being completely vertical, as well as a vertical in the sense of being within a tolerance generally acceptable in the technical field to which the technology of the present disclosure belongs, and which does not violate the spirit of the technology of the present disclosure.

[0028] 1, the flying imaging device 1 has a flying function and an imaging function, and while flying, captures an image of a wall surface 2A of an object 2. In the description of this specification, the concept of "flying" includes not only the meaning of the flying imaging device 1 moving in the air, but also the meaning of the flying imaging device 1 standing still in the air.

[0029] As an example, the wall surface 2A is a plane. A plane refers to a two-dimensional surface (i.e., a surface extending in two dimensions). In addition, in the description of this specification, the concept of "plane" does not include the meaning of a mirror surface. In this embodiment, for example, the wall surface 2A is a plane defined by the horizontal and vertical directions (i.e., a surface extending in the horizontal and vertical directions). The wall surface 2A includes irregularities. The irregularities referred to here include, for example, irregularities caused by the material forming the wall surface 2A, as well as irregularities caused by defects and / or flaws. As an example, the object 2 having the wall surface 2A is a pier provided on a bridge. The pier is made of, for example, reinforced concrete. Here, a pier is given as an example of the object 2, but the object 2 may be an object other than a pier (e.g., a tunnel or a dam).

[0030] The flight function of the flight imaging device 1 (hereinafter also simply referred to as "flight function") is a function that causes the flight imaging device 1 to fly based on a flight instruction signal. A flight instruction signal refers to a signal that instructs the flight imaging device 1 to fly. The flight instruction signal is transmitted, for example, from a transmitter 20 for controlling the flight imaging device 1. The transmitter 20 is operated by a user (not shown). The transmitter 20 includes a control unit 22 for controlling the flight imaging device 1, and a display device 24 for displaying various images and / or information, etc. The display device 24 is, for example, a liquid crystal display.

[0031] Note that, although an example is given here in which the flight instruction signal is transmitted from the transmitter 20, the flight instruction signal may also be transmitted from a base station (not shown) or the like that sets a flight route for the flight imaging device 1. The imaging function of the flight imaging device 1 (hereinafter also simply referred to as the "imaging function") is a function in which the flight imaging device 1 captures an image of a subject (for example, the wall surface 2A of the target object 2).

[0032] The flight imaging device 1 comprises an air vehicle 10 and an imaging device 30. The air vehicle 10 is, for example, an unmanned aerial vehicle such as a drone. The flight function is realized by the air vehicle 10. The air vehicle 10 has multiple propellers 12, and flies when the multiple propellers 12 rotate. The flight of the air vehicle 10 is synonymous with the flight of the flight imaging device 1. The air vehicle 10 is an example of a "moving body" according to the technology of the present disclosure.

[0033] The imaging device 30 is, for example, a digital camera or a video camera. The imaging function is realized by the imaging device 30. The imaging device 30 is mounted on the flying object 10. Specifically, the imaging device 30 is provided on the lower part of the flying object 10. Here, an example is given in which the imaging device 30 is provided on the lower part of the flying object 10, but the imaging device 30 may also be provided on the upper part or front part of the flying object 10, etc.

[0034] The flying imaging device 1 sequentially captures images of multiple imaging target areas 3 on the wall surface 2A. The imaging target areas 3 are areas determined by the angle of view of the flying imaging device 1. In the example shown in FIG. 1 , a rectangular area is shown as an example of the imaging target area 3. Multiple images to be combined 92 are obtained by sequentially capturing images of the multiple imaging target areas 3 using the imaging device 30. A composite image 90 is generated by combining the multiple images to be combined 92. The multiple images to be combined 92 are combined so that adjacent images to be combined 92 partially overlap each other. An example of the composite image 90 is a two-dimensional panoramic image. The two-dimensional panoramic image is merely an example, and a three-dimensional image (e.g., a three-dimensional panoramic image) may be generated as the composite image 90 in a similar manner to how a two-dimensional panoramic image is generated as the composite image 90.

[0035] The composite image 90 may be generated each time a second or subsequent frame of a compositing image 92 is obtained, or may be generated after a plurality of compositing images 92 are obtained for the wall surface 2A. The process of generating the composite image 90 may be executed by the flight imaging device 1, or may be executed by an external device (not shown) communicatively connected to the flight imaging device 1. The composite image 90 is used, for example, to inspect or measure the wall surface 2A of the object 2.

[0036] 1 shows an example in which each imaging target area 3 is imaged by the imaging device 30 with the optical axis OA of the imaging device 30 perpendicular to the wall surface 2A. The following description is based on the assumption that each imaging target area 3 is imaged by the imaging device 30 with the optical axis OA of the imaging device 30 perpendicular to the wall surface 2A.

[0037] The multiple imaging target areas 3 are imaged so that adjacent imaging target areas 3 partially overlap. The multiple imaging target areas 3 are imaged so that adjacent imaging target areas 3 partially overlap in order to synthesize compositing images 92 corresponding to the adjacent imaging target areas 3 based on feature points included in the overlapping parts of the adjacent imaging target areas 3. Hereinafter, the partial overlap of adjacent imaging target areas 3 and the partial overlap of adjacent compositing images 92 may each be referred to as "overlap."

[0038] As an example, the flight imaging device 1 moves in a zigzag pattern by alternately repeating horizontal and vertical movements. As a result, multiple imaging target areas 3 connected in a zigzag pattern are sequentially imaged. As an example, a tape measure 4 is provided on both horizontal ends of the wall surface 2A. The tape measure 4 hangs down from above the object 2. The tape measure 4 is provided on both horizontal sides of the multiple imaging target areas 3. The user moves the flight imaging device 1 in the horizontal and vertical directions by maneuvering the flight imaging device 1 based on the scales on the tape measure 4.

[0039] As an example, as shown in FIG. 2, the imaging device 30 includes a computer 32, an image sensor 34, an image sensor driver 36, an imaging lens 38, and an input / output I / F 40.

[0040] The computer 32 includes a processor 42, a storage 44, and a RAM 46. The computer 32 is an example of an "imaging control device" and a "computer" according to the techniques of the present disclosure. The processor 42 is an example of a "processor" according to the techniques of the present disclosure. The processor 42, the storage 44, and the RAM 46 are connected to one another via a bus 48, and the bus 48 is connected to an input / output I / F 40. The input / output I / F 40 is also connected to an image sensor 34, an image sensor driver 36, and an imaging lens 38.

[0041] The processor 42 has, for example, a CPU and controls the entire imaging device 30. The storage 44 is a non-volatile storage device that stores various programs, various parameters, etc. Examples of the storage 44 include an HDD and / or a flash memory (for example, an EEPROM and / or an SSD).

[0042] The RAM 46 is a memory that temporarily stores information and is used as a work memory by the processor 42. The RAM 46 may be, for example, a DRAM and / or an SRAM.

[0043] The image sensor 34 is connected to an image sensor driver 36. The image sensor driver 36 controls the image sensor 34 in accordance with instructions from the processor 42. The image sensor 34 is, for example, a CMOS image sensor. Note that, although a CMOS image sensor is exemplified as the image sensor 34 here, the technology of the present disclosure is not limited to this, and other image sensors may be used. Under the control of the image sensor driver 36, the image sensor 34 captures an image of a subject (for example, the wall surface 2A of the target 2) and outputs image data obtained by capturing the image.

[0044] The imaging lens 38 is disposed closer to the subject (object side) than the image sensor 34. The imaging lens 38 captures subject light, which is light reflected from the subject, and forms an image of the captured subject light on the imaging surface of the image sensor 34. The imaging lens 38 includes multiple optical elements (not shown), such as a focus lens, a zoom lens, and an aperture. The imaging lens 38 is connected to the computer 32 via an input / output I / F 40. Specifically, the multiple optical elements included in the imaging lens 38 are connected to the input / output I / F 40 via a drive mechanism (not shown) having a power source. The multiple optical elements included in the imaging lens 38 operate under the control of the computer 32. In the imaging device 30, adjustment of focus, optical zoom, exposure, and the like is achieved by operating the multiple optical elements included in the imaging lens 38 (e.g., various lenses, an aperture, etc.).

[0045] 3 shows, as an example, a first imaging target area 3A, a second imaging target area 3B, and a third imaging target area 3C that are consecutive in the horizontal direction among the multiple imaging target areas 3. A portion of the first imaging target area 3A overlaps with a portion of the second imaging target area 3B, and a portion of the second imaging target area 3B overlaps with a portion of the third imaging target area 3C.

[0046] The flight imaging device 1 captures images at a timing when it determines that a predetermined imaging condition is met. Examples of the predetermined imaging condition include a condition that the amount of overlap between portions of adjacent imaging target areas 3 is within a predetermined range. The predetermined range is set in consideration of the efficiency of sequentially capturing images of multiple imaging target areas 3 and the number of feature points required to composite adjacent composite images 92.

[0047] When the positioning of the flight imaging device 1 is stable and the flight imaging device 1 is moving normally, the second imaging target area 3B is imaged when part of the second imaging target area 3B overlaps with part of the first imaging target area 3A, and the third imaging target area 3C is imaged when part of the third imaging target area 3C overlaps with part of the second imaging target area 3B. As a result, the flight imaging device 1 sequentially images the first imaging target area 3A, the second imaging target area 3B, and the third imaging target area 3C.

[0048] However, when the flight imaging device 1 is moving, for example, external disturbances such as wind may act on the flight imaging device 1, causing the positioning of the flight imaging device 1 to become unstable. If the positioning of the flight imaging device 1 becomes unstable in this way, it is expected that the flight imaging device 1 will fail to capture an image of the second imaging target area 3B after capturing an image of the first imaging target area 3A (hereinafter referred to as the "overlap imaging process"). An example of a failed overlap imaging process is when the distance traveled by the flight imaging device 1 exceeds the distance from the position where the first imaging target area 3A is captured to the position where the second imaging target area 3B is captured before it is determined that the predetermined imaging conditions are met.

[0049] Here, it is conceivable to have the flying imaging device 1 capture the third imaging target area 3C on the condition that the overlapping imaging process for the second imaging target area 3B is successful. However, in this case, if the overlapping imaging process for the second imaging target area 3B fails, there is the inconvenience that the third imaging target area 3C cannot be captured (i.e., imaging of the third imaging target area 3C cannot be continued). Therefore, in order to resolve this inconvenience, the processor 42 executes the following imaging process.

[0050] 4 , an imaging program 50 is stored in the storage 44. The imaging program 50 is an example of a “program” according to the technology of the present disclosure. The processor 42 reads the imaging program 50 from the storage 44 and executes the read imaging program 50 on the RAM 46. The processor 42 performs imaging processing in accordance with the imaging program 50 executed on the RAM 46.

[0051] The imaging process is started each time the flight imaging device 1 starts moving horizontally. As an example, the following describes a case where the flight imaging device 1 starts moving horizontally in response to receiving a flight instruction signal that moves at a constant speed from the transmitter 20 (see FIG. 1).

[0052] The imaging process is realized by the processor 42 operating in accordance with the imaging program 50 as a first imaging control unit 52, a second imaging control unit 54, a first overlap determination unit 56, a lost determination unit 58, a third imaging control unit 60, a second overlap determination unit 62, a first image storage control unit 64, an interval imaging determination unit 66, a fourth imaging control unit 68, an image quality determination unit 70, a second image storage control unit 72, and a lost information storage control unit 74.

[0053] As an example, as shown in FIG. 5 , when the flying imaging device 1 starts moving in the horizontal direction, the first imaging control unit 52 outputs a first imaging instruction signal to the image sensor 34, causing the image sensor 34 to capture an image of the first imaging target area 3A, which is the first imaging target area 3. The first imaging target area 3A is captured under the control of the first imaging control unit 52, thereby obtaining first synthesis image data. The first synthesis image data is image data representing a first synthesis image 92A, which is a synthesis image 92 corresponding to the first imaging target area 3A. The first synthesis image data is stored in the storage 44. The first imaging target area 3A is an example of a "first imaging target area" according to the technology of the present disclosure. The first synthesis image 92A is an example of a "first image" according to the technology of the present disclosure.

[0054] The second imaging control unit 54 outputs a second imaging instruction signal to the image sensor 34 while the flight imaging device 1 is moving, thereby causing the image sensor 34 to capture an image of the second imaging target area 3B. This results in the acquisition of overlap determination image data. The overlap determination image data is image data representing the overlap determination image 94. The overlap determination image 94 may be, for example, an image for display (e.g., a live view image or a post-view image), and the overlap determination image data may be output to a display device (not shown) provided in the imaging device 30 and / or a display device 24 (see FIG. 1) provided in the transmitter 20.

[0055] In the following description, unless there is an explanation that "images are taken under the control of the second image capture control unit 54," "image capture" refers to image capture for obtaining the composite image 92.

[0056] The first overlap determination unit 56 determines whether the area of ​​the overlap region where a portion of the first synthesis image 92A overlaps a portion of the overlap determination image 94 (hereinafter referred to as the "first overlap amount") is within a first specified range.

[0057] The first predetermined range is set taking into consideration the efficiency of sequentially capturing images of a plurality of imaging target areas 3 and the number of feature points required to composite adjacent compositing images 92 (see FIG. 1 ). For example, the upper limit of the first predetermined range is set to a value that is 50% or less of the area of ​​the compositing image 92, taking into consideration the efficiency of sequentially capturing images of a plurality of imaging target areas 3. Furthermore, for example, the lower limit of the first predetermined range is set to a value that is 30% or more of the area of ​​the compositing image 92, taking into consideration the number of feature points required to composite adjacent compositing images 92.

[0058] The movement speed of the aircraft 10 is set to a speed at which the first overlap determination unit 56 makes a determination at least once between the time when the first overlap amount falls below the upper limit value of the first predetermined range and the time when it falls below the lower limit value of the first predetermined range.

[0059] The first overlap amount is an example of a "second overlap amount" according to the technology of the present disclosure. The first predetermined range is an example of a "second predetermined range" according to the technology of the present disclosure. The first synthesis image 92A is an example of a "fourth image" according to the technology of the present disclosure. The overlap determination image 94 is an example of a "fifth image" according to the technology of the present disclosure.

[0060] 6 shows an example in which the first overlap amount exceeds the upper limit of the first predetermined range. When the first overlap amount exceeds the upper limit of the first predetermined range, the first overlap determination unit 56 determines that the first overlap amount is not within the first predetermined range.

[0061] If the first overlap determination unit 56 determines that the first overlap amount is not within the first predetermined range, the lost determination unit 58 determines whether the time that has elapsed since the first timing at which the first imaging target area 3A was imaged (hereinafter referred to as the "elapsed time") has exceeded a first predetermined time. The first predetermined time is set to, for example, the time from when the first imaging target area 3A is imaged until the first overlap amount reaches the lower limit of the first predetermined range when the flight imaging device 1 moves at a constant speed.

[0062] 6 shows an example in which the elapsed time does not exceed the first predetermined time. When the loss determination unit 58 determines that the elapsed time does not exceed the first predetermined time, the second imaging control unit 54 outputs a second imaging instruction signal to the image sensor 34, causing the image sensor 34 to capture an image of the second imaging target area 3B. This results in new overlap determination image data.

[0063] As an example, FIG. 7 shows an example in which the first overlap amount is within the first predetermined range. When the first overlap determination unit 56 determines that the first overlap amount is within the first predetermined range, the third imaging control unit 60 executes overlap imaging processing. That is, the third imaging control unit 60 outputs a third imaging instruction signal to the image sensor 34, causing the image sensor 34 to capture the second imaging target area 3B. The second imaging target area 3B is captured under the control of the third imaging control unit 60, thereby obtaining second compositing image data. The second compositing image data is image data representing a second compositing image 92B, which is a compositing image 92 corresponding to the second imaging target area 3B. The second imaging target area 3B is an example of a "second imaging target area" according to the technology of the present disclosure. The second compositing image 92B is an example of a "second image" according to the technology of the present disclosure.

[0064] As an example, as shown in FIG. 8 , the second overlap determination unit 62 determines whether the area of ​​the overlap region where a portion of the first compositing image 92A and a portion of the second compositing image 92B overlap (hereinafter referred to as the "second overlap amount") is within a second predetermined range. The second predetermined range is set, for example, to the same upper and lower limit values ​​as the first predetermined range. Note that the second predetermined range may also be set to upper and lower limit values ​​different from those of the first predetermined range. The second overlap amount is an example of a "first overlap amount" according to the technology of the present disclosure. The second predetermined range is an example of a "first predetermined range" according to the technology of the present disclosure.

[0065] A factor that may cause the second overlap amount to exceed the upper limit of the second predetermined range after the first overlap determination unit 56 makes a determination is, for example, an increase in the second overlap amount due to a change in the orientation of the flight imaging device 1 caused by a disturbance such as wind after the first overlap determination unit 56 makes a determination. Here, if the second compositing image data is stored in the storage 44 even though the second overlap amount exceeds the upper limit of the second predetermined range, the number of compositing image data stored in the storage 44 increases compared to when the second compositing image data is stored in the storage 44 on the condition that the second overlap amount is within the second predetermined range. Therefore, in this embodiment, an upper limit is set for the second predetermined range in order to limit the number of compositing image data stored in the storage 44.

[0066] Furthermore, factors that may cause the second overlap amount to fall below the lower limit of the second predetermined range after the first overlap determination unit 56 makes a determination include, for example, a decrease in the second overlap amount due to a change in the orientation of the flight imaging device 1 caused by external disturbances such as wind after the first overlap determination unit 56 makes a determination, or a decrease in the second overlap amount due to a delay between the output of the third imaging instruction signal from the third imaging control unit 60 to the image sensor 34 and the capture of an image by the image sensor 34. Here, if the second compositing image data is stored in the storage 44 even though the second overlap amount is below the lower limit of the second predetermined range, there is a risk that the number of feature points required to composite adjacent compositing images 92 will be insufficient. Therefore, in this embodiment, a lower limit is set for the second predetermined range to ensure the number of feature points required to composite adjacent compositing images 92.

[0067] 8 shows an example in which the second overlap amount is within the second predetermined range. When the second overlap determination unit 62 determines that the second overlap amount is within the second predetermined range, the first image storage control unit 64 outputs the second combining image data to the storage 44. As a result, the second combining image data is stored in the storage 44.

[0068] In addition, when the second synthesis image data is stored in the storage 44 by the first image storage control unit 64, the second imaging target area 3B imaged under the control of the above-mentioned third imaging control unit 60 is treated as the first imaging target area 3A, and the second synthesis image data obtained by imaging the second imaging target area 3B under the control of the third imaging control unit 60 is treated as the first synthesis image data.

[0069] When the second combining image data is stored in the storage 44 by the first image storage control unit 64, the second imaging control unit 54 outputs a second imaging instruction signal to the image sensor 34, causing the image sensor 34 to image a new second imaging target area 3B, thereby obtaining new overlap determination image data.

[0070] As an example, Figure 9 shows an example in which the elapsed time exceeds the first predetermined time. The first position indicates the position of the center of the flight imaging device 1 when the first imaging target area 3A is imaged. The second position indicates the position of the center of the flight imaging device 1 when the first overlap amount, which is the overlap amount between a portion of the first synthesis image 92A (see Figure 7) and a portion of the overlap determination image 94 (see Figure 7), reaches the lower limit of the first predetermined range. The first position is an example of a "first position" related to the technology of the present disclosure. The second position is an example of a "second position" related to the technology of the present disclosure.

[0071] If the elapsed time exceeds the first predetermined time, the distance traveled by the flight imaging device 1 from the first position exceeds the distance from the first position to the second position. In this case, the imaging device 30 loses the opportunity to capture the second imaging target area 3B, and the overlap imaging process fails. In other words, this means that the second synthesis image 92B corresponding to the second imaging target area 3B has been lost as one of the images used in generating the composite image 90.

[0072] When the lost determination unit 58 determines that the elapsed time has exceeded the first predetermined time, the interval imaging determination unit 66 determines whether the elapsed time has reached a second predetermined time. If the second predetermined time is T2, the second predetermined time T2 is determined by the following formula (1) when the flight imaging device 1 moves at a constant speed, for example.

[0073]

[0074] where T1 is the first preset time. As described above, the first preset time is set, for example, to the time from when the first imaging target area 3A is imaged until the first overlap amount reaches the lower limit of the first preset range. Furthermore, T3 is the third preset time. The third preset time is set, for example, to the same time as the time from when the first imaging target area 3A is imaged until the first overlap amount reaches the upper limit of the first preset range. When the flight imaging device 1 moves at a constant speed, when the elapsed time reaches the second preset time, the flight imaging device 1 reaches a position where it can image the third imaging target area 3C (see Figure 3). The second preset time is an example of the "first preset time" related to the technology of the present disclosure.

[0075] As an example, Figure 10 shows an example in which the second overlap amount is outside the second predetermined range (specifically, the second overlap amount is below the lower limit of the second predetermined range). If the second overlap amount is below the lower limit of the second predetermined range, it may be impossible to combine the first and second compositing images 92A and 92B based on the feature points contained as images in a portion of the first and second compositing images 92A and 92B. In this case, the imaging device 30 fails to perform the overlap imaging process because it is unable to obtain a second compositing image 92B that can be combined with the first compositing image 92A. This means that the second compositing image 92B corresponding to the second imaging target area 3B has been lost as one of the images used in generating the composite image 90.

[0076] If the second overlap determining section 62 determines that the second overlap amount is not within the second predetermined range, the interval image capture determining section 66 determines whether the elapsed time has reached the second predetermined time.

[0077] In the above description, examples of cases in which the overlap image capturing process fails include a case in which the elapsed time exceeds the first predetermined time and a case in which the second overlap amount is outside the second predetermined range. However, other cases may also be included in cases in which the overlap image capturing process fails, such as a case in which the image capturing device 30 does not determine to capture an image because a certain condition (for example, a condition in which the image capturing device 30 is not in focus when the autofocus mode is set as the operating mode) is not satisfied, or a case in which the image data to be combined is not properly stored in the storage 44.

[0078] As an example, FIG. 11 shows an example in which the elapsed time has reached the second predetermined time. When the elapsed time has reached the second predetermined time, the distance traveled by the flight imaging device 1 from the first position reaches the first predetermined travel distance. The third position indicates the position of the center of the flight imaging device 1 when the elapsed time has reached the second predetermined time. The first predetermined travel distance is the distance from the first position to the third position. For example, the first predetermined travel distance is twice the distance from the first position to the second position. The first predetermined travel distance is an example of a "first predetermined travel distance" according to the technology of the present disclosure. The third position is an example of a "third position" according to the technology of the present disclosure. The second position is an example of a "fourth position" according to the technology of the present disclosure.

[0079] The fourth imaging control unit 68 executes interval imaging processing when the interval imaging determination unit 66 determines that the elapsed time has reached the second preset time. That is, the fourth imaging control unit 68 outputs a fourth imaging instruction signal to the image sensor 34, causing the image sensor 34 to capture an image of the third imaging target area 3C. The third combining image data is obtained by capturing an image of the third imaging target area 3C under the control of the fourth imaging control unit 68. The third combining image data is image data representing a third combining image 92C, which is a combining image 92 corresponding to the third imaging target area 3C. The third imaging target area 3C is an example of a "third imaging target area" according to the technology of the present disclosure.

[0080] As an example, as shown in FIG. 12 , the image quality determination unit 70 determines whether the third compositing image 92C satisfies a predetermined image quality. The predetermined image quality is set based on, for example, the amount of blur, exposure, artifacts (e.g., geometric, illumination, or color artifacts), and / or the amount of blur. The fact that the third compositing image 92C does not satisfy the predetermined image quality means that feature points (i.e., feature points to be matched between images) required for generating the composite image 90 cannot be extracted from the third compositing image 92C. If the third compositing image 92C does not satisfy the predetermined image quality, the third compositing image 92C cannot be obtained as one of the images required for generating the composite image 90, and the imaging device 30 fails the interval imaging process.

[0081] If the image quality determination section 70 determines that the third synthesis image 92C does not satisfy the predetermined image quality, the interval image capture determination section 66 again determines whether the elapsed time has reached the second predetermined time.

[0082] In addition, the second predetermined time T2 when the inverter ball imaging process fails in addition to the overlap imaging process is determined by the following equation (2), for example, when the flying imaging device 1 moves at a constant speed.

[0083]

[0084] where T1 is the first predetermined time. As described above, the first predetermined time is set to, for example, the time from when the first imaging target area 3A is imaged until the first overlap amount reaches the lower limit of the first predetermined range. Furthermore, N is a natural number indicating the number of times the overlap imaging process and the inverter image capturing process have failed. Furthermore, T3 is the third predetermined time. As described above, the third predetermined time is set to, for example, the same time as the time from when the first imaging target area 3A is imaged until the first overlap amount reaches the upper limit of the first predetermined range.

[0085] When the image quality determination unit 70 determines that the third compositing image 92C satisfies the predetermined image quality, the second image storage control unit 72 outputs the third compositing image data to the storage 44. As a result, the third compositing image data is stored in the storage 44.

[0086] The lost information storage control unit 74 acquires position information regarding the position of the second imaging target area 3B (hereinafter referred to as the "lost position") when the overlap imaging process or the interval imaging process fails, first image information regarding the first synthesis image 92A, and second image information regarding the third synthesis image 92C.

[0087] The position information regarding the lost position is information indicating the order of imaging of the second imaging target area 3B corresponding to the lost position, counted from the first imaging target area 3 (see FIG. 5 ). The first image information regarding the first compositing image 92A is, for example, identification information that can identify the first compositing image 92A corresponding to the first imaging target area 3A captured immediately before the second imaging target area 3B corresponding to the lost position. The second image information regarding the third compositing image 92C is, for example, identification information that can identify the third compositing image 92C corresponding to the third imaging target area 3C captured by interval imaging processing.

[0088] The lost information storage control unit 74 generates lost information in which the first image information and the second image information are associated with the location information, and stores the lost information in the storage 44. The location information may be associated with only one of the first image information and the second image information. The location information is an example of "location information" according to the technology of the present disclosure. The first image information is an example of "first image information" according to the technology of the present disclosure. The second image information is an example of "second image information" according to the technology of the present disclosure. The storage 44 is an example of "memory" according to the technology of the present disclosure. The first compositing image 92A is an example of a "seventh image" according to the technology of the present disclosure. The third compositing image 92C is an example of an "eighth image" according to the technology of the present disclosure.

[0089] In addition, when the lost information is stored in the storage 44 by the lost information storage control unit 74, the third imaging target area 3C imaged under the control of the above-mentioned fourth imaging control unit 68 is treated as the first imaging target area 3A, and the third synthesis image data obtained by imaging the third imaging target area 3C under the control of the fourth imaging control unit 68 is treated as the first synthesis image data.

[0090] 13 shows an example in which the overlapping image capture process and the first interval image capture process have consecutively failed, but the second interval image capture process has succeeded. In the example shown in Fig. 13, the overlapping image capture process and the first interval image capture process have consecutively failed, and then the interval image capture determination unit 66 determines that the elapsed time has reached the second predetermined time, and the fourth image capture control unit 68 executes the second interval image capture process.

[0091] When the second interval imaging process is executed, the distance traveled by the flying imaging device 1 from the first position reaches the second preset travel distance. The fourth position indicates the position of the center of the flying imaging device 1 when the elapsed time reaches the second preset time. The second preset travel distance is the distance from the first position to the fourth position. For example, the second preset travel distance is three times the distance from the first position to the second position. The second preset travel distance is an example of a "first preset travel distance" according to the technology of the present disclosure. The fourth position is an example of a "third position" according to the technology of the present disclosure. The second position is an example of a "fourth position" according to the technology of the present disclosure.

[0092] In the example shown in Figure 13, the second synthesis image data corresponding to the overlapping imaging process and the third synthesis image data corresponding to the first interval imaging process are not obtained, but the first synthesis image data corresponding to the first imaging target area 3A and the third synthesis image data corresponding to the second interval imaging process are obtained.

[0093] In the example shown in Figure 13, the first imaging target area 3A, which was imaged just before the second imaging target area 3B corresponding to the overlapping imaging process, can be considered an example of the "first imaging target area" according to the technology of the present disclosure, the second imaging target area 3B corresponding to the overlapping imaging process can be considered an example of the "second imaging target area" according to the technology of the present disclosure, and the third imaging target area 3C corresponding to the first interval imaging process can be considered an example of the "third imaging target area" according to the technology of the present disclosure.

[0094] 13 , the second imaging target area 3B corresponding to the overlapping imaging process can be considered as an example of the “first imaging target area” according to the technology of the present disclosure, the third imaging target area 3C corresponding to the first interval imaging process can be considered as an example of the “second imaging target area” according to the technology of the present disclosure, and the third imaging target area 3C corresponding to the second interval imaging process can be considered as an example of the “third imaging target area” according to the technology of the present disclosure. In this case, the first interval imaging process is an example of the “overlap imaging process” according to the technology of the present disclosure. Furthermore, the reason for the failure of the first interval imaging process may be that the third compositing image 92C obtained by capturing the third imaging target area 3C corresponding to the first interval imaging process by the imaging device 30 did not meet the predetermined image quality. In this case, the third compositing image 92C is an example of the “third image” according to the technology of the present disclosure.

[0095] As described above, if the overlapping imaging process fails but the interval imaging process succeeds, a second imaging target area 3B for which no compositing image data could be obtained will exist between the third imaging target area 3C corresponding to the successful interval imaging process and the first imaging target area 3A. If a second imaging target area 3B for which no compositing image data could be obtained exists, there is a problem in that a missing area will be present in part of the composite image 90 (see FIG. 1 ) generated by combining multiple compositing images 92. Therefore, to resolve this problem, the processor 42 executes the following re-imaging process.

[0096] 14 , a re-imaging program 100 is stored in the storage 44. The processor 42 reads the re-imaging program 100 from the storage 44 and executes the read re-imaging program 100 on the RAM 46. The processor 42 performs re-imaging processing in accordance with the re-imaging program 100 executed on the RAM 46.

[0097] In the re-imaging process, the flying imaging device 1 starts moving horizontally from the same position as when the imaging process started. Also, in the re-imaging process, the flying imaging device 1 moves at the same movement speed as the movement speed in the imaging process. The re-imaging process starts when the flying imaging device 1 starts moving horizontally.

[0098] The re-imaging process is realized by the processor 42 operating as a first information acquisition unit 102, an arrival determination unit 104, a fifth imaging control unit 106, a third overlap determination unit 108, a sixth imaging control unit 110, a fourth overlap determination unit 112, a third image storage control unit 114, a second information acquisition unit 116, a fifth overlap determination unit 118, and an alarm control unit 120 in accordance with the re-imaging program 100.

[0099] 15 as an example, the first information acquisition unit 102 acquires, from the lost information stored in the storage 44, position information regarding the position of the second imaging target area 3B corresponding to the lost position, and first image information regarding the first compositing image 92A. Based on the position information, for example, the imaging order counted from the first imaging target area 3 (see FIG. 5) for the second imaging target area 3B corresponding to the lost position is identified. Furthermore, based on the first image information regarding the first compositing image 92A, for example, the first compositing image 92A (see FIG. 16) corresponding to the first imaging target area 3A captured immediately before the second imaging target area 3B corresponding to the lost position is identified.

[0100] The arrival determination unit 104 determines whether the flight imaging device 1 has reached the first imaging target area 3A that was imaged immediately before the second imaging target area 3B that corresponds to the lost position (hereinafter also referred to as the "first imaging target area 3A immediately before the lost position"). If the time required for the flight imaging device 1 to reach the first imaging target area 3A immediately before the lost position after starting the re-imaging process is T4, the required time T4 is determined by the following equation (3), for example, when the flight imaging device 1 moves at a constant speed.

[0101]

[0102] where T1 is the first predetermined time. As described above, the first predetermined time is set to, for example, the time from when the first imaging target area 3A is imaged until the first overlap amount reaches the lower limit of the first predetermined range. Furthermore, M is a natural number equal to or greater than 2 that indicates the order of imaging, counting from the first imaging target area 3, of the second imaging target area 3B corresponding to the lost position.

[0103] When the arrival determination unit 104 determines that the flight imaging device 1 has reached the first imaging target area 3A immediately before the lost position, the fifth imaging control unit 106 outputs a fifth imaging instruction signal to the image sensor 34, causing the image sensor 34 to capture an image of the second imaging target area 3B. This provides overlap determination image data.

[0104] 16 , the third overlap determination unit 108 determines whether a first overlap amount, which is the amount of overlap between a portion of the first synthesis image 92A specified by the first image information and a portion of the overlap determination image 94, is within a first predetermined range. The first predetermined range is the same as the first predetermined range in the imaging process.

[0105] 16 shows an example in which the first overlap amount is within the first predetermined range. When the third overlap determination unit 108 determines that the first overlap amount is within the first predetermined range, the sixth imaging control unit 110 executes overlap imaging processing. The overlap imaging processing is similar to the overlap imaging processing in the imaging processing. That is, the sixth imaging control unit 110 outputs a sixth imaging instruction signal to the image sensor 34, causing the image sensor 34 to capture an image of the second imaging target area 3B. This results in the acquisition of second compositing image data representing the second compositing image 92B corresponding to the lost position.

[0106] 17 , the fourth overlap determination unit 112 determines whether a second overlap amount, which is the overlap amount between a portion of the first combination image 92A and a portion of the second combination image 92B, is within a second predetermined range. The second overlap amount is the same as the second overlap amount in the image capture process, and the second predetermined range is the same as the second predetermined range in the image capture process.

[0107] 17 shows an example in which the second overlap amount is within the second predetermined range. When the fourth overlap determination unit 112 determines that the second overlap amount is within the second predetermined range, the third image storage control unit 114 outputs the second combining image data to the storage 44. As a result, the second combining image data is stored in the storage 44.

[0108] 18 shows an example in which the second overlap amount is outside the second predetermined range (specifically, the second overlap amount is below the lower limit of the second predetermined range). When the fourth overlap determination unit 112 determines that the second overlap amount is not within the second predetermined range, the notification control unit 120 performs notification processing. Examples of the notification processing include activating a notification device (not shown) provided in the imaging device 30 and / or the transmitter 20. Examples of the notification device include a speaker, a light, or a display device. Examples of the notification content provided by the notification device include urging the user to retry the imaging process.

[0109] 19 as an example, the second information acquisition unit 116 acquires second image information about the third compositing image 92C from the lost information stored in the storage 44. Based on the second image information about the third compositing image 92C, for example, the third compositing image 92C corresponding to the third imaging target area 3C captured by the interval imaging process is identified.

[0110] The fifth overlap determination unit 118 determines whether the area of ​​the overlap region where a portion of the second compositing image 92B and a portion of the third compositing image 92C overlap (hereinafter referred to as the "third overlap amount") is within a third predetermined range. The third overlap amount is the same as the second overlap amount in the imaging process, and the third predetermined range is the same as the second predetermined range in the imaging process.

[0111] 18 shows an example in which the third overlap amount is within the third predetermined range. When the third overlap amount is within the third predetermined range, the second imaging target area 3B between the first imaging target area 3A and the third imaging target area 3C is fully imaged. Therefore, in this case, the re-imaging process ends.

[0112] On the other hand, if the third overlap amount is outside the third predetermined range, the fifth overlap determination unit 118 determines that the third overlap amount is not within the third predetermined range. In this case, the second imaging target area 3B that is not imaged exists between the first imaging target area 3A and the third imaging target area 3C. Therefore, in this case, the processing by the fifth imaging control unit 106, the third overlap determination unit 108, the sixth imaging control unit 110, the fourth overlap determination unit 112, the third image storage control unit 114, the second information acquisition unit 116, and the fifth overlap determination unit 118 is executed again.

[0113] In addition, if the fifth overlap determination unit 118 determines that the third overlap amount is not within the third predetermined range, the second imaging target area 3B imaged under the control of the sixth imaging control unit 110 described above will thereafter be treated as the first imaging target area 3A, and the second synthesis image data obtained by imaging the second imaging target area 3B under the control of the sixth imaging control unit 110 will be treated as the first synthesis image data.

[0114] Next, the operation of the flight imaging device 1 according to this embodiment will be described with reference to Fig. 20 and Fig. 21. Fig. 20 shows an example of the flow of imaging processing according to this embodiment, and Fig. 21 shows an example of the flow of re-imaging processing according to this embodiment. First, the imaging processing shown in Fig. 20 will be described.

[0115] 20 , first, in step ST10, the first imaging control unit 52 causes the image sensor 34 to capture an image of the first imaging target area 3A, which is the first imaging target area 3 (see FIG. 5 ). This results in first synthesis image data representing a first synthesis image 92A. After the processing of step ST10 is executed, the imaging processing proceeds to step ST12.

[0116] In step ST12, the second imaging control unit 54 causes the image sensor 34 to capture an image of the second imaging target area 3B (see FIG. 5 ). This produces overlap determination image data representing the overlap determination image 94. After the processing of step ST12 is executed, the imaging processing proceeds to step ST14.

[0117] In step ST14, the first overlap determination unit 56 determines whether a first overlap amount, which is the overlap amount between a portion of the first synthesis image 92A obtained in step ST10 and a portion of the overlap determination image 94 obtained in step ST12, is within a first predetermined range (see FIG. 5 ). If the first overlap amount is outside the first predetermined range in step ST14, the determination is negative, and the imaging process proceeds to step ST16. If the first overlap amount is within the first predetermined range in step ST14, the determination is positive, and the imaging process proceeds to step ST18.

[0118] In step ST16, the lost determination unit 58 determines whether the elapsed time since the first timing at which the first imaging target area 3A was imaged in step ST10 has exceeded a first predetermined time (see FIG. 6 ). If the elapsed time has not exceeded the first predetermined time in step ST16, the determination is negative, and the imaging process proceeds to step ST12. If the elapsed time has exceeded the first predetermined time in step ST16 (i.e., if the overlap imaging process for the second imaging target area 3B has failed), the determination is positive, and the imaging process proceeds to step ST24.

[0119] In step ST18, the third imaging control unit 60 causes the image sensor 34 to capture an image of the second imaging target area 3B (see FIG. 7). This executes overlap imaging processing, and second synthesis image data representing the second synthesis image 92B is obtained. After the processing of step ST18 is executed, the imaging processing proceeds to step ST20.

[0120] In step ST20, the second overlap determination unit 62 determines whether a second overlap amount, which is the overlap amount between a portion of the first compositing image 92A obtained in step ST10 and a portion of the second compositing image 92B obtained in step ST18, is within a second predetermined range (see FIG. 8 ). If the second overlap amount is within the second predetermined range in step ST20, the determination is affirmative, and the imaging process proceeds to step ST22. If the second overlap amount is outside the second predetermined range in step ST20 (i.e., if the overlap imaging process for the second imaging target area 3B has failed), the determination is negative, and the imaging process proceeds to step ST24.

[0121] In step ST22, the first image storage control unit 64 stores the second combining image data obtained in step ST18 in the storage 44 (see FIG. 8 ). After the processing of step ST22 is executed, the imaging processing proceeds to step ST12. When the second combining image data is stored in the storage 44 in step ST22, thereafter, the second imaging target area 3B imaged in step ST18 is treated as the first imaging target area 3A, and the second combining image data obtained in step ST18 is treated as the first combining image data.

[0122] In step ST24, the interval image capture determination unit 66 determines whether the elapsed time has reached the second predetermined time (see FIGS. 9 and 10). If the elapsed time has not reached the second predetermined time in step ST24, the determination is negative, and the process of step ST24 is executed again. If the elapsed time has reached the second predetermined time in step ST24, the determination is positive, and the image capture process proceeds to step ST26.

[0123] In step ST26, the fourth imaging control unit 68 causes the image sensor 34 to capture an image of the third imaging target area 3C (see FIG. 11 ). This executes interval imaging processing, and third synthesis image data representing the third synthesis image 92C is obtained. After the processing of step ST26 is executed, the imaging processing proceeds to step ST28.

[0124] In step ST28, the image quality determination unit 70 determines whether the third compositing image 92C obtained in step ST26 satisfies the default image quality (see FIG. 12 ). If the third compositing image 92C satisfies the default image quality in step ST26, the determination is affirmative, and the imaging process proceeds to step ST30. If the third compositing image 92C does not satisfy the default image quality in step ST28 (i.e., if the interval imaging process for the third imaging target area 3C has failed), the determination is negative, and the imaging process proceeds to step ST24.

[0125] In step ST30, the second image storage control unit 72 stores the third synthesis image data obtained in step ST26 in the storage 44 (see FIG. 12 ). After the processing of step ST30 is executed, the imaging processing proceeds to step ST32.

[0126] In step ST32, the lost information storage control unit 74 acquires position information regarding the position of the second imaging target area 3B corresponding to the lost position when the overlap imaging process or the interval imaging process fails, first image information regarding the first compositing image 92A obtained in step ST10 or step ST18, and second image information regarding the third compositing image 92C obtained in step ST26 (see FIG. 12 ). Then, the lost information storage control unit 74 generates lost information by associating the position information with the first image information and the second image information, and stores the lost information in the storage 44 (see FIG. 12 ). If the lost information is stored in the storage 44 in step ST32, the third imaging target area 3C captured in step ST26 is subsequently treated as the first imaging target area 3A, and the third compositing image data obtained in step ST26 is subsequently treated as the first compositing image data.

[0127] In step ST34, the processor 42 determines whether a condition for terminating the imaging process (termination condition) is met. Examples of the termination condition include a condition that the user has given the imaging device 30 an instruction to terminate the imaging process, or a condition that the number of compositing images 92 has reached the number specified by the user. If the termination condition is not met in step ST34, the determination is negative, and the imaging process proceeds to step ST12. If the termination condition is met in step ST34, the determination is positive, and the imaging process ends.

[0128] Next, the re-imaging process shown in FIG. 21 will be described.

[0129] 21 , first, in step ST40, the first information acquisition unit 102 acquires, from the lost information stored in the storage 44, position information on the position of the second imaging target area 3B corresponding to the lost position and first image information on the first synthesis image 92A corresponding to the first imaging target area 3A immediately preceding the lost position (see FIG. 15 ). After the processing of step ST40 is executed, the re-imaging processing proceeds to step ST42.

[0130] In step ST42, the arrival determination unit 104 determines whether the flight imaging device 1 has reached the first imaging target area 3A immediately before the lost position (see FIG. 15). In step ST42, if the flight imaging device 1 has not reached the first imaging target area 3A immediately before the lost position, the processing of step ST42 is executed again. In step ST42, if the flight imaging device 1 has reached the first imaging target area 3A immediately before the lost position, the re-imaging processing proceeds to step ST44.

[0131] In step ST44, the fifth imaging control unit 106 causes the image sensor 34 to capture an image of the second imaging target area 3B. This produces overlap determination image data representing the overlap determination image 94. After the processing of step ST44 is executed, the re-imaging processing proceeds to step ST46.

[0132] In step ST46, the third overlap determination unit 108 determines whether the first overlap amount, which is the overlap amount between the portion of the first synthesis image 92A identified by the first image information obtained in step ST40 and the portion of the overlap determination image 94 obtained in step ST44, is within a first predetermined range (see FIG. 16 ). If the first overlap amount is outside the first predetermined range in step ST46, the determination is negative, and the re-imaging process proceeds to step ST44. If the first overlap amount is within the first predetermined range in step ST46, the determination is positive, and the re-imaging process proceeds to step ST48.

[0133] In step ST48, the sixth imaging control unit 110 causes the image sensor 34 to capture an image of the second imaging target area 3B (see FIG. 16 ). This executes overlap imaging processing, and second synthesis image data representing the second synthesis image 92B is obtained. After the processing of step ST48 is executed, the re-imaging processing proceeds to step ST50.

[0134] In step ST50, the fourth overlap determination unit 112 determines whether a second overlap amount, which is the overlap amount between a portion of the first compositing image 92A identified by the first image information obtained in step ST40 and a portion of the second compositing image 92B obtained in step ST48, is within a second predetermined range (see FIG. 17 ). If the second overlap amount is within the second predetermined range in step ST50, the determination is affirmative, and the re-imaging process proceeds to step ST52. If the second overlap amount is outside the second predetermined range in step ST50 (i.e., if the overlap imaging process for the second imaging target area 3B has failed), the determination is negative, and the re-imaging process proceeds to step ST58.

[0135] In step ST52, the third image storage control unit 114 stores the second combining image data obtained in step ST48 in the storage 44 (see FIG. 17 ). After the processing of step ST52 is executed, the re-imaging processing proceeds to step ST54.

[0136] In step ST54, the second information acquisition unit 116 acquires second image information related to the third synthesis image 92C from the lost information stored in the storage 44 (see FIG. 19 ). After the processing of step ST54 is executed, the re-imaging processing proceeds to step ST56.

[0137] In step ST56, the fifth overlap determination unit 118 determines whether a third overlap amount, which is the overlap amount between a portion of the second compositing image 92B obtained in step ST48 and a portion of the third compositing image 92C identified by the second image information obtained in step ST54, is within a third predetermined range (see FIG. 19 ). If the third overlap amount is outside the third predetermined range in step ST56, the determination is negative, and the re-imaging process proceeds to step ST44. If the determination is negative in step ST56 and the re-imaging process proceeds to step ST44, the second imaging target area 3B captured in step ST48 is treated as the first imaging target area 3A, and the second compositing image data obtained in step ST48 is treated as the first compositing image data. If the third overlap amount is within the third predetermined range in step ST50, the determination is positive, and the re-imaging process ends.

[0138] In step ST58, the notification control unit 120 performs notification processing (see FIG. 18). After the processing in step ST58 is executed, the re-imaging processing ends.

[0139] The imaging control method described above as the operation of the flying imaging device 1 is an example of an "imaging control method" according to the technology of the present disclosure.

[0140] As described above, in the flight imaging device 1 according to this embodiment, the processor 42 causes the imaging device 30 to capture the first imaging target area 3A. If a portion of the second imaging target area 3B overlaps with a portion of the first imaging target area 3A during the flight imaging device 1's movement, the processor 42 performs an overlap imaging process in which the imaging device 30 captures the second imaging target area 3B (see FIGS. 5 to 8). If the overlap imaging process fails, the processor 42 performs an interval imaging process in which the imaging device 30 captures the third imaging target area 3C, provided that the distance traveled by the flight imaging device 1 from the first position where the first imaging target area 3A was captured by the imaging device 30 reaches a first predetermined distance (see FIGS. 9 to 11). Therefore, even if the overlap imaging process fails, the third imaging target area 3C can still be captured. In other words, even if the overlap imaging process fails, the imaging process can continue.

[0141] A case where the overlapping imaging process has failed includes a case where the second imaging target area 3B has not been imaged by the imaging device 30 (i.e., before the second imaging target area 3B has been imaged) and the travel distance of the flying imaging device 1 exceeds the distance from the first position to the second position where the second imaging target area 3B is imaged (see FIG. 9). Therefore, even if the overlapping imaging process has failed because the travel distance exceeds the distance from the first position to the second position, it is possible to image the third imaging target area 3C.

[0142] A case where the overlap imaging process has failed includes a case where the second imaging target area 3B is imaged by the imaging device 30, and the second overlap amount, which is the overlap amount between a portion of the first compositing image 92A obtained by imaging the first imaging target area 3A and a portion of the second compositing image 92B obtained by imaging the second imaging target area 3B, is outside the second predetermined range (see FIG. 10 ). Therefore, even if the overlap imaging process has failed because the second overlap amount is outside the second predetermined range, it is possible to image the third imaging target area 3C.

[0143] A portion of the third imaging target area 3C overlaps a portion of the second imaging target area 3B (see FIG. 11 ). Therefore, for example, when the second compositing image 92B is obtained by capturing an image of the second imaging target area 3B in a re-imaging process (see FIG. 17 ), a portion of the third compositing image 92C corresponding to the third imaging target area 3C can be made to overlap with a portion of the second compositing image 92B corresponding to the second imaging target area 3B.

[0144] The first predetermined movement distance is the distance from the first position to the third position where a part of the third imaging target area 3C overlaps a part of the second imaging target area 3B (see FIG. 11 ). Therefore, when the interval imaging process is performed on the condition that the movement distance of the flight imaging device 1 from the first position reaches the first predetermined movement distance, the third imaging target area 3C can be imaged so that a part of the third imaging target area 3C overlaps a part of the second imaging target area 3B.

[0145] The first predetermined movement distance is a natural number multiple of 2 or greater than the distance from the first position to the second position where the second imaging target area 3B is imaged. Here, for example, when the natural number is 2 (see FIG. 11 ), when interval imaging processing is performed, a space equivalent to one second imaging target area 3B exists between the first imaging target area 3A and the third imaging target area 3C. Therefore, for example, in the re-imaging processing, the second imaging target area 3B is imaged at the second position, thereby obtaining multiple composite images 92 that are continuous in the movement direction of the flight imaging device 1.

[0146] Furthermore, for example, when the natural number greater than or equal to 2 is 3 (see FIG. 13 ), when interval imaging processing is performed, there is a space between the first imaging target area 3A and the third imaging target area 3C, which is the equivalent of two imaging target areas 3. Therefore, for example, in the re-imaging processing, the imaging target area 3 is imaged at the second position and the third position (i.e., a position separated from the second position by the distance from the first position to the second position), thereby making it possible to obtain a plurality of compositing images 92 that are continuous in the direction of movement of the flight imaging device 1.

[0147] The overlapping imaging process is performed on the condition that a first overlap amount, which is the amount of overlap between a portion of the first compositing image 92A obtained by imaging the first imaging target area 3A and a portion of the overlap determination image 94 obtained by imaging the second imaging target area 3B, is within a first predetermined range (see FIG. 7 ). Therefore, a second overlap amount, which is the amount of overlap between a portion of the second compositing image 92B obtained by the overlapping imaging process and a portion of the first compositing image 92A, can be kept within a second predetermined range.

[0148] The fact that the travel distance has reached the first preset travel distance is determined on the condition that, when the flight imaging device 1 is moving at a constant speed, the time that has elapsed since the first timing at which the imaging device 30 captured an image of the first imaging target area 3A reaches a first preset time (see FIG. 11 ). Therefore, interval imaging processing can be performed based on the time that has elapsed since the first timing.

[0149] When the overlapping imaging process fails, the processor 42 acquires position information regarding the position of the second imaging target area 3B, first image information regarding the first compositing image 92A, and second image information regarding the third compositing image 92C (see FIG. 12 ). The position information regarding the position of the second imaging target area 3B is associated with the first image information and the second image information and stored in the storage 44. Therefore, by executing the re-imaging process based on the position information, the first image information, and the second image information, it is possible to obtain multiple compositing images 92 that are continuous in the movement direction of the flying imaging device 1.

[0150] In the above embodiment, the lost determination unit 58 determines whether the elapsed time from the first timing at which the first imaging target area 3A was imaged exceeds a first predetermined time (see FIG. 9 ). However, as an example shown in FIG. 22 , the lost determination unit 58 may determine whether the travel distance of the flying imaging device 1 from the first position corresponding to the first timing exceeds a third predetermined travel distance.

[0151] The travel distance is calculated based on the travel speed and elapsed time of the flying imaging device 1. The travel speed of the flying imaging device 1 is calculated based on, for example, the acceleration indicated by acceleration data input to the processor 42 from the acceleration sensor 80 mounted on the imaging device 30 (i.e., the acceleration measured by the acceleration sensor 80). The acceleration sensor 80 is an example of an "acceleration sensor" according to the technology of the present disclosure.

[0152] The third predetermined movement distance is the distance from the first position to the second position. As described above, the second position indicates the position of the center of the flying imaging device 1 when the overlap amount between a portion of the first synthesis image 92A and a portion of the overlap determination image 94 reaches the lower limit of the first predetermined range (see FIG. 7 ). If the movement time exceeds the third predetermined movement distance, the imaging device 30 loses an opportunity to capture the second imaging target area 3B, and therefore fails the overlap imaging process.

[0153] In this way, when the travel distance is derived based on the acceleration measured by the acceleration sensor 80, it is determined whether or not the overlapping image capture process has failed by taking into account the change in acceleration of the flight imaging device 1. Therefore, it is possible to improve the accuracy of the determination compared to, for example, when it is determined whether or not the overlapping image capture process has failed without taking into account the change in acceleration of the flight imaging device 1.

[0154] In the above embodiment, the interval image capture determination unit 66 determines whether the elapsed time from the first timing at which the first image capture target area 3A was captured has reached a second predetermined time (see FIG. 11 ). However, as an example shown in FIG. 23 , the interval image capture determination unit 66 may determine whether the travel distance of the flight image capture device 1 from the first position corresponding to the first timing has reached a first predetermined travel distance.

[0155] The first predetermined movement distance is the distance from the first position to the third position. The third position indicates the position of the center of the flight imaging device 1 when, assuming that the second compositing image 92B is obtained by capturing an image of the second imaging target area 3B, the second overlap amount between the part of the third compositing image 92C corresponding to the third imaging target area 3C and the part of the second compositing image 92B reaches the upper limit of the second predetermined range. If the interval imaging determination unit 66 determines that the movement distance exceeds the first predetermined movement distance, the interval imaging process is executed.

[0156] In this way, when the travel distance is derived based on the acceleration measured by the acceleration sensor 80, it is determined whether or not to perform interval imaging processing by taking into account the change in acceleration of the flight imaging device 1. Therefore, it is possible to improve the accuracy of the determination compared to, for example, when it is determined whether or not to perform interval imaging processing without taking into account the change in acceleration of the flight imaging device 1.

[0157] 22 and 23, the acceleration sensor 80 is mounted on the imaging device 30, but it may also be mounted on the flying object 10. Alternatively, the acceleration sensor 80 may be mounted on each of the flying object 10 and the imaging device 30, and the moving speed may be derived based on the average value of the acceleration measured by each acceleration sensor 80.

[0158] 22 and 23, the movement speed of the flight imaging device 1 is derived based on the acceleration measured by the acceleration sensor 80. However, as shown in FIG. 24, the processor 42 may operate as a movement speed derivation unit 76, and the movement speed derivation unit 76 may derive the movement speed of the flight imaging device 1 based on the first synthesis image 92A and the overlap determination image 94.

[0159] For example, the movement speed of the flying imaging device 1 is derived as follows: First, the movement distance (hereinafter referred to as the "inter-image movement distance") of a feature point commonly included in the first compositing image 92A and the overlap determination image 94 from a position included in the first compositing image 92A to a position included in the overlap determination image 94 is derived. The first compositing image 92A and the overlap determination image 94 are examples of a "sixth image" according to the technology of the present disclosure.

[0160] Next, based on the focal length when the first synthesis image 92A and the overlap determination image 94 are obtained and the inter-image movement distance, the movement distance (hereinafter referred to as the "inter-area movement distance") of the second imaging target area 3B corresponding to the overlap determination image 94 relative to the first imaging target area 3A corresponding to the first synthesis image 92A is derived. Also, the time interval (hereinafter referred to as the "inter-image time interval") when the first synthesis image 92A and the overlap determination image 94 are obtained is derived. Then, based on the inter-area movement distance and the inter-image time interval, the movement speed of the flying imaging device 1 is derived.

[0161] 24 , the lost determination unit 58 may acquire the moving speed derived by the moving speed derivation unit 76, and derive the moving distance of the flight imaging device 1 based on the moving speed and the elapsed time. Then, the lost determination unit 58 may determine whether the moving distance exceeds a third predetermined moving distance.

[0162] In this way, even when the movement speed of the flight imaging device 1 is derived based on the first synthesis image 92A and the overlap determination image 94, whether or not the overlap imaging process has failed is determined taking into account the acceleration change of the flight imaging device 1. Therefore, for example, the accuracy of the determination can be improved compared to when whether or not the overlap imaging process has failed is determined without taking into account the acceleration change of the flight imaging device 1.

[0163] 25 , the interval image capture determination unit 66 may acquire the movement speed derived by the movement speed derivation unit 76, and derive the movement distance of the flight image capture device 1 based on the movement speed and the elapsed time. Then, the interval image capture determination unit 66 may determine whether the movement distance has reached the first predetermined movement distance.

[0164] In this way, even when the movement speed of the flight imaging device 1 is derived based on the first synthesis image 92A and the overlap determination image 94, a determination is made as to whether or not to perform interval imaging processing taking into account changes in acceleration of the flight imaging device 1. Therefore, for example, the accuracy of the determination can be improved compared to when a determination is made as to whether or not to perform interval imaging processing without taking into account changes in acceleration of the flight imaging device 1.

[0165] 24 and 25 , the processor 42 may output moving speed data indicating the moving speed to the transmitter 20. The transmitter 20 may display the moving speed indicated by the moving speed data input from the imaging device 30 on the display device 24.

[0166] In this way, when the movement speed is displayed on the display device 24, the user can adjust the movement speed of the flight imaging device 1 based on the movement speed displayed on the display device 24. The movement speed data is an example of "movement speed data" according to the technology of the present disclosure. The first synthesis image 92A and the overlap determination image 94 are an example of a "ninth image" according to the technology of the present disclosure.

[0167] In the above embodiment, the flying imaging device 1 is assumed to alternately move horizontally and vertically to capture images of multiple imaging target areas 3 while moving in a zigzag pattern (see FIG. 1). However, the flying imaging device 1 may also capture images of the wall surface 2A of the target object 2 while moving in any direction along the wall surface 2A.

[0168] In addition, although the above embodiment gives an example in which the imaging device 30 is mounted on the flying body 10, the imaging device 30 may also be mounted on various moving bodies (e.g., a gondola, an automatic transport robot, an unmanned transport vehicle, or a high-altitude inspection vehicle), etc.

[0169] Furthermore, in the above embodiment, the composite image 92 is stored in the storage 44, but may be stored in a storage medium other than the storage 44. Furthermore, in the above embodiment, the lost information is stored in the storage 44, but may be stored in a storage medium other than the storage 44. The storage medium may be provided in a device other than the flight imaging device 1 (for example, a server and / or a personal computer, etc.). Examples of the storage medium include computer-readable non-transitory storage media such as a USB memory, an SSD, an HDD, an optical disk, and a magnetic tape.

[0170] Furthermore, in each of the above embodiments, processor 42 is exemplified, but instead of processor 42, or together with processor 42, at least one other CPU, at least one GPU, and / or at least one TPU may be used.

[0171] Furthermore, in the above-described embodiments, an example has been described in which the imaging program 50 and the re-imaging program 100 are stored in the storage 44, but the technology of the present disclosure is not limited to this. For example, the imaging program 50 and / or the re-imaging program 100 may be stored in a storage medium other than the storage 44. Furthermore, the imaging program 50 and / or the re-imaging program 100 stored in a storage medium may be installed in the computer 32 of the imaging device 30.

[0172] Alternatively, the imaging program 50 and / or the re-imaging program 100 may be stored in a storage device such as another computer or server device connected to the imaging device 30 via a network, and the imaging program 50 and / or the re-imaging program 100 may be downloaded and installed on the computer 32 in response to a request from the imaging device 30.

[0173] Furthermore, it is not necessary to store the entire imaging program 50 and / or the re-imaging program 100 in a storage device such as another computer or server device connected to the imaging device 30, or in the storage 44; only a part of the imaging program 50 and / or the re-imaging program 100 may be stored therein.

[0174] Furthermore, although the imaging device 30 has a built-in computer 32 , the technology of the present disclosure is not limited to this. For example, the computer 32 may be provided outside the imaging device 30 .

[0175] In addition, in each of the above embodiments, the computer 32 includes the processor 42, the storage 44, and the RAM 46, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 32. Furthermore, a combination of a hardware configuration and a software configuration may be used instead of the computer 32.

[0176] Furthermore, the hardware resources that execute the various processes described in the above embodiments can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource that executes various processes by executing software, i.e., a program. Examples of processors include dedicated electronic circuits, such as FPGAs, PLDs, and ASICs, which are processors with a circuit configuration specifically designed to execute specific processes. Each processor has built-in or connected memory, and each processor executes various processes by using the memory.

[0177] The hardware resources that execute various processes may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resources that execute various processes may be a single processor.

[0178] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes various processes. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute various processes, on a single IC chip, as typified by SoC. In this way, various processes are realized using one or more of the above-mentioned various processors as hardware resources.

[0179] Furthermore, the hardware structure of these various processors can be, more specifically, electronic circuits that combine circuit elements such as semiconductor devices. The above-described gaze detection process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0180] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0181] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0182] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A processor is provided. The processor, causing an imaging device to capture an image of a first imaging target region; performing overlapping imaging processing for causing the imaging device to capture an image of the second imaging target area when a part of the second imaging target area overlaps a part of the first imaging target area during a process in which the moving body on which the imaging device is mounted moves; performing an interval imaging process for causing the imaging device to image a third imaging target area on condition that a moving distance of the moving body from a first position where the first imaging target area is imaged by the imaging device reaches a first preset moving distance when the overlap imaging process has failed; if the overlapping imaging process fails, acquiring position information on a position of the second imaging target area, first image information on a seventh image obtained by imaging the first imaging target area by the imaging device, and second image information on an eighth image obtained by imaging the third imaging target area by the imaging device; Position information regarding the position of the second imaging target area is stored in a memory in association with at least one of the first image information and the second image information. Imaging control device.

2. The case where the overlap imaging process has failed includes a case where the second imaging target area is not imaged by the imaging device and the movement distance exceeds a distance from the first position to a second position where the second imaging target area is imaged by the imaging device. The imaging control device according to claim 1 .

3. The case where the overlap imaging process has failed includes a case where the second imaging target area is imaged by the imaging device, and a first overlap amount between a part of a first image obtained by imaging the first imaging target area and a part of a second image obtained by imaging the second imaging target area is outside a first predetermined range. The imaging control device according to claim 1 .

4. The case where the overlap imaging process has failed includes a case where the third image obtained by imaging the second imaging target area by the imaging device does not satisfy a predetermined image quality. The imaging control device according to claim 1 .

5. A portion of the third imaging target area overlaps a portion of the second imaging target area. The imaging control device according to claim 1 .

6. The first predetermined movement distance is a distance from the first position to a third position where a portion of the third imaging target area overlaps a portion of the second imaging target area. The imaging control device according to claim 1 .

7. The first predetermined moving distance is a natural number multiple of a distance from the first position to a fourth position where the second imaging target area is imaged by the imaging device. The imaging control device according to claim 1 .

8. The overlapping imaging process is performed on the condition that a second overlap amount between a part of a fourth image obtained by imaging the first imaging target area with the imaging device and a part of a fifth image obtained by imaging the second imaging target area is within a second predetermined range. The imaging control device according to claim 1 .

9. The moving distance is derived based on acceleration measured by an acceleration sensor mounted on the imaging device and / or the moving object. The imaging control device according to claim 1 .

10. The fact that the moving distance has reached the first preset moving distance is determined on the condition that, when the moving body moves at a constant speed, the time that has elapsed since the first timing at which the first imaging target area is imaged by the imaging device reaches a first preset time. The imaging control device according to claim 1 .

11. The moving distance is derived based on a moving speed of the moving object derived based on a plurality of sixth images obtained by capturing images by the imaging device, and on a time interval when the plurality of sixth images are captured. The imaging control device according to claim 1 .

12. The processor, Acquire a moving speed of the moving object; outputting moving speed data indicating the moving speed; The moving speed is derived based on a plurality of ninth images obtained by capturing images by the imaging device. The imaging control device according to claim 1 .

13. Having an imaging device capture an image of a first imaging target area; performing overlapping imaging processing for causing the imaging device to capture an image of the second imaging target area when a part of the second imaging target area overlaps a part of the first imaging target area during a process in which the moving body on which the imaging device is mounted moves; performing an interval imaging process for causing the imaging device to image a third imaging target area on condition that a moving distance of the moving body from a first position where the first imaging target area is imaged by the imaging device reaches a first preset moving distance when the overlap imaging process fails; and When the overlapping imaging process fails, acquiring position information on the position of the second imaging target area, first image information on a seventh image obtained by imaging the first imaging target area by the imaging device, and second image information on an eighth image obtained by imaging the third imaging target area by the imaging device. Equipped with Position information regarding the position of the second imaging target area is stored in a memory in association with at least one of the first image information and the second image information. Imaging control method.

14. A program for causing a computer to execute processing, comprising: The process comprises: causing an imaging device to capture an image of a first imaging target region; performing overlapping imaging processing for causing the imaging device to capture an image of the second imaging target area when a part of the second imaging target area overlaps a part of the first imaging target area during a process in which the moving body on which the imaging device is mounted moves; performing an interval imaging process for causing the imaging device to image a third imaging target area on condition that a moving distance of the moving body from a first position where the first imaging target area is imaged by the imaging device reaches a first preset moving distance when the overlap imaging process fails; and When the overlapping imaging process fails, acquiring position information on the position of the second imaging target area, first image information on a seventh image obtained by imaging the first imaging target area by the imaging device, and second image information on an eighth image obtained by imaging the third imaging target area by the imaging device. Including, Position information regarding the position of the second imaging target area is stored in a memory in association with at least one of the first image information and the second image information. program.

Images