Method and device for merging images

Abstract

A method for merging images according to some embodiments of the present invention may comprise the steps of: acquiring static images by photographing at least a part of an object by means of a camera of an aerial vehicle; generating partial images by merging at least two of the static images by using at least one feature point included in each of the static images; and generating a block image by merging at least two of the partial images by using at least one feature point included in each of the partial images.

Description

Method and device for merging images

[0001] It relates to a method and device for merging images.

[0002]

[0003] Aerial vehicles (AVs) fly to approach a target, and various information about the target can be collected through the vehicle's camera. Aerial photography technology is being integrated into various fields and is even being used for the inspection of industrial structures. When the target is an industrial structure, the aerial vehicle's camera can capture images of the exterior or interior, allowing for the identification of any malfunctions or damage to the structure.

[0004] At this time, there is an increasing demand for technology that merges and outputs images captured by aircraft to make it easier for users to check and manage them.

[0005] Meanwhile, when an aircraft photographs a subject, it is necessary to capture images with overlapping layers to avoid missing any portion of the subject. Furthermore, stitching multiple images requires a high overlap rate. When an aircraft photographs a subject with a high overlap rate, multiple images are generated. These images may contain unnecessary information. Furthermore, storing multiple images requires more storage space, and processing them requires more resources (GPU, CPU, etc.).

[0006]

[0007] The present invention provides a method and device for merging images. Furthermore, the present invention provides a computer-readable recording medium containing a program for executing the method on a computer. The technical problems to be solved are not limited to the technical problems described above, and other technical problems may exist.

[0008]

[0009] A method for merging images according to some embodiments of the present invention may include the steps of: acquiring static images by photographing at least a portion of an object through a camera of an aircraft; generating partial images by merging at least two or more of the static images using at least one feature point included in each of the static images; and generating a block image by merging at least two or more of the partial images using at least one feature point included in each of the partial images.

[0010] In addition, the step of photographing may be configured to photograph the object by changing the angle of the camera while the aircraft is positioned at the n-th point, and the step of generating the partial images may be configured to merge static images photographed at the n-th point to generate one partial image, where n may be a natural number greater than or equal to 1.

[0011] In addition, the step of generating the partial images may include the steps of identifying at least one common feature point in a first static image and a second static image selected from among the static images, determining at least one candidate solution for matching the at least one common feature point, and merging the first static image and the second static image based on an optimal solution selected from among the at least one candidate solution.

[0012] Additionally, the candidate solution may include a method of aligning the at least one common feature point by at least one of translation, rotation and scaling of at least one of the first static image and the second static image.

[0013] Additionally, the optimal solution may be selected from among the at least one candidate solution based on at least one preset criterion.

[0014] In addition, the identifying step may include a step of identifying a predetermined object in each of the static images using a machine learning model, and a step of identifying at least one common feature point in a portion of each of the static images excluding the predetermined object.

[0015] In addition, the step of generating the block image may include the steps of identifying at least one common feature point in a first partial image and a second partial image selected from among the partial images, determining at least one candidate solution for matching the at least one common feature point, and merging the first partial image and the second partial image based on an optimal solution selected from among the at least one candidate solution.

[0016] Additionally, the static images can be obtained through one flight of the aircraft.

[0017] Additionally, the partial images may be generated from static images acquired through at least one flight of the aircraft.

[0018] In addition, the method may further include a step of matching at least one block image to an item corresponding to the target object and a step of outputting at least one defect (default) identified from at least one of the static images on the item.

[0019] In addition, the method may further include a step of determining at least one of the partial image and the block image as a merged image, a step of determining a duplicate image among the static images using stitching information, and a step of removing the determined duplicate image.

[0020] A computer-readable recording medium according to some embodiments of the present invention may include a recording medium having recorded thereon a program for executing the above-described method on a computer.

[0021] A device for merging images according to some embodiments of the present invention comprises at least one memory and at least one processor, wherein the at least one processor is configured to acquire static images by photographing at least a portion of an object through a camera of an aircraft, generate partial images by merging at least two or more of the static images using at least one feature point included in each of the static images, and generate a block image by merging at least two or more of the partial images using at least one feature point included in each of the partial images.

[0022] In addition, the at least one processor controls the camera to change the angle of the camera to capture the object while the aircraft is positioned at a point n, where n may be a natural number greater than or equal to 1.

[0023] Additionally, the at least one processor may identify at least one common feature point in a first static image and a second static image selected from among the static images, determine at least one candidate solution for matching the at least one common feature point, and merge the first static image and the second static image based on an optimal solution selected from among the at least one candidate solution.

[0024] Additionally, the candidate solution may include a method of aligning the at least one common feature point by at least one of translation, rotation and scaling of at least one of the first static image and the second static image.

[0025] In addition, the optimal solution may be selected from among the at least one candidate solution based on at least one preset criterion, and a candidate solution that matches with a rotation angle exceeding a threshold value may be excluded from among the at least one candidate solution.

[0026] Additionally, the at least one processor can identify a predetermined object in each of the static images using a machine learning model, and identify the at least one common feature point in the remaining portion of each of the static images excluding the predetermined object.

[0027] In addition, the at least one processor may identify at least one common feature point in a first partial image and a second partial image selected from among the partial images, determine at least one candidate solution for matching the at least one common feature point, merge the first partial image and the second partial image based on an optimal solution selected from among the at least one candidate solution, and exclude a candidate solution including a method of matching by enlarging or reducing the at least one candidate solution by exceeding a threshold value.

[0028] Additionally, the static images can be acquired through a single flight of the aircraft.

[0029] Additionally, the partial images may be generated from static images acquired through at least one flight of the aircraft.

[0030] According to some embodiments of the present invention, an aircraft includes at least one camera, at least one memory, and at least one processor, wherein the at least one processor is configured to capture at least a portion of an object through the camera to obtain static images, generate partial images by merging at least two or more of the static images using at least one feature point included in each of the static images, generate a block image by merging at least two or more of the partial images using at least one feature point included in each of the partial images, acquire the static images by changing an angle of the camera in a first direction, generate the partial images by merging the static images in the first direction, and generate the block image by merging the partial images in a second direction perpendicular to the first direction.

[0031]

[0032] FIG. 1 is a drawing for explaining an example of photographing an object using an aircraft according to one embodiment.

[0033] FIG. 2 is a drawing for explaining the relationship between an aircraft, a controller, a server, and a station according to one embodiment.

[0034] Figure 3 is a schematic diagram illustrating an example of an aircraft according to one embodiment.

[0035] FIG. 4 is a flowchart illustrating an example of a method for merging images according to one embodiment.

[0036] FIG. 5 is a diagram illustrating an example of a processor obtaining a static image according to one embodiment.

[0037] FIG. 6 is a diagram illustrating a processor according to another embodiment obtaining a static image.

[0038] FIG. 7 is a flowchart illustrating an example of a processor generating a merged image according to one embodiment.

[0039] FIG. 8 is a flowchart illustrating an example of a processor generating partial images according to one embodiment.

[0040] FIG. 9 is a diagram illustrating an example of a processor identifying common features in images according to one embodiment.

[0041] FIG. 10 is a diagram illustrating an example in which a processor according to one embodiment identifies feature points in only a portion of an image.

[0042] FIG. 11 is a diagram illustrating an example of a processor identifying feature points excluding a portion of an image according to one embodiment.

[0043] FIG. 12 is a diagram illustrating an example of a processor determining a candidate solution according to one embodiment.

[0044] FIG. 13 is a diagram for explaining the relationship between a static image, a partial image, and a block image according to one embodiment.

[0045] FIG. 14 is a flowchart illustrating an example of a processor generating a merged image according to another embodiment.

[0046] FIG. 15 is a flowchart illustrating an example of a processor performing post-processing related to a merged image according to one embodiment.

[0047] FIG. 16 is a flowchart illustrating an example of a processor outputting a defect to an item according to one embodiment.

[0048] FIG. 17 is a diagram illustrating an example of a processor matching a block image and an item according to one embodiment.

[0049] FIG. 18 is a diagram illustrating an example of a processor outputting a defect according to one embodiment.

[0050] FIG. 19 is a diagram illustrating another example in which a processor outputs a defect according to one embodiment.

[0051] FIG. 20 is a diagram illustrating an example in which a processor indicates a defect on an item when the target object is a military facility according to one embodiment.

[0052] FIG. 21 is a flowchart illustrating an example of a processor removing duplicate images according to one embodiment.

[0053] FIG. 22 is a diagram illustrating a method for determining a duplicate image according to one embodiment.

[0054] Figure 23 is a drawing for explaining the result of removing duplicate images according to one embodiment.

[0055] FIG. 24 is a diagram illustrating a method for determining a duplicate image according to one embodiment.

[0056] FIG. 25 is a drawing for explaining a method for removing duplicate images according to one embodiment.

[0057] FIG. 26 is a drawing for explaining a method for removing duplicate images according to one embodiment.

[0058] Fig. 27 is a schematic diagram illustrating an example of a device for merging images according to one embodiment.

[0059]

[0060] The terms used in the examples are selected from widely used, current terms, as much as possible. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, in which case their meanings will be described in detail in the relevant description. Therefore, the terms used in the specification should be defined based on their intended meaning and the overall content of the specification, rather than simply their names.

[0061] When a part of a specification is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0062] Additionally, terms including ordinal numbers, such as "first" or "second," used in the specification may be used to describe various components, but the components should not be limited by the terms. The terms may be used to distinguish one component from another.

[0063] Below, embodiments are described in detail with reference to the attached drawings. However, the embodiments may be implemented in various different forms and are not limited to the examples described herein.

[0064]

[0065] FIG. 1 is a drawing for explaining an example of photographing an object using an aircraft according to one embodiment.

[0066] The aircraft (10) may include any aircraft capable of flight, including a drone, an Unmanned Aerial Vehicle (UAV), an Unmanned Aerial Mobility (UAM), an aircraft, a helicopter, etc.

[0067] The aircraft (10) can fly alone or with multiple aircraft capable of collaborating. Furthermore, the aircraft (10) can also collaborate with other types of devices, such as vehicles and robots. Furthermore, the aircraft (10) can automatically fly around the target object (20) along a predetermined flight path, or can manually fly around the target object (20) under user control.

[0068] The aircraft (10) can capture images of the object (20) while flying around the object (20). For example, the captured images of the object (20) can be used to determine whether the object (20) has a defect (e.g., breakdown, damage, etc.). The user can detect, recognize, and / or identify the object (20) through the aircraft (10).

[0069] As an example, the aircraft (10) can photograph the body or blades of a wind turbine. Specifically, the aircraft (10) can fly around the wind turbine to photograph the body or blades. For example, the aircraft (10) can take off from a station at a starting point and fly to the nose of the wind turbine. Then, the aircraft (10) can start photographing from the nose of the wind turbine and can photograph the blades while flying autonomously (or manually) along the blades. Typically, a wind turbine can include three blades, and the aircraft (10) can photograph the three blades to acquire images. At this time, the images photographed by the aircraft (10) can be utilized for inspection of the blades.

[0070] As another example, the aircraft (10) can photograph the surface of a large building. For example, the aircraft (10) can acquire images by photographing the surface while flying around the perimeter of the large building. In this case, the images captured by the aircraft (10) can be utilized for inspection of the surface of the large building.

[0071] As another example, the aircraft (10) can photograph various structures within a military facility. For example, a military facility may include barbed wire fences, buildings, and exterior walls, and the aircraft (10) can fly over various points within the military facility and photograph the surfaces of the structures. In this case, the images captured by the aircraft (10) can be utilized for inspection of the military facility.

[0072] Although wind turbines, bridges, large buildings, and military facilities are depicted as objects (20) in FIG. 1, the objects are not limited thereto. In other words, any structure having a shape may be applied to the objects (20) without limitation. For example, the objects (20) may be structures in the industrial field. The objects (20) may be structures for power generation (e.g., wind turbines, thermal power plants, hydroelectric power plants, nuclear power plants, solar power plants, etc.), large buildings (e.g., factories, exhibition halls, stadiums, etc.), bridges, dams, power lines, roads, etc., but are not limited thereto. As another example, the objects (20) may be structures that must be detected or information collected in the security and military fields (e.g., barbed wire fences, ammunition depots, exterior walls, etc.). As another example, anything that is difficult for a user to inspect entirely with the naked eye, dangerous, or requires a lot of manpower and cost for inspection may be applied to the objects (20).

[0073]

[0074] FIG. 2 is a drawing for explaining the relationship between an aircraft, a controller, a server, and a station according to one embodiment.

[0075] Referring to FIG. 2, the server (30), the controller (40), and the station (50) can independently or jointly control the aircraft (10). For example, the server (30), the controller (40), and the station (50) can control the operation (e.g., movement, rotation, etc.) of the aircraft (10) or control the filming of the aircraft (10).

[0076] The aircraft (10) includes at least one camera, and can capture images of an object (20) using the camera. For example, the camera can be installed at a location advantageous for capturing images during flight of the aircraft (10) (e.g., an area not obscured by a propeller, etc., at the front or lower front of the aircraft).

[0077] For example, the aircraft (10) can fly using a global navigation satellite system (GNSS) and / or an inertial navigation system (INS).

[0078] For example, the aircraft (10) can transmit and receive data with a server (30), a controller (40), and / or a station (50). In addition, the controller (40) and the server (30), the server (30) and the station (50), and the station (50) and the controller (40) can transmit and receive data with each other.

[0079] Here, the data may include data required to control the flight of the aircraft (10), data on a flight image of the aircraft (10), data on an image taken of an object (20) by the aircraft (10), etc.

[0080] A flight image represents the field of view of an aircraft (10) when the aircraft (10) is flying. For example, the flight image may be a dynamic image acquired in real time, but is not limited thereto.

[0081] The image captured by the aircraft (10) of the target object (20) refers to an image captured by the aircraft (10) while flying around the target object (20). At this time, the image captured by the aircraft (10) of the target object (20) can be used as an image to check for defects in the target object (20). In this case, the image captured by the aircraft (10) of the target object (20) can be referred to as an inspection image.

[0082] For example, the flight image may be an image with a relatively low resolution (or a low Ground Sampling Distance (GSD)) compared to the inspection image, and the inspection image may be an image with a relatively high resolution (or a high GSD) compared to the flight image. Meanwhile, depending on the type of aircraft (10), the cameras that generate the flight image and the inspection image may be the same or different.

[0083] A user can control the aircraft (10) using the interface of the controller (40). For example, the controller (40) can generate a control signal based on user input received through the interface and transmit the control signal to the aircraft (10). The controller (40) can transmit the control signal to the aircraft (10) via wireless communication. The control signal may be a signal that controls the flight, attitude, navigation, etc. of the aircraft.

[0084] The aircraft (10) can control the motor to rotate the propeller according to the control signal received from the controller (40). The aircraft (10) can move, rotate, etc. by changing the speed and / or attitude, etc. by the rotation of the propeller. Here, the attitude of the aircraft can be expressed as pitch (Y), roll (X), yaw (Z), etc. In addition, the aircraft can perform photographing of the target object (20), etc. according to the control signal received from the controller (40).

[0085] The controller (40) may further include a display device, and the user may check the flight image and / or inspection image of the aircraft (10) through the display device.

[0086] The controller (40) may be a device on which an application for controlling an aircraft (10) is installed. For example, the device on which the application is installed may be a variety of portable devices such as a smartphone, tablet, smart pad, laptop, or wearable device.

[0087] The server (30) or station (50) can control the aircraft by directly transmitting a control signal to the aircraft (10). In addition, the aircraft (10) can transmit a flight image and / or an inspection image to the server (30), controller (40), or station (50).

[0088] The aircraft (10), server (30), controller (40), and station (50) can each analyze the inspection image. For example, the analysis of the inspection image may be to determine whether there is a defect in the target object (20) in the inspection image through an algorithm such as machine learning or deep learning. The aircraft (10) may directly determine whether there is a defect in the target object (20), or may transmit the inspection image to the server (30), controller (40), or station (50). The server (30), controller (40), or station (50) may analyze the inspection image received from the aircraft (10) to determine whether there is a defect in the target object (20).

[0089] Meanwhile, in the case of large structures, there may be areas that are difficult for humans to inspect. Furthermore, it may take a lot of time for humans to inspect the entire large structure, or the inspection process may be dangerous. Accordingly, when inspecting large structures using an aircraft (10), inspection time is shortened and there is a possibility of no casualties. Meanwhile, to facilitate user inspection and management, the images captured by the aircraft (10) need to be merged and output. However, in the case of large structures, the merged result of the images captured by the aircraft (10) may be distorted due to various variables. For example, when the aircraft (10) photographs a large building with a lot of glass, if an object reflected in the glass is used as a feature point, distortion may occur in the merged image. Merging images may mean creating a single image by matching the common parts contained in two or more images. Merging may be expressed as stitching.

[0090] The user can check and manage the target object (20) through the merged image. The user can visually check the target object (20) at a glance through the merged image. In addition, the target object (20) can be photographed periodically (e.g., once a week, once a month, once a year, etc.) using the aircraft (10), and the merged images of the target object (20) can be stored. Accordingly, the user can check the change in defects of the target object (20) over time through the merged images, and repair / manage the defects.

[0091] In one embodiment, the processor can merge images without distortion, regardless of the type of object. For example, even if the object contains a lot of glass, the processor can generate a merged image with minimal distortion. Furthermore, the processor can generate a merged image with minimal distortion even when merging images using a machine learning model.

[0092]

[0093] Figure 3 is a schematic diagram illustrating an example of an aircraft according to one embodiment.

[0094] Referring to FIG. 3, the aircraft (10) may include a sensor (110), a camera (120), a memory (130), a driving device (140), a communication device (150), and a processor (160). However, the components of the aircraft (10) are not limited to those illustrated in FIG. 3. In other words, the aircraft (10) may include at least one more component in addition to the components illustrated in FIG. 3, or at least one of the components illustrated in FIG. 3 may be excluded.

[0095] The sensor (110) detects various information necessary for the operation of the aircraft (10) (e.g., flight, photography, etc.), such as the aircraft (10) itself, the surrounding environment of the aircraft (10), identification of the target (20), and confirmation of the distance between the aircraft (10) and the target (20). The sensor (110) may include, but is not limited to, a gyro sensor, a barometer, an ultrasonic sensor, a magnetic sensor, an acceleration sensor, a proximity sensor, a lidar, a radar, and / or a GPS sensor.

[0096] For example, a gyro sensor and / or an acceleration sensor can measure the three-axis angular velocity of the aircraft (10). A barometer can measure pressure changes and / or air pressure in the atmosphere around the aircraft (10). Since air pressure varies with altitude, the aircraft (10) can also measure the altitude of the aircraft (10) using the barometer. An ultrasonic sensor can measure the distance between the aircraft (10) and the ground or an object (20). A magnetic sensor is a type of terrestrial magnetism sensor (compass sensor) and can detect geomagnetic information.

[0097] For example, a proximity sensor can measure the proximity state of an object (20) to an aircraft (10), the distance between the aircraft (10) and the object (20), and can include an ultrasonic sensor that can measure the distance to the object (20) from a signal reflected from the object (20) by outputting ultrasonic waves. A GPS sensor can calculate the current coordinates (x, y, z) of the aircraft (10) using GPS signals.

[0098] The sensor (110) may include an attitude and heading reference system (AHSR). For example, the attitude and heading reference system may include an inertial sensor or an inertial measurement unit (IMU). For example, the attitude and heading reference system may include a gyro sensor, an acceleration sensor, and a magnetic sensor, and fuse the sensor values ​​to obtain the attitude value of the aircraft (10). ) can be output. Here, the detailed value ( ) can be an angle based on three-dimensional coordinates (x-axis coordinate, y-axis coordinate, z-axis coordinate) according to GPS coordinates.

[0099] The camera (120) can photograph the object (20) according to instructions from the processor (160). For example, the aircraft (10) can include at least one camera, and can include a low-resolution camera and / or a high-resolution camera.

[0100] The camera (120) can be combined with a gimbal whose angle can be adjusted. Accordingly, the shooting angle of the camera (120) can be adjusted by the gimbal.

[0101] The memory (130) may include any non-transitory computer-readable recording medium. As an example, the memory (130) may include a non-permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, the non-permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be a separate permanent storage device distinct from the memory. In addition, the memory (130) may store an operating system (OS) and at least one program code (e.g., a code for the processor (160) to perform an operation to be described later with reference to FIGS. 4 to 12).

[0102] These software components may be loaded from a computer-readable recording medium separate from the memory (130). This separate computer-readable recording medium may be a recording medium that can be directly connected to a computer, and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. Alternatively, the software components may be loaded into the memory (130) via a communication device (150) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (130) based on a computer program (e.g., a computer program for the processor (160) to perform the operations described below with reference to FIGS. 4 to 26) that is installed by files provided by developers or a file distribution system that distributes installation files of applications via the communication device (150).

[0103] The driving device (140) controls the driving of the motor at a speed and direction according to instructions from the processor (160), and accordingly, the rotational speed and direction of the propeller connected to the motor can be controlled. For example, the driving device (140) may include a motor and a propeller.

[0104] The communication device (150) performs data communication between the aircraft (10) and an external device. For example, the communication device (150) may communicate with the controller (40), server (30), and / or station (50) using various communication methods such as infrared communication, RF (Radio Frequency) communication, Wi-Fi communication, ZigBee communication, Bluetooth communication, laser communication, UWB (Ultra-Wideband) communication, LTE, 5G, 6G, and wireless LAN. However, the communication method employed in the communication device (150) is not limited to the above-described method.

[0105] The processor (160) can process commands of a computer program by performing basic arithmetic, logic, and input / output operations. Here, the commands can be provided from memory (130) or an external device (e.g., a server (30), a controller (40), a station (50), etc.). In addition, the processor (160) can generally control the operations of other components included in the aircraft (10).

[0106] For example, the functions performed by each module included in the processor (160) may be performed by one processor or by separate processors. The processor (160) may perform calculations or data processing related to control and / or communication of at least one other component of the aircraft (10).

[0107] The processor (160) acquires static images by photographing at least a portion of the object (20) through the camera (120) of the aircraft (10).

[0108] For example, the processor (160) can control the camera (120) to change the angle of the camera (120) to capture the target object (20) while the aircraft (10) is located at the n-th point (where n is a natural number greater than or equal to 1). In other words, the aircraft (10) does not generate a static image while moving, but can generate at least one static image at one point and at least one static image at another point.

[0109] Then, the processor (160) generates a partial image by merging at least two or more of the static images using at least one feature point included in each of the static images. For example, the processor (160) can identify at least one common feature point in a first static image and a second static image selected from among the static images. Thereafter, the processor (160) can determine at least one candidate solution for matching the at least one common feature point. Thereafter, the processor (160) can merge the first static image and the second static image based on the optimal solution selected from among the at least one candidate solution. In one example, the processor (160) can generate a partial image by matching at least four common features in the first static image and the second static image. The first static image and the second static image are images taken by the aircraft (10) at the same point.

[0110] For example, the processor (160) can use a machine learning model to identify a predetermined object (e.g., glass, etc.) in each of the static images, and identify common features in the remaining portions of each of the static images excluding the predetermined object.

[0111] Here, the candidate solution includes a method for matching common feature points by at least one of translation, rotation, and scaling of at least one of the first and second static images. Furthermore, the optimal solution includes a method for selecting at least one candidate solution based on at least one pre-determined criterion.

[0112] Then, the processor (160) generates a block image by merging at least two or more of the partial images using at least one feature point included in each of the partial images. For example, the processor (160) can identify at least one common feature point in a first partial image and a second partial image selected from among the partial images. Thereafter, the processor (160) can determine at least one candidate solution for matching the at least one common feature point. Thereafter, the processor (160) can merge the first partial image and the second partial image based on an optimal solution selected from among the at least one candidate solution.

[0113] A detailed description of the operations of the aircraft (10) described above with reference to FIG. 3 will be described later with reference to FIGS. 4 to 27.

[0114] Meanwhile, the processor (160) may be implemented as an array of a plurality of logic gates, or may be implemented as a combination of a general-purpose microprocessor and a memory storing a program that can be executed on the microprocessor. For example, the processor (160) may include a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the processor (160) may include an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. For example, the processor (160) may also refer to a combination of processing devices, such as a combination of a digital signal processor (DSP) and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors coupled with a digital signal processor (DSP) core, or a combination of any other such configurations.

[0115] Meanwhile, the operation of the processor (160) described above with reference to FIG. 3 may be implemented by a separate device. In this case, the separate device (hereinafter referred to as an “image merging device”) may be included in at least one of the aircraft (10), the server (30), the controller (40), the station (50), and another external device. When the image merging device is included in at least one of the server (30), the controller (40), the station (50), and another external device, the aircraft (10) may capture static images and dynamic images, and transmit the captured images to the image merging device. An example of the image merging device is described below with reference to FIG. 27.

[0116]

[0117] FIG. 4 is a flowchart illustrating an example of a method for merging images according to one embodiment. The method illustrated in FIG. 4 comprises steps that are processed sequentially in the aircraft (10) or processor (160) illustrated in FIGS. 1 to 3. Therefore, even if omitted below, the content described above regarding the aircraft (10) or processor (160) can also be applied to the method illustrated in FIG. 4. Furthermore, the operations of the processor (160) described below can also be performed by the image merging device illustrated in FIG. 27.

[0118] Referring to FIGS. 3 and 4, first, the processor (160) can obtain static images by photographing at least a portion of the target object (20) through the camera (120) of the aircraft (10) (S100).

[0119] Additionally, the processor (160) can obtain information about the position of the aircraft (10) at the time the images are captured, information about the distance between the aircraft (10) and the target (20), information about the angle of view of the camera (120), and information about the direction in which the camera (120) captures the images (i.e., the capturing angle of the camera (120)).

[0120] Hereinafter, with reference to FIGS. 5 and 6, an example of a processor (160) obtaining a static image will be described.

[0121]

[0122] FIG. 5 is a diagram illustrating an example of a processor obtaining a static image according to one embodiment.

[0123] Referring to FIGS. 3 and 5, the processor (160) can photograph the target object (20) while changing the angle of the camera (120) while the aircraft (10) is located at the n-th point (where n is a natural number greater than or equal to 1).

[0124] The first to nth points may be positions having the same altitude, and the aircraft (10) may photograph the object (20) at the first to nth points while moving in the horizontal direction. Conversely, the first to nth points may be positions having different altitudes, and the aircraft (10) may photograph the object (20) at the first to nth points while moving in the vertical direction.

[0125] In one example, the direction of movement of the aircraft (10) and the direction in which the angle of the camera (120) changes may be perpendicular to each other. For example, if the aircraft (10) hops horizontally, the angle of the camera (120) may change vertically. Conversely, if the aircraft (10) hops vertically, the angle of the camera (120) may change horizontally. Hopping of the aircraft (10) means moving from one point to another.

[0126] In another example, the direction of movement of the aircraft (10) and the direction in which the angle of the camera (120) changes may be the same. For example, if the aircraft (10) hops horizontally, the angle of the camera (120) may change horizontally. Conversely, if the aircraft (10) hops vertically, the angle of the camera (120) may change vertically.

[0127] In another example, the aircraft (10) may include a plurality of cameras (120). The cameras (120) may be arranged vertically or horizontally. The cameras (120) may be attached to the aircraft (10) so that their angles of view overlap by a predetermined ratio. When the aircraft (10) captures static images using the plurality of cameras (120), there is no need to change the angle of the cameras (120) while capturing the target (20). Therefore, the time required for the aircraft (10) to capture the target (20) can be shortened.

[0128] First, the aircraft (10) is positioned at a first point, and can capture a portion (510) of the target object (20) by changing the angle of the camera (120) at the first point to generate multiple static images. Accordingly, at least one image representing the portion (510) can be acquired. The angle of the camera (120) can be changed up and down or left and right. When the aircraft (10) includes multiple cameras (120), the aircraft (10) can acquire multiple static images at one point without changing the angle of the camera (120).

[0129] Thereafter, the aircraft (10) can move from the first point to the second point, and change the angle of the camera (120) at the second point to capture a portion (520) of the target object (20). Accordingly, at least one image representing the portion (520) can be acquired. In FIG. 5, for convenience of explanation, the areas captured by the aircraft (10) at the first point and the second point are depicted as being spaced apart, but the areas captured by the aircraft (10) at the first point and the second point may overlap each other.

[0130] In the same manner, the aircraft (10) may move from a second point to a third point and photograph a portion (530), thereby obtaining at least one image representing the portion (530). In FIG. 5, for convenience of explanation, the areas photographed by the aircraft (10) at the second point and the third point are depicted as being spaced apart, but the areas photographed by the aircraft (10) at the second point and the third point may overlap each other.

[0131] Static images can be acquired during a single flight of the aircraft (10). Specifically, the images generated by photographing portion (510) can all be acquired during the same flight. Similarly, the images generated by photographing portion (520) and the images generated by photographing portion (530) can all be acquired during the same flight.

[0132] Of course, some of the images generated as portions (510) to (530) are captured may be acquired in other flights. However, capturing a portion is accomplished in a single flight (i.e., one flight). One flight may refer to the period from when the aircraft (10) takes off to begin capturing images until it lands.

[0133] The processor (160) may determine the number of images acquired for each section (510, 520, 530) by considering the overlap rate between images, the angle of view of the camera (120), the required resolution of the images, etc. For example, the required value for the GSD of the images may be determined by the distance between the aircraft (10) and the target (20) and the number of pixels of the camera, but is not limited thereto. The number of images acquired for each section (510, 520, 530) may be 1 to 7, but is not limited thereto. In addition, the processor (160) may determine the distance to the target (20) by considering the GSD of the images, the specifications of the camera, and / or the overlap rate between the images. For example, in order to easily perform merging of images, the aircraft (10) may generate an odd number of images at one point.

[0134] In addition, although FIG. 5 illustrates an aircraft (10) photographing an object (20) while moving in a horizontal direction, the present invention is not limited thereto. For example, the aircraft (10) may also photograph an object (20) while moving in a vertical direction.

[0135]

[0136] FIG. 6 is a diagram illustrating a processor according to another embodiment for acquiring a static image. The aircraft (10) can capture an object (20) while maintaining the capturing direction and changing the capturing position. The moving direction of the aircraft (10) may be perpendicular to the capturing direction, but may vary depending on the angle of the camera (120) or the posture of the aircraft (10). FIG. 6 illustrates examples of the object (20) and the capturing positions (L1, L2) of the aircraft (10) when viewing the object (20) from the side. Although FIG. 6 illustrates only the capturing positions (L1, L2), the aircraft (10) can capture the object (20) from more capturing positions.

[0137] Referring to FIGS. 3 and 6, the aircraft (10) can photograph the object (20) at the photographing position (L1) and move to the photographing position (L2) to photograph the object (20). In this manner, the aircraft (10) can photograph the object (20) or elements (21, 22, 23) while moving up and down. For example, when the object (20) is a wind turbine, the aircraft (10) can photograph the wind turbine or blades at multiple photographing positions while moving from the root to the tip of the blade or from the tip to the root.

[0138] Meanwhile, the number of shooting positions can be set in various ways so that the front of the object (20) or elements (21, 22, 23) is captured without omission. In other words, the processor (160) can set the number of shooting positions so that the front of the elements (21, 22, 23) is captured without omission by considering the areas (R1, R2) captured at the shooting positions (L1, L2).

[0139] In FIG. 6, regions (R1, R2) are shown spaced apart for convenience of explanation, but regions (R1, R2) may overlap, and although only two shooting positions (L1, L2) are shown, the aircraft (10) may shoot elements (21, 22, 23) at a greater number of shooting positions.

[0140] At this time, the processor (160) can control the aircraft (10) to move up and down while maintaining the same distance (D) as the target (20).

[0141] When photographing a wind turbine (20), a merged image can be generated for each blade. For example, a first merged image can be generated for the first blade (21), a second merged image can be generated for the second blade (22), and a third merged image can be generated for the third blade (23).

[0142] Accordingly, when performing a process of removing duplicate images as described below, the processor (160) can remove duplicate images for each blade. For example, the processor (160) can generate a first merged image for the first blade (21) and remove duplicate images from among the static images included in the first merged image.

[0143]

[0144] Referring again to FIGS. 3 and 4, the processor (160) can then generate a merged image based on the static images (S200).

[0145] At this time, the merged image may include a partial image or a block image as described below. In other words, the processor (160) may generate partial images and block images based on static images, and may determine at least one of the generated partial images and block images as the merged image.

[0146] For example, when a target object (20) is photographed in the same manner as in FIG. 5 and static images are generated, the processor (160) can stitch the static images to generate a partial image and stitch the partial images to generate a merged image.

[0147] As another example, when a subject (20) is photographed in the same manner as in FIG. 6 and static images are generated, the processor (160) can stitch static images of a portion of the subject (20) to generate a merged image.

[0148] Meanwhile, the processor (160) can obtain information about the position of the aircraft (10) at the time when the static images are captured, information about the distance between the aircraft (10) and the target (20), information about the angle of view of the camera (120), and information about the direction in which the camera (120) captures the image (i.e., the shooting angle of the camera (120). The information obtained at the time when the image is captured can be displayed as shooting information, and the shooting information is stored in a manner matched with the static image.

[0149] Hereinafter, the process of generating a merged image will be described in more detail with reference to FIG. 7.

[0150]

[0151] FIG. 7 is a flowchart illustrating an example of a processor generating a merged image according to one embodiment.

[0152] Referring to FIGS. 3 and 7, first, the processor (160) can generate partial images by merging at least two or more static images using at least one feature point included in each of the static images (S210).

[0153] As some examples, the processor (160) can determine the direction in which to merge the static images depending on the direction in which the angle of the camera (120) changes. If it is assumed that the processor (160) captures the object (20) while adjusting the angle of the camera (120) in the vertical direction while the aircraft (10) is positioned at the same point, the processor (160) can merge the static images in the vertical direction. In other words, the processor (160) can merge the static images captured while changing the angle of the camera (120) in the vertical direction to generate a partial image. In the same manner, if the processor (160) captures the object while adjusting the angle of the camera (120) in the horizontal direction, the processor (160) can merge the static images in the horizontal direction. In other words, the processor (160) can merge the static images captured while changing the angle of the camera (120) in the horizontal direction to generate a partial image.

[0154] Since a partial image is generated by merging multiple static images, the area of ​​the object (20) included in the partial image is wider than the area of ​​the object (20) included in the static image. In other words, by connecting parts of the object (20) included in the static image, the partial image can include an area wider than the static image.

[0155] In the same vein, the area of ​​the object (20) included in the block image is wider than the area of ​​the object (20) included in the partial image. The method by which the partial images are merged to create a block image will be described later with reference to step (S220).

[0156] Hereinafter, with reference to FIGS. 8 to 12, examples of the processor (160) generating partial images will be described.

[0157]

[0158] FIG. 8 is a flowchart illustrating an example of a processor generating partial images according to one embodiment.

[0159] Referring to FIGS. 3, 7, and 8, first, the processor (160) can identify at least one common feature point in a first static image and a second static image selected from among the static images (S211).

[0160] The first static image and the second static image may be sequentially captured images or overlapping images. For example, assuming that five images were acquired while changing the angle of the camera (120) while the aircraft (10) was positioned at the same point, and that the first static image was the third captured image, the second static image may be the fourth captured image.

[0161] Hereinafter, with reference to FIGS. 9 to 11, an example in which the processor (160) identifies common features in a first static image and a second static image will be described.

[0162]

[0163] FIG. 9 is a diagram illustrating an example of a processor identifying common features in images according to one embodiment.

[0164] Referring to FIG. 9, the processor (160) selects a first static image (720) and a second static image (730) from among the static images (710). At this time, the first static image (720) and the second static image (730) may be images captured continuously. Accordingly, the first static image (720) and the second static image (730) may commonly include the same portion of the target object (20).

[0165] The processor (160) identifies at least one feature point (721, 722, 723) in the first static image (720). For example, the processor (160) may identify at least one feature point (721, 722, 723) by considering the type of the object (20), the external features of the object (20), the external environment (weather, time, etc.) at the time the object (20) was photographed, etc.

[0166] In the same manner as described above, the processor (160) identifies at least one feature point (731, 732, 733) in the second static image (730).

[0167] The method by which the processor (160) identifies feature points in the first static image (720) and the second static image (730) is the same as the conventional method by which feature points in an image are identified, so a detailed description thereof is omitted.

[0168] The processor (160) identifies at least one common feature point among the feature points (721, 722, 723) and the feature points (731, 732, 733). Taking FIG. 9 as an example, the processor (160) can identify that the feature points (722, 723) of the first static image (720) and the feature points (731, 732) of the second static image (730) are common feature points.

[0169] Meanwhile, the processor (160) may identify feature points by checking only a portion of the static images (720, 730).

[0170] Below, an example in which the processor (160) identifies feature points only in some of the static images (720, 730) is described with reference to FIG. 10.

[0171]

[0172] FIG. 10 is a diagram illustrating an example in which a processor according to one embodiment identifies feature points in only a portion of an image.

[0173] Referring to FIG. 10, the processor (160) selects a first static image (1820) and a second static image (1830) from among static images (1810). Then, the processor (160) identifies feature points in each of the first static image (1820) and the second static image (1830), and then extracts feature points common to each other.

[0174] At this time, the processor (160) can identify feature points by checking only the area (821) of the first static image (1820) and the area (831) of the second static image (1830). If the first static image (1820) and the second static image (1830) are continuously captured images, the first static image (1820) and the second static image (1830) may include a common area of ​​the target object (20).

[0175] In this case, the processor (160) can identify feature points by checking only the areas (i.e., areas (821), (831)) containing common parts of the object (20) in the first static image (1820) and the second static image (1830). Accordingly, the processor (160) can quickly identify feature points in the images (1820, 1830).

[0176] Meanwhile, the processor (160) may identify feature points excluding some of the images (1820, 1830) by considering the external features of the target object (20).

[0177] Below, an example in which the processor (160) identifies feature points excluding some of the images (720, 730) is described with reference to FIG. 11.

[0178]

[0179] FIG. 11 is a diagram illustrating an example of a processor identifying feature points excluding a portion of an image according to one embodiment.

[0180] Referring to FIG. 11, the processor (160) selects a first static image (920) and a second static image (930) from among the static images (910). Then, the processor (160) identifies feature points in each of the first static image (920) and the second static image (930), and then extracts feature points common to each other.

[0181] For example, the processor (160) can identify glass among static images (910). If a portion reflected on the glass in the first static image (920) and the second static image (930) is selected as a feature point, the image resulting from the merge of the first static image (920) and the second static image (930) is likely to be distorted. Therefore, the processor (160) identifies the glass and extracts feature points from an area excluding the glass.

[0182] The processor (160) can identify a specific object (921, 931) among static images (910) using a machine learning model. For example, the machine learning model can be trained using labeled data that distinguishes between glass and non-glass portions in the image.

[0183] In addition, the processor (160) can identify at least one common feature point in the remaining portion (922, 932) excluding a predetermined object (921, 931) in each of the static images (910). Accordingly, the image in which the first static image (920) and the second static image (930) are merged has little distortion or no distortion exceeding a standard.

[0184]

[0185] Referring again to FIGS. 3, 7 and 8, the processor (160) may then determine at least one candidate solution for matching at least one common feature point (S212).

[0186] For example, a candidate solution includes a method of matching common features by at least one of translation, rotation and scaling of at least one of a first static image and a second static image.

[0187] Hereinafter, with reference to FIG. 12, an example in which the processor (160) determines a candidate solution will be described.

[0188]

[0189] FIG. 12 is a diagram illustrating an example of a processor determining a candidate solution according to one embodiment.

[0190] Referring to FIG. 12, feature points (1011, 1012) of the first static image (1010) and feature points (1021, 1022) of the second static image (1020) correspond to common feature points. As the feature points (1011, 1012) and feature points (1021, 1022) are aligned, the first static image (1010) and the second static image (1020) can be merged.

[0191] The processor (160) determines a candidate solution (1030, 103k) (wherein, k is a natural number greater than or equal to 2) with which the feature points (1011, 1012) and the feature points (1021, 1022) can be aligned. For example, the processor (160) may determine a candidate solution (1030, 103k) that manipulates at least one of the first static image (1010) and the second static image (1020) so that the positions and directions of the feature points (1011, 1012) and the feature points (1021, 1022) can be aligned identically.

[0192] Here, the manipulation includes moving, rotating, and / or resizing at least one of the first static image (1010) and the second static image (1020). Resizing also includes enlarging or reducing the size of the image.

[0193] For example, the processor (160) can use a machine learning model to determine a candidate solution (1030, 103k).

[0194]

[0195] Referring again to FIGS. 3, 7 and 8, the processor (160) may then align the first static image and the second static image based on an optimal solution selected from at least one candidate solution (S213).

[0196] As described above with reference to FIG. 12, the processor (160) may determine at least one candidate solution (1030, 103k) with which common features can be aligned. However, considering the conditions (or method) under which the aircraft (10) captured the static images, some of the candidate solutions (1030, 103k) may not be suitable for image merging. Therefore, the processor (160) selects an optimal solution from among the candidate solutions (1030, 103k).

[0197] For example, the optimal solution can be selected from among candidate solutions based on at least one predefined criterion. The criterion can be predefined in relation to the translation, rotation, and / or scaling of the image. Furthermore, the criterion can be set differently depending on whether the image is to be merged horizontally or vertically. Furthermore, the criterion can be set differently depending on how the image is captured.

[0198] As an example, the processor (160) may exclude from among the candidate solutions (1030, 103k) a solution whose image rotation angle exceeds a threshold value. If the processor (160) acquires the first static image (1010) and the second static image (1020) by only changing the angle of the camera (120), the processor (160) may exclude (or assign a low weight to) a candidate solution that includes a method of matching with a rotation angle exceeding a threshold value from among the candidate solutions.

[0199] As another example, the processor (160) may exclude from among candidate solutions (1030, 103k) solutions whose degree of image resizing exceeds a threshold value.

[0200] As another example, the processor (160) may exclude from among candidate solutions (1030, 103k) solutions whose image movement degree exceeds a threshold value.

[0201]

[0202] Referring again to FIGS. 3 and 7, the processor (160) can then generate a block image by merging at least two or more of the partial images using at least one feature point included in each of the partial images (S220).

[0203] For example, a partial image may be generated from static images acquired through at least one flight of the aircraft (10). As described above with reference to FIG. 5, the static images may be acquired through one flight (i.e., a single flight) of the aircraft (10). However, the partial image is not limited to static images acquired during one flight of the aircraft (10).

[0204] As an example, one of the partial images may be generated from images acquired during the flight at 13:00 on January 1, 2024, and one of the partial images may be generated from images acquired during the flight at 15:00 on January 1, 2024. As another example, one of the partial images may be generated from images acquired during the flight at 13:00 on January 1, 2024, and one of the partial images may be generated from images acquired during the flight at 13:00 on January 2, 2024.

[0205] That is, a single partial image is generated only from static images acquired during a single flight (i.e., one flight), but each of the partial images need not be generated from static images acquired during a single flight.

[0206] The method by which the processor (160) generates a block image is the same as the method by which the processor (160) generates a partial image. In other words, the processor (160) identifies at least one common feature point in a first partial image and a second partial image selected from among the partial images. Then, the processor (160) determines at least one candidate solution for matching the at least one common feature point. Then, the processor (160) merges the first partial image and the second partial image based on the optimal solution selected from among the at least one candidate solution.

[0207] If the processor (160) acquires the first partial image and the second partial image while maintaining the distance from the target object (20), the processor (160) may exclude (or assign a low weight to) a candidate solution that includes a method of matching by enlarging or reducing the image by exceeding a threshold value among the candidate solutions.

[0208] Accordingly, the examples of step (S210) described above with reference to FIGS. 8 to 12 can be equally applied to step (S220).

[0209] Meanwhile, a single block image may include all or part of the object (20). For example, if the object (20) is a building, the block image may be generated for each floor of the building. Alternatively, if the exterior of the object (20) includes a characteristic object (e.g., a pillar, etc.), the block image may be generated for each region that can be segmented based on the characteristic object.

[0210]

[0211] FIG. 13 is a diagram for explaining the relationship between a static image, a partial image, and a block image according to one embodiment.

[0212] In Fig. 13, static images (1110), partial images (1120), and block images (1130) are illustrated. The number of static images (1110), partial images (1120), and block images (1130) illustrated in Fig. 11 is merely an example and is not limited thereto.

[0213] The processor (160) merges static images (1110) to generate partial images (1120). For example, the processor (160) may merge static images (1111, 1112, 1113) to generate partial images (1121). At this time, the static images (1111, 1112, 1113) may be images generated through a single flight. In one example, the static images (1111, 1112, 1113) may be images sequentially captured while changing the angle of the camera (120) in the vertical direction while the aircraft (10) is hovering.

[0214] In this way, the processor (160) generates partial images (1121, …, 112q) (where q is a natural number greater than or equal to 2) using static images (1111, …, 111p) (where p is a natural number greater than or equal to 2).

[0215] A partial image may be a merged image generated by matching static images captured by the aircraft (10) at one point. For example, the partial image (1121) may be a merged image generated by matching static images captured by the aircraft (10) at a first point, the partial image (1122) may be a merged image generated by matching static images captured by the aircraft (10) at a second point, and the partial image (112q) may be a merged image generated by matching static images captured by the aircraft (10) at a q-th point.

[0216] Then, the processor (160) merges partial images (1120) to generate block images (1130). For example, the processor (160) may merge partial images (1121, 1122) to generate block image (1131). At this time, the partial images (1121, 1122) do not need to be images generated through a common flight.

[0217] In this way, the processor (160) generates block images (1131, …, 113r) (wherein, r is a natural number greater than or equal to 2) using partial images (1121, …, 112q) (wherein, q is a natural number greater than or equal to 2).

[0218]

[0219] Referring again to FIGS. 3 and 7, the processor (160) may then determine at least one of the partial image and the block image as a merged image (S230).

[0220] In some examples, the processor (160) may determine a partial image, a block image, or a merged image of a partial image and a block image.

[0221]

[0222] FIG. 14 is a flowchart illustrating an example of a processor generating a merged image according to another embodiment.

[0223] Referring to FIG. 14, the processor (160) can merge static images of the target object (20) captured while the aircraft (10) hops.

[0224] More specifically, first, the processor (160) can generate a first partial image by merging static images captured while changing the angle of the camera (120) at a first point (S240). Therefore, the first partial image includes only static images captured by the aircraft (10) at one location.

[0225] Next, the processor (160) can merge static images captured while changing the angle of the camera at a second point to generate a second partial image (S250). Accordingly, the second partial image also includes only static images captured by the aircraft (10) at one location.

[0226] The angle of the camera (120) changes in the first direction, the first point and the second point are located in the second direction, and the first direction and the second direction are perpendicular to each other. For example, if the first direction is vertical, the second direction is horizontal. Also, if the first direction is horizontal, the second direction is horizontal.

[0227] Next, the processor (160) can merge the first partial image and the second partial image to generate a block image (S260). The processor (160) merges the first partial image and the second partial image according to the movement direction of the aircraft (10). For example, when the aircraft (10) moves in the right direction, the processor (160) matches the right area of ​​the first partial image and the left area of ​​the second partial image to merge the first partial image and the second partial image.

[0228] When a target object (20) is photographed at N locations, the processor (160) can merge N partial images to generate one block image. At this time, the N locations can be located on one straight line. N is a natural number greater than or equal to 2.

[0229] Next, the processor (160) can determine at least one of the first partial image, the second partial image, and the block image as a merged image (S270).

[0230] As some examples, the processor (160) may determine any one of the first partial image, the second partial image, the block image, the first partial image and the second partial image, the first partial image and the block image, the second partial image and the block image, and the first partial image, the second partial image and the block image as the merged image.

[0231]

[0232] Referring again to FIGS. 3 and 4, the processor (160) may then perform post-processing related to the merged image (S300).

[0233] In some examples, the processor (160) may output an item corresponding to the merged image with an identified defect or remove duplicate images from the merged image.

[0234] Hereinafter, step (S300) according to some embodiments of the present invention will be described in more detail with reference to FIGS. 15 to 26.

[0235]

[0236] FIG. 15 is a flowchart illustrating an example of a processor performing post-processing related to a merged image according to one embodiment.

[0237] Referring to FIGS. 3, 4, and 15, as an example, the processor (160) may identify a defect based on at least one of a static image and a merged image in step (S300), and output the identified defect to an item corresponding to the merged image (S310).

[0238] As another example, the processor (160) can determine duplicate images among static images in step (S300) and remove the determined duplicate images (S320).

[0239] Although FIG. 15 illustrates that both steps (S310) and (S320) are performed after step (S200), this is only for convenience of explanation, and either step (S310) or step (S320) may be omitted.

[0240] Hereinafter, step (S310) will be described in detail with reference to FIGS. 16 to 20, and step (S320) will be described in detail with reference to FIGS. 21 to 26.

[0241]

[0242] FIG. 16 is a flowchart illustrating an example of a processor outputting a defect to an item according to one embodiment.

[0243] Referring to FIG. 3 and FIG. 16, first, the processor (160) can match at least one block image to an item corresponding to the target object (20) (S311).

[0244] For example, the item may include a two-dimensional image of the entire appearance of the object (20) or a three-dimensional rendered image corresponding to the entire appearance of the object (20). However, the item is not limited to the above, and may include various types of images representing the entire appearance of the object (20). For example, the item may be an image of any one of the front, back, and side of the object (20).

[0245] If the item is a two-dimensional image of the object (20), the resolution of the item may be relatively lower than that of the static image. Since the processor (160) determines whether the object (20) contains defects from the static image, the static image must be a high-resolution image. On the other hand, since it is sufficient for the item to identify the entire appearance of the object (20), the resolution of the item does not necessarily need to be high.

[0246] Additionally, when the item is a two-dimensional image of the object (20), the static images and the item may be acquired during the same flight of the aircraft (10) or may be acquired during different flights of the aircraft (10).

[0247] If the item is a three-dimensional rendering image corresponding to the entire appearance of the object (20), the processor (160) may create the item by combining static images, or may create the item through an image captured by the aircraft (10) of the entire object (20).

[0248] For example, the processor (160) can match a block image and an item based on at least one of feature points included in at least one block image and an item, shooting information of static images, and location information of an aircraft.

[0249] Hereinafter, with reference to FIG. 17, examples of matching a block image and an item by the processor (160) are described.

[0250]

[0251] FIG. 17 is a diagram illustrating an example of a processor matching a block image and an item according to one embodiment.

[0252] Referring to FIG. 17, an item (1210) may represent the entire appearance of a target object (20). The processor (160) may match a block image (1220) with an item (1210) to determine which part (1230) of the item (1210) the block image (1220) corresponds to. In FIG. 17, a part (1230) corresponding to the block image (1220) is indicated by a gray box.

[0253] As an example, the processor (160) can match the block image (1220) and the item (1210) based on the feature points included in the block image (1220) and the item (1210). The item (1210) can be an image representing the object (20). Accordingly, the item (1210) can include feature points. The processor (160) can identify feature points common to each other among the feature points on the block image (1220) and the feature points on the item (1210). In addition, the processor (160) can identify a portion (1230) corresponding to the block image (1220) within the item (1210) by matching the common feature points.

[0254] As another example, the processor (160) can match the block image (1220) with the item (1210) based on the shooting information of the static images and the location information of the aircraft (10). When the aircraft (10) shoots the static images, the shooting information and the location information of the aircraft (10) can be acquired. For example, the shooting information can include information about the distance between the aircraft (10) and the target object (20), information about the angle of view of the camera (120), information about the direction in which the camera (120) shoots the image (i.e., the shooting angle of the camera (120), etc. Accordingly, the processor (160) can determine which part of the target object (20) the static image was shot from by checking the shooting information and the location information of the aircraft (10). Since the block image (1220) is generated by merging static images, the processor (160) can identify a portion (1230) corresponding to the block image (1220) within the item (1210).

[0255] As another example, the processor (160) can match the block image (1220) and the item (1210) based on the feature points included in the block image (1220) and the item (1210) and the location information of the aircraft (10). For example, the location information of the aircraft (10) may include GPS information at the time when the aircraft (10) captures the static images. Accordingly, the processor (160) can confirm the approximate location range of the object (20) at the time when the static images constituting the block image (1220) were captured. For example, the processor (160) can confirm which floor of the object (20) the static images were captured on, etc. The processor (160) can identify feature points that are common to each other among the feature points on the block image (1220) and the feature points in the location range. And, the processor (160) can identify a portion (1230) corresponding to the block image (1220) within the location range by matching common feature points.

[0256] Meanwhile, the processor (160) can identify feature points on the item (1210) and feature points on the block image (1220) through a machine learning model. In addition, the processor (160) can also identify common feature points among the feature points on the item (1210) and feature points on the block image (1220) through the machine learning model.

[0257]

[0258] Referring again to FIGS. 3 and 16, the processor (160) may then output at least one defect identified from at least one of the static images onto the item (S312).

[0259] Here, defects refer to problems that can be identified from the exterior of the object (20). For example, if the object (20) is a building, defects may include damage to the exterior wall, damage to the paint, or internal elements of the object (20) exposed to the exterior (e.g., electrical wiring, plumbing, etc.). Furthermore, if the object (20) is a military facility, defects may include, in addition to the examples described above, damage to barbed wire fences, etc.

[0260] In some examples, the processor (160) may analyze static images or merged images to identify defects and output the identified defects to an area corresponding to the matching result of step (S311). In other words, the processor (160) may identify defects based on at least one of the static images and the merged image, and output the identified defects to an area of ​​an item corresponding to the merged image.

[0261] To this end, the processor (160) can first analyze static images or merged images to identify defects. For example, the processor (160) can determine whether the images contain defects using deep learning or the like.

[0262] Next, the processor (160) may display the identified defect in an area of ​​the item. At this time, the processor (160) may display and output the defect on the item, or may output a static image or block image showing the defect together with the item.

[0263] For example, the processor (160) may indicate a defect by displaying a shape representing the defect in a determined area on the item. When the processor (160) matches the merged image with the item, the processor (160) may determine which location on the item the defect included in the merged image corresponds to, and accordingly, the processor (160) may display a shape representing the defect in the determined location on the item. The shape representing the defect may vary depending on the size, type, etc. of the defect. For example, the processor (160) may display a larger shape as the size of the defect increases. Furthermore, the processor (160) may display the defect with different shapes depending on the type of defect. For example, the processor (160) may display a first defect as a circle, a second defect as a triangle, or a third defect as a square. Furthermore, the processor (160) may assign a sequence number to the defects and display them in order to identify them.

[0264] As another example, the processor (160) may display the identified defect by outputting a merged image to a determined area on the item. For example, the processor (160) may output a block image to a portion of the item. For example, the processor (160) may replace a portion of the item with a block image. In this case, the processor (160) may display the defect on the block image and output the block image with the defect displayed on it by overlapping it with the item. If there are multiple defects, the processor (160) may display all of the multiple defects on the item, and the user may select any one of the displayed defects. In this case, the processor (160) may output a static image or block image showing the defect selected by the user together with the item.

[0265] Hereinafter, with reference to FIGS. 18 and 19, examples of a processor (160) outputting a defect of a target object (20) will be described.

[0266]

[0267] FIG. 18 is a diagram illustrating an example of a processor outputting a defect according to one embodiment.

[0268] Referring to FIG. 18, the processor (160) can identify a defect (1311) of the object (20) on the block image (1310). As described above with reference to step (S311), the processor (160) can identify which part of the item (1320) the block image (1310) corresponds to. Accordingly, the processor (160) can identify a location on the item (1320) corresponding to the defect (1311) and display the defect (1321) on the item (1320).

[0269] In addition, the processor (160) may not only display and output a defect (1311) on the item (1320), but may also output a block image (1310) in which the defect (1311) appears together with the item (1320). For example, the processor (160) may output the block image (1310) by overlapping it on the item (1320). In this case, the user may confirm the location of the defect (1321) on the item (1320) based on the location of the defect (1311) on the block image (1310).

[0270]

[0271] FIG. 19 is a diagram illustrating another example in which a processor outputs a defect according to one embodiment.

[0272] Referring to FIG. 19, if there are multiple defects in the object (20), the processor (160) can display all of the multiple defects on the item (1410). At this time, the user can select any one of the multiple defects (1411), and the processor (160) can output static images (1441, 1442, 1443) representing the selected defect (1411). The processor (160) can display the order of the defects to identify the multiple defects. In FIG. 19, the first defect (1411) is displayed as number 1, and the second defect is displayed as number 2.

[0273] For example, the processor (160) may sequentially list static images (1441, 1442, 1443) including a defect (1411) and output them (1420). Alternatively, the processor (160) may overlap static images (1441, 1442, 1443) based on the defect (1411) and output them (1430).

[0274] Meanwhile, although FIG. 19 illustrates a plurality of static images (1441, 1442, 1443) including a defect (1411), this is not limited. There may be only one static image including a defect (1411).

[0275] As another example, if a user selects a defect (1411), the processor (160) may output a partial image or block image that includes the defect (1411).

[0276] The processor (160) may also output previous images (static images, partial images, or block images) that match the location of the defect (1411). For example, if the defect (1411) existed at the same location for three years and the aircraft (10) photographed the defect (1411) every year, three block images containing the defect (1411) may have been generated. Accordingly, the processor (160) may output the block images generated annually. By the processor (160) outputting previously generated images together, the user can check changes in the same defect over time at a glance.

[0277]

[0278] FIG. 20 is a diagram illustrating an example in which a processor indicates a defect on an item when the target object is a military facility according to one embodiment.

[0279] FIG. 20 illustrates an example in which the location of a defect (1521) is indicated on an item (1520) when a defect (e.g., damage to the fence) (black box) is found on a fence included in a military facility.

[0280] As described above with reference to FIGS. 1 to 19, an aircraft (10) can fly over a military facility and photograph various parts of the military facility. The processor (160) can merge static images of the military facility to generate a block image (1510). In addition, the processor (160) can match the block image (1510) with an item (1520) corresponding to the military facility. Accordingly, if a defect (black box) is identified in the block image (1510) and / or the static images constituting the block image (1510), the processor (160) can display a location (1521) where the defect exists on the item (1520).

[0281] Additionally, as described above with reference to FIGS. 18 and 19, the processor (160) may output not only the item (1520), but also static images and / or block images (1510) in which defects appear.

[0282]

[0283] FIG. 21 is a flowchart illustrating an example of a processor removing duplicate images according to one embodiment.

[0284] Referring to FIG. 3 and FIG. 21, first, the processor (160) can determine duplicate images among static images using stitching information (S321).

[0285] Stitching information is information about the result of the processor (160) stitching static images. In other words, the stitching information represents the result of moving, rotating, resizing, etc. the static images. For example, the stitching information may be the position, size, etc. of the static images on the merged image. Additionally, the stitching information may be the coordinates of the static images on a two-dimensional plane.

[0286] The processor (160) can determine which static images to remove from among the stitched static images. The processor (160) can remove unnecessary static images from among the static images. The processor (160) can remove static images that include overlapping areas with adjacent static images. In addition, the processor (160) can determine whether another static image includes the object (20) included in the static image, and determine overlapping images based on the determination result.

[0287] The processor (160) can determine a duplicate image by considering whether the object (20) is missing. The processor (160) can identify the object (20) included in a static image, and can determine whether the object (20) is missing when the duplicate image is removed. The processor (160) can determine only static images in which the object (20) is not missing even when removed as duplicate images.

[0288] Hereinafter, the method by which the processor (160) determines duplicate images will be described in detail through FIGS. 22 to 24.

[0289]

[0290] FIG. 22 is a diagram illustrating a method for determining a duplicate image according to one embodiment.

[0291] Referring to FIG. 3, FIG. 21, and FIG. 22, the processor (160) can determine duplicate images (1820, 1840, 1860) among the static images (1810 to 1860). The duplicate images (1820, 1840, 1860) represent static images to be removed among the static images (1810 to 1860). In FIG. 22, the duplicate images (1820, 1840, 1860) are indicated by dotted lines.

[0292] The static images (1810 to 1860) are images captured of the object (1800). The processor (160) stitches the static images (1810 to 1860) to create a merged image. For convenience of explanation, FIG. 8 illustrates the result of stitching the static images (1810 to 1860).

[0293] The processor (160) determines whether the object (1800) is not missed even if the duplicate images (1820, 1840, 1860) are removed. When the duplicate images (1820, 1840, 1860) are removed, the processor (160) may remove the duplicate images (1820, 1840, 1860) if other static images (1810, 1830, 1850) all contain the object (1800).

[0294]

[0295] Figure 23 is a drawing for explaining the result of removing duplicate images according to one embodiment.

[0296] Referring to FIGS. 3 and 21 to 23, FIG. 23 illustrates an example in which duplicate images (1820, 1840, 1860) are removed from among static images (1810 to 1860), leaving only three static images (1810, 1830, 1850). As illustrated in FIG. 23, even if duplicate images (1820, 1840, 1860) are removed, the remaining three static images (1810, 1830, 1850) include all areas of the target object (1800). Therefore, by removing duplicate images (1820, 1840, 1860), storage space for storing many images can be reduced, and resources for processing many images can be reduced.

[0297]

[0298] FIG. 24 is a diagram illustrating a method for determining a duplicate image according to one embodiment.

[0299] Referring to FIGS. 3 and 24, the processor (160) can determine an overlapping image by comparing the degree to which three adjacent static images (i, i+1, i+2) overlap with each other. The processor (160) can determine an overlapping image by using the area of ​​the area in which the three static images (i, i+1, i+2) overlap with each other and / or the area of ​​each of the static images (i, i+1, i+2).

[0300] The processor (160) calculates the coordinates of the static images (i, i+1, i+2) and confirms the coordinates of the vertices of the static images (i, i+1, i+2). The processor (160) can confirm the area where the static images (i, i+1, i+2) overlap through the coordinates of the vertices of the static images (i, i+1, i+2).

[0301] As an example, the processor (160) can calculate the IOU (Intersection of Unit) as in <Mathematical Formula 1> below.

[0302] <Mathematical Formula 1>

[0303]

[0304] In Fig. 24, the area where static images (i, i+2) overlap is marked in gray.

[0305] The processor (160) can compare the ratio of the area of ​​the overlapping region of the static images (i, i+2) to the area of ​​the static image (i+1) with a threshold value to determine whether to remove the static image (i+1). The processor (160) can remove the static image (i+1) if the IoU exceeds the threshold value. The threshold value can be a number smaller than 1. For example, the threshold value can be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, etc. The larger the threshold value, the fewer static images are removed, and the smaller the threshold value, the more static images are removed. Therefore, the user can determine how many static images are to be removed by setting the size of the threshold value. The size of the threshold value can be determined by the capacity of the storage space or the resources for processing data, etc.

[0306] As another example, the processor (160) can calculate IOU as in <Mathematical Formula 2> below.

[0307] <Mathematical Formula 2>

[0308]

[0309] The processor (160) can compare the ratio of the area of ​​the overlapping area of ​​the static images (i, i+2) to the area of ​​the static image (i) or the static image (i+2) with a threshold value to determine whether to remove the static image (i+1).

[0310] As another example, the processor (160) can calculate IOU as in <Mathematical Formula 3> below.

[0311] <Mathematical Formula 3>

[0312]

[0313] The processor (160) can compare the ratio of the area of ​​the area where the static images (i, i+2) overlap and the area of ​​the union of the static image (i) and the static image (i+2) with a threshold value to determine whether to remove the static image (i+1).

[0314] As another example, the processor (160) can calculate IOU as in <Mathematical Formula 4> below.

[0315] <Mathematical Formula 4>

[0316]

[0317] The processor (160) can compare the ratio of the area of ​​the area where the static images (i, i+2) overlap and the area of ​​the union of the static image (i) and the static image (i+2) with a threshold value to determine whether to remove the static image (i+1).

[0318] As another example, the processor (160) can calculate IOU as in <Mathematical Formula 5> below.

[0319] <Mathematical Formula 5>

[0320]

[0321] The processor (160) can compare the ratio of the area of ​​the overlapping region of static images (i, i+1, i+2) and the area of ​​the intersection of static image (i) and static image (i+2) and the union of static image (i+1) with a threshold value to determine whether to remove static image (i+1).

[0322] Although the above-described <Mathematical Formula 1> to <Mathematical Formula 5> described a method in which the processor (160) determines a duplicate image based on an area where static images overlap, the processor (160) may also determine a duplicate image based on whether or not the object (20) is missing. The processor (160) may identify the object (20) included in the static images and determine a duplicate image based on whether or not the object (20) overlaps.

[0323] The processor (160) may determine a static image that does not contain an object among the static images as a duplicate image, or may determine a static image that contains the same object as another static image as a duplicate image.

[0324] For example, if the object (20) included in the static image (i+1) among three static images (i to i+2) is also included in the static images (i, i+2), the processor (160) may remove the static image (i+1). Alternatively, if the object (20) included in the static image (i) is also included in the static images (i+1, i+2), the processor (160) may remove the static image (i).

[0325] Additionally, the processor (160) can determine a static image that does not include the object (20) as a duplicate image. The processor (160) can immediately remove a static image if the static image does not include the object (20), regardless of whether it overlaps with another static image.

[0326]

[0327] Referring again to FIG. 3 and FIG. 21, the processor (160) can then remove the determined duplicate image (S322).

[0328] Eliminating duplicate images may mean determining which static images among the static images will not be stored, or determining which static images will not be used when generating a merged image. By removing unnecessary duplicate images, the processor (160) can reduce the number of static images stored in the memory (130) and reduce the resources required to analyze static images to find defects.

[0329] Hereinafter, the duplicate image removal process of the processor (160) according to some embodiments of the present invention will be described in more detail with further reference to FIGS. 25 and 26.

[0330]

[0331] FIG. 25 is a drawing for explaining a method for removing duplicate images according to one embodiment.

[0332] Referring to FIG. 3, FIG. 21 and FIG. 25, the processor (160) can sequentially compare the degree to which three adjacent static images overlap to determine overlapping images.

[0333] Figure 25 illustrates removing static image (i+1) from among five static images (i to i+4), followed by removing static image (i+3).

[0334] The processor (160) can sequentially compare static images (i to i+4). Sequentially comparing may mean that the processor (160) repeats the process of comparing a predetermined number of static images.

[0335] The processor (160) first compares three static images (i to i+2) to determine whether to remove the static image (i+1). If the static image (i+1) is removed, the processor (160) compares the static images (i, i+2, i+3) to determine whether to remove the static image (i+2). If the static image (i+2) is not removed, the processor (160) compares the static images (i+2, i+3, i+4) to determine whether to remove the static image (i+3). If the static image (i+3) is removed, the processor (160) ends the operation of removing duplicate images. Therefore, the processor (160) determines that two static images (i+1, i+3) are removed from among the five static images (i to i+4), and two static images (i+1, i+3) are determined to be duplicate images.

[0336] In another embodiment, the processor (160) may determine a duplicate image based on the results of comparing at least four static images. If static image (i) and static image (i+3) sufficiently overlap, the processor (160) may simultaneously remove two static images (i+1, i+2).

[0337]

[0338] FIG. 26 is a drawing for explaining a method for removing duplicate images according to one embodiment.

[0339] Referring to FIG. 3, FIG. 21, FIG. 25 and FIG. 26, in FIG. 25 described above, the processor (160) sequentially compares static images to remove duplicate images, but in FIG. 26, the processor (160) can remove duplicate images after comparing a predetermined number of static images.

[0340] The processor (160) can determine duplicate images among N static images included in the merged image. N is a natural number greater than or equal to 3. For example, the processor (160) can determine which static image to remove among the four or more static images by determining how much the four or more static images overlap each other. In addition, the processor (160) can also determine duplicate images by analyzing all static images to be used to generate the merged image.

[0341] FIG. 26 illustrates that the processor (160) calculates the overlapping area of ​​five static images (i to i+4) and removes two static images (i+1, i+3). The processor (160) can remove static images that overlap with adjacent static images without causing the target object (20) to be omitted from among the five static images (i to i+4). For example, the processor (160) can determine static images in the order of the largest overlapping area with adjacent static images, and determine the static images as duplicate images in the determined order. At this time, the processor (160) can determine whether the target object (20) is omitted while determining the duplicate images in the determined order. If the processor (160) determines that the target object (20) is omitted, it does not determine the image as a duplicate, but performs a decision on the next static image in the order.

[0342] If the target object (20) is a blade, the processor (160) can analyze all static images captured of a blade to determine duplicate images. For example, the processor (160) can determine duplicate images among 100 static images acquired for the first blade. The processor (160) can simultaneously remove the determined duplicate images.

[0343]

[0344] Fig. 27 is a schematic diagram illustrating an example of a device for merging images according to one embodiment.

[0345] Referring to FIG. 27, the device (2000) for merging images includes a processor (2100), a memory (2200), and a communication device (2300). However, the components of the device (2000) are not limited to those illustrated in FIG. 12. In other words, the device (1400) may include at least one more component in addition to the components illustrated in FIG. 12, or at least one of the components illustrated in FIG. 12 may be excluded.

[0346] As described above with reference to FIG. 3, the device (2000) may be included in at least one of the aircraft (10), the server (30), the controller (40), the station (50), and other external devices. Accordingly, the processor (160), the memory (130), and the communication device (150) described above with reference to FIG. 3 may correspond to the processor (2100), the memory (2200), and the communication device (2300) of the device (2000), respectively. Therefore, a detailed description of the processor (2100), the memory (2200), and the communication device (2300) is omitted.

[0347] As described above, the processor (160, 2100) can generate a merged image with low distortion or a distortion level below a reference value regardless of the type of object. Furthermore, the processor (160, 2100) can generate a merged image with low distortion even when merging images using a machine learning model. Furthermore, as described above, after the merged image is generated, the processor (160, 2100) can display the identified defect on the corresponding item in the merged image or remove duplicate images from the merged image.

[0348] Meanwhile, the above-described method can be written as a program that can be executed on a computer, and can be implemented on a general-purpose digital computer that runs the program using a computer-readable recording medium. In addition, the structure of the data used in the above-described method can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium includes storage media such as magnetic storage media (e.g., ROM, RAM, USB, floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD, etc.).

[0349] Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from the essential characteristics of the above-described invention. Therefore, the disclosed methods should be considered illustrative rather than restrictive. The scope of the claims, not the foregoing description, is defined by the scope of the patent, and should be interpreted to encompass all differences within the scope equivalent thereto.

Claims

1. A step of acquiring static images by photographing at least a part of an object through a camera of an aircraft; A step of generating partial images by merging at least two or more of the static images using at least one feature point included in each of the static images; and A step of generating a block image by merging at least two or more of the partial images using at least one feature point included in each of the partial images, The step of acquiring the above static images comprises: acquiring the static images by changing the angle of the camera in a first direction; The step of generating the above partial images comprises: generating the partial images by merging the static images in the first direction; The step of generating the block image comprises generating the block image by merging the partial images in a second direction perpendicular to the first direction. How to merge images.

2. In paragraph 1, The above-mentioned photographing step is to photograph the object by changing the angle of the camera while the aircraft is located at the nth point, The step of generating the above partial images is to generate one partial image by merging static images captured at the nth point, The above n is a natural number greater than or equal to 1 How to merge images.

3. In paragraph 1, The steps of generating the above partial images are: A step of identifying at least one common feature point in a first static image and a second static image selected from among the above static images; A step of determining at least one candidate solution for matching at least one common feature point; A step of merging the first static image and the second static image based on an optimal solution selected from the at least one candidate solution. How to merge images.

4. In paragraph 3, The above candidate solutions are: A method of aligning at least one common feature point by at least one of translation, rotation and scaling of at least one of the first static image and the second static image. How to merge images.

5. In paragraph 3, The above optimal solution is, is selected from among the at least one candidate solution based on at least one pre-established criterion, Excluding candidate solutions that align with a rotation angle exceeding a threshold value from among at least one candidate solution above. How to merge images.

6. In paragraph 3, The above identifying step is, A step of identifying a specific object in each of the static images using a machine learning model, A step of identifying at least one common feature point in each of the static images, excluding the predetermined object, How to merge images.

7. In paragraph 1, The step of generating the above block image is: A step of identifying at least one common feature point in a first partial image and a second partial image selected from among the above partial images; A step of determining at least one candidate solution for matching at least one common feature point; A step of merging the first partial image and the second partial image based on an optimal solution selected from at least one candidate solution, Excluding candidate solutions that include a method of matching by enlarging or reducing by exceeding a threshold value among at least one candidate solution above. How to merge images.

8. In paragraph 1, The above static images are obtained through one flight of the aircraft. How to merge images.

9. In paragraph 1, The above partial images are generated by static images acquired through at least one flight of the aircraft. How to merge images.

10. In paragraph 1, A step of matching at least one block image to an item corresponding to the target object; and Further comprising the step of outputting at least one defect (default) identified from at least one of the static images on the item. How to merge images.

11. In paragraph 1, A step of determining at least one of the partial image and the block image as a merged image; A step of determining duplicate images among the static images using stitching information; and Further comprising a step of removing the determined duplicate images. How to merge images.

12. A computer-readable program for executing the method of paragraph 1 on a computer. Recording medium.

13. At least one memory; and Contains at least one processor, At least one processor, Obtaining static images by photographing at least a portion of the object through the camera of the aircraft, Generating partial images by merging at least two of the static images using at least one feature point included in each of the static images, Merge images to create a block image by merging at least two of the partial images using at least one feature point included in each of the partial images, Obtaining the static images by changing the angle of the camera in the first direction, and generating the partial images by merging the static images in the first direction, Generating the block image by merging the partial images in a second direction perpendicular to the first direction A device that merges images.

14. In paragraph 13, The at least one processor controls the camera to change the angle of the camera to capture the object while the aircraft is located at the nth point, The above n is a natural number greater than or equal to 1 A device that merges images.

15. In paragraph 13, At least one processor, Identifying at least one common feature point in a first static image and a second static image selected from among the above static images, Determine at least one candidate solution for matching at least one common feature point, Merging the first static image and the second static image based on an optimal solution selected from at least one candidate solution. A device that merges images.

16. In paragraph 15, The above candidate solutions are: A method of aligning at least one common feature point by at least one of translation, rotation and scaling of at least one of the first static image and the second static image. A device that merges images.

17. In paragraph 15, The above optimal solution is, is selected from among the at least one candidate solution based on at least one pre-established criterion, Excluding candidate solutions that align with a rotation angle exceeding a threshold value from among at least one candidate solution above. A device that merges images.

18. In paragraph 15, At least one processor, Using a machine learning model, identify a specific object in each of the above static images, Identifying at least one common feature point in each of the static images except for the predetermined object A device that merges images.

19. In paragraph 13, At least one processor, Identifying at least one common feature point in a first partial image and a second partial image selected from among the above partial images, Determine at least one candidate solution for matching at least one common feature point, Merging the first partial image and the second partial image based on an optimal solution selected from at least one candidate solution, Excluding candidate solutions that include a method of matching by enlarging or reducing by exceeding a threshold value among at least one candidate solution above. A device that merges images.

20. In paragraph 13, The above static images are obtained through one flight of the aircraft. A device that merges images.

21. In paragraph 13, The above partial images are generated by static images acquired through at least one flight of the aircraft. A device that merges images.

22. At least one camera; At least one memory; and Contains at least one processor, At least one processor, Obtaining static images by photographing at least a part of the object through the above camera, Generating partial images by merging at least two of the static images using at least one feature point included in each of the static images, A block image is generated by merging at least two of the partial images using at least one feature point included in each of the partial images, Obtaining the static images by changing the angle of the camera in the first direction, and generating the partial images by merging the static images in the first direction, Generating the block image by merging the partial images in a second direction perpendicular to the first direction Aircraft.

Images