Method and system for generating true ortho image

Abstract

The present disclosure relates to a method for generating a true ortho image. The method comprises the steps of: acquiring a plurality of images, position information about each of the plurality of images, and height information about an object appearing in each of the plurality of images; generating, on the basis of the height information, a plurality of patches corresponding to each of the plurality of images from the plurality of images; combining the plurality of patches on the basis of the position information so as to generate the true ortho image; and adjusting the color of the true ortho image on the basis of the color of a plurality of boundary areas adjacent to the boundary line of the plurality of patches, and the height information about the object appearing in the plurality of boundary areas.

Description

Method and system for generating a true image

[0001] The present disclosure relates to a method and system for generating a true image.

[0002] True ortho images, used in digital twin solutions and precision mapping, are created using multiple images or videos of a target area. These images are typically taken from a vertical perspective, looking down at the surface of the target area. These images can be combined in a mosaic fashion by a computing device to create a true ortho image.

[0003] Meanwhile, color differences may exist between the multiple images that form the basis of a true story image. For example, color differences may arise if the images were taken at different times. Furthermore, each image may contain objects of varying heights, such as buildings, trees, and traffic lights, which can create differences in shadows or lighting between the images.

[0004] As described above, when generating a true image using multiple images with color differences due to various factors, these color differences remain in the true image. Therefore, various techniques are being developed to address these color differences before and / or after generating the true image.

[0005] The present disclosure provides a method and system for generating a true image to solve the above problems.

[0006] The present disclosure can be implemented in various ways, including as a method, a computer program stored on a readable storage medium, or a system (device).

[0007] A method for generating a true image according to one embodiment of the present disclosure may include a step of obtaining a plurality of images, position information of each of the plurality of images, and height information of an object appearing in each of the plurality of images, a step of generating a plurality of patches corresponding to each of the plurality of images from the plurality of images based on the height information, a step of generating a true image by combining the plurality of patches based on the position information, and a step of adjusting a color of the true image based on colors of a plurality of boundary areas adjacent to boundary lines of the plurality of patches and height information of an object appearing in the plurality of boundary areas.

[0008] A device for generating a true image according to one embodiment of the present disclosure includes a memory and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program may include commands for obtaining a plurality of images, position information of each of the plurality of images and height information of an object appearing in each of the plurality of images, generating a plurality of patches corresponding to each of the plurality of images from the plurality of images based on the height information, combining the plurality of patches based on the position information to generate a true image, and adjusting a color of the true image based on colors of a plurality of boundary areas adjacent to boundary lines of the plurality of patches and height information of an object appearing in the plurality of boundary areas.

[0009] Methods, systems and computer programs according to various embodiments of the present disclosure can provide a color-adjusted true image based on height information.

[0010] The method, system and computer program according to various embodiments of the present disclosure can make the degree of color adjustment relatively large for a portion in a true image where the height difference is relatively small, and make the degree of color adjustment relatively small for a portion in a true image where the height difference is relatively large.

[0011] The method, system and computer program according to various embodiments of the present disclosure can reduce the problem of the image becoming blurry by maintaining the color of the area close to the original color of the patch when the boundary of the patch occurs in an area with a large height difference in the true image.

[0012] Methods, systems and computer programs according to various embodiments of the present disclosure can mosaic so that patch boundaries do not occur in areas where there is a lot of height information by using height information in the process of generating a true image.

[0013] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs (referred to as “one skilled in the art”) from the description of the claims.

[0014] Embodiments of the present disclosure will be described below with reference to the accompanying drawings, wherein like reference numerals represent similar elements, but are not limited thereto.

[0015] FIG. 1 is an exemplary diagram showing how a true image generation service is provided according to one embodiment of the present disclosure.

[0016] FIG. 2 is a schematic diagram showing a configuration in which an information processing system is connected to enable communication with a plurality of image providing devices and a plurality of user terminals in order to provide a true image generation service according to one embodiment of the present disclosure.

[0017] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure.

[0018] FIG. 4 is an exemplary diagram illustrating a process of generating a true image from multiple images according to one embodiment of the present disclosure.

[0019] FIG. 5 is an exemplary diagram illustrating a process of generating a true image by combining multiple patches generated from multiple images according to one embodiment of the present disclosure.

[0020] FIG. 6 is an exemplary diagram illustrating a process of generating a true image by combining multiple patches according to one embodiment of the present disclosure.

[0021] FIG. 7 is an exemplary diagram illustrating a process of adjusting the color of a true image based on height information according to one embodiment of the present disclosure.

[0022] FIG. 8 is a flowchart illustrating a method for generating a true image according to one embodiment of the present disclosure.

[0023] FIG. 9 is an example of a color-adjusted true image based on height information according to one embodiment of the present disclosure.

[0024] Figure 10 is an example of a color-adjusted true image without considering height information according to a comparative example.

[0025] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0026] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0027] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0028] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0029] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0030] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0031] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0032] In this disclosure, 'color balancing' refers to an image processing technique that adjusts the color gradient between adjacent pixels to achieve color balance in a target image. This technique analyzes the color change pattern of an image to locally correct color imbalances, rather than directly adjusting pixel values. First, the color gradient of an image is calculated to identify areas with abrupt color changes and derive the color change pattern of the image. Next, the color gradient of each pixel of the image is adjusted by considering object height information within the image, image labeling information, weights, etc. to alleviate areas with abrupt color changes in a specific direction and minimize the color gradient between adjacent pixels. Through this, the color imbalance of the entire image is adjusted, and the image color tone is naturally adjusted near the border.

[0033] In this disclosure, a "Digital Surface Model (DSM)" is a three-dimensional model that provides elevation information for all objects on the surface of the earth. A DSM can accurately represent the surface of a terrain by providing elevation information for each point, including all structures and vegetation on the surface, such as buildings, trees, and bridges. DSMs are primarily used in remote sensing and geographic information systems (GIS) and are generated based on data acquired through remote sensing techniques such as aerial photography, satellite imagery, and LiDAR. These models are utilized in various fields, including urban planning, environmental research, disaster management, and communications. For example, in urban planning, they are used to establish development strategies by analyzing building heights and density, while in environmental research, they are used to monitor forest management and ecosystem changes. Furthermore, in disaster management, they play a crucial role in predicting and preparing for natural disasters such as floods and landslides. In the communications field, they are used to select optimal locations for communication towers through radio wave path analysis. DSM acquires multiple images using various methods such as LiDAR, aerial and satellite photos, and radar (SAR), and provides accurate terrain information including all structures on the ground surface within the acquired multiple images.

[0034] FIG. 1 is an exemplary diagram showing how a true image generation service is provided according to one embodiment of the present disclosure.

[0035] The server (100) according to the present disclosure may be a server that provides services related to authenticated images. For example, the server (100) may be a server that generates and provides authenticated images. Alternatively, the server (100) may be a server that provides a map application based on authenticated images.

[0036] In the present disclosure, a "True Ortho" may refer to an orthophoto image generated by combining multiple images (e.g., aerial images, satellite images). The process of generating a true orthophoto may include precisely aligning the positions and angles of each of the multiple images, and correcting distortions according to the height of objects appearing in the multiple images, for example, using a digital ground surface model (DSM). The aligned and corrected images are synthesized into a single integrated image, so that the positional information of each pixel can be accurately expressed. True orthophoto images are utilized in various fields such as urban planning, land management, disaster management, and map making, and are particularly useful for accurately providing topographic information in complex urban environments.

[0037] The server (100) according to the present disclosure can generate a true image of a target area using multiple images. To generate the true image, the server (100) can use location information of each of the multiple images and / or height information of objects appearing in each of the multiple images. In addition, the server (100) can process the true image to adjust its color.

[0038] In one embodiment, the server (100) may receive multiple images of a target area. Here, the multiple images may include at least one of an aerial photograph, a drone photograph, or a satellite photograph obtained by photographing the target area by at least one image providing device (e.g., a camera device).

[0039] In one embodiment, the server (100) may receive location information for each of the plurality of images. The location information may be received together with the plurality of images from an image providing device that acquired the plurality of images, but is not limited thereto. The location information may also be received separately from the plurality of images from an external device separate from the image providing device. Alternatively, the server (100) may receive the plurality of images from the image providing device and then analyze the plurality of images to extract location information for each of the plurality of images.

[0040] In one embodiment, the image providing device captures and acquires a target area, and each of the multiple images received by the server (100) may include an object in the target area. Here, the object is an object having height information from the ground in the target area, and may include structures or facilities such as buildings, traffic lights, roads, trees, ground surfaces, sidewalks, towers, or signs located in the target area. In addition to the aforementioned artifacts, the object may also include living beings such as people, animals, or plants that may have height information from the ground in the target area.

[0041] In one embodiment, the server (100) may obtain height information of each object of the target area included in each of the plurality of images. Here, the height information of each object may refer to height information of the object from the ground of the target area. In addition, the height information of each object may be viewed as height information of a pixel of each of the plurality of images. This height information of the object may be calculated by an image providing device or another device while photographing the target area, but is not limited thereto, and the server (100) or another computing device that receives the plurality of images from the image providing device may calculate the height information by analyzing each of the plurality of images. In addition, the height information of the object may be calculated by a separate analysis device (not shown) other than the plurality of image providing devices that obtain the plurality of images of the target area or the server (100) that generates a true image and adjusts the color, and transmitted to the server (100).

[0042] The server (100) according to the present disclosure can generate a true image (110) with improved quality. The true image may exhibit color differences at the boundaries between multiple images or patches that form the basis of the true image. The server (100) can process the portions of the true image where color differences appear to reduce the color differences. This will be described in detail below in FIG. 2.

[0043] The server (100) can provide the generated real-time image (110) to the user terminal (102). The user terminal (102) can display the real-time image (110) in various forms. For example, the user can execute a map application installed on the user terminal (102). In addition, the user terminal (102) can receive the real-time image (110) for the target area from the server (100) in response to the location information of the user terminal (102) or the user's input, and output it on the display of the user terminal (102).

[0044] The true image (110) output to the user terminal (102) may be an image in which the color difference in the boundary area between the plurality of images or the plurality of patches used to generate the true image (110) is reduced. In addition, the true image (110) output to the user terminal (102) may be an image in which the color difference is adjusted based on the height difference of the object appearing in the boundary area. For example, when the height difference is large, the degree of color difference adjustment in the boundary area may be relatively small, and when the height difference is small, the degree of color difference adjustment in the boundary area may be relatively large. The user of the user terminal (102) may be provided with a true image (110) of improved quality.

[0045] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) is connected to a plurality of image providing devices (210_1, 210_2) and a plurality of user terminals (240_1, 240_2, 240_3) so as to be able to communicate with each other in order to provide a true image generation service according to one embodiment of the present disclosure.

[0046] The information processing system (230) may include a system capable of providing a true image generation service. In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to the true image generation service, or one or more distributed computing devices and / or distributed databases based on cloud computing services. For example, the information processing system (230) may include separate systems (e.g., servers) for the true image generation service.

[0047] A plurality of image providing devices (210_1, 210_2) can transmit a plurality of captured images to an information processing system (230) via a network (220). The information processing system (230) can generate a true image using the plurality of images received from the plurality of image providing devices (210_1, 210_2). Here, a true image generating service, etc. provided by the information processing system (230) can be provided to users through applications, web browsers, etc. installed on each of a plurality of user terminals (240_1, 240_2, 240_3).

[0048] A plurality of image providing devices (210_1, 210_2) and a plurality of user terminals (240_1, 240_2, 240_3) can communicate with an information processing system (230) via a network (220). The network (220) can be configured to enable communication between the plurality of image providing devices (210_1, 210_2) and the plurality of user terminals (240_1, 240_2, 240_3) and the information processing system (230). Here, the network (220) can be configured as a wired network (220) such as Ethernet, a wired home network (220) (Power Line Communication), a telephone line communication device, and RS-serial communication, a wireless network (220) such as a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof, depending on the installation environment. The communication method is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network (220) may include, but also short-range wireless communication between an image providing device (210_1, 210_2) or a user terminal (240_1, 240_2, 240_3) and an information processing system (230).

[0049] In FIG. 2, a mobile phone terminal (240_1), a tablet terminal (240_2), and a PC terminal (240_3) are illustrated as examples of user terminals, but are not limited thereto, and the user terminals (240_1, 240_2, 240_3) may be any computing devices capable of wired and / or wireless communication. For example, the user terminals may include smartphones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, etc. In addition, although FIG. 2 illustrates three user terminals (240_1, 240_2, 240_3) communicating with the information processing system (230) via the network (220), this is not limited thereto, and a different number of user terminals may be configured to communicate with the information processing system (230) via the network (220).

[0050] In FIG. 2, a satellite (210_1) and a digital camera (210_2) are illustrated as examples of image providing devices, but the present invention is not limited thereto, and the image providing devices (210_1, 210_2) may include drone devices, aerial photography devices, satellites, etc. that are capable of wired and / or wireless communication and that photograph the sky above a target area including a subject. For example, the image providing devices may include drones, smartphone cameras, panoramic cameras, LiDAR (Light Detection and Ranging) systems, satellite radars, UAVs (Unmanned Aerial Vehicles), reconnaissance aircraft, aerial photography cameras, observation balloons, airships, etc. In addition, although FIG. 2 illustrates two image providing devices (210_1, 210_2) communicating with an information processing system (230) via a network (220), the present invention is not limited thereto, and two or more image providing devices may be configured to communicate with an information processing system (230) via a network (220).

[0051] In FIG. 2, the information processing system (230) can obtain multiple images, location information of each of the multiple images, and height information of objects appearing in each of the multiple images from multiple image providing devices (210_1, 210_2). As an example, the location information or height information may exist in the form of metadata for each of the multiple images. As another example, the location information or height information may be generated by the multiple image providing devices (210_1, 210_2) or another device and then transmitted to the information processing system (230). As another example, the location information and height information may be generated from multiple images by the information processing system (230).

[0052] The information processing system (230) can generate a plurality of patches corresponding to each of a plurality of images, and can generate a true image by combining the plurality of patches. Additionally or alternatively, the information processing system (230) can receive a plurality of patches corresponding to each of a plurality of images or a true image generated by combining the plurality of patches from a plurality of image providing devices (210_1, 210_2). Based on this, the information processing system (230) can adjust the color of the true image based on the plurality of images, position information of each of the plurality of images, height information of objects appearing in each of the plurality of images, a plurality of patches corresponding to each of the plurality of images, color information of a plurality of boundary areas adjacent to boundaries of the plurality of patches, or a true image generated by combining the plurality of patches.

[0053] In one embodiment, the height information of each object may be acquired based on a digital surface model (DSM) generated from multiple images, but is not limited thereto. For example, the depth values ​​of pixels included in multiple images corresponding to objects in a target area may be acquired, and the height information of the object may be calculated based on the acquired depth values. In addition, distance information to the object may be received from at least one of a lidar and an infrared sensor, and the height information of the object may be acquired based on the distance information.

[0054] In one embodiment, the height information of each object can be calculated using stereo vision, LiDAR, time of flight (ToF), structured light, a machine learning model using an artificial intelligence algorithm, or GPS. For example, stereo vision is a method of calculating depth information (or height information) of an object by placing two cameras at a certain interval to capture the same target area or the same object, and using triangulation based on the parallax of the two images acquired from each camera. In addition, LiDAR is a method of calculating height information of an object by firing a laser pulse at a target area or object, measuring the time it takes for it to reflect and return, converting the time it takes for it to arrive into a distance, and generating a three-dimensional (3D) point cloud. In addition, the time of flight method is a method of calculating height information of an object by firing light from a ToF camera to a target area or object, measuring the time it takes for it to reflect and return, and converting the time it takes for it to arrive into a distance. Structured light is a method of projecting patterned light, such as a grid pattern, onto a target area or object and analyzing the reflected, distorted pattern to derive height information about the target object. Furthermore, using artificial neural network models such as U-NET or RseNET, object height information can be extracted from a single image.

[0055] In one embodiment, the information processing system (230) may generate a plurality of patches corresponding to each of the plurality of images based on the plurality of images, position information of each of the plurality of images, or height information of objects included in the plurality of images. As an example, the plurality of images may correspond to images captured using perspective projection, and the plurality of patches corresponding to each of the plurality of images generated by the information processing system (230) from the plurality of images may correspond to images obtained using parallel projection. That is, the information processing system (230) may generate the plurality of patches by converting the plurality of images, which are images of a perspective view, into images of a parallel view. As another example, the plurality of images may be images captured using a line scan method.

[0056] In one embodiment, the information processing system (230) can generate a true image by combining multiple generated patches. That is, the true image can be generated by combining multiple patches with parallel viewpoints by converting them into parallel projections, rather than combining multiple images with perspective viewpoints by having multiple image providing devices capture a target area or object in perspective projection. As another example, the true image can be generated by combining multiple patches with parallel viewpoints by converting the images into parallel projections of the target area or object by multiple image providing devices in line scan. In this case, the images captured in line scan can be used to generate a true image as multiple patches, or can be converted into multiple patches and then used to generate a true image. A detailed description of a method for generating a true image by combining multiple patches will be described later with reference to FIGS. 4 to 7.

[0057] The information processing system (230) according to the present disclosure can adjust the color of a true image. According to one embodiment, the information processing system (230) can adjust the color of at least a portion of a true image generated by combining a plurality of patches by considering labeling information of the plurality of patches, color information of a plurality of boundary regions adjacent to the boundaries of the plurality of patches, height information of objects appearing in the plurality of boundary regions, etc. Here, adjusting the color of a specific region may mean adjusting the color value of a pixel included in the specific region or a pixel value representing a color. In addition, the degree of color adjustment may mean an adjustment value of a color value or pixel value.

[0058] In one embodiment, the information processing system (230) can identify boundaries between patches within a true image generated by combining multiple patches. The information processing system (230) can set a predetermined boundary area based on the boundary area. The information processing system (230) can set the boundary area within a variety of ranges. As an example, the information processing system (230) can set an area having a predetermined distance (e.g., a predetermined number of pixels) from the boundary area as the boundary area. As another example, the information processing system (230) can set the entire area of ​​a patch as the boundary area.

[0059] The information processing system (230) can identify objects included in a plurality of boundary areas (a first boundary area and a second boundary area) arranged on both sides of a boundary line, and calculate the height difference of each object. The information processing system (230) can determine the degree of color adjustment of the first boundary area and the second boundary area based on the calculated height difference of the objects. The information processing system (230) can set weights for the color values ​​of each boundary area according to the height difference of the objects included in the first boundary area and the second boundary area. For example, the greater the height difference of the objects between the boundary areas, the greater the weight can be set for each boundary area. The greater the weight set for each boundary area, the less the adjustment value of each boundary area can be in the color adjustment step. Accordingly, the color value before adjustment for boundary areas with large weights, i.e., boundary areas with large object height differences, can be maintained as much as possible. Details of adjusting the color of a true-image by color value weights in consideration of labeling information, color information, or object height information will be described later with reference to FIGS. 6 and 7.

[0060] Although the information processing system (230) is described above as providing a true image generation service, the present disclosure is not limited thereto, and hardware / software for providing a true image generation service may be equipped in a user terminal (240_1, 240_2, 240_3).

[0061] FIG. 3 is a block diagram showing the internal configuration of a user terminal (240) and an information processing system (230) according to one embodiment of the present disclosure.

[0062] The user terminal (240) may refer to any computing device capable of executing an image color adjustment application, a web browser, etc., and capable of wired / wireless communication, and may include, for example, a mobile phone terminal (240_1), a tablet terminal (240_2), a PC terminal (240_3) of FIG. 2. As illustrated, the user terminal (240) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (240) and the information processing system (230) may be configured to communicate information and / or data via a network (220) using their respective communication modules (316, 336). Additionally, the input / output device (320) may be configured to input information and / or data to the user terminal (240) or output information and / or data generated from the user terminal (240) via the input / output interface (318).

[0063] The memory (312, 332) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the user terminal (240) or the information processing system (230) as a separate permanent storage device distinct from the memory. In addition, an operating system and at least one program code may be stored in the memory (312, 332).

[0064] These software components may be loaded from a computer-readable recording medium separate from the memory (312, 332). This separate computer-readable recording medium may include a recording medium directly connectable to the user terminal (240) and the information processing system (230), and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (312, 332) through a communication module (316, 336) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (312, 332) based on a computer program that is installed by files provided by developers or a file distribution system that distributes installation files of applications through a network (220).

[0065] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by a memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a storage device such as the memory (312, 332).

[0066] The communication module (316, 336) may provide a configuration or function for the user terminal (240) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (240) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system, etc.). For example, a request or data generated by the processor (314) of the user terminal (240) according to a program code stored in a recording device such as a memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) may be received by the user terminal (240) via the communication module (316) of the user terminal (240) via the communication module (336) and the network (220).

[0067] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera, a keyboard, a microphone, a mouse, etc., including an audio sensor and / or an image sensor, and the output device may include a device such as a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface (318) may be a means for interfacing with a device that has a configuration or function integrated into one for performing input and output, such as a touch screen. In FIG. 3, the input / output device (320) is illustrated not to be included in the user terminal (240), but is not limited thereto and may be configured as a single device with the user terminal (240). In addition, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is illustrated as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0068] The user terminal (240) and the information processing system (230) may include more components than those shown in FIG. 3. However, there is no need to explicitly illustrate most of the conventional components. In one embodiment, the user terminal (240) may be implemented to include at least some of the input / output devices (320) described above. In addition, the user terminal (240) may further include other components, such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, etc. For example, if the user terminal (240) is a smartphone, it may include components that a smartphone generally includes, and various components, such as an acceleration sensor, a gyro sensor, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration, may be implemented to be further included in the user terminal (240).

[0069] FIG. 4 is an exemplary diagram illustrating a process of generating a true image from multiple images according to one embodiment of the present disclosure.

[0070] In one embodiment, the processor (334) of the information processing system (230) may receive multiple images (410) from an image providing device and generate a true image (450) based on the multiple images (410). The multiple images (410) may include a first image (412), a second image (414), and a third image (416). Here, the first image (412), the second image (414), and the third image (416) may correspond to images taken of multiple areas of a target area, which is a subject. In addition, the first image (412), the second image (414), and the third image (416) may each correspond to at least one of an aerial photograph, a drone photograph, or a satellite photograph. In FIG. 4, three images (412, 414, 416) are illustrated as multiple images (410), but the present disclosure is not limited thereto, and the number of multiple images (410) may be any number.

[0071] In one embodiment, the processor may generate a digital surface model (DSM) from a plurality of images (410). The DSM may include height information (420) for each object included in each of the plurality of images. Furthermore, the DSM may be implemented in the form of a three-dimensional image or generated in the form of two-dimensional data.

[0072] Additionally, the processor may generate a plurality of patches (430) corresponding to each of the plurality of images based on the DSM including the height information (420). For example, the processor may obtain a first patch (432), a second patch (434), and a third patch (436) from the first image (412), the second image (414), and the third image (416), respectively, based on the height information (420) obtained from the DSM.

[0073] The plurality of images (410) may correspond to perspective view images acquired by taking pictures using a perspective projection method such as aerial photography, drone photography, or satellite photography, or images taken using a line scan method. The plurality of patches (430) may correspond to parallel view images converted or adjusted using a parallel projection method. That is, the plurality of patches (430) may correspond to images of the ground surface viewed from vertically above the target area. Here, at least one of the plurality of images from a perspective view may include an occluded area that is not visible, such as the back of a building, depending on the shooting position or view. If any of the plurality of images from a perspective view includes an occluded area, a patch for the area may be generated from another image in which the area is properly taken, and thus the area is likely to become a boundary between the plurality of patches.

[0074] In one embodiment, the DSM may include location information (440) for each of the plurality of images. The location information (440) may include coordinate information for a predetermined area of ​​each of the plurality of images. Additionally, the location information (440) may include type information for the predetermined area of ​​each of the plurality of images. For example, the type information included in the location information (440) may include information that can identify which object, such as a road, a building, a field, a sign, or a billboard, corresponds to a predetermined area of ​​each of the plurality of images. Additionally, the coordinate information included in the location information (440) may include a location value of a specific object.

[0075] In one embodiment, the processor may generate a true image (450) by combining a plurality of patches (430) based on a DSM including height information (420) and / or position information (440) of a plurality of images (410) through a series of processes. A detailed description of the series of processes for generating a true image is provided below in FIG. 5.

[0076] FIG. 5 is an exemplary diagram illustrating a process of generating a true image by combining multiple patches generated from multiple images according to one embodiment of the present disclosure.

[0077] In one embodiment, the processor may obtain a plurality of images (510) from an image providing device. Here, the plurality of images (510) may correspond to the plurality of images (410) of FIG. 4.

[0078] In one embodiment, the processor may extract a DSM (520) from a plurality of images (510), which includes location information of an object or a predetermined area within the plurality of images and / or height information of an object included in the plurality of images. Although FIG. 5 illustrates the DSM (520) as a two-dimensional image, the present invention is not limited thereto and may be expressed as two-dimensional data, 2.5-dimensional data, three-dimensional data, or the like. The DSM (520) may be image data having a height value at a location corresponding to each pixel.

[0079] In one embodiment, the processor can generate multiple patches (530) from multiple images and extract multiple partial images within a given region from the multiple patches. The processor can then mosaic the multiple partial images based on their locations. Details of the method for extracting multiple partial images within a given region from the multiple patches are described in detail later in FIG. 7.

[0080] In one embodiment, when a plurality of patches (530) corresponding to each of the plurality of images are generated from a plurality of images, at least some of the plurality of patches (530) may include a predetermined common area. A plurality of partial images including such a common area may be extracted. Among the plurality of partial images including the common area, a first partial image may include an occluded area of ​​a specific object, and a second partial image may include color information corresponding to the occluded area. That is, the processor may mosaic the plurality of partial images according to their positions by taking into account whether the common area includes an occluded area, etc.

[0081] FIG. 6 is an exemplary diagram illustrating a process of generating a true image by combining multiple patches according to one embodiment of the present disclosure.

[0082] In one embodiment, a processor can generate a true image from multiple patches. There may be a common area overlapping between the multiple patches. In one embodiment, to generate a true image for the common area or a portion of the common area, the processor can select any one of the multiple patches. In other words, for a common area overlapping between multiple patches, the processor can generate a true image for the common area using one patch or multiple patches.

[0083] In one embodiment, the processor can generate a true image from multiple patches using a mosaic technique. The processor can extract multiple partial images within a given area from the multiple patches. The processor can generate a true image by mosaicking the extracted multiple partial images according to location.

[0084] As illustrated in FIG. 6, the processor can obtain multiple patches (620, 630) for a predetermined area (610) from multiple images. The predetermined area (610) includes a first area (610a) in which a rooftop of a building appears and a second area (610b) in which the ground and a vehicle appear. The predetermined area (610) shows a boundary (b) of the rooftop of the building, and the first area (610a) and the second area (610b) are divided based on the boundary (b). The first patch (620) shows the entire first area (610a) and a part of the second area (610b). The second patch (630) shows a part of the first area (610a) and the entire second area (610b). The processor can use the first patch (620) and the second patch (630) to generate a true image for the predetermined area (610).

[0085] In one embodiment, the processor may set boundaries for each patch for generating a true image. The area within the boundaries of each patch may be used for generating a true image. Conversely, the area outside the boundaries of each patch may not be used for generating a true image.

[0086] The processor according to the present disclosure can use height information during the mosaic process to mosaic without creating patch boundaries in areas with significant height differences. For example, this mosaic method may be feasible for small areas where occlusion areas do not occur significantly.

[0087] In one embodiment, the processor may set multiple candidate boundary lines to define the boundaries of a patch. The processor may identify multiple candidate partial images adjacent to the candidate boundary lines. The processor may calculate the height difference between the multiple candidate partial images that touch the candidate boundary lines. For example, the processor may calculate the height difference of an object appearing in a portion of the multiple candidate partial images that touch the candidate boundary lines.

[0088] In one embodiment, the processor can determine whether the calculated height difference is less than or equal to a threshold value. If the height difference between objects appearing in multiple candidate partial images that touch the candidate boundary line is less than or equal to the threshold value, the processor can extract the multiple candidate partial images as multiple partial images. Conversely, if the height difference between objects appearing in multiple candidate partial images that touch the candidate boundary line exceeds the threshold value, the processor can exclude the multiple candidate partial images from the multiple partial images. In other words, the candidate boundary line is excluded, and the above-described process can be performed centered on another candidate boundary line.

[0089] In one embodiment, the processor may use a method other than a threshold value to determine the magnitude of the calculated height difference. For example, the processor may determine whether the calculated height difference is greater or lesser based on a preset algorithm or mathematical formula. In this case, the processor may determine whether the calculated height difference is determined to be a boundary line or extracted as a partial image based on the result of the judgment.

[0090] As illustrated in FIG. 6, the processor may set a plurality of candidate boundary lines (640). The plurality of candidate boundary lines (640) may include, but are not limited to, a first candidate boundary line (640a) and a second candidate boundary line (640b). The processor may identify a plurality of neighboring candidate partial images centered around the second candidate boundary line (640b). The processor may calculate a height difference between the plurality of candidate partial images that touch the second candidate boundary line (640b). The height difference calculated by the processor may correspond to a height difference between a building and the ground or a building and a vehicle. If the processor determines that the height difference exceeds a threshold value, the processor may exclude the plurality of candidate partial images that touch the second candidate boundary line (640b).

[0091] The processor can identify a plurality of neighboring candidate partial images centered on the first candidate boundary line (640a). The processor can calculate a height difference between the plurality of candidate partial images that touch the first candidate boundary line (640a). In some areas of the plurality of candidate partial images that touch the first candidate boundary line (640a), a rooftop of a building at substantially the same level appears in common. Therefore, the processor can calculate a height difference close to '0'. The processor determines that the height difference is less than or equal to a threshold value and can extract the plurality of candidate partial images that touch the first candidate boundary line (640a) as a plurality of partial images. That is, the processor can extract a plurality of partial images (650, 660) with respect to a predetermined area (610) based on the first candidate boundary line (640a). The extracted plurality of partial images (650, 660) can be mosaicked according to their positions to generate a true image for the predetermined area (610).

[0092] FIG. 7 is an exemplary diagram illustrating a process of adjusting the color of a true image based on height information according to one embodiment of the present disclosure.

[0093] A processor according to the present disclosure can adjust the color of a true image based on the color of a plurality of boundary regions adjacent to the boundary lines of a plurality of patches and the height information of an object appearing in the boundary regions. In one embodiment, the processor can generate an improved true image by adjusting the weights of a portion with a large height difference by reflecting the height value of the DSM so that the color of the original image or patch can be maintained. That is, the processor can adjust the color in a portion with little height difference in the same way as before, and adjust the weights in a portion with a large height difference so that the color of the original image or patch can be maintained more.

[0094] In one embodiment, the processor may calculate the height difference between objects appearing in each of a plurality of boundary regions adjacent to the boundary line. Based on the calculated height difference, the processor may assign a weight corresponding to the height difference to each of the plurality of boundary regions. Furthermore, the processor may determine a color adjustment value for each of the plurality of boundary regions based on the weight.

[0095] In one embodiment, the weight and the height difference may have a positive correlation. For example, the weight may increase as the height difference increases, and the weight may decrease as the height difference decreases. As another example, the height difference may be divided into sections, and different weights may be applied to each section, but the present disclosure is not limited thereto.

[0096] In one embodiment, the color adjustment value and the weight may have a negative correlation. For example, as the weight increases, the color adjustment value may decrease, and as the weight decreases, the color adjustment value may increase. As another example, the weight may be divided into sections, and different color adjustment values ​​or adjustment ratios may be applied to each section, but the present disclosure is not limited thereto.

[0097] As illustrated in FIG. 7, a building rooftop (710) appears as an object in the image of the real-time image. A plurality of patches (720, 730, 740) may be used to generate the image of the real-time image. The plurality of patches (720, 730, 740) illustrated in FIG. 7 may be part of a plurality of patches converted from a plurality of images, but are not limited thereto.

[0098] The processor can calculate a height difference between objects appearing in a first-first boundary area (722) of a first patch (720) adjacent to a first boundary line (b1) and a second boundary area (732) of a second patch (730). In addition, the processor can calculate a height difference between objects appearing in a first-second boundary area (724) of a first patch (720) adjacent to a second boundary line (b2) and a third boundary area (742) of a third patch (740). According to one embodiment, the height difference can be calculated based on height information of an object appearing in a specific area of ​​a boundary area. The boundary area can be a portion of a patch or the entire area. The boundary area can be an area within a predetermined distance (e.g., a predetermined number of pixels) from a boundary line, but is not limited thereto.

[0099] The height of the object appearing in the first boundary area (722) may correspond to the height of the building rooftop. The height of the object appearing in the second boundary area (732) may correspond to the height of the ground or a vehicle. The processor may calculate a value corresponding to the height difference between the building rooftop and the ground or the vehicle, and assign a weight corresponding to the value to the first boundary area (722) and the second boundary area (732). The processor may assign a relatively large weight to the first boundary area (722) and the second boundary area (732). The color adjustment value for the first boundary area (722) and the second boundary area (732) may be relatively small.

[0100] The heights of objects appearing in the first-second boundary area (724) and the third boundary area (742) may correspond to the same height of the building rooftop. The processor may calculate a height difference that is substantially close to '0' and assign a weight corresponding to the value to the first-second boundary area (724) and the third boundary area (742). The processor may assign a relatively small weight to the first-second boundary area (724) and the third boundary area (742). The color adjustment values ​​for the first-second boundary area (724) and the third boundary area (742) may be relatively large.

[0101] In one embodiment, a patch transformed from a perspective viewpoint to a balanced viewpoint may have occluded areas, such as the back of a building, that are not visible depending on the shooting location. The processor can mosaic these occluded areas into other patches. In this case, the area between the rooftop of the building and the ground, for example, near the second boundary line (b2), is likely to be the boundary of the mosaicked patch. When patches are mosaicked along the boundary of the building, the rooftop and the ground of the building become the boundary of the patch and are typically of different colors. In some embodiments, the processor can exclude these boundary areas from color balancing.

[0102] In one embodiment, with respect to excluding from the color adjustment described above, the processor may determine whether the height difference exceeds a threshold value. If the height difference exceeds the threshold value, the processor may maintain the colors of multiple boundary areas without adjustment. In one embodiment, the processor may determine that the height difference between the first-first boundary area (722) and the second boundary area (732) bordering the second boundary line (b2) exceeds the threshold value. In this case, the processor may maintain the colors of the first-first boundary area (722) and the second boundary area (732) without adjustment.

[0103] FIG. 8 is a flowchart illustrating a method for generating a true image according to one embodiment of the present disclosure.

[0104] According to one embodiment, the method for generating a true image may be executed by a processor included in an information processing system. According to another embodiment, the method for generating a true image may be executed by a processor installed or built into a computing device included in a user terminal.

[0105] The method (800) may be initiated by a processor obtaining (S810) a plurality of images, location information of each of the plurality of images, and height information of an object appearing in each of the plurality of images. In one embodiment, the processor may obtain height information of an object appearing in each of the plurality of images, including generating a digital surface model (DSM) from the plurality of images and obtaining height information of the object based on the DSM. Additionally or alternatively, the processor may obtain height information of an object appearing in each of the plurality of images, including obtaining a depth value for a pixel corresponding to the object, and obtaining height information of the object based on the depth value. Furthermore, additionally or alternatively, the processor may obtain height information of an object appearing in each of the plurality of images, including receiving distance information to the object from at least one of a lidar and an infrared sensor, and obtaining height information of the object based on the distance information. Here, the plurality of images may include at least one of an aerial photograph, a drone photograph, and a satellite photograph. Furthermore, the object may include at least one of a building, a ground surface, and a tree.

[0106] The processor can generate a plurality of patches corresponding to each of the plurality of images from the plurality of images (S820). In one embodiment, the processor can generate the plurality of patches by acquiring the plurality of images captured in a line scan manner, or by converting the plurality of images, which are perspective view images, into parallel view images.

[0107] The processor may generate a true image for a common area by combining a plurality of patches based on location information (S830). In one embodiment, the processor may generate a true image by extracting a plurality of partial images within a predetermined area from the plurality of patches and mosaicking the plurality of partial images according to locations. Here, extracting the plurality of partial images may include identifying a plurality of neighboring candidate partial images centered on a candidate boundary line within the predetermined area from the plurality of patches, determining whether a height difference between the plurality of candidate partial images that touch the candidate boundary line is less than or equal to a threshold value, and extracting the plurality of candidate partial images as the plurality of partial images if the height difference is less than or equal to the threshold value. Additionally or alternatively, extracting the plurality of partial images may include excluding the plurality of candidate partial images if the height difference exceeds the threshold value.

[0108] The processor may adjust the color of the tactile image based on the color of the plurality of boundary areas adjacent to the boundary lines of the plurality of patches and the height information of the objects appearing in the plurality of boundary areas (S840). In one embodiment, the processor may adjust the color of the tactile image by calculating the height difference between the objects appearing in each of the first boundary area and the second boundary area adjacent to a predetermined boundary line among the plurality of boundary areas, and determining the degree of color adjustment of the first boundary area and the second boundary area based on the height difference. Additionally or alternatively, determining the degree of color adjustment may include assigning a weight corresponding to the height difference to the first boundary area and the second boundary area, and determining an adjustment value of the color value of the first boundary area and the second boundary area based on the weight. Here, the weight may increase as the height difference increases, and the adjustment value may decrease as the weight increases. Additionally or alternatively, determining the degree of color adjustment may include determining whether the height difference exceeds a threshold, and if the height difference exceeds the threshold, determining to maintain the color of the first boundary area and the second boundary area without adjustment.

[0109] FIG. 9 is an example diagram of a color-adjusted true image based on height information according to one embodiment of the present disclosure. FIG. 10 is an example diagram of a color-adjusted true image without considering height information according to a comparative example.

[0110] The comparative example according to Fig. 10 may represent a true-color image generated by performing color adjustment according to a conventional method. The first portion (1010) represents a thin portion with a large color difference, such as a traffic sign. The traffic sign shown in the first portion (1010) may have a large height difference from the road. If color adjustment according to a conventional method is performed, a phenomenon in which heterogeneous colors spread widely may occur in the first portion (1010). Furthermore, although a very rare case, an issue may occur in which a very distinct color difference spreads widely and can be easily detected.

[0111] The second portion (1020) represents the area where the building rooftop and the ground connect. The building rooftop shown in the second portion (1020) may have a significant height difference from the ground. If color adjustment is performed according to a conventional method, the second portion (1020) may suffer from a problem of imbalance. For example, the ground portion may be affected by the bright color of the building's rooftop, while the rooftop portion may be affected by the dark color of the ground, resulting in a slightly blurred appearance.

[0112] On the other hand, FIG. 9 may represent a color-adjusted true image based on height information according to an embodiment of the present disclosure. Looking at the portion corresponding to the first portion (1010) of FIG. 9 in FIG. 10 , it can be seen that the phenomenon of heterogeneous colors spreading widely between traffic signs and roads does not occur or is reduced. In the true image generated according to the present disclosure, color adjustment is performed based on the height difference between the traffic sign and the road, and as a result, the colors of the traffic sign and the road can be maintained close to their original colors.

[0113] In addition, when examining the portion corresponding to the second portion (1020) of FIG. 10 in FIG. 9, it can be seen that the phenomenon of the colors of the building rooftop and the road becoming hazy does not occur or is reduced. In the true-color image generated according to the present disclosure, color adjustment is performed according to the height difference between the building rooftop and the road, and as a result, the colors of the building rooftop and the road can be maintained close to their original colors.

[0114] The image color adjustment method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0115] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0116] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0117] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0118] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0119] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0120] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

[0121] [Explanation of symbols]

[0122] 100: Processor

[0123] 102: User terminal

[0124] 110: Color-adjusted true image

Claims

1. A step of obtaining a plurality of images, location information of each of the plurality of images, and height information of an object appearing in each of the plurality of images; A step of generating a plurality of patches corresponding to each of the plurality of images from the plurality of images based on the height information; A step of generating a true image by combining the plurality of patches based on the above location information; and A step of adjusting the color of the true image based on the color of a plurality of boundary areas adjacent to the boundary lines of the plurality of patches and the height information of the object appearing in the plurality of boundary areas. A method for generating a true image, comprising:

2. In paragraph 1, The steps for adjusting the color of the above image are: A step of calculating a height difference between objects appearing in each of the first boundary area and the second boundary area adjacent to a predetermined boundary line among the plurality of boundary areas; and A method for generating a true image, comprising a step of determining the degree of color adjustment of the first boundary area and the second boundary area based on the height difference.

3. In paragraph 2, The step of determining the degree of color adjustment is: A step of assigning a weight corresponding to the height difference to the first boundary area and the second boundary area; and A method for generating a true image, comprising the step of determining color adjustment values ​​of the first boundary area and the second boundary area based on the weights.

4. In paragraph 3, The above weight increases as the height difference increases, A method for generating a true image, wherein the above adjustment value decreases as the above weight increases.

5. In paragraph 2, The step of determining the degree of color adjustment is: a step of determining whether the height difference exceeds a threshold value; and A method for generating a true image, comprising a step of determining to maintain the colors of the first boundary area and the second boundary area without adjustment when the height difference exceeds the threshold value.

6. In paragraph 1, The steps of generating the above true image are: A step of extracting a plurality of partial images within a predetermined area from the plurality of patches; and A method for generating a true image, comprising a step of mosaicking the plurality of partial images according to their positions.

7. In paragraph 6, The step of extracting the above multiple partial images is: A step of identifying a plurality of neighboring candidate partial images centered on a candidate boundary line within the predetermined area from the plurality of patches; A step of determining whether the height difference of the plurality of candidate partial images that touch the candidate boundary line is less than or equal to a threshold value; and A method for generating a true image, comprising a step of extracting the plurality of candidate partial images as the plurality of partial images when the height difference is less than or equal to the threshold value.

8. In paragraph 7, The step of extracting the above multiple partial images is: A method for generating a true image, further comprising a step of excluding the plurality of candidate partial images when the height difference exceeds the threshold value.

9. In paragraph 1, The step of generating the above multiple patches is: A method for generating a true image, comprising the step of converting the plurality of images, which are perspective view images, into parallel view images.

10. In paragraph 1, The step of obtaining the above height information is: A step of generating a digital surface model (DSM) from the plurality of images; and A method for generating a true image, comprising the step of obtaining height information of the object based on the digital ground surface model.

11. In paragraph 1, The step of obtaining the above height information is: A step of obtaining a depth value for a pixel corresponding to the object; and A method for generating a true image, comprising a step of obtaining height information of the object based on the depth value.

12. In paragraph 1, The step of obtaining the above height information is: A step of receiving distance information to the object from at least one of a lidar and an infrared sensor; and A method for generating a true image, comprising the step of obtaining height information of the object based on the distance information.

13. In paragraph 1, A method for generating a true image, wherein the plurality of images include at least one of an aerial photograph, a drone photograph, and a satellite photograph.

14. In paragraph 1, A method for generating a true image, wherein the object comprises at least one of a building, a ground surface, and a tree.

15. A computer-readable recording medium storing a computer program for executing the method according to paragraph 1 on a computer.

16. As a true image generation system, memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, Acquire a plurality of images, location information of each of the plurality of images, and height information of an object appearing in each of the plurality of images, Based on the above height information, a plurality of patches corresponding to each of the plurality of images are generated from the plurality of images, Based on the above location information, a true image is generated by combining the plurality of patches, A system for generating a true image, comprising commands for adjusting the color of the true image based on the color of a plurality of boundary areas adjacent to the boundary lines of the plurality of patches and the height information of objects appearing in the plurality of boundary areas.

Images