Image color adjustment method and system

Abstract

The present disclosure relates to an image color adjustment method. The method comprises the steps of: identifying a first image and a second image continuously arranged on the basis of a seam line; dividing each of the first image and the second image into a plurality of nodes on the basis of the seam line; setting a weight of each of the plurality of nodes for the first image and the second image on the basis of a distance from the seam line; and adjusting pixel values of the plurality of nodes for the first image and the second image on the basis of the weight of each of the plurality of nodes, wherein an adjustment value of the pixel value of each of the plurality of nodes is associated with the weight of each of the plurality of nodes.

Description

Image color adjustment method and system

[0001] The present disclosure relates to a method and system for adjusting image color, and more particularly, to a method and system for adjusting the color of a target image based on a boundary in a target image in which a plurality of images are mixed.

[0002] True ortho images, used in digital twin solutions and precision map creation, are created by combining multiple aerial photos in a mosaic fashion. This process can introduce color differences and color inconsistencies at edges. These issues degrade image quality, so various image processing techniques are used to address them.

[0003] One conventional image processing technique is to blend the edges of a mosaic image using a buffer area to mitigate color differences at the edges. However, if the color difference is significant, the image may still appear blotchy, and ghosting can occur when moving objects are present.

[0004] The present disclosure provides an image color adjustment method and device (system) to solve the above problems.

[0005] The present disclosure can be implemented in various ways, including as a method, a device (system), or a computer program stored on a readable storage medium.

[0006] According to one embodiment of the present disclosure, an image color adjustment method performed by at least one processor includes the steps of identifying a first image and a second image that are sequentially arranged based on a seam line, dividing each of the first image and the second image into a plurality of nodes based on the seam line, setting weights of each of the plurality of nodes of the first image and the second image based on a distance from the seam line, and adjusting pixel values ​​of each of the plurality of nodes of the first image and the second image based on the weights of each of the plurality of nodes, wherein the adjusted values ​​of the pixel values ​​of each of the plurality of nodes are associated with the weights of each of the plurality of nodes.

[0007] An image color adjustment system according to one embodiment of the present disclosure includes a memory and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for identifying a first image and a second image that are sequentially arranged based on a boundary line, dividing each of the first image and the second image into a plurality of nodes based on the boundary line, setting weights of each of the plurality of nodes of the first image and the second image based on a distance from the boundary line, and adjusting pixel values ​​of each of the plurality of nodes of the first image and the second image based on the weights of each of the plurality of nodes, wherein the adjusted values ​​of the pixel values ​​of each of the plurality of nodes are associated with the weights of each of the plurality of nodes.

[0008] In various embodiments of the present disclosure, a method, system, and computer program can be provided that can perform color balancing centered on a boundary between images while maintaining the unique color of each image.

[0009] In various embodiments of the present disclosure, weights are set for a plurality of nodes of a target image based on their distance from a boundary line, with the weights being set smaller the closer to the boundary line and larger the farther from the boundary line, so that nodes or pixels near the boundary line undergo sufficient color change to maintain image color correction performance, while at the same time, color change of nodes or pixels far from the boundary line is sufficiently limited to maintain the color of the entire image.

[0010] In various embodiments of the present disclosure, a target image is divided into a plurality of nodes by grouping at least one pixel of the target image based on a boundary line, such that the size of each node is smaller the closer it is to the boundary line, and the size of each node is larger the farther it is from the boundary line, thereby performing basic work for performing image balancing for each node based on the distance from the boundary line in the image color adjustment step.

[0011] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs (referred to as “one skilled in the art”) from the description of the claims.

[0012] Embodiments of the present disclosure will be described below with reference to the accompanying drawings, wherein like reference numerals represent similar elements, but are not limited thereto.

[0013] FIG. 1 is a drawing showing an example of color correction of an image according to one embodiment of the present disclosure.

[0014] FIG. 2 is a schematic diagram showing a configuration in which an information processing system (230) is connected to a plurality of image providing devices (210_1, 210_2) and a plurality of user terminals (240_1, 240_2, 240_3) so as to be able to communicate with each other in order to provide an image color adjustment service according to one embodiment of the present disclosure.

[0015] FIG. 3 is a block diagram showing the internal configuration of a user terminal (240) and an information processing system (230) according to one embodiment of the present disclosure.

[0016] FIG. 4 is an example diagram of a target image (400) that combines a plurality of captured images including a first image and a second image according to one embodiment of the present disclosure.

[0017] FIG. 5 is an example diagram of labeling information of a target image according to one embodiment of the present disclosure.

[0018] FIG. 6 is a diagram illustrating an example of a method for dividing a target image (600) into multiple nodes according to one embodiment of the present disclosure.

[0019] FIG. 7 is a diagram illustrating an example of a method for dividing a target image into multiple nodes according to one embodiment of the present disclosure.

[0020] FIG. 8A and FIG. 8B are exemplary diagrams illustrating a method for setting weights for each of a plurality of nodes of a target image according to one embodiment of the present disclosure.

[0021] FIG. 9 is an exemplary diagram illustrating a method for adjusting pixel values ​​according to one embodiment of the present disclosure.

[0022] FIG. 10 is a flowchart illustrating an image color adjustment method according to one embodiment of the present disclosure.

[0023] FIGS. 11 to 13 are exemplary diagrams showing the effect of image color adjustment considering weights set based on a boundary line according to one embodiment of the present disclosure.

[0024] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0025] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0026] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0027] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0028] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0029] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0030] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0031] FIG. 1 is a diagram illustrating an example of color correction of an image according to one embodiment of the present disclosure. According to one embodiment, a processor (100) may generate a color-adjusted correction image (110) based on a boundary from a target image including boundary information, and provide the correction image (110) to a user via a user terminal (102). Here, the processor (100) may be a processor included in a server belonging to a provider area that provides an image color adjustment service and is connected to the user terminal (102) via network communication. However, the present disclosure is not limited thereto, and the processor (100) may be included in the user terminal (102).

[0032] According to one embodiment, the processor (100) may receive a target image including a boundary line. Here, the target image may include a first image and a second image that are arranged consecutively based on the boundary line. The processor (100) may identify the first image and the second image from at least one received image. Here, the fact that the first image and the second image are arranged consecutively based on the boundary line may mean that the first image and the second image share the boundary line. According to one embodiment, the first image and the second image may mean that there is a common part between the photographed object, environment, subject, building, etc. In this case, the first image and the second image may represent a common object, and at least a portion of each of the first image and the second image may be combined with each other based on the common object. According to another embodiment, the first image and the second image may not have a common part with respect to the photographed object. In this case, at least a portion of each of the first image and the second image may be combined with each other based on a specific object or specific information included in each of the first image and the second image.

[0033] According to one embodiment, the processor (100) may receive a target image (or composite image) in which at least a portion of each of a plurality of images including the first and second images is synthesized or mosaicked, and / or labeling information of the target image, in order to identify each of the first image and the second image, which are sequentially arranged based on a boundary line from the received target image. Here, the labeling information of the target image may mean source information of each pixel included in the target image.

[0034] According to one embodiment, the processor (100) may receive a plurality of images including the first and second images and / or labeling information for each of the plurality of images, in order to identify each of the first and second images sequentially arranged based on a boundary line from the received target image. Here, the labeling information of the target image may refer to source information of each pixel included in each of the plurality of images.

[0035] In one embodiment, the labeling information may be expressed in various forms. For example, the labeling information may include an image (labeling image) that expresses image information representing the source of each pixel in the target image or multiple images as pixel values.

[0036] In one embodiment, the labeling information of the target image may include not only source information of each pixel of the target image, i.e., where each pixel of the target image comes from among a plurality of images including the first image and the second image, but also boundary information included in the target image. That is, in a target image in which at least a portion of each of a plurality of images including the first image and the second image is combined with each other, the boundary may be viewed as a point where the source of each pixel of the target image changes. For example, if the target image is a combination of a portion of the first image and a portion of the second image, the boundary may be viewed as a set of pixels of the first image combined with a portion of the second image, or a set of pixels of the second image combined with a portion of the first image. Furthermore, in the above case, if the target image includes a first pixel and a second pixel, the first pixel comes from the first image, the second pixel comes from the second image, and the first pixel and the second pixel are adjacent to each other, the first pixel and the second pixel may be viewed as touching the boundary. In some embodiments, the boundary may be viewed as including the first pixel and the second pixel. A detailed description of the labeling information is provided later in Fig. 5.

[0037] In one embodiment, a target image in which at least a portion of each of a plurality of images, including a first image and a second image, is combined with each other may include one or more boundary lines. For example, as illustrated in FIG. 1, the target image may include a first boundary line, a second boundary line, and a third boundary line. Here, the first boundary line may correspond to a boundary where at least a portion of each of the first image and the second image is combined. In addition, the second boundary line may correspond to a boundary where at least a portion of the second image and the third image is combined, and the third boundary line may correspond to a boundary where at least a portion of the third image and the fourth image is combined. The boundary line information is not limited thereto, and in another embodiment, the second boundary line may correspond to a boundary where at least a portion of the second image and the first image are combined, and the third boundary line may correspond to a boundary where at least a portion of the first image and the third image are combined.

[0038] In one embodiment, it is necessary to adjust a color imbalance that occurs based on a boundary line of a target image. For example, if the target image is a combination of at least a portion of each of a plurality of images including a first image and a second image, each of the plurality of images including the first image and the second image may correspond to images of a common subject captured from various viewpoints and in various environments. That is, the pixel values ​​(i.e., color values ​​representing the color of each pixel) of each pixel of the first image and the second image, which are different from each other in terms of the subject of capture, capture time (time), capture environment, or capture time (view), may not be uniformly distributed. That is, if a target image is generated by physically combining at least a portion of each of the first image and the second image with a common subject portion of the first image and the second image as a boundary line, the target image may have a color difference based on the boundary line. A detailed description of the color imbalance that occurs based on the boundary line is described later in FIG. 4.

[0039] In one embodiment, the processor (100) can identify a first image and a second image from a target image, extract boundary information, and correct the pixel values ​​of all pixels so that the difference in pixel values ​​(or color values) between each pixel of the target image and its adjacent pixels is minimized based on the boundary. However, in order to generate a corrected image (110) with a minimum difference in pixel values ​​at the boundary, pixels included in the target image that are adjacent to each other around the boundary and have different image sources can be actively corrected to perform color balancing. On the other hand, adjacent pixels included in the target image that are derived from the same image can be regarded as having already been color balanced and can be passively corrected. Accordingly, the processor (100) can use the labeling information to determine whether each pixel and its adjacent pixels of the target image are derived from the same image or different images, and if the former, the pixel values ​​of each pixel and its adjacent pixels are maintained, but if the latter, the pixel values ​​of each pixel can be adjusted so that the difference in pixel values ​​of the adjacent pixels is minimized. Additionally, the processor (100) may adjust the pixel value of a pixel of the target image based on the distance the pixel is from the boundary line, based on the labeling information and the boundary line information. That is, the processor (100) according to one embodiment may set a weight that takes into account the distance of each pixel of the target image from the boundary line, so as to actively adjust the pixel value of a pixel close to the boundary line and passively adjust the pixel value of a pixel far from the boundary line.Accordingly, in order to adjust the color imbalance between pixels near the boundary in the target image as well as the overall color of the target image, the processor (100) adjusts the target image pixel by pixel based on the boundary or adjusts it node by node by grouping a plurality of pixels into one node (or group), and divides the nodes based on the distance between each pixel or node and the boundary, and further sets the weight of each pixel or each node based on the distance between the boundary lines to adjust the pixel value of each pixel. That is, a detailed description of a method for dividing a target image into a plurality of nodes based on the boundary line and a method for setting the weight of each pixel or each node based on the boundary line will be described later with reference to FIGS. 6 to 8B. In addition, a method for adjusting the color of the target image by minimizing the difference in the pixel value of each pixel or node of the target image will be described later with reference to FIG. 9.

[0040] In one embodiment, the processor (100) can directly synthesize the target image. That is, the processor (100) may receive a target image in which at least a portion of each of a plurality of images, including a first image and a second image, is combined based on a boundary line, and correct the pixel values ​​of the target image through a series of steps. Alternatively, the processor (100) may directly synthesize a target image by combining at least a portion of each of a plurality of images, including a first image and a second image, based on a boundary line, and correct the pixel values ​​of the target image using the synthesized target image. In addition, prior to synthesizing the first image and the second image, the processor (100) may preprocess the pixel values ​​of each of the plurality of images including a common subject using an average value, a median value, a mode value, etc., and then select the first image and the second image to be combined based on the same boundary line. Accordingly, prior to color correction for minimizing the difference in pixel values ​​of adjacent images based on the boundary line, in the step of generating the target image, a filtering step is performed so that adjacent images to be combined based on the boundary line have as similar color tones as possible, thereby generating a corrected image (110) in which the difference in pixel values ​​based on the boundary line is further minimized. A detailed description of this is described in Figs. 11 to 13.

[0041] FIG. 2 is a schematic diagram illustrating a configuration in which an information processing system (230) is connected to a plurality of image providing devices (210_1, 210_2) and a plurality of user terminals (240_1, 240_2, 240_3) so as to be able to communicate with each other in order to provide an image color adjustment service according to one embodiment of the present disclosure. The information processing system (230) may include a system capable of providing an image color adjustment service. In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to the image color adjustment service, or one or more distributed computing devices and / or distributed databases based on a cloud computing service. For example, the information processing system (230) may include separate systems (e.g., servers) for the image color adjustment service. The plurality of image providing devices (210_1, 210_2) can transmit the plurality of captured images or the target image synthesized from the plurality of images to the information processing system (230) via the network (220). The information processing system (230) can provide an image color adjustment service that creates a corrected image with the color of the target image adjusted based on the boundary of the target image received from the plurality of image providing devices (210_1, 210_2). Here, the image color adjustment service, etc. provided by the information processing system (230) can be provided to the user through an application, web browser, etc. installed on each of the plurality of user terminals (240_1, 240_2, 240_3).

[0042] A plurality of image providing devices (210_1, 210_2) and a plurality of user terminals (240_1, 240_2, 240_3) can communicate with an information processing system (230) via a network (220). The network (220) can be configured to enable communication between the plurality of image providing devices (210_1, 210_2) and the plurality of user terminals (240_1, 240_2, 240_3) and the information processing system (230). Here, the network (220) can be configured as a wired network (220) such as Ethernet, a wired home network (220) (Power Line Communication), a telephone line communication device, and RS-serial communication, a wireless network (220) such as a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof, depending on the installation environment. The communication method is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network (220) may include, but also short-range wireless communication between an image providing device (210_1, 210_2) or a user terminal (240_1, 240_2, 240_3) and an information processing system (230).

[0043] In FIG. 2, a mobile phone terminal (240_1), a tablet terminal (240_2), and a PC terminal (240_3) are illustrated as examples of user terminals, but are not limited thereto, and the user terminals (240_1, 240_2, 240_3) may be any computing devices capable of wired and / or wireless communication. For example, the user terminals may include smartphones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, etc. In addition, although FIG. 2 illustrates three user terminals (240_1, 240_2, 240_3) communicating with the information processing system (230) via the network (220), this is not limited thereto, and a different number of user terminals may be configured to communicate with the information processing system (230) via the network (220).

[0044] In one embodiment, the information processing system (230) may receive a target image including a plurality of captured images including a first image and a second image, or at least a portion of each of the plurality of captured images combined along a boundary line, and labeling information of the target image including boundary line information from a plurality of image providing devices (210_1, 210_2). The plurality of image providing devices (210_1, 210_2) may include a drone device, an aerial photography device, a satellite, etc. that captures the sky above a target area including a subject. In FIG. 2, the plurality of image providing devices (210_1, 210_2) are illustrated as existing outside the information processing system (230) as a subject that generates a target image by combining (synthesizing) at least a portion of each of the plurality of captured images along a boundary line. However, the present invention is not limited thereto, and a synthesis module (not shown) as a subject that generates the target image may be located and used within the information processing system (230). In another embodiment, a separate image synthesis device (not shown) is located outside the information processing system (230), and the separate image synthesis device receives a plurality of captured images captured by the image providing devices (210_1, 210_2) via a network (220), generates a target image along a boundary line of each portion of the plurality of captured images, transmits the generated target image to the information processing system (230) via the network (220), and the information processing system (230) can perform color adjustment on the target image received from the separate image synthesis device.

[0045] In FIG. 2, the information processing system (230) is illustrated as generating a color-adjusted image and providing it to a user terminal after receiving a plurality of captured images or target images from a plurality of image providing devices (210_1, 210_2), but this is not limited thereto, and hardware / software for providing an image color adjustment service may be provided in the user terminal.

[0046] FIG. 3 is a block diagram showing the internal configuration of a user terminal (240) and an information processing system (230) according to one embodiment of the present disclosure. The user terminal (240) may refer to any computing device capable of executing an image color adjustment application, a web browser, etc., and capable of wired / wireless communication, and may include, for example, a mobile phone terminal (240_1), a tablet terminal (240_2), a PC terminal (240_3), etc. of FIG. 2. As illustrated, the user terminal (240) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (240) and the information processing system (230) may be configured to communicate information and / or data via a network (220) using respective communication modules (316, 336). In addition, the input / output device (320) may be configured to input information and / or data to the user terminal (240) or output information and / or data generated from the user terminal (240) via the input / output interface (318).

[0047] The memory (312, 332) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the user terminal (240) or the information processing system (230) as a separate permanent storage device distinct from the memory. In addition, an operating system and at least one program code may be stored in the memory (312, 332).

[0048] These software components may be loaded from a computer-readable recording medium separate from the memory (312, 332). This separate computer-readable recording medium may include a recording medium directly connectable to the user terminal (240) and the information processing system (230), and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (312, 332) through a communication module (316, 336) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (312, 332) based on a computer program that is installed by files provided by developers or a file distribution system that distributes installation files of applications through a network (220).

[0049] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by a memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a storage device such as the memory (312, 332).

[0050] The communication module (316, 336) may provide a configuration or function for the user terminal (240) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (240) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system, etc.). For example, a request or data generated by the processor (314) of the user terminal (240) according to a program code stored in a recording device such as a memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) may be received by the user terminal (240) via the communication module (316) of the user terminal (240) via the communication module (336) and the network (220).

[0051] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera, a keyboard, a microphone, a mouse, etc., including an audio sensor and / or an image sensor, and the output device may include a device such as a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface (318) may be a means for interfacing with a device that has a configuration or function integrated into one for performing input and output, such as a touch screen. In FIG. 3, the input / output device (320) is illustrated as not being included in the user terminal (240), but is not limited thereto and may be configured as a single device with the user terminal (240). In addition, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is illustrated as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0052] The user terminal (240) and the information processing system (230) may include more components than those shown in FIG. 3. However, there is no need to explicitly illustrate most of the conventional components. In one embodiment, the user terminal (240) may be implemented to include at least some of the input / output devices (320) described above. In addition, the user terminal (240) may further include other components, such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, etc. For example, if the user terminal (240) is a smartphone, it may include components that a smartphone generally includes, and various components, such as an acceleration sensor, a gyro sensor, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration, may be implemented to be further included in the user terminal (240).

[0053] FIG. 4 is an exemplary diagram of a target image (400) that combines multiple captured images, including a first image and a second image, according to one embodiment of the present disclosure. In one embodiment, the processor may receive a target image (400) that will be subject to image color adjustment from an image providing device. Here, the target image (400) may include an image generated by synthesizing multiple captured images of a target area captured from above the target area in a mosaic manner as a true ortho image.

[0054] In one embodiment, the processor may receive multiple captured images of a target area from an image providing device, synthesize the multiple captured images, and generate a target image (400). In addition, using an image synthesis device separately located outside the processor, the image synthesis device may receive multiple captured images, synthesize the multiple captured images, generate a target image (400), and transmit the generated target image (400) to a processor that performs image color adjustment.

[0055] In one embodiment, the image providing device can capture multiple captured images by photographing the target area from above. For example, as illustrated in FIG. 4, the image providing device can capture a road from above a road on which a vehicle is passing. Furthermore, the image providing device can divide the road into multiple sections and capture images, and capture images of each section as multiple captured images. Here, each section may include common parts, but is not limited thereto, and may not include common parts. However, in order to create a true image of the captured object, it is preferable that at least the entire set of each section includes the entire subject without omitting any part of the subject.

[0056] In one embodiment, the image providing device may generate a target image (400) using a plurality of captured images. Here, the target image (400) may include an image generated by combining a plurality of partial images, each of which is at least a portion of a plurality of captured images, based on a boundary line. In addition, the boundary line may correspond to one of the sets of pixels of a common portion of the plurality of captured images. For example, as illustrated in FIG. 4, the image providing device may combine a first partial image (410), which is at least a part of a first captured image that captures a first section of a target road, and a second partial image (420), which is at least a part of a second captured image that captures a second section, based on a first boundary line (412), combine a second partial image (420), which is at least a part of a second captured image that captures a second section, and a third partial image (430), which is at least a part of a third captured image that captures a third section, based on a second boundary line (422), and combine a third partial image (430), which is at least a part of a third captured image that captures a third section, and a fourth partial image (440), which is at least a part of a fourth captured image that captures a fourth section, based on a third boundary line (432), to generate a target image (400) of a target area. However, the first image, the second image, the third image, and the fourth image may all be different from each other, but this is not limited thereto. For example, the first image and the third image may correspond to the same image, the second image may correspond to an image different from the first image and the third image, and the fourth image may correspond to an image different from the first image, the second image, and the third image.That is, the first partial image (410) and the third partial image (430) may correspond to a part of the first image (or the third image), the second partial image (420) may correspond to a part of a second image that is different from the first image or the third image, and the fourth partial image (440) may correspond to a part of a fourth image that is different from the first image (= the third image) and the second image.

[0057] In one embodiment, each boundary line included in the target image (400) may correspond to one of the sets of pixels common to adjacent captured images based on each boundary line among a plurality of captured images. For example, as illustrated in FIG. 4, a first partial image (410) and a second partial image (420) may be adjacent to each other based on a first boundary line (412) included in the target image (400). According to some embodiments, the first boundary line may correspond to one of the sets of common pixels in which a common portion is captured in each subject that is the shooting target of each of the first image including the first partial image (410) and the second image including the second partial image (420). That is, when the first image and the second image are images acquired by an image providing device by shooting a first zone and a second zone of a specific road, images in which common portions in the first zone and the second zone are captured may be included in each of the first image and the second image. In this case, parts of the first image and the second image can be combined based on some of the common parts in the first and second regions to synthesize one image of the first and second regions, and the boundary where parts of the first and second images are combined can correspond to the first boundary line.

[0058] In one embodiment, the image providing device may capture the same road to obtain a first image and a second image. However, the first image and the second image may be images captured of the first section and the second section of the same road, respectively, in terms of the shooting target, images captured at different times in terms of the shooting time (time), images captured from different angles or heights in terms of the shooting view, and images captured in different weather, seasons, or under different conditions such as shadows or lighting in terms of the shooting external environment. For example, as illustrated in FIG. 4, the first partial image (410) may correspond to a portion of the first image captured of the first section of the car road, which is the shooting target, and the second partial image (420) may correspond to a portion of the second image captured of the second section, and the first section and the second section may include the same subject in common. Here, the first image may correspond to an image of the first area taken by a drone from 100 meters above the ground during a sunny lunchtime in summer, and the second image may correspond to an image of the second area taken by a person with a camera from 200 meters above the ground on an autumn day during the sunset. In this case, the first and second images may have been taken in different shooting environments and thus may have different overall color tones. However, since they include a common part of each subject, the first partial image (410), which is a part of the first image, and the second partial image (420), which is a part of the second image, may be combined by using a part of the common part as the first boundary line (412). The remaining part of the target image (400) may be generated in the same manner.

[0059] In another embodiment, the image providing device may divide the photographing target into a plurality of areas and photograph the area to obtain a plurality of photographed images including a first photographed image and a second photographed image, and transmit the obtained plurality of photographed images to an image synthesizing device separate from the image providing device. In this case, the image synthesizing device may generate a target image (400) by combining a plurality of partial images including a first partial image and a second partial image, which are at least a portion of each of the plurality of photographed images including the first photographed image and the second photographed image, based on a boundary line as described above. The image synthesizing device may provide the generated target image (400) to a processor, and the processor may perform image color adjustment based on a boundary line included in the target image (400).

[0060] FIG. 5 is an exemplary diagram of labeling information of a target image according to one embodiment of the present disclosure. In one embodiment, the processor may receive labeling information of the target image. Here, the target image may include an image generated by combining partial images, including a first partial image and a second partial image, which are each part of a plurality of images, including a first image and a second image. In addition, the labeling information of the target image may include source information of each of the plurality of partial images, including the first partial image and the second partial image, which constitute the target image.

[0061] In the present disclosure, the labeling information may include any format capable of indicating the source information for each of the multiple images. In one embodiment, the labeling information may be a mask image indicating the source information for each of the multiple images. In another embodiment, the labeling information may be metadata indicating the source information for each of the multiple images. However, the format of the labeling information in the present disclosure is not limited to a mask image or metadata.

[0062] In one embodiment, the labeling information may include a labeling image (500) that expresses the source of each of a plurality of partial images, including a first partial image and a second partial image constituting the target image, as pixel values. For example, the first partial image (410), the second partial image (420), the third partial image (430), and the fourth partial image (440) constituting the target image of FIG. 4 may each be derived from the first image, the second image, the third image, and the fourth image. Furthermore, the first image, the second image, the third image, and the fourth image may be different from each other. Here, the labeling image (500) included in the labeling information of the target image (400) may include an image composed of a first label image (510) in which pixels included in a first partial image (410) derived from a first image in the target image are colored with a first color, a second label image (520) in which pixels included in a second partial image (420) derived from a second image are colored with a second color different from the first color, a third label image (530) in which pixels included in a third partial image (430) derived from a third image are colored with a third color different from the first color and the second color, and a fourth label image (540) in which pixels included in a fourth partial image (440) derived from a fourth image are colored with a fourth color different from the first color, the second color, and the third color. That is, the labeling image (500) may include an image in which the photographed image that is the source of each pixel of the target image is identified, and a plurality of pixels in the target image that are the same source photographed image are painted with the same color, and pixels in the different source photographed images are painted with different colors. However, the labeling image (500) is not limited thereto, and may include an image in which a plurality of pixels in the target image that are the same source are painted with the same pattern, and a plurality of pixels in the target image that are different sources are painted with different patterns.

[0063] In one embodiment, the labeling information of the target image is not limited to the labeling image (500), and may include metadata about the source information of each pixel of the target image. For example, the labeling information may be expressed in the form of a pixel source map, a metadata file in XML, JSON, CSV, YAML format, or other formats, or embedded metadata such as EXIF ​​or XMP.

[0064] In one embodiment, the labeling information of the target image may include labeling data that numerically represents the image information that is the source of each pixel included in the target image. For example, the labeling data may include a data set that combines the coordinate values ​​of each pixel included in the target image and the image information that is the source of each pixel. For example, if the target image is an image in which at least a portion of a first image, a second image, and a third image is combined, and the target image includes a first pixel, a second pixel, a third pixel, and a fourth pixel, and the first pixel and the second pixel of the target image are from the first image, the third pixel is from the second image, and the fourth pixel is from the third image, the labeling data may be a set including 'ab00' which combines the coordinate values ​​(a, b) of the first pixel and the first image identification value (00), 'cd00' which combines the coordinate values ​​(c, d) of the second pixel and the first image identification value (00), 'ef01' which combines the coordinate values ​​(e, f) of the third pixel and the second image identification value (01), and 'gh10' which combines the coordinate values ​​(g, h) of the fourth pixel and the third image identification value (10), respectively, with the identification values ​​of the first image, the second image, and the third image as 00, 01, and 10, respectively.

[0065] In one embodiment, the labeling information of the target image may include not only source information of each pixel of the target image, but also boundary information. For example, as illustrated in FIG. 5, the boundary lines of the first label image (510) and the second label image (520) in the labeling image (500) included in the labeling information may correspond to the first boundary line (412) that is the boundary of the first partial image (410) and the second partial image (420) constituting the target image. That is, since the first partial image and the second partial image in the target image include pixels of various colors, it may be difficult to extract the pixels included in the first boundary line based only on the target image, the first partial image, or the second partial image. In this case, if labeling information in which the same data (for example, the same pixel value in the labeling image (500)) is assigned to pixels of the same source in the target image is used together, the boundary information can be extracted more easily. For example, in a labeling image (500) as illustrated in FIG. 5, a boundary line may be determined as a set of pixels in which the pixel values ​​assigned to each pixel differ between adjacent pixels. In this way, boundary line information may be expressed as a set of pixels in which the labeling information assigned to each pixel differs between adjacent pixels.

[0066] By this configuration, the processor can extract boundary information of the target image based on the target image and labeling information, identify a plurality of partial images including a first partial image and a second partial image constituting the target image based on the boundary information, and further identify a captured image including each partial image.

[0067] In one embodiment, the processor may receive labeling information or location information for each of a plurality of images included in a target image. Based on at least one of the labeling information and the location information, the processor may identify each image included in the target image and a boundary between the images.

[0068] In one embodiment, the target image may be an image obtained by synthesizing a second image based on a first image. Labeling information and positional information of each image may be used to synthesize the first and second images. The processor may use the labeling information as well as the positional information for the first and second images to identify the first image, the second image, and the boundary. For example, the first image may be a true image of a specific city, and the second image may be a true image of a specific sphere within the specific city. The positional information of the first image and / or the second image may be absolute or relative information.

[0069] FIG. 6 is a diagram illustrating an example of a method for dividing a target image (600) into a plurality of nodes according to one embodiment of the present disclosure. In one embodiment, the processor may divide each of a first image and a second image adjacent to a first boundary line included in the target image into a plurality of nodes based on the boundary line. Here, a node may correspond to a single pixel or a group including a plurality of pixels. In addition, the sizes of each of the plurality of nodes may be the same or different from each other. Here, the size of a node may be determined in proportion to the number of pixels included in the node as the area of ​​the node.

[0070] In one embodiment, the processor may identify a first image (610), a boundary (612), and a second image (620) in a target image (600). In one embodiment, the processor may segment a first region (630) in the first image (610) that touches the boundary (612) into a plurality of first nodes (632). In addition, the processor may segment a second region (640) in the first image that touches the first region (630) and is further from the boundary (612) than the first region (630) into a plurality of second nodes (642). In addition, the processor may segment a third region (650) in the first image that touches the second region (640) and is further from the boundary (612) than the second region (640) into a plurality of third nodes (652). Here, each of the first nodes may have a first pixel unit, each of the second nodes may have a second pixel unit larger than the first pixel unit, and each of the third nodes may have a third pixel unit larger than the second pixel unit.

[0071] In one embodiment, the processor may divide a first region (630) that touches a boundary line (612) in a first image (610) into a plurality of first nodes (632). Then, a second region (640) that touches the plurality of first nodes (632) in the first image and is further from the boundary line (612) than the plurality of first nodes (632) may be divided into a plurality of second nodes (642). Furthermore, a third region (650) that touches the plurality of second nodes (642) in the first image and is further from the boundary line (612) than the plurality of second nodes (642) may be divided into a plurality of third nodes (652). Here, each of the first nodes may have a first pixel unit, each of the second nodes may have a second pixel unit that is larger than the first pixel unit, and each of the third nodes may have a third pixel unit that is larger than the second pixel unit. For example, the first pixel unit may be a 1*1 pixel unit, the second pixel unit may be a 2*2 pixel unit, and the third pixel unit may be a 4*4 pixel unit.

[0072] In one embodiment, the processor may identify a plurality of pixels of a target image (or a first image, a second image) bordering a boundary as a first pixel unit and define them as a plurality of first nodes. For example, as illustrated in FIG. 6, the processor may identify a plurality of pixels (630) of a first image (610) bordering a boundary as a first pixel unit (a*a pixels) and define them as a plurality of first nodes (632). Then, the processor may merge a plurality of pixels of the first image bordering the plurality of first nodes as a second pixel unit and define them as a plurality of second nodes. For example, as illustrated in FIG. 6, a plurality of pixels (640) of the first image (610) bordering the plurality of first nodes (632) may be merged as a second pixel unit (ba*ba) and define them as a plurality of second nodes (642). Then, the processor can define a plurality of third nodes by merging a plurality of pixels that are adjacent to a plurality of second nodes and are further from the boundary than the plurality of second nodes into a third pixel unit. For example, the processor can define a plurality of third nodes (652) by merging a plurality of pixels (650) that are adjacent to a plurality of second nodes (642) and are further from the boundary than the plurality of second nodes into a third pixel unit (cb*cb). Here, the first pixel unit (a*a) may be smaller than the second pixel unit (ba*ba), and the second pixel unit (ba*ba) may be smaller than the third pixel unit (cb*cb). However, it is not limited thereto, and for example, if the processor defines up to an arbitrary Nth node with the same rule as above, the first pixel unit to the Mth pixel unit, which is a predetermined critical minimum pixel unit - where M and N are natural numbers and M is smaller than N - are of the same size, and the M+1th pixel unit may be smaller than the M+2nd pixel unit, and the M+2nd pixel unit may be smaller than the M+3rd pixel unit.

[0073] Through this configuration, by grouping at least one pixel of the target image based on the boundary, the target image is divided into multiple nodes, with the size of each node being smaller the closer it is to the boundary, and the size of each node being larger the farther it is from the boundary, thereby performing the basic work of performing image balancing for each node based on the distance from the boundary in the image color adjustment step. A more detailed description of this will be provided later in Fig. 7.

[0074] FIG. 7 is a diagram illustrating an example of a method for dividing a target image into multiple nodes according to one embodiment of the present disclosure. In one embodiment, the processor can divide the target image into multiple nodes by associating the size of each node with its distance from a boundary line. For example, the node division method illustrated in FIG. 6 can be applied to divide the target image illustrated in FIG. 4 into multiple nodes. Furthermore, the node division method can be applied as is even if the target image has multiple boundary lines.

[0075] In one embodiment, partitioning the target image into a plurality of nodes by relating the size of each node to its distance from the boundary may include partitioning the image such that nodes closer to the boundary include a smaller number of pixels and nodes farther from the boundary include a relatively larger number of pixels, such that nodes farther from the boundary are larger than nodes closer to the boundary.

[0076] In one embodiment, as illustrated in FIG. 7, with 1*1 pixel as the first pixel unit, a region consisting of a plurality of pixels adjacent to the boundary by a first pixel unit is defined as a first region, and the first region can be divided into a plurality of first nodes (i.e., one first node is equivalent to one first pixel) by the first pixel unit. Then, with 2*2 pixels as the second pixel unit, a region adjacent to the first region by a second pixel unit and further from the boundary than the first region can be defined as a second region, and the second region can be divided into a plurality of second nodes by the second pixel unit. With 4*4 pixels as the third pixel unit, a region adjacent to the second region by a third pixel unit and further from the boundary than the second region can be defined as a third region, and the third region can be divided into a plurality of third nodes by the third pixel unit. Accordingly, the target image can be divided into a first region, a second region, and a third region in a direction away from the boundary line, and the first region can include a plurality of first nodes each having a size of 1*1 pixels, the second region can include a plurality of second nodes each having a size of 2*2 pixels, and the third region can include a plurality of third nodes each having a size of 4*4 pixels.

[0077] In another embodiment, partitioning the target image into a plurality of nodes by relating the size of each node to its distance from the boundary may include partitioning such that nodes closer to the boundary include a smaller number of pixels, nodes farther from the boundary include a relatively larger number of pixels, nodes closer to the boundary by a threshold minimum distance have the same size, and nodes farther from the boundary have a larger size than nodes closer to the boundary in an area exceeding the threshold minimum distance from the boundary.

[0078] In another embodiment, the target image may be divided into multiple nodes in the shape of an n*n pixel unit square (n is a natural number) as shown in FIG. 7, but may be divided into multiple nodes in the shape of an m*n pixel unit rectangle (m and n are different natural numbers).

[0079] With this configuration, the target image can be segmented such that the size of nodes farther from the boundary is larger than that of nodes closer to the boundary, including quadtree nodes, and the pixel values ​​of pixels closer to the boundary can be actively adjusted, and the pixel values ​​of pixels farther from the boundary can be passively adjusted, with weights set based on the distance from the boundary.

[0080] FIGS. 8A and 8B are exemplary diagrams illustrating a method for setting weights for each of a plurality of nodes of a target image according to one embodiment of the present disclosure. In one embodiment, the processor may divide the target image into a plurality of nodes based on distances from a boundary line, and then set weights for each of the plurality of nodes based on the distances from the boundary line. Here, the distance from the boundary line, which serves as a criterion for dividing the target image into a plurality of nodes, and the distance from the boundary line, which serves as a criterion for setting weights for each of the plurality of nodes, may be the same or different.

[0081] In one embodiment, the processor can identify a boundary of a target image, and identify a first region bordering the boundary, a second region bordering the first region and further from the boundary than the first region, and a third region bordering the second region and further from the boundary than the second region. Furthermore, the processor can divide the first region into a plurality of first nodes each having a first pixel unit, a plurality of second nodes each having a second pixel unit larger than the first pixel unit, and a plurality of third nodes each having a third pixel unit larger than the second pixel unit. That is, the processor can determine that, with respect to the distance from the boundary, the size (in pixels) of the node can be larger as the distance between each node and the boundary increases, and the size (in pixels) of the node can be smaller as the distance between each node and the boundary decreases.

[0082] In one embodiment, the processor may identify a boundary of a target image, and based on a distance from the boundary, a weight assigned to each node may be set such that the greater the distance between each node and the boundary, the greater the weight assigned to each node, and the smaller the distance between each node and the boundary, the smaller the weight assigned to each node. In this case, the tendency to set weights to each node based on the distance from the boundary and the tendency to split into multiple nodes in relation to the distance from the boundary may be the same.

[0083] In one embodiment, the processor may identify a boundary of a target image, and set first weights for a plurality of first nodes located within a first distance from the boundary among a plurality of nodes of the target image (or the first image, the second image). For example, as illustrated in FIG. 8A, the boundary of the target image (800) may be identified, and first weights may be set for a plurality of first nodes (812) located within a first distance (a) (810) from the boundary. Furthermore, the processor may set second weights greater than the first weights for a plurality of second nodes located within a second distance greater than the first distance from the boundary among a plurality of nodes outside the first distance (a) from the boundary of the target image. For example, as illustrated in FIG. 8A, the processor may set second weights greater than the first weights for a plurality of second nodes (822) located within a second distance (b) greater than the first distance (a) from the boundary among a plurality of nodes outside the first distance (a) from the boundary. Additionally, the processor may set a third weight greater than the second weight for a plurality of second nodes located within a third distance greater than the second distance from the boundary line among a plurality of nodes outside the second distance from the boundary line of the target image. For example, as illustrated in FIG. 8A, a third weight greater than the second weight may be set for a plurality of third nodes (832) located within a third distance (c) greater than the second distance (b) from the boundary line among a plurality of nodes outside the second distance (b) from the boundary line (830). Here, the second distance being greater than the first distance means that not only are a plurality of sets of nodes within the first distance from the boundary line included in a plurality of sets of nodes within the second distance from the boundary line, but also at least the second distance is greater than twice the first distance.Similarly, the third distance being greater than the second distance means that not only are a plurality of sets of nodes within the second distance from the boundary included in a plurality of sets of nodes within the third distance from the boundary, but also that the third distance is at least twice as large as the second distance.

[0084] Although FIG. 8a assumes that the first distance (a), the second distance (b), and the third distance (c) are each a single number, and depicts a set of pixels of a target image that are spaced apart from a boundary line by the first distance (a), the second distance (b), and the third distance (c) in a straight line, the present invention is not limited thereto. For example, at least one of the first distance (a), the second distance (b), or the third distance (c) may be expressed as a set of various numbers, and thus, the set of pixels spaced apart from a boundary line by at least one of the first distance, the second distance, or the third distance may take the form of a curve. However, even in this case, the second distance must be greater than the first distance, and the third distance must be greater than the second distance.

[0085] In one embodiment, the second distance (b) associated with the second weight may be three times the first distance (a) associated with the first weight, and the third distance (c) associated with the third weight may be seven times the first distance (a). In this case, the Nth distance (not shown) associated with the Nth weight from the boundary may be two times the first distance (a). N - It may correspond to 1 times. In addition, if the first pixel unit, which is the size of each of the plurality of first nodes for which the first weight is set, corresponds to (a*a), the second pixel unit, which is the size of each of the plurality of second nodes for which the second weight is set, corresponds to (2a*2a), the third pixel unit, which is the size of each of the plurality of third nodes for which the third weight is set, corresponds to (4a*4a), and the Nth pixel unit, which is the size of each of the plurality of Nth nodes for which the Nth weight is set, corresponds to (2 N -1a*2 N -1a) may apply.

[0086] In one embodiment, the same specific weight may be set to each of a plurality of nodes of the same size, but the present invention is not limited thereto, and in some cases, the same weight may be set to a plurality of groups of nodes of different sizes. That is, the same weight may be set to each of a plurality of nodes of the same size, but different weights may be set to a plurality of nodes of different sizes, and on the other hand, at least some of the plurality of nodes of different sizes may be set to the same weight. However, even in this case, the size of the weight set to a node with a larger pixel size must be greater than or at least equal to the weight set to a node with a smaller pixel size. For example, as illustrated in FIG. 8A, an area located within 1 pixel from a boundary in a target image may be divided into 1*1 pixel units, an area located within 3 pixels of an area located outside 1 pixel from the boundary may be divided into 2*2 pixel units, and an area located within 7 pixels of an area located outside 3 pixels from the boundary may be divided into 4*4 pixel units. Here, different weights may be set for multiple nodes divided into 1*1 pixel units, multiple nodes divided into 2*2 pixel units, and multiple nodes divided into 4*4 pixel units, or the same weight may be set for at least some of them. However, the weights set for multiple nodes divided into 4*4 pixel units cannot be smaller than the weights set for multiple nodes divided into 2*2 pixel units, and the weights set for multiple nodes divided into 2*2 pixel units cannot be smaller than the weights set for multiple nodes divided into 1*1 pixel units.

[0087] In one embodiment, the region bordering the boundary and the region not bordering the boundary can be segmented to have the same pixel unit. The target image (850) illustrated in FIG. 8B is divided into sections based on the boundary (860). Some sections of the boundary (860) are set in a diagonal direction. In this case, nodes corresponding to some regions (e.g., R7 / C10) that do not directly border the boundary can have the same 1*1 pixel unit as the nodes corresponding to the region bordering the boundary.

[0088] FIG. 9 is an exemplary diagram illustrating a method for adjusting pixel values ​​according to one embodiment of the present disclosure. In one embodiment, the processor may adjust the pixel value (or pixel values) of each node of the target image (or the first image, the second image) based on a weight set for each of a plurality of nodes of the target image. When the first image and the second image are combined based on a boundary line to generate a target image, the pixel value of each of the plurality of nodes of the target image may be adjusted so as to reduce the difference in pixel values ​​(or pixel values) between the nodes of the first image that touch the boundary line and the nodes of the second image that touch the boundary line.

[0089] Here, the adjustment value of the pixel value of each of the plurality of nodes may be associated with the weight of each of the plurality of nodes. In one embodiment, the larger the weight of each of the plurality of nodes, the smaller the adjustment value of the pixel value of each of the plurality of nodes. For example, the weight and the adjustment value of the pixel value may have a negative correlation.

[0090] The processor may sequentially adjust pixel values ​​for each node group in the image. For example, the processor may first adjust pixel values ​​for a first node group in the image, and then adjust pixel values ​​for a second node group adjacent to the first node group but further from the boundary than the first node group.

[0091] In one embodiment, the processor may adjust pixel values ​​of the first node group by a first adjustment value corresponding to a first weight so as to reduce the difference in pixel values ​​between the first node group of the first image and the nodes adjacent to the first node group. As illustrated in FIG. 9, the first node group (910) adjacent to the boundary in the target image (900) has predetermined pixel values ​​(P11-P17, P21-P27, P31-P37, P41-P47). The processor may adjust the pixel values ​​of the pixels of the first node group (910) so as to reduce the difference in pixel values ​​between the first node group (910) and the nodes (940) adjacent to the first node group (910).

[0092] For example, the processor may adjust the pixel values ​​of the pixels of the 1-1 node (912) so that the difference in pixel values ​​(e.g., the difference between P17 and P18, the difference between P27 and P28, the difference between P37 and P38, the difference between P47 and P48) between the 1-1 node (912) and the nodes adjacent to the 1-1 node (912) among the 1st node group (910) is reduced. After the adjustment, the pixels of the 1-1 node (962) in the target image (950) may have the adjusted pixel values ​​(P17', P27', P37', P47'). Here, the 1-1 node (962) may have a first weight, and the adjusted pixel values ​​for the 1-1 node (e.g., P17'-P17, P27'-P27, P37'-P37, P47'-P47) may correspond to the first weight. Similarly, the pixels of the node (990) bordering the second image also have adjusted pixel values ​​(P18', P28', P38', P48').

[0093] As illustrated in FIG. 9, in the target image (900), the 1-2 node (914) which is adjacent to the 1-1 node (912) and is further from the boundary than the 1-1 node (912) has predetermined pixel values ​​(P15, P16, P25, P26, P35, P36, P45, P46). After adjusting the 1-1 node (912), the processor can adjust the pixel values ​​of the pixels of the 1-2 node (914) so ​​that the difference in pixel values ​​between the 1-2 node (914) and the 1-1 node (912) adjacent to the 1-2 node (914) is reduced. After the adjustment, the pixels of the 1-2 node (964) in the target image (950) can have the adjusted pixel values ​​(P15', P16', P25', P26', P35', P36', P45', P46'). Here, the 1st-2nd node (964) may have a first weight, and the adjustment values ​​of the pixel values ​​for the 1st-2nd node (e.g., P15'-P15, P16'-P16, P25'-P25, P26'-P26, P35'-P35, P36'-P36, P45'-P45, P46'-P46) may correspond to the first weight.

[0094] In a similar manner, the processor may adjust pixel values ​​in the same manner for other nodes belonging to the first node group (e.g., the first-3 node (966)) after adjusting the first-1 node (962) and the first-2 node (964). The first node group may have a first weight, and the adjusted pixel values ​​for the first node group may correspond to the first weight.

[0095] After adjusting the first node group (910, 960), the processor can adjust the pixel values ​​of the second node group (920, 970), which is adjacent to the first node group and is further from the boundary than the first node group. After adjusting the second node group (920, 970), the processor can sequentially execute adjustments for the third node group (930, 980), etc. Here, the second node group can have a second weight, and the third node group can have a third weight.

[0096] In one embodiment, the first weight may be less than the second weight, and the first adjustment value may be greater than the second adjustment value. For example, the first adjustment value may be inversely proportional to the first weight, and the second adjustment value may be inversely proportional to the second weight.

[0097] In one embodiment, the second adjustment value may be calculated according to the following mathematical expression 1. Here, P1 is the average pixel value of the first node group, P2 is the average pixel value of the second node group, w1 is the weight of the first node group, w2 is the weight of the second node group, and a may be a coefficient.

[0098]

[0099] In one embodiment, the processor may perform pixel value adjustments multiple times. For example, the processor may sequentially perform pixel value adjustments from the first node group to the n-th node group, and then, if a specific condition is not satisfied, perform pixel value adjustments again sequentially from the first node group to the n-th node group.

[0100] In one embodiment, the processor can repeatedly execute the process until the difference in pixel values ​​between the nodes of the first image and the nodes of the second image bordering the boundary is less than or equal to a preset threshold.

[0101] In one embodiment, the processor may utilize a weighted least squares method in the gradient domain for color balancing. For example, the processor may use a sparse solver to solve the equation Ax = B to find x that minimizes the error. Here, the sparse matrix A may be configured in a weighted format to indicate how each node is related to other nodes. In addition, the sparse vector B may represent the difference in pixel values ​​(color values) between each node, such that the difference in pixel values ​​at the boundary may be represented and the rest may be initialized to 0.

[0102] FIG. 10 is a flowchart illustrating an image color adjustment method according to one embodiment of the present disclosure. According to one embodiment, the image color adjustment method may be executed by a processor included in an information processing system. According to another embodiment, the image color adjustment method may be executed by a processor mounted or built into a computing device included in a user terminal. The image color adjustment method may be initiated by a step (S1010) of identifying a first image and a second image that are sequentially arranged based on a boundary line. To this end, the processor may receive a target image in which a plurality of images, including the first image and the second image, are mosaicked. Then, the processor may receive labeling information for each of the plurality of images and identify the first image, the second image, and the boundary line in the target image based on the labeling information.

[0103] The processor may divide the first image into a plurality of nodes based on the boundary line (S1020). In addition, the processor may divide the second image into a plurality of nodes based on the boundary line. Here, the size of each of the plurality of nodes may be related to the distance from the boundary line. To this end, the processor may divide a first area that touches the boundary line into a plurality of first nodes, divide a second area that touches the plurality of first nodes and is further from the boundary line than the plurality of first nodes into a plurality of second nodes, and divide a third area that touches the plurality of second nodes and is further from the boundary line than the plurality of second nodes into a third node. Additionally or alternatively, the processor may divide the first image into a plurality of nodes such that each of the plurality of first nodes has a first pixel unit, each of the plurality of second nodes has a second pixel unit larger than the first pixel unit, and each of the plurality of third nodes has a third pixel unit larger than the second pixel unit.

[0104] The processor may set a weight for each of a plurality of nodes of the first image based on the distance from the boundary line (S1030). In addition, the processor may set a weight for each of a plurality of nodes of the second image based on the distance from the boundary line. To this end, the processor may set a first weight for each of a plurality of first nodes, a second weight greater than the first weight for each of a plurality of second nodes, and a third weight greater than the second weight for each of a plurality of third nodes. In addition, or alternatively, the processor may set a first weight for each of a plurality of first nodes located at a first distance from the boundary line among the plurality of nodes, a second weight greater than the first weight for each of a plurality of second nodes located within a second distance greater than the first distance from the boundary line among the plurality of nodes, and a third weight greater than the second weight for each of a plurality of third nodes located outside the second distance from the boundary line among the plurality of nodes.

[0105] The processor may adjust pixel values ​​of a plurality of nodes of the first image based on weights of each of the plurality of nodes (S1040). To this end, the processor may adjust the pixel values ​​of the plurality of nodes so that the difference in pixel values ​​between nodes of the first image and nodes of the second image that border the boundary line is reduced. In one embodiment, the processor may adjust the pixel values ​​of the first node group by a first adjustment value corresponding to the first weight so that the difference in pixel values ​​between the first node group of the first image and the nodes bordering the first node group is reduced. Then, based on the first adjustment value, the pixel values ​​of the second node group of the first image that borders the first node group and is farther from the boundary line than the first node group may be adjusted by a second adjustment value corresponding to the second weight.

[0106] FIGS. 11 to 13 are exemplary diagrams showing the effect of image color adjustment considering weights set based on a boundary line according to one embodiment of the present disclosure.

[0107] FIG. 11 is a diagram illustrating a target image (1100) in which a first image (1110) and a second image (1120) having different colors are synthesized (mosaiced) around a boundary line, according to one embodiment. In one embodiment, the processor may acquire a target image based on the first image and the second image captured in different shooting environments for the same subject. Accordingly, the first image and the second image may have different colors and tones for the same subject. In addition, in the target image in which at least a portion of the first image and the second image are combined around a boundary line, pixels derived from the first image and pixels derived from the second image adjacent to the boundary line may have different colors.

[0108] Fig. 12 shows the result of adjusting the color of the target image without setting the weights. In one embodiment, the image color adjustment may be performed on all pixels of the image so that the node size is divided into larger ones as the distance from the boundary is larger and smaller ones as the distance from the boundary is smaller, and the difference in pixel values ​​of adjacent pixels is minimized as the distance from the boundary is smaller, per pixel. Accordingly, as illustrated in Fig. 12, although the pixels near the boundary of the target image (1200) are adjusted seamlessly, the adjustment effect of the pixels near the boundary is also propagated to the pixels far from the boundary without filtering, so that the pixel value difference of the pixels far from the boundary is also adjusted to minimize the pixel value difference from the adjacent pixels to the same extent as the pixel value difference of the pixels near the boundary is minimized, and accordingly, the color tone of all pixels of the target image may not be restricted from changing significantly. In other words, the problem may arise that the pixel values ​​are unnecessarily actively adjusted not only near the boundary where active color adjustment is required in the target image, but also far from the boundary where color balancing can be considered to have already been performed, thereby changing the color tone of the entire image.

[0109] Figure 13 illustrates the result of color adjustment of a target image considering weights set based on the distance between boundaries. In one embodiment, the processor may divide the target image into a plurality of nodes based on the boundaries. Here, the plurality of nodes may be divided based on the distance from the boundaries, such that the closer the distance from the boundaries, the smaller the node, and the farther the distance from the boundaries, the larger the node. However, the present invention is not limited thereto, and for example, nodes located within a first threshold distance from the boundaries may be divided into equal sizes, nodes located beyond the first threshold distance and within a second threshold distance may be divided into larger nodes as the distance from the boundaries increases, and nodes may be divided into constant sizes when the distance from the second threshold distance increases. In addition, the processor may set a weight for each node based on the distance from the boundaries. Here, the weight may be set such that the closer the distance from each node to the boundaries, the greater the change in the pixel value (e.g., pixel value) of the corresponding node, and the farther the distance from each node to the boundaries, the less the change in the pixel value of the corresponding node. Accordingly, as illustrated in FIG. 13, the pixel values ​​of pixels near the boundary of the target image (1300) are relatively actively adjusted to be large, thereby being judged as seamless, while the pixel values ​​of pixels farther from the boundary are relatively passively adjusted to be small, thereby maintaining the color of the original image of the corresponding pixels as much as possible. With this configuration, image color adjustment is performed on a pixel / node basis, but since the weight of each node tends to increase as the size of each node increases as it gets farther from the boundary, the pixels farther from the boundary can serve as anchors that limit color change.

[0110] In one embodiment, prior to performing image color adjustment, image-level preprocessing may be performed first to enable image color adjustment to be performed more effectively. For example, assume that a first image photographed in a first area and a second image photographed in a second area are combined along a boundary line to generate a target image. Here, when the processor selects the first image and the second image to constitute the target image from among the plurality of first images and the plurality of second images, the processor may calculate a first average pixel value (e.g., pixel value) for each of the plurality of first images and a second average pixel value for each of the plurality of second images. Then, the processor may extract the first image and the second image, respectively, from the plurality of first images and the plurality of second images, which minimize the difference between the first average pixel value and the second average pixel value. The processor may generate a target image using the extracted first image and the second image, and perform color adjustment on the target image.

[0111] Through this configuration, nodes close to the boundary in the target image are split into smaller sizes and assigned smaller weights, while nodes far from the boundary are split into larger sizes and assigned larger weights. Therefore, when color balancing is performed so that the difference in pixel values ​​between each pixel and its neighbors is minimized, each pixel value is based on the pixel value that reflects the weight. Therefore, the pixel values ​​of pixels near the boundary that require color balancing are relatively adjusted to a large extent, while the pixel values ​​of pixels far from the boundary that can already be considered color balanced are maintained or relatively limited in color change. This prevents the aftereffects of adjusting the pixel values ​​of pixels near the boundary that require color balancing from propagating to pixels far from the boundary.

[0112] The image color adjustment method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0113] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0114] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0115] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0116] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0117] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0118] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

Claims

1. A method for adjusting image color, performed by at least one processor, A step of identifying a first image and a second image that are arranged continuously based on a seam line; A step of dividing each of the first image and the second image into a plurality of nodes based on the boundary line; A step of setting weights of each of the plurality of nodes of the first image and the second image based on the distance from the boundary line; and A step of adjusting pixel values ​​of a plurality of nodes of the first image and the second image based on the weights of each of the plurality of nodes, An image color adjustment method, wherein the adjustment value of the pixel value of each of the plurality of nodes is associated with the weight of each of the plurality of nodes.

2. In paragraph 1, The step of identifying the first image and the second image comprises: A step of receiving a plurality of images including the first image and the second image; A step of receiving labeling information or location information for each of the plurality of images; and An image color adjustment method, comprising a step of identifying the first image, the second image, and the boundary line based on at least one of the labeling information and the location information.

3. In paragraph 1, The step of dividing into the above multiple nodes is: A step of dividing a first area bordering the above boundary into a plurality of first nodes; A step of dividing a second region that is adjacent to the plurality of first nodes and is further from the boundary line than the plurality of first nodes into a plurality of second nodes; and A step of dividing a third region that is adjacent to the plurality of second nodes and is further from the boundary line than the plurality of second nodes into a plurality of third nodes, Each of the plurality of first nodes has a first pixel unit, Each of the plurality of second nodes has a second pixel unit larger than the first pixel unit, A method for adjusting image color, wherein each of the plurality of third nodes has a third pixel unit larger than the second pixel unit.

4. In paragraph 3, The above first pixel unit is 1x1 pixel, The above second pixel unit is 2x2 pixels, A method for adjusting image color, wherein the third pixel unit is 4x4 pixels.

5. In paragraph 1, The step of dividing into the above multiple nodes is: A step of defining a plurality of first nodes by identifying a plurality of pixels adjacent to the above boundary line as a first pixel unit; A step of defining a plurality of second nodes by merging a plurality of pixels adjacent to the plurality of first nodes into second pixel units; and A step of defining a plurality of third nodes by merging a plurality of pixels adjacent to the plurality of second nodes and further from the boundary line than the plurality of second nodes into third pixel units, The first pixel unit is smaller than the second pixel unit, A method for adjusting image color, wherein the second pixel unit is smaller than the third pixel unit.

6. In paragraph 3, The step of setting the above weights is: A step of setting a first weight for the plurality of first nodes; A step of setting a second weight greater than the first weight for the plurality of second nodes; and An image color adjustment method, comprising a step of setting a third weight greater than the second weight for the plurality of third nodes.

7. In paragraph 1, The step of setting the above weights is: A step of setting a first weight for a plurality of first nodes located within a first distance from the boundary line among the plurality of nodes; A step of setting a second weight greater than the first weight for a plurality of second nodes located within a second distance greater than the first distance from the boundary line among the plurality of nodes; and An image color adjustment method, comprising a step of setting a third weight greater than the second weight for a plurality of third nodes located outside the second distance from the boundary line among the plurality of nodes.

8. In paragraph 1, The step of adjusting the above pixel values ​​is: An image color adjustment method, comprising a step of adjusting pixel values ​​of the plurality of nodes so that the difference in pixel values ​​between the nodes of the first image and the nodes of the second image that touch the boundary line is reduced.

9. In paragraph 8, The step of adjusting the above pixel values ​​is: A step of adjusting the pixel values ​​of the first node group by a first adjustment value corresponding to the first weight so that the difference in pixel values ​​between the first node group of the first image and the nodes adjacent to the first node group is reduced; and Based on the first adjustment value, a step of adjusting the pixel values ​​of a second node group of the first image that is adjacent to the first node group and is further from the boundary line than the first node group is included by a second adjustment value corresponding to a second weight, The first weight is smaller than the second weight, An image color adjustment method, wherein the first adjustment value is greater than the second adjustment value.

10. In paragraph 9, The above first adjustment value is inversely proportional to the above first weight, An image color adjustment method, wherein the second adjustment value is inversely proportional to the second weight.

11. In paragraph 9, The above second adjustment value is calculated according to mathematical formula 1, The above mathematical expression 1 is, And, The above P1 is the average pixel value of the first node group, The above P2 is the average pixel value of the second node group, The above w1 is the weight of the first node group, The above w2 is the weight of the second node group, A method for adjusting image color, wherein the above a is a coefficient.

12. In paragraph 8, A method for adjusting image color, wherein the step of adjusting pixel values ​​of the plurality of nodes is repeatedly performed until the difference in pixel values ​​between the nodes of the first image and the nodes of the second image becomes less than a preset threshold value.

13. In paragraph 1, A step of calculating a first average pixel value for the first image and a second average pixel value for the second image; and An image color adjustment method further comprising the step of adjusting pixel values ​​of the first image and pixel values ​​of the second image so as to reduce a difference between the first average pixel value and the second average pixel value.

14. In paragraph 1, A step of dividing the second image into a plurality of nodes based on the boundary line, wherein the size of each of the plurality of nodes of the second image is related to the distance from the boundary line; A step of setting a weight of each of the plurality of nodes of the second image based on the distance from the boundary line; and An image color adjustment method further comprising a step of adjusting pixel values ​​of a plurality of nodes of the second image based on the weights of each of the plurality of nodes of the second image.

15. In paragraph 1, A method for adjusting image color, wherein the size of each of the plurality of nodes is related to the distance from the boundary line.

16. A computer-readable recording medium storing a computer program for executing the method according to paragraph 1 on a computer.

17. As an image color adjustment system, memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, A method for identifying a first image and a second image that are sequentially arranged based on a boundary line, dividing each of the first image and the second image into a plurality of nodes based on the boundary line, setting weights of each of the plurality of nodes of the first image and the second image based on a distance from the boundary line, and adjusting pixel values ​​of each of the plurality of nodes of the first image and the second image based on the weights of each of the plurality of nodes, comprising: An image color adjustment system, wherein the adjustment value of the pixel value of each of the plurality of nodes is associated with the weight of each of the plurality of nodes.

Images