Color transfer method, device, equipment and storage medium

Abstract

The present application relates to a color transfer method, device, equipment and storage medium, which are applied in the field of image processing, including obtaining reference images corresponding to different scene types, and building a dictionary library based on the reference images, the dictionary library including scene dictionaries corresponding to different scene types; obtaining original visible light images and original infrared images, and performing grayscale multi-scale fusion to generate grayscale fusion images; dividing the original visible light image and the grayscale fusion image into blocks according to a preset image block size, generating a number of visible light image blocks and a number of grayscale fusion image blocks; judging the types of a number of visible light image blocks according to the visible light image blocks and the dictionary library, and transferring the color of the reference image corresponding to the type of the visible light image block to the grayscale fusion image block using a color transfer algorithm, generating a number of color fusion image blocks; splicing and fusing a number of color fusion image blocks to generate a color image. The technical effects of the present application are: accurate color transfer and reduction of color assignment errors.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a color transfer method, device, equipment and storage medium. Background Art

[0002] The color processing of infrared and visible light image fusion belongs to pseudo color processing. Color transfer is to transfer the color distribution of the reference image to the fused grayscale image to achieve the purpose of highlighting the target and improving the contrast. At present, the fusion color processing is mainly through the global statistics and calculation of the reference image and the image to be processed; for example, the TNO method is to calculate the corresponding value of each pixel after processing by counting the global mean and variance of each channel of the RGB or YUV image, and then calculate the corresponding value of each pixel through the transfer formula; the Welsh algorithm is also for global image processing.

[0003] The processing idea of ​​the existing technology is mainly through calculation of the global color, which also determines that the processing method of the existing technology is better for images with a single color; when the scene content in the image is complex, the use of the existing technology for color transmission will cause color bleeding or turbid scene colors, which will cause the image clarity to decrease. The decrease in image clarity may lead to a reduction in the speed and accuracy of observation. Summary of the invention

[0004] In order to help solve the problem that when the scene content in the image is complex, the use of existing technology for color transfer will cause color bleeding or scene color turbidity, thereby causing a decrease in image clarity. The decrease in image clarity may lead to a reduction in observation speed and accuracy. The application provides a color transfer method, device, equipment and storage medium.

[0005] In a first aspect, the present application provides a color transfer method, which adopts the following technical solution: the method comprises:

[0006] Acquire reference images corresponding to different scene types, and construct a dictionary library according to the reference images, wherein the dictionary library includes scene dictionaries corresponding to different scene types;

[0007] Acquire an original visible light image and an original infrared image, perform grayscale multi-scale fusion on the original visible light image and the original infrared image, and generate a grayscale fused image;

[0008] The original visible light image and the grayscale fused image are processed by block division according to a preset image block size to generate a plurality of visible light image blocks and a plurality of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number of the blocks is the same;

[0009] Determine the types of the visible light image blocks according to the visible light image blocks and the dictionary library, and transfer the color of the reference image corresponding to the type of the visible light image block to the grayscale fused image block by using a color transfer algorithm to generate a plurality of color fused image blocks;

[0010] A plurality of the color fusion image blocks are spliced ​​and fused to generate a color image.

[0011] In a specific implementation manner, constructing a dictionary library according to the reference image includes:

[0012] Constructing the correspondence between grayscale information and gradient information in reference images corresponding to different scene types, and generating a scene dictionary;

[0013] The dictionary library is formed by combining scene dictionaries corresponding to different scene types.

[0014] In a specific implementation manner, acquiring the original visible light image and the original infrared image comprises:

[0015] Obtain visible light video stream data and infrared video stream data;

[0016] Frame image extraction is performed on the visible light video stream data and the infrared video stream data to obtain the original visible light image and the original infrared image.

[0017] In a specific implementation manner, before the original visible light image and the grayscale fused image are divided into blocks according to a preset image block size, the method further includes:

[0018] Performing global color transfer processing on the grayscale fused image according to a preset color mode corresponding to the scene type to generate an initial color fused image;

[0019] The block processing of the original visible light image and the grayscale fused image according to a preset image block size to generate a plurality of visible light image blocks and a plurality of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number of the visible light image blocks is the same, comprises:

[0020] The original visible light image and the initial color fused image are divided into blocks according to a preset image block size to generate a plurality of visible light image blocks and a plurality of initial color fused image blocks, wherein the visible light image blocks correspond to the initial color fused image blocks and the number of the blocks is the same.

[0021] In a specific implementation manner, the determining the types of the plurality of visible light image blocks according to the visible light image block and the dictionary library, and transferring the color of the reference image corresponding to the type of the visible light image block to the grayscale fused image block using a color transfer algorithm comprises:

[0022] Calculating the correspondence between the grayscale information and the gradient information of the visible light image block;

[0023] calculating the Euclidean distance between the visible light image block and scene dictionaries corresponding to different scene types according to the correspondence between the grayscale information and the gradient information of the visible light image block, and setting the calculated minimum value of the Euclidean distance as the type of the visible light image block;

[0024] Calculating and determining the types of all the visible light image blocks in sequence;

[0025] Determining whether the color of the corresponding initial color fused image block is accurate according to the type of the visible light image block;

[0026] If the color of the initial color fused image block is inaccurate, the color of the reference image corresponding to the visible light image block type is transferred to the initial color fused image block using a color transfer algorithm.

[0027] In a specific implementation manner, after the color image is generated by splicing and fusing a plurality of the color fused image blocks, the method further includes:

[0028] A filtering algorithm is used to filter the fused color image to generate a color clear image.

[0029] In a second aspect, the present application provides a color transfer device, which adopts the following technical solution: the device comprises:

[0030] A dictionary library construction module, used to obtain reference images corresponding to different scene types, and to construct a dictionary library according to the reference images, wherein the dictionary library includes scene dictionaries corresponding to different scene types;

[0031] A grayscale image generation module is used to obtain an original visible light image and an original infrared image, perform grayscale multi-scale fusion on the original visible light image and the original infrared image, and generate a grayscale fused image;

[0032] An image segmentation module, used for performing block processing on the original visible light image and the grayscale fused image according to a preset image block size to generate a plurality of visible light image blocks and a plurality of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number of the blocks is the same;

[0033] A color transfer module, used for determining the types of the visible light image blocks according to the visible light image blocks and the dictionary library, and transferring the color of the reference image corresponding to the type of the visible light image block to the grayscale fused image block by using a color transfer algorithm, so as to generate a plurality of color fused image blocks;

[0034] The color image generation module is used to splice and fuse a plurality of the color fusion image blocks to generate a color image.

[0035] In a third aspect, the present application provides a computer device, which adopts the following technical solution: it includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute any of the above-mentioned color transfer methods.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: storing a computer program that can be loaded by a processor and execute any of the above-mentioned color transfer methods.

[0037] In summary, this application has the following beneficial technical effects:

[0038] By processing the image in blocks, determining the type of each image block, and then transferring colors according to the type of the image block, the mutual interference of the scenes in the scene can be eliminated, and at the same time, the boundary between different scenes can be determined, so that the assigned color can be close to the real color of the natural scene, and the error of color assignment can be reduced. In the prior art, based on global calculation and color transfer of the global image, the present application scheme realizes local color transfer of the image by means of image blocks, so as to improve the accuracy of color transfer, make the local color closer to the real scene, and especially make the color of the natural scene more real. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a color transfer method in an embodiment of the present application;

[0040] Figure 2 is a schematic diagram of an original visible light image in an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of an original infrared image in an embodiment of the present application;

[0042] Figure 4 is a color image after FPGA global color processing in an embodiment of the present application;

[0043] Figure 5 is a color image adjusted by FPGA local color processing in an embodiment of the present application;

[0044] Figure 6 is a schematic diagram of a color transfer device in an embodiment of the present application;

[0045] Figure 7 It is a schematic diagram used to embody a computer device in an embodiment of the present application.

[0046] Attached figure numerals: 601, dictionary library construction module; 602, grayscale image generation module; 603, image segmentation module; 604, color transfer module; 605, color image generation module. DETAILED DESCRIPTION

[0047] The following combination Figure 1-Figure 7 This application is described in further detail.

[0048] The embodiment of the present application discloses a color transfer method, which can be used to determine the type of image blocks and perform color transfer of a corresponding reference image according to the type of image blocks, so that local colors are closer to real scenes, especially the colors of natural scenes are more realistic, thereby avoiding the problem of color bleeding or scene color turbidity when the image scene content is complex.

[0049] The color processing of infrared and visible light image fusion belongs to pseudo color processing. Color transfer is to transfer the color distribution of the reference image to the fused grayscale image to achieve the purpose of highlighting the target and improving the contrast. The current color processing methods include: linear transfer in RGB, Lab, YUV and other color spaces, sparse coding color reconstruction, direct color channel mapping method, etc.

[0050] Commonly used color transfer algorithms include TNO color transfer algorithm and Welsh color transfer algorithm. The linear transfer of color for image fusion was mainly proposed by TNO; after years of development, multiple transfer, multi-scale transfer and other methods have also evolved. This method can facilitate the real-time implementation.

[0051] Welsh color migration method: First, convert the reference image and grayscale image from RGB space to lαβ color space respectively. Second, remap the brightness of the reference image according to the brightness and standard deviation of the grayscale image. Since the histogram values ​​of the reference image and the grayscale image are not necessarily in the same range, if a reference image with a very low brightness value is compared with a histogram of a target image with a very high brightness value, a large error will occur. Therefore, it is necessary to remap the brightness of the reference image. The remapping method can be expressed as: L = (nl' / nl)* (l – ml) + ml'; where l is the data of the l channel of the source image, L is the value of the l channel of the new source image after the transformation, ml and ml' are the means of the l channels of the source image and the colored image, respectively, and nl and nl' represent their l channel standard deviations. Third, randomly select a batch of sample points from the reference image, and use the linear combination of the brightness of the pixel and the standard deviation of the brightness in the neighborhood as the weight. The calculation formula can be expressed as: W = l / 2 + σ / 2; where w is the weight, l is the brightness of the pixel, and σ is the standard deviation of the brightness in a neighborhood around the pixel. Regarding the size of the neighborhood, Welsh pointed out that it is generally 5×5, and a larger neighborhood will have better results for individual images. Fourth, scan the grayscale image row by row, and for each pixel, calculate its weight by the formula, and find the closest sample point among the sample points of the reference image, that is, the best matching point, and assign the value of the color channel of the matching point to the corresponding pixel of the grayscale image, thereby realizing color transfer. Fifth, convert the reference image and grayscale image from lαβ space to RGB space.

[0052] From the above TNO algorithm and Welsh algorithm, we can see that the current fusion color processing is mainly through global statistics and calculations of the reference image and the image to be processed; for example, the TNO method is to calculate the global mean and variance of each channel of the RGB or YUV image, and then calculate the corresponding value of each pixel after processing through the transfer formula; the Welsh algorithm is also for global image processing.

[0053] The processing idea of ​​the prior art through the calculation of global color determines that the processing method of the prior art has a better effect on images with a single color; when the scene content in the image is complex, the use of the prior art for color transfer will cause color bleeding or turbid scene color, which will cause the image clarity to decrease. The decrease in image clarity may lead to a decrease in the speed and accuracy of observation; in addition, due to global statistical calculations, the processed color of the scene depends on the result of global color balance, resulting in large color errors in the coloring process, which cannot maintain authenticity and is unnatural, and it is difficult to restore the natural color of the original scene. In order to help avoid the problems of color bleeding and turbid scene color, as well as large color errors in the coloring process that are unrealistic and unnatural, the present application provides a color transfer method.

[0054] Reference Figure 1 , the method comprises the following steps:

[0055] S10, obtaining reference images corresponding to different scene types, and constructing a dictionary library according to the reference images, wherein the dictionary library includes scene dictionaries corresponding to different scene types.

[0056] Specifically, in order to improve the accuracy of color migration, the accuracy of color migration can be improved by judging the image type before color migration. At the same time, the local color of the image after color transfer can also be closer to the real scene, making the color of the scene more realistic. Scenery types can be divided into natural background scenes and artificial background scenes. Natural background scenes can include the sky, plants, bare land, water surface, desert, shallow grass, the junction of heaven and earth, the junction of sea and sky, etc.; artificial background scenes can include asphalt roads, buildings, etc., and scene dictionaries are constructed for different types of scenes. The scene dictionaries of all scene types are combined to form a dictionary library; wherein, the scene types used to construct the dictionary library can be selected by the user according to actual needs. Specifically, the construction of scene dictionaries corresponding to different types of scenes is based on reference images in advance. The reference image can be understood as an image of the standard color of the scene. Through color transfer, the standard color of the scene is transferred to the grayscale image to achieve coloring of the grayscale image.

[0057] S20, acquiring an original visible light image and an original infrared image, performing grayscale multi-scale fusion on the original visible light image and the original infrared image, and generating a grayscale fused image.

[0058] Specifically, obtain the original visible light image, refer to Figure 2 , and the original infrared image reference Figure 3 ; Among them, the original visible light image and the original infrared image can be understood as the visible light image and infrared image to be processed, and the original visible light image and the original infrared image are two images taken at the same time and at the same position; because the visible light image has richer texture but is easily affected by environmental factors, while the infrared image is less affected by environmental factors, it can still accurately capture the characteristic information of significant targets even in complex environments such as darkness and dense fog, but the infrared image has relatively less texture information. Therefore, by fusing the original visible light image and the original infrared image, an image with richer image information and higher definition can be obtained, so that the targets in the image can be identified and detected later. Since the color processing of the fusion of infrared and visible light images belongs to pseudo-color processing, the original infrared image and the original visible light image are subjected to grayscale multi-scale fusion processing to obtain a grayscale fused image, and then the grayscale fused image is assigned color to obtain a color image with higher definition.

[0059] S30, performing block processing on the original visible light image and the grayscale fused image according to a preset image block size to generate a plurality of visible light image blocks and a plurality of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number of the visible light image blocks is the same.

[0060] Specifically, in the embodiment of the present application, color transfer is performed in the form of image blocks, and the original visible light image and the grayscale fused image are divided into blocks according to a preset image block size to generate a number of visible light image blocks and a number of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number is the same. It can also be understood that the visible light image and the grayscale fused image are divided into blocks in the same manner and according to the same image block size, so that the visible light image blocks and the grayscale fused image blocks can correspond one to one, which is convenient for subsequent color transfer of the grayscale image.

[0061] It should be noted that the preset image block size and image segmentation method can be selected and set by the user according to actual needs. In the embodiment of the present application, the image is segmented by a sliding window. For example, assuming that the preset image block size is set to 8*8, the offset step of each small block is 4 during the sliding segmentation process, and there is an overlapping area between each small block, so as to avoid missing information such as image features and details between image blocks as much as possible, and ensure the richness of the obtained image information. In actual applications, users can make specific settings based on the computing resources of the actual processing equipment, the computing amount of the task, etc.

[0062] S40, determining types of several visible light image blocks according to the visible light image blocks and the dictionary library, and transferring the color of the reference image corresponding to the visible light image block type to the grayscale fused image block using a color transfer algorithm to generate several color fused image blocks.

[0063] Specifically, the types of several visible light image blocks are determined based on the visible light image blocks and the dictionary library. Since the texture information in the visible light image is relatively rich, the use of visible light images for type determination can improve the accuracy of type determination. The dictionary library contains a relatively comprehensive range of scene types. The visible light image blocks of the type to be determined are compared with the dictionary, so that the types of the visible light image blocks can be determined. After the types of all visible light images are determined, the colors of the reference images corresponding to the types of the visible light image blocks are transferred to the grayscale fusion image blocks using a color transfer algorithm to generate several color fusion image blocks. Specifically, the color transfer algorithm used can be the TNO color transfer algorithm. At this time, the color transfer only transfers the color, and the image grayscale does not need to be transferred.

[0064] S50, splicing and fusing a plurality of color fusion image blocks to generate a color image.

[0065] Specifically, the generated color fusion image blocks are stitched or fused together to generate a final color image, thereby completing the color assignment process of the grayscale image.

[0066] In the present application, the image is processed in blocks, the type of each image block is determined, and then color transfer is performed according to the type of the image block, which can eliminate the mutual interference of the scenes in the scene, and at the same time, the boundary of different scenes can be determined, so that the assigned color can be close to the real color of the natural scene, and the error of color assignment can be reduced. In the prior art, based on global calculation and color transfer of the global image, the present application realizes local color transfer of the image by image blocks compared to the prior art, so as to improve the accuracy of color transfer, make the local color closer to the real scene, and especially make the color of the natural scene more real.

[0067] In one embodiment, considering that in various application scenarios, it is necessary to realize real-time processing of video data, the amount of data processed by the algorithm on the host computer is large, and it is difficult to realize real-time performance. Therefore, in order to realize the real-time performance of video data processing, the step of constructing a dictionary library according to the reference image can be specifically performed as follows:

[0068] The process of constructing a dictionary library is implemented by using FPGA. First, the correspondence between the grayscale information and the gradient information in the reference image corresponding to different scene types is constructed to generate a scene dictionary; since the number of selected scene types is related to the processing speed, the more scene types are selected, the slower the FPGA real-time processing speed will be. Therefore, in the embodiment of the present application, the scene types are selected to be commonly used or have more obvious features, mainly including plants, roads, skies, water surfaces and buildings. Afterwards, the scene dictionaries corresponding to different scene types are combined to form the dictionary library.

[0069] In the present application scheme, the correspondence between the grayscale information and the gradient information in the reference image is directly established to construct a scene dictionary, which reduces the types of scenes and can reduce the FPGA calculation time, thereby improving the real-time processing.

[0070] In one embodiment, the step of acquiring the original visible light image and the original infrared image may be specifically performed as follows:

[0071] Obtain visible light video stream data and infrared video stream data; perform frame image extraction on the visible light video stream data and infrared video stream data to obtain original visible light images and original infrared images.

[0072] FPGA is used in real-time processing scenarios. Therefore, what FPGA obtains is generally video stream data. The obtained video stream data is subjected to frame image extraction to obtain infrared images and visible light images for computational processing.

[0073] In one embodiment, considering that it takes a lot of time to divide the image into blocks and perform color transfer on each image block, the following steps may be performed before dividing the original visible light image and the grayscale fused image into blocks according to the preset image block size:

[0074] According to the color mode corresponding to the preset scene type, the grayscale fusion image is processed with global color transfer to generate the initial color fusion image. Figure 4 Specifically, the existing global color processing algorithm can be used to directly process the grayscale fusion image for color transfer, which can reduce the calculation time and improve the real-time performance. The grayscale fusion image is divided into blocks according to the preset image block size to generate a number of grayscale fusion image blocks, including: the initial color fusion image is divided into blocks according to the preset image block size to generate a number of initial color fusion image blocks, wherein the visible light image blocks correspond to the initial color fusion image blocks and the number is the same.

[0075] In one embodiment, the steps of determining the types of several visible light image blocks according to the visible light image blocks and the dictionary library, and transferring the color of the reference image corresponding to the visible light image block type to the grayscale fused image block using a color transfer algorithm may be specifically performed as follows:

[0076] After global color processing of the grayscale image and dividing the initial color fused image into blocks, first calculate the correspondence between the grayscale information and gradient information of the visible light image block; then, based on the correspondence between the grayscale information and gradient information of the visible light image block, calculate the Euclidean distance between the visible light image block and the scene dictionary corresponding to different scene types, and set the calculated minimum Euclidean distance as the type of the visible light image block. Calculate and determine the types of all visible light image blocks in sequence; determine whether the color of the corresponding initial color fused image block is accurate based on the type of the visible light image block; if the color of the initial color fused image block is inaccurate, transfer the color of the reference image corresponding to the visible light image block type to the initial color fused image block using a color transfer algorithm, and finally fuse and splice to generate a color image, refer to Figure 5 .

[0077] It should be noted that the 8*8 block method is used in the embodiment of the present application. Therefore, when the FPGA performs type judgment on the visible light image during processing, it first obtains 8 lines of image blocks. In the subsequent processing, a 7-line buffer will be set to reserve 7 lines of processing time to increase the calculation speed and reduce the calculation time, thereby improving the real-time performance of the processing. For real-time calculation, the total delay of the method of this embodiment is 11 lines × line time, which can be ignored in actual processing, and the real-time performance is good.

[0078] In the present application scheme, considering that it will take a lot of time to judge the type of each image block and perform color migration on each image block, it is difficult to meet the real-time requirements. Therefore, the grayscale image is first globally colored, and then the local color is fine-tuned according to the type of the image block, thereby increasing the processing speed and improving real-time performance.

[0079] In one embodiment, considering that the image blocks used in the present application scheme for color transfer processing may produce a block effect, after a plurality of color fused image blocks are spliced ​​and fused to generate a color image, the following steps may be performed:

[0080] A filtering algorithm is used to filter the fused color image to generate a color clear image. The filtering algorithm can be a median filtering algorithm or a multi-scale method.

[0081] In the present application scheme, considering that the image blocks used in the present application scheme for color transfer processing will produce block effects, and interference may occur between image blocks, the block effects are eliminated by filtering to improve the clarity and quality of the image.

[0082] Figure 1 FIG. 1 is a flow chart of a color transfer method in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows; unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps may be executed in other orders; and Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0083] Based on the above method, an embodiment of the present application also discloses a color transfer device.

[0084] Reference Figure 6 , the device includes the following modules:

[0085] A dictionary library construction module 601 is used to obtain reference images corresponding to different scene types and construct a dictionary library according to the reference images, wherein the dictionary library includes scene dictionaries corresponding to different scene types;

[0086] A grayscale image generation module 602 is used to obtain an original visible light image and an original infrared image, perform grayscale multi-scale fusion on the original visible light image and the original infrared image, and generate a grayscale fused image;

[0087] An image segmentation module 603 is used to perform block processing on the original visible light image and the grayscale fused image according to a preset image block size to generate a number of visible light image blocks and a number of grayscale fused image blocks, where the visible light image blocks correspond to the grayscale fused image blocks and the number of the blocks is the same;

[0088] A color transfer module 604 is used to determine the types of several visible light image blocks according to the visible light image blocks and the dictionary library, and transfer the color of the reference image corresponding to the type of the visible light image block to the grayscale fused image block using a color transfer algorithm to generate several color fused image blocks;

[0089] The color image generation module 605 is used to splice and fuse a number of color fusion image blocks to generate a color image.

[0090] In one embodiment, the dictionary library construction module 601 is further used to construct the correspondence between grayscale information and gradient information in the reference image corresponding to different scene types to generate a scene dictionary; and to combine scene dictionaries corresponding to different scene types to form a dictionary library.

[0091] In one embodiment, the grayscale image generation module 602 is further used to obtain visible light video stream data and infrared video stream data; perform frame image extraction on the visible light video stream data and infrared video stream data to obtain original visible light images and original infrared images.

[0092] In one embodiment, the image segmentation module 603 is further used to perform global color transfer processing on the grayscale fused image according to a color mode corresponding to a preset scene type to generate an initial color fused image; block processing is performed on the original visible light image and the grayscale fused image according to a preset image block size to generate a number of visible light image blocks and a number of grayscale fused image blocks, and the visible light image blocks correspond to the grayscale fused image blocks and the number is the same. The method includes: block processing is performed on the original visible light image and the initial color fused image according to a preset image block size to generate a number of visible light image blocks and a number of initial color fused image blocks, and the visible light image blocks correspond to the initial color fused image blocks and the number is the same.

[0093] In one embodiment, the color transfer module 604 is further used to calculate the correspondence between the grayscale information and the gradient information of the visible light image block; calculate the Euclidean distance between the visible light image block and the scene dictionary corresponding to different scene types according to the correspondence between the grayscale information and the gradient information of the visible light image block, and set the calculated minimum Euclidean distance as the type of the visible light image block; calculate and determine the types of all visible light image blocks in sequence; determine whether the color of the corresponding initial color fusion image block is accurate according to the type of the visible light image block; if the color of the initial color fusion image block is inaccurate, transfer the color of the reference image corresponding to the type of the visible light image block to the initial color fusion image block using a color transfer algorithm.

[0094] In one embodiment, the color image generation module 605 is further configured to use a filtering algorithm to filter the fused color image to generate a color clear image.

[0095] The color transfer device provided in the embodiment of the present application can be applied to the color transfer method provided in the above embodiment. For relevant details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0096] It should be noted that: when the color transfer device provided in the embodiment of the present application performs color transfer, only the division of the above-mentioned functional modules / functional units is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the color transfer device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the color transfer method provided in the above method embodiment and the implementation method of the color transfer device provided in this embodiment belong to the same concept. The specific implementation process of the color transfer device provided in this embodiment is detailed in the above method embodiment, which will not be repeated here.

[0097] The embodiment of the present application also discloses a computer device.

[0098] Specifically, if Figure 7As shown, the computer device can be a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. The processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above chips.

[0099] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0100] The embodiment of the present application also discloses a computer-readable storage medium.

[0101] Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art can understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include the process of the implementation of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0102] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A color transfer method, characterized in that: The method comprises: Acquire reference images corresponding to different scene types, and construct a dictionary library according to the reference images, wherein the dictionary library includes scene dictionaries corresponding to different scene types; Acquire an original visible light image and an original infrared image, perform grayscale multi-scale fusion on the original visible light image and the original infrared image, and generate a grayscale fused image; The original visible light image and the grayscale fused image are processed by block division according to a preset image block size to generate a plurality of visible light image blocks and a plurality of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number of the blocks is the same; Determine the types of the visible light image blocks according to the visible light image blocks and the dictionary library, and transfer the color of the reference image corresponding to the type of the visible light image block to the grayscale fused image block by using a color transfer algorithm to generate a plurality of color fused image blocks; Splicing and fusing a plurality of the color fused image blocks to generate a color image; Before the original visible light image and the grayscale fused image are divided into blocks according to a preset image block size, the method further includes: Performing global color transfer processing on the grayscale fused image according to a preset color mode corresponding to the scene type to generate an initial color fused image; The block processing of the original visible light image and the grayscale fused image according to a preset image block size to generate a plurality of visible light image blocks and a plurality of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number of the visible light image blocks is the same, comprises: The original visible light image and the initial color fused image are processed by block division according to a preset image block size to generate a plurality of visible light image blocks and a plurality of initial color fused image blocks, wherein the visible light image blocks correspond to the initial color fused image blocks and the number of the blocks is the same; The step of determining the types of the visible light image blocks according to the visible light image blocks and the dictionary library, and transferring the colors of the reference images corresponding to the types of the visible light image blocks to the grayscale fused image blocks by using a color transfer algorithm comprises: Calculating the correspondence between the grayscale information and the gradient information of the visible light image block; calculating the Euclidean distance between the visible light image block and scene dictionaries corresponding to different scene types according to the correspondence between the grayscale information and the gradient information of the visible light image block, and setting the calculated minimum value of the Euclidean distance as the type of the visible light image block; Calculating and determining the types of all the visible light image blocks in sequence; Determining whether the color of the corresponding initial color fused image block is accurate according to the type of the visible light image block; If the color of the initial color fused image block is inaccurate, the color of the reference image corresponding to the visible light image block type is transferred to the initial color fused image block using a color transfer algorithm; The constructing a dictionary library according to the reference image comprises: Constructing the correspondence between grayscale information and gradient information in reference images corresponding to different scene types, and generating a scene dictionary; The dictionary library is formed by combining scene dictionaries corresponding to different scene types.

2. The method according to claim 1, characterized in that: The obtaining of the original visible light image and the original infrared image comprises: Obtain visible light video stream data and infrared video stream data; Frame image extraction is performed on the visible light video stream data and the infrared video stream data to obtain the original visible light image and the original infrared image.

3. The method according to claim 1, characterized in that: After the color fusion image blocks are spliced ​​and fused to generate a color image, the method further includes: A filtering algorithm is used to filter the fused color image to generate a color clear image.

4. A color transfer device, applied to the color transfer method according to any one of claims 1 to 3, characterized in that: The device comprises: A dictionary library construction module (601) is used to obtain reference images corresponding to different scene types and construct a dictionary library based on the reference images, wherein the dictionary library includes scene dictionaries corresponding to different scene types; A grayscale image generation module (602) is used to obtain an original visible light image and an original infrared image, perform grayscale multi-scale fusion on the original visible light image and the original infrared image, and generate a grayscale fused image; An image segmentation module (603) is used to perform block processing on the original visible light image and the grayscale fused image according to a preset image block size to generate a plurality of visible light image blocks and a plurality of grayscale fused image blocks, wherein the visible light image blocks correspond to the grayscale fused image blocks and the number of the blocks is the same; A color transfer module (604) is used to determine the types of the visible light image blocks according to the visible light image blocks and the dictionary library, and transfer the color of the reference image corresponding to the type of the visible light image block to the grayscale fused image block using a color transfer algorithm to generate a plurality of color fused image blocks; The color image generation module (605) is used to splice and fuse a plurality of the color fusion image blocks to generate a color image.

5. The device according to claim 4, characterized in that: The dictionary library building module (601) is used to: Constructing the correspondence between grayscale information and gradient information in reference images corresponding to different scene types, and generating a scene dictionary; The scene dictionaries corresponding to different scene types are combined to form a dictionary library.

6. The device according to claim 4, characterized in that: The grayscale image generation module (602) is used to: Obtain visible light video stream data and infrared video stream data; Frame image extraction is performed on the visible light video stream data and the infrared video stream data to obtain original visible light images and original infrared images.

7. The device according to claim 4, characterized in that: The image segmentation module (603) is used to: Performing global color transfer processing on the grayscale fused image according to the color mode corresponding to the preset scene type to generate an initial color fused image; The original visible light image and the grayscale fused image are divided into blocks according to a preset image block size to generate a number of visible light image blocks and a number of grayscale fused image blocks. The visible light image blocks correspond to the grayscale fused image blocks and the number is the same, including: The original visible light image and the initial color fused image are divided into blocks according to a preset image block size to generate a number of visible light image blocks and a number of initial color fused image blocks, wherein the visible light image blocks correspond to the initial color fused image blocks and the number of the blocks is the same.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method according to any one of claims 1 to 3.

9. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method as claimed in any one of claims 1 to 3.

Images