A Multi-source Image Fusion Method

Abstract

The present invention discloses a multi-source image fusion method, which relates to the field of image processing technology, including collecting multi-source images, pre-processing the multi-source image data and integrating them into a database; extracting multi-source image features through a database, and constructing an adaptive image feature fusion algorithm; reconstructing the fused image features into a final fused image; and evaluating the quality of the fused image after optimizing the fused image. The present invention extracts image features through a database and constructs an adaptive image feature fusion algorithm, so that the fusion process can dynamically adjust the fusion strategy according to different image features and fusion requirements, which can effectively improve the quality and naturalness of the fused image, solves the problem of insufficient adaptability and flexibility of the prior art in the image fusion process, and improves the computing efficiency while ensuring the fusion effect.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a multi-source image fusion method. Background Art

[0002] In the field of digital image processing, multi-source image fusion is an important technology that aims to fuse multiple images from different sensors, different perspectives or different time points into an image with richer information. With the development of computer vision and artificial intelligence technology, image fusion technology has been widely used in remote sensing, medical imaging, monitoring systems, military reconnaissance and other fields. Traditional image fusion methods mainly include pixel-based fusion, feature-based fusion and decision-based fusion. Although the existing technology has made significant progress, there are still some shortcomings. Most of the existing image fusion algorithms are overly simplified in image preprocessing and feature integration, ignoring the multi-dimensional information of the image, resulting in the fused image being insufficient in detail retention and information integrity. When processing multi-source images in high dynamic range and complex scenes, it is often difficult to achieve the ideal fusion effect. This is mainly because they lack sufficient adaptability and flexibility in the design of image feature extraction and fusion strategies. Summary of the invention

[0003] In view of the problems existing in the above-mentioned existing multi-source image fusion method, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that most existing image fusion algorithms often find it difficult to achieve an ideal fusion effect when processing multi-source images in high dynamic range and complex scenes. This is mainly because they lack sufficient adaptability and flexibility in the design of image feature extraction and fusion strategies.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a multi-source image fusion method, which includes collecting multi-source images, pre-processing the multi-source image data and integrating them into a database; extracting multi-source image features through the database and constructing an adaptive image feature fusion algorithm; reconstructing the fused image features into a final fused image; and evaluating the quality of the fused image after optimizing the fused image.

[0006] As a preferred solution of the multi-source image fusion method described in the present invention, the multi-source image acquisition includes acquiring image metadata, image clarity, contrast, information entropy, texture complexity and edge density in image feature data, environmental data and spatial data.

[0007] As a preferred solution of the multi-source image fusion method described in the present invention, the multi-source image data is preprocessed and integrated into the database to perform geometric correction on the image, eliminate distortion, spatially align images from different sources and convert all images into a unified format and resolution, import the preprocessed images and their metadata into the database, and establish data query and accessibility.

[0008] As a preferred solution of the multi-source image fusion method of the present invention, wherein: the extracting multi-source image features through a database is to extract multi-source image features through a database based on data collected in the database:

[0009]

[0010] Where F represents the feature vector extracted and synthesized from all source images, N is the total number of source images, σ(·) is the activation function, Represents image I i Apply k-fold gradient operations, The standard deviation is σ k Gaussian filter, w k and b are the weight and bias terms of the model respectively, k is the index variable, and K represents the total number of gradient operations.

[0011] As a preferred solution of the multi-source image fusion method of the present invention, the construction of the adaptive image feature fusion algorithm is to construct an adaptive image feature fusion algorithm based on the extracted feature vectors to fuse the extracted feature vectors:

[0012] Construct an image quality assessment function:

[0013]

[0014] in, Represents image I i Laplace operator, Var(·) represents variance calculation, I i,m Yes Yes Image I i The mth pixel value of image I i The average pixel value of image I i The total number of pixels in, p q is image I i The probability of the qth gray level appearing in the image, Q is the total number of gray levels in the image, α, β and γ are weight coefficients;

[0015] Construct an image content complexity evaluation function:

[0016]

[0017] Among them, P x,yis image I i The gray-level co-occurrence matrix, x and y are gray levels, N e is image I i The number of pixels identified as edges by the Canny algorithm, N o is image I i The total number of pixels in, δ and ∈ are weight coefficients;

[0018] Construct the feature difference function:

[0019]

[0020] Where A is the dimension of the feature vector, F i [j] represents the jth eigenvalue in the eigenvector of the i-th image, F a [j] represents the jth eigenvalue in the average of all image feature vectors, w j is the weight of the j-th dimension feature;

[0021] Comprehensively construct an adaptive image feature fusion algorithm:

[0022]

[0023] Among them, F f is the fused image feature vector, N is the total number of source images, I i is the i-th source image, F i is the feature vector of the i-th image, q(I i ) is the image quality assessment function, C(I i ) is the image content complexity evaluation function, D(F i ,F a ) is the feature difference function and λ is the adjustment parameter.

[0024] As a preferred solution of the multi-source image fusion method described in the present invention, wherein: the reconstructing the fused image features into the final fused image is to use the generative adversarial network in the deep learning model to reconstruct the image, including using the source image, feature vector and target image as training data of the model for training. After the training is completed, the fused image feature vector is input into the generative adversarial network to output the reconstructed image.

[0025] As a preferred solution of the multi-source image fusion method described in the present invention, the optimization processing of the fused image is to use a sharpening filter and a local contrast enhancement technique to enhance the details, adjust the color temperature and color saturation according to the overall hue of the image and application requirements, and use histogram equalization to adjust the brightness and contrast range of the image.

[0026] As a preferred solution of the multi-source image fusion method of the present invention, wherein: the quality of the fused image is evaluated by evaluating the quality of the fused image after optimization processing:

[0027] If the quality of the fused image meets the standards, the fused image quality is tested, the latest image processing technology and algorithms are tracked, the fusion algorithm is updated regularly, and the visual appeal of the image is further enhanced by fine-tuning the color balance and contrast;

[0028] If the quality of the fused image does not meet the standards, a comprehensive quality assessment is performed on the fused image to identify specific quality issues, the fusion algorithm is iteratively updated, new technologies and methods are introduced, and post-processing techniques are applied to the fused image that does not meet the standards, including local contrast enhancement, sharpening filtering, and dynamic range adjustment.

[0029] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein the processor implements the steps of a multi-source image fusion method when executing the computer program.

[0030] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of a multi-source image fusion method when executed by a processor.

[0031] The beneficial effects of the present invention are as follows: the present invention extracts image features through a database and constructs an adaptive image feature fusion algorithm, so that the fusion process can dynamically adjust the fusion strategy according to different image features and fusion requirements, which can effectively improve the quality and naturalness of the fused image, solves the problem of insufficient adaptability and flexibility of the prior art in the image fusion process, and improves computing efficiency while ensuring the fusion effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0033] Figure 1 Schematic diagram of the process of multi-source image fusion method.

[0034] Figure 2 Schematic diagram of the workflow of the multi-source image fusion method. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0036] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0037] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0038] Example 1

[0039] Reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a multi-source image fusion method, the multi-source image fusion method comprising the following steps:

[0040] S1, collect multi-source images, pre-process the multi-source image data and integrate them into the database;

[0041] Specifically, collecting multi-source images includes collecting image metadata, image clarity, contrast, information entropy, texture complexity and edge density in image feature data, environmental data and spatial data.

[0042] By collecting image metadata, image feature data (such as image clarity, contrast, etc.), environmental data and spatial data, comprehensive acquisition of images from different sources and types is achieved, providing rich and detailed original data for image fusion, thereby ensuring that the fusion process can be performed based on the most complete information, ultimately improving the quality of the fused image.

[0043] Furthermore, the multi-source image data are preprocessed and integrated into the database to geometrically correct the images, eliminate distortion, spatially align the images from different sources and convert all the images into a uniform format and resolution, import the preprocessed images and their metadata into the database, and establish data queryability and accessibility.

[0044] By performing geometric correction, distortion elimination, spatial alignment, and format and resolution unification on images, we have achieved standardization and consistency of image data from different sources. The preprocessed images and their metadata are imported into the database, and data query and accessibility are established, which facilitates the implementation of subsequent feature extraction and fusion algorithms and improves the manageability of data and the efficiency of subsequent processing.

[0045] S2, extract multi-source image features through the database and build an adaptive image feature fusion algorithm;

[0046] Specifically, extracting multi-source image features through a database is to extract multi-source image features through a database based on data collected in the database:

[0047]

[0048] Wherein, F represents the feature vector extracted and synthesized from all source images, N is the total number of source images, σ(·) is the activation function, which is preferably the ReLU function in this embodiment, Represents image I i Apply k-fold gradient operations, which are implemented by calculating the first-order or higher-order derivatives of the image to extract edge information of the image. The standard deviation is σ k Gaussian filter is used to reduce noise while retaining gradient features, w k and b are the weight and bias of the model, respectively, which are usually learned through the training process in deep learning models. k is an index variable used to traverse these K different gradient operations in the summation expression. K represents the total number of gradient operations.

[0049] Based on the data collected in the database, advanced algorithms are used to extract image features, including the application of k-gradient operations and Gaussian filtering techniques, to achieve in-depth analysis and accurate acquisition of image features, provide key feature information for image fusion, make the fusion process more precise and effective, improve the accuracy and representativeness of the features, and lay a solid foundation for high-quality image fusion.

[0050] Furthermore, an adaptive image fusion algorithm is constructed to construct an adaptive image feature fusion algorithm. An adaptive image feature fusion algorithm is constructed based on the extracted feature vectors to fuse the extracted feature vectors:

[0051] Construct an image quality assessment function:

[0052]

[0053] in, Represents image I i The Laplace operator is a second-order differential operator used to measure the speed of brightness change in an image, thereby detecting the edge of the region and used for edge detection. Var(·) represents variance calculation, which is used to evaluate the clarity of the edge of the image. i,m Yes Yes Image I i The mth pixel value of image I i The average pixel value of image I iThe total number of pixels in, p q is image I i The probability of the qth gray level appearing in the image, Q is the total number of gray levels in the image, α, β and γ are weight coefficients;

[0054] Construct an image content complexity evaluation function:

[0055]

[0056] Among them, P x,y is image I i The gray-level co-occurrence matrix, x and y are gray levels, N e is image I i The number of pixels in the image that are identified as edges by the Canny algorithm, which is a widely used image edge detection technology that aims to accurately detect edge information from an image and locate these edges as accurately as possible. o is image I i The total number of pixels in, δ and ∈ are weight coefficients;

[0057] Construct the feature difference function:

[0058]

[0059] Where A is the dimension of the feature vector, F i [j] represents the jth eigenvalue in the eigenvector of the i-th image, F a [j] represents the jth eigenvalue in the average of all image feature vectors, w j is the weight of the j-th dimension feature, which is set according to the importance of the feature;

[0060] Comprehensively construct an adaptive image feature fusion algorithm:

[0061]

[0062] Among them, F f is the fused image feature vector, N is the total number of source images, I i is the i-th source image, F i is the feature vector of the i-th image, q(I i ) is the image quality assessment function, C(I i ) is the image content complexity evaluation function, D(F i ,F a ) is the feature difference function, is an exponential decay factor that adjusts the weight of each image according to the difference between the feature vectors, and λ is a regulation parameter that controls the effect of feature differences on the overall weight.

[0063] By constructing image quality evaluation function, image content complexity evaluation function and feature difference function, a comprehensive adaptive image feature fusion algorithm is implemented. The algorithm can dynamically adjust based on image quality, content complexity and feature difference, optimize the fusion process, and achieve highly adaptive and comprehensive performance-optimized image fusion, improve the quality and naturalness of the fused image, enhance the flexibility and adaptability of the algorithm, and significantly improve the visual effect and application value of the fused image.

[0064] S3, reconstructing the fused image features into a final fused image;

[0065] Specifically, reconstructing the fused image features into a final fused image is to use a generative adversarial network in a deep learning model to reconstruct the image, including using a source image, a feature vector, and a target image as training data for the model. After the training is completed, the fused image feature vector is input into the generative adversarial network to output the reconstructed image.

[0066] Generative adversarial network is a deep learning model composed of two neural networks. It is mainly used to generate data, especially in the field of image generation. It consists of two parts: generator and discriminator. It reconstructs the fused image features by using the generative adversarial network (GAN) in the deep learning model to achieve high-quality image generation. The source image, feature vector and target image are used as the training data of the model. After training, the fused image feature vector is input into the generative adversarial network, and the reconstructed image is output. The reconstructed image is closer to the original multi-source image in details and overall quality, achieving seamless visual fusion, improving the authenticity and naturalness of the fused image, and providing more accurate and delicate image resources for remote sensing, medical imaging, monitoring systems and other fields.

[0067] S4, evaluating the quality of the fused image after optimizing the fused image;

[0068] Specifically, the fused image is optimized by using a sharpening filter and local contrast enhancement technology to enhance details, adjusting the color temperature and color saturation according to the overall tone of the image and application requirements, and using histogram equalization to adjust the brightness and contrast range of the image.

[0069] By using sharpening filters and local contrast enhancement technology to enhance details, adjusting color temperature and color saturation according to the overall tone of the image and application requirements, and using histogram equalization to adjust the brightness and contrast range of the image, the clarity and visual effect of the fused image are improved, making the image more vivid and layered, enhancing the usability and aesthetics of the image, and making the fused image more in line with the visual habits and aesthetic needs of the human eye, suitable for scenes such as display, analysis, and decision support.

[0070] Furthermore, evaluating the quality of the fused image is to evaluate the quality of the optimized fused image:

[0071] If the quality of the fused image meets the standards, the fused image quality is tested, the latest image processing technology and algorithms are tracked, the fusion algorithm is updated regularly, and the visual appeal of the image is further enhanced by fine-tuning the color balance and contrast;

[0072] If the quality of the fused image does not meet the standards, a comprehensive quality assessment is performed on the fused image to identify specific quality issues, the fusion algorithm is iteratively updated, new technologies and methods are introduced, and post-processing techniques are applied to the fused image that does not meet the standards, including local contrast enhancement, sharpening filtering, and dynamic range adjustment.

[0073] By evaluating the quality of the optimized fused image, the present invention ensures that the quality of the fused image meets the predetermined standard. Through continuous quality control and optimization, the continuous optimization and improvement of the fused image is ensured, and the reliability and adaptability of the image fusion technology are improved, so that it can meet the ever-changing application requirements and quality standards.

[0074] Example 2

[0075] This is the second embodiment of the present invention, and the difference between this embodiment and the previous embodiment is that:

[0076] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0078] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0079] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A multi-source image fusion method, characterized in that: include, Collect multi-source images, pre-process the multi-source image data and integrate them into the database; Extract multi-source image features through the database and build an adaptive image feature fusion algorithm; Reconstruct the fused image features into a final fused image; Evaluate the quality of the fused image after optimizing the fused image; The collecting of multi-source images includes collecting image metadata, image clarity, contrast, information entropy, texture complexity and edge density, environmental data and spatial data in image feature data; The preprocessing of multi-source image data and integration into the database is to perform geometric correction on the image, eliminate distortion, spatially align images from different sources and convert all images into a unified format and resolution, import the preprocessed images and their metadata into the database, and establish data query and accessibility; The method of extracting multi-source image features through a database is to extract multi-source image features through a database based on data collected in the database: Where F represents the feature vector extracted and synthesized from all source images, N is the total number of source images, σ(·) is the activation function, Represents image I i Apply k-fold gradient operations, The standard deviation is σ k Gaussian filter, w k and b are the weight and bias of the model respectively, k is the index variable, and K represents the total number of gradient operations; The construction of the adaptive image feature fusion algorithm is to construct an adaptive image feature fusion algorithm based on the extracted feature vectors to fuse the extracted feature vectors: Construct an image quality assessment function: in, Represents image I i Laplace operator, Var(·) represents variance calculation, I i,m Yes Yes Image I i The mth pixel value of image I i The average pixel value of image I i The total number of pixels in, p q is image I i The probability of the qth gray level appearing in the image, Q is the total number of gray levels in the image, α, β and γ are weight coefficients; Construct an image content complexity evaluation function: Among them, P x,y is image I i The gray-level co-occurrence matrix, x and y are gray levels, N e is image I i The number of pixels identified as edges by the Canny algorithm, N o is image I i The total number of pixels in, δ and ∈ are weight coefficients; Construct the feature difference function: Where A is the dimension of the feature vector, F i [j] represents the jth eigenvalue in the eigenvector of the i-th image, F a [j] represents the jth eigenvalue in the average of all image feature vectors, w j is the weight of the j-th dimension feature; Comprehensively construct an adaptive image feature fusion algorithm: Among them, F f is the fused image feature vector, N is the total number of source images, I i is the i-th source image, F i is the feature vector of the i-th image, q(I i ) is the image quality assessment function, C(I i ) is the image content complexity evaluation function, D(F i ,F a ) is the feature difference function and λ is the adjustment parameter.

2. The multi-source image fusion method according to claim 1, characterized in that: The method of reconstructing the fused image features into a final fused image is to use a generative adversarial network in a deep learning model to perform image reconstruction, including using a source image, a feature vector and a target image as training data for the model. After the training is completed, the fused image feature vector is input into the generative adversarial network to output a reconstructed image.

3. The multi-source image fusion method according to claim 2, characterized in that: The optimization processing of the fused image is to use a sharpening filter and a local contrast enhancement technology to enhance details, adjust the color temperature and color saturation according to the overall hue of the image and application requirements, and use histogram equalization to adjust the brightness and contrast range of the image.

4. The multi-source image fusion method according to claim 3, characterized in that: The quality evaluation of the fused image is to evaluate the quality of the optimized fused image: If the quality of the fused image meets the standards, the fused image quality is tested, the latest image processing technology and algorithms are tracked, the fusion algorithm is updated regularly, and the visual appeal of the image is further enhanced by fine-tuning the color balance and contrast; If the quality of the fused image does not meet the standards, a comprehensive quality assessment is performed on the fused image to identify specific quality issues, the fusion algorithm is iteratively updated, new technologies and methods are introduced, and post-processing techniques are applied to the fused image that does not meet the standards, including local contrast enhancement, sharpening filtering, and dynamic range adjustment.

5. A computer device comprising: A memory and a processor; the memory stores a computer program, wherein the processor implements the steps of the multi-source image fusion method according to any one of claims 1 to 4 when executing the computer program.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-source image fusion method according to any one of claims 1 to 4 are implemented.

Images