Multi-source image fusion method based on adaptive cross window weighted link strength and related device

Abstract

The invention discloses a multi-source image fusion method based on adaptive cross window weighted link strength and a related device. Comprising a multi-scale transformation decomposition link of multi-source image data, a fusion link of adopting a # imgabs0 # model of adaptive cross window weighted link strength for a low-frequency sub-segment, a fusion link of adopting a region average energy contrast combined region variance weighted adaptive algorithm for a high-frequency sub-segment, and a multi-scale decomposition inverse transformation image fusion link. The fusion rate can be effectively improved by adopting a # imgabs1 # model of adaptive cross window weighted link strength in the low-frequency sub-segment fusion, and the detail texture information of the high-frequency sub-segment can be reserved to the maximum extent by adopting an average energy contrast combined region variance weighted adaptive strategy in the high-frequency sub-segment. Finally, the rapidity of the algorithm is improved while the fusion quality of the multi-source image information is reserved as much as possible by the multi-source image data.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology and relates to a multi-source image fusion method based on adaptive cross window weighted link strength and a related device. Background Art

[0002] Image fusion refers to the integration of information from two or more multi-source images. Its purpose is to extract information from multiple images, concentrate or integrate complementary data to improve the information availability of images, and obtain a more accurate, comprehensive and reliable image description of the same target. The fused image is more in line with the visual characteristics of humans or machines, so as to facilitate further analysis, understanding, detection and recognition of the image.

[0003] Image fusion technology can be divided into three levels: pixel level, feature level and decision level. Among them, pixel-level image fusion technology directly fuses on the original data layer. Although it can keep as much original data as possible, the source images must be accurately registered before image fusion, so it has limitations. In addition, the pixel-level image fusion technology has a large amount of data, slow processing speed and poor real-time performance. Feature-level image fusion technology belongs to the middle layer. It first extracts features from the original image, and then comprehensively analyzes and processes the extracted features, achieving objective information compression, which is conducive to real-time processing. Decision-level fusion technology is a high-level fusion. During the fusion process, each sensor first establishes preliminary judgments and conclusions on the same target, then performs relevant processing on the decisions from each sensor, and finally performs decision-level fusion processing. Decision-level fusion technology has good real-time and fault tolerance, but the preprocessing cost is high.

[0004] In the prior art, there have been many studies on image fusion techniques. Among them, the average fusion algorithm replaces the grayscale values ​​of relevant pixels in the image to be fused with the average value of the grayscale values ​​of the relevant pixels. The fused image generated by this method contains not only clear information in the image, but also blurred information in the image, so that effective and invalid information are combined together, and a good fusion effect cannot be obtained. The low-pass ratio pyramid image fusion algorithm is a method that is compatible with people's visual system model and can efficiently retain the detail features in the image. Although it conforms to the visual characteristics of people, it is sensitive to noise and unstable because the local contrast of noise is generally large. The image fusion method based on HIS transformation transforms an image to be fused from RGB space to HIS space, and then replaces the I component of the previous image with the intensity of another image to be fused. It is best to reversely transform the replaced HIS image into an RGB image, that is, the fused image. Since the I component of an image to be fused in the HIS space and the grayscale value of another image to be fused are two channel data with different spectral intensity characteristics, this replacement causes the fused image to produce spectral degradation and serious loss of spectral information.

[0005] In summary, the image fusion technology in the prior art has large data volume, slow processing speed, is sensitive to noise and produces poor fusion effect. Therefore, proposing a new image fusion algorithm is the key to solving the above problems. Summary of the invention

[0006] The purpose of this application is to solve the problems of large data volume, slow processing speed and poor fusion effect in the existing image fusion technology, and to provide a multi-source image fusion method and related device based on adaptive cross window weighted link strength. In order to achieve the above objectives, this application adopts the following technical solutions: In a first aspect, the present application provides a multi-source image fusion method based on adaptive cross window weighted link strength, comprising the following steps: Acquire the registered multispectral images to form a multi-source image data set; Decomposing the image data in the multi-source image data set into high-frequency sub-band images and low-frequency sub-band images by a multi-scale transformation algorithm; Input the low frequency subband image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, and then the low-frequency fusion image is obtained; When the algorithm model calculates the low-frequency sub-segment fusion weight, it includes determining an adaptive cross window based on each pixel point in the low-frequency sub-band image, and calculating the weighted link strength of the corresponding window under the adaptive cross window; The high-frequency sub-segment fusion of high-frequency sub-band images is performed by combining regional average energy contrast with regional variance weighted adaptive strategy to obtain a high-frequency fused image. A multi-scale transformation algorithm is used to inversely transform the low-frequency fusion image and the high-frequency fusion image to achieve image fusion.

[0007] Furthermore, the present application describes inputting the low-frequency sub-band image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, specifically: 1) Generate pixels Belonging adaptive cross window: The adaptive cross window is composed of a horizontal gradient filter operator With vertical filtering operator It consists of two parts: determining the current pixel point based on the image structure and color information The size of the cross-intersection area is determined by the arm length parameters in four directions. The arm length parameters in the four directions are divided into two filtering operators: horizontal and vertical directions. and The arm lengths in the four directions are adaptively adjusted according to the structure of the image and the discrimination criteria of the color information. The specific discrimination criteria are as follows: Principle 1:

[0008] Guideline 2:

[0009] Principle 3:

[0010] in, Pixel With pixels The spatial distance difference, is the pixel difference, , Represents the element value of a color channel at the pixel point; is the preset upper limit of the color threshold. is the preset color threshold lower limit, is the preset upper limit of the distance threshold, is the preset lower distance threshold, and , ; 2) The pixel point can be determined by the discrimination criterion The size of the horizontal filter operator and the size of the vertical filter operator of the corresponding cross window are then determined with weighted coefficients; The weighting coefficient of the horizontal filtering operator is:

[0011] The weighted coefficient of the vertical filtering operator is:

[0012] in, and are the maximum weights in the horizontal and vertical directions respectively, , , the minimum weight is always ; 3) Fill the pixels around the cross window with a constant value of 0, and finally get the determined pixels The horizontal gradient filter operator required by the adaptive cross window With vertical filtering operator , whose size is ,in , ; 4) Horizontal gradient filtering operator With vertical filtering operator Get the adaptive cross window :

[0013]

[0014]

[0015] in, Represents pixels The surrounding size is The pixel area, Indicates that the area is filtered by the vertical filter operator The window matrix after the action, Indicates that the area is filtered by the horizontal filter operator The window matrix after the action, Indicates the pixel obtained by considering both Adaptive cross window matrix; 5) Based on pixels The adaptive cross-window of Calculate the weighted link strength under this window:

[0016]

[0017] in, Represents pixel The weighted link strength under the adaptive cross window where it is located, where represents the average gradient in this adaptive cross window; 6) Use the above weighted link strength to calculate the low-frequency sub-segment fusion weight.

[0018] Furthermore, the present application The algorithm model uses the image spatial frequency taking into account the main and secondary diagonals as input variables:

[0019]

[0020]

[0021]

[0022]

[0023]

[0024] in, represents the feedback input of the PCNN model, represents the number of iterations, represents the spatial frequency of the image taking into account the main and secondary diagonals, represents the gradient energy in the vertical direction, represents the gradient energy in the horizontal direction, represents the gradient energy in the main diagonal direction, represents the gradient energy on the sub-diagonal, Represents pixel elements in the neighborhood.

[0025] Furthermore, the present application The algorithm model uses an adaptive weight matrix to dynamically adjust the coupling between adjacent neurons:

[0026] in, represents the weight matrix, Represents the connection range of the neuron.

[0027] Furthermore, the multispectral image described in the present application includes a visible light image and an infrared light image. After the algorithm model obtains the ignition mapping diagram of visible light and infrared light, the fusion coefficient is selected according to the following selection rules:

[0028] in, represents the low-frequency coefficient of visible light, represents the low-frequency coefficient of infrared light, represents the fused low-frequency coefficients, Indicates the number of infrared light ignitions. Indicates the number of visible light ignitions.

[0029] Furthermore, the present application adopts the regional average energy contrast combined with the regional variance weighted adaptive strategy to perform high-frequency sub-segment fusion on the high-frequency sub-band image, including:

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] in, Representing images In pixels The central area Regional energy, Indicates area The point in Represents the average energy contrast of the area corresponding to infrared light, represents a positive integer, represents the energy of the visible light image area, Represents a visible light image, Represents the pixel elements in the visible light image neighborhood, Represents the average energy contrast of the region corresponding to visible light, represents the energy of the infrared image area, Indicates infrared light images. Represents the pixel elements in the infrared light image neighborhood, represents the regional variance corresponding to infrared light, represents the regional variance corresponding to visible light, Indicates After layer decomposition, Localized order of directions; The selection rules of fusion coefficients under the regional average energy contrast combined with regional variance weighted adaptive strategy are as follows:

[0036] and is the weight coefficient and satisfies:

[0037] in, Represents the fused image The high frequency coefficients.

[0038] In a second aspect, the present application provides a multi-source image fusion system based on adaptive cross window weighted link strength, comprising: An image acquisition module is used to acquire multispectral images that have been registered and processed to form a multi-source image data set; A data decomposition module, used for decomposing image data in a multi-source image data set into high-frequency sub-band images and low-frequency sub-band images by a multi-scale transformation algorithm; The low-frequency sub-segment fusion module is used to input the low-frequency sub-band image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, and then the low-frequency fusion image is obtained; A high-frequency sub-segment fusion module is used to fuse high-frequency sub-segments of high-frequency sub-band images using a regional average energy contrast combined with a regional variance weighted adaptive strategy to obtain a high-frequency fused image; The inverse transformation module is used to perform inverse transformation on the low-frequency fusion image and the high-frequency fusion image by adopting a multi-scale transformation algorithm to achieve image fusion.

[0039] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0041] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. A processor of a computer device reads the computer instructions, and the processor of the computer device executes the computer instructions to implement the steps of the above method.

[0042] Compared with the prior art, this application has the following beneficial effects: The present invention adopts adaptive cross window weighted link strength in low frequency sub-segment fusion. The model can effectively improve the fusion rate. By adopting the average energy contrast combined with the regional variance weighted adaptive strategy in the high-frequency sub-segments, the detailed texture information of the high-frequency sub-segments can be retained to the greatest extent. Ultimately, the multi-source image data can be improved while retaining the multi-source image information as much as possible and improving the speed of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A flowchart of the method of the present application.

[0045] Figure 2 Schematic diagram for this application Figure 3 This is the logic block diagram of the system of this application.

[0046] Figure 4 These are experimental effect diagrams of the embodiments of the present application, where (a) is the experimental result of the algorithm of the present application, (b) is the experimental result of the traditional NSST-PCNN algorithm, (c) is the experimental result of the wavelet fusion algorithm, and (d) is the experimental result of the mean fusion algorithm. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0049] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0050] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0051] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0052] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0053] Algorithmic model, that is, based on the link coefficient The optimized Pulse Coupled Neural Network (PCNN) algorithm model is an algorithm with significant advantages in the field of image processing.

[0054] The present application is further described in detail below with reference to the accompanying drawings: See also Figure 1 The embodiment of the present application discloses a multi-source image fusion method based on adaptive cross window weighted link strength, comprising the following steps: S1 acquires the registered multispectral images to form a multi-source image data set; S2 decomposes the image data in the multi-source image data set into high-frequency sub-band images and low-frequency sub-band images through a multi-scale transformation algorithm; S3 inputs the low-frequency subband image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, and then the low-frequency fusion image is obtained; When the algorithm model calculates the low-frequency sub-segment fusion weight, it includes determining an adaptive cross window based on each pixel point in the low-frequency sub-band image, and calculating the weighted link strength of the corresponding window under the adaptive cross window; S4 adopts the regional average energy contrast combined with the regional variance weighted adaptive strategy to fuse the high-frequency sub-segments of the high-frequency sub-band image to obtain a high-frequency fused image; S5 uses a multi-scale transformation algorithm to inversely transform the low-frequency fusion image and the high-frequency fusion image to achieve image fusion.

[0055] like Figure 2 As shown, in a feasible embodiment of the present application, step S3 inputs the low-frequency sub-band image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained. The specific method is as follows: The tradition The link strength based on the fixed window in the algorithm is replaced by the weighted link strength based on the adaptive cross window, as follows: First, the pixels The adaptive cross window is generated and determined as follows: The adaptive cross window is mainly composed of the horizontal gradient filter operator With vertical filtering operator It consists of two parts. First, the current pixel is determined based on the image structure and color information. A cross-section area, the size of which is determined by the arm length parameters in four directions It is decided to divide it into two filtering operators in the horizontal direction and the vertical direction and , the arm length is adaptively adjusted according to the structure and color information of the image, and the specific judgment criteria are as follows: Principle 1:

[0056] Guideline 2:

[0057] Principle 3:

[0058] in Pixel and The spatial distance difference is is the pixel difference, specifically defined as ,and , are the preset color threshold and distance threshold. Criterion 1 is mainly used to limit and The color difference, and The color difference between the pixel on its right is mainly to prevent the arm from being too long and crossing the boundary; criteria 2 and 3 mainly consider the spatial distance, and the maximum spatial distance is , but when the spatial distance is When the color difference is between the two, it is necessary to limit the threshold of color difference.

[0059] The above rules can be used to determine the pixel points The corresponding horizontal filter operator size and vertical filter operator size of the cross window are then further determined. The closer the distance to the center pixel point, the greater the contribution to the link strength. The farther the distance to the center pixel point, the smaller the contribution to the link strength. The maximum weight is , , the minimum weight is always , with linear decay according to the inverse distance.

[0060] The weighting coefficient of the horizontal filtering operator is:

[0061] The weighted coefficient of the vertical filtering operator is:

[0062] Finally, fill the pixels around the cross window with a constant value of 0 to get the final pixel. The horizontal gradient filter operator required by the adaptive cross window With vertical filtering operator , whose size is ,in , .

[0063] By the horizontal gradient filter operator With vertical filtering operator You can get an adaptive cross window :

[0064]

[0065]

[0066] in Represents pixels The surrounding size is The pixel area, Indicates that the area is filtered by the vertical filter operator The window matrix after the action, Indicates that the area is filtered by the horizontal filter operator The window matrix after the action, Indicates the pixel obtained by considering both Adaptive cross window matrix.

[0067] In determining the pixel After the adaptive cross window is calculated, the weighted link strength under this window can be calculated:

[0068] in Represents pixel The weighted link strength under the adaptive cross window where it is located, where It represents the average gradient in the adaptive cross window. The specific calculation formula is as follows:

[0069] In practical applications, The algorithm model uses the image spatial frequency taking into account the main and secondary diagonals as input variables:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] in, represents the spatial frequency of the image taking into account the main and secondary diagonals, represents the gradient energy in the vertical direction, represents the gradient energy in the horizontal direction, represents the gradient energy in the main diagonal direction, represents the gradient energy on the sub-diagonal, Represents pixel elements in the neighborhood.

[0076] In practical applications, The algorithm model uses an adaptive weight matrix to dynamically adjust the coupling between adjacent neurons:

[0077] In the formula, Represents the connection range of the neuron.

[0078] In practical applications, After obtaining the ignition mapping diagram of visible light and infrared light in the algorithm model, the selection rule of the fusion coefficient is:

[0079] represents the low-frequency coefficient of visible light, represents the low-frequency coefficient of infrared light, represents the fused low-frequency coefficients, Indicates the number of infrared light ignitions. Indicates the number of visible light ignitions.

[0080] In another feasible embodiment of the present application, the high-frequency sub-band image obtained in step S2 adopts a regional average energy contrast combined with a regional variance weighted adaptive strategy to perform high-frequency sub-segment fusion, specifically including:

[0081]

[0082]

[0083]

[0084]

[0085]

[0086] In the formula, Representing images In pixels The central area Regional energy, Represents the average energy contrast of the area corresponding to infrared light, Represents the average energy contrast of the region corresponding to visible light, represents the regional variance corresponding to infrared light, represents the regional variance corresponding to visible light, Indicates After layer decomposition, The localization degree of the direction.

[0087] The selection rules of fusion coefficients under the regional average energy contrast combined with regional variance weighted adaptive strategy are as follows:

[0088] in satisfy:

[0089] Represents the fused image The high frequency coefficients of is the weight coefficient.

[0090] like Figure 3 As shown, the embodiment of the present application discloses a multi-source image fusion system based on adaptive cross window weighted link strength, including: An image acquisition module is used to acquire multispectral images that have been registered and processed to form a multi-source image data set; A data decomposition module, used for decomposing image data in a multi-source image data set into high-frequency sub-band images and low-frequency sub-band images by a multi-scale transformation algorithm; The low-frequency sub-segment fusion module is used to input the low-frequency sub-band image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, and then the low-frequency fusion image is obtained; A high-frequency sub-segment fusion module is used to fuse high-frequency sub-segments of high-frequency sub-band images using a regional average energy contrast combined with a regional variance weighted adaptive strategy to obtain a high-frequency fused image; The inverse transformation module is used to perform inverse transformation on the low-frequency fusion image and the high-frequency fusion image by adopting a multi-scale transformation algorithm to achieve image fusion.

[0091] Example In order to illustrate the effectiveness of the multi-source image fusion method based on adaptive cross window weighted link strength proposed in this application, the following experimental demonstration is conducted: 1460 sets of registered infrared and visible light image data sets with a size of 400×400 pixels are selected for the experiment. By comparing the image fusion algorithm proposed in this application with the wavelet fusion algorithm and the traditional mean fusion algorithm, the effects of various fusions are compared from four dimensions: standard deviation, information entropy, average gradient, and spatial frequency: Table 1 Objective price comparison table:

[0092] like Figure 4 As shown, Figure 4 The experimental effect diagrams are shown in Figures (a), (b), (c), and (d) respectively, which are the experimental results of different algorithms. It can be seen from the visual effect that the fused image obtained by the algorithm of this application can better eliminate the problem of false information easily introduced by other multi-resolution methods during fusion, and the detail information is better preserved, making the reconstructed image look more satisfactory to the human eye's visual perception of the image. It can also be seen from the table that the use of the improved NSST-PCNN algorithm for image fusion has advantages in standard deviation, information entropy, average gradient, and spatial frequency that other algorithms do not have.

[0093] Therefore, this application solves the problem of information loss and spatial distortion in fused images, achieves good results in both subjective visual effects and objective evaluation, and effectively preserves the spectral information of multispectral images while enhancing the spatial detail expression capability and improving the spatial resolution. A computer device is provided in one embodiment of the present application. The computer device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0094] The computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present application.

[0095] The computer device may be a desktop computer, a notebook computer, a palm computer, a cloud server, etc. The computer device may include, but is not limited to, a processor and a memory.

[0096] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0097] The memory may be used to store the computer programs and / or modules, and the processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0098] If the module / unit integrated in the computer device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0099] The present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 1 The methods provided in the various optional methods are therefore not described in detail here.

[0100] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.

[0101] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0102] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0103] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 The computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device, or are transmitted through a computer-readable storage medium. Computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The instruction device is implemented in the process Figure 1 A process or multiple processes and / or structures Figure 1These computer program instructions can also be loaded onto a computer or other programmable device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0104] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0105] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0106] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A multi-source image fusion method based on adaptive cross window weighted link strength, characterized in that: The following steps are involved: Acquire the registered multispectral images to form a multi-source image data set; Decomposing the image data in the multi-source image data set into high-frequency sub-band images and low-frequency sub-band images by a multi-scale transformation algorithm; Input the low frequency subband image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, and then the low-frequency fusion image is obtained; When the algorithm model calculates the low-frequency sub-segment fusion weight, it includes determining an adaptive cross window based on each pixel point in the low-frequency sub-band image, and calculating the weighted link strength of the corresponding window under the adaptive cross window; The high-frequency sub-segment fusion of high-frequency sub-band images is performed by combining regional average energy contrast with regional variance weighted adaptive strategy to obtain a high-frequency fused image. A multi-scale transformation algorithm is used to inversely transform the low-frequency fusion image and the high-frequency fusion image to achieve image fusion.

2. The multi-source image fusion method based on adaptive cross window weighted link strength according to claim 1 is characterized in that: The low frequency sub-band image is input into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, specifically: 1) Generate pixels Belonging adaptive cross window: The adaptive cross window is composed of a horizontal gradient filter operator With vertical filtering operator It consists of two parts: determining the current pixel point based on the image structure and color information The size of the cross-intersection area is determined by the arm length parameters in four directions. The arm length parameters in the four directions are divided into two filtering operators: horizontal and vertical directions. and The arm lengths in the four directions are adaptively adjusted according to the structure of the image and the discrimination criteria of the color information. The specific discrimination criteria are as follows: Principle 1: Guideline 2: Principle 3: in, Pixel With pixels The spatial distance difference, is the pixel difference, , Represents the element value of a color channel at the pixel point; is the preset upper limit of the color threshold. is the preset color threshold lower limit, is the preset upper limit of the distance threshold, is the preset lower distance threshold, and , ; 2) The pixel point can be determined by the discrimination criterion The size of the horizontal filter operator and the size of the vertical filter operator of the corresponding cross window are then used to determine the weighting coefficients therein; The weighting coefficient of the horizontal filtering operator is: The weighted coefficient of the vertical filtering operator is: in, and are the maximum weights in the horizontal and vertical directions respectively, , , the minimum weight is always ; 3) Fill the pixels around the cross window with a constant value of 0, and finally get the determined pixels The horizontal gradient filter operator required by the adaptive cross window With vertical filtering operator , whose size is ,in , ; 4) Horizontal gradient filtering operator With vertical filtering operator Get the adaptive cross window : in, Represents pixels The surrounding size is The pixel area, Indicates that the area is filtered by the vertical filter operator The window matrix after the action, Indicates that the area is filtered by the horizontal filter operator The window matrix after the action, Indicates the pixel obtained by considering both Adaptive cross window matrix; 5) Based on pixels The adaptive cross-window of Calculate the weighted link strength under this window: in, Represents pixel The weighted link strength under the adaptive cross window where it is located, where represents the average gradient in this adaptive cross window; 6) Use the above weighted link strength to calculate the low-frequency sub-segment fusion weight.

3. The multi-source image fusion method based on adaptive cross window weighted link strength according to claim 2 is characterized in that: Said The algorithm model uses the image spatial frequency taking into account the main and secondary diagonals as input variables: in, represents the feedback input of the PCNN model, represents the number of iterations, represents the spatial frequency of the image taking into account the main and secondary diagonals, represents the gradient energy in the vertical direction, represents the gradient energy in the horizontal direction, represents the gradient energy in the main diagonal direction, represents the gradient energy on the sub-diagonal, Represents pixel elements in the neighborhood.

4. The multi-source image fusion method based on adaptive cross window weighted link strength according to claim 2 is characterized in that: Said The algorithm model uses an adaptive weight matrix to dynamically adjust the coupling between adjacent neurons: in, represents the weight matrix, Represents the connection range of the neuron.

5. The multi-source image fusion method based on adaptive cross window weighted link strength according to claim 2 is characterized in that: The multispectral image includes a visible light image and an infrared light image. After the algorithm model obtains the ignition mapping diagram of visible light and infrared light, the fusion coefficient is selected according to the following selection rules: in, represents the low-frequency coefficient of visible light, represents the low-frequency coefficient of infrared light, represents the fused low-frequency coefficients, Indicates the number of infrared light ignitions. Indicates the number of visible light ignitions.

6. The multi-source image fusion method based on adaptive cross window weighted link strength according to claim 1 is characterized in that: The method of using the regional average energy contrast combined with the regional variance weighted adaptive strategy to fuse the high-frequency sub-segments of the high-frequency sub-band image includes: in, Representing images In pixels The central area Regional energy, Indicates area The point in Represents the average energy contrast of the area corresponding to infrared light, represents a positive integer, represents the energy of the visible light image area, Represents a visible light image, Represents the pixel elements in the visible light image neighborhood, Represents the average energy contrast of the region corresponding to visible light, represents the energy of the infrared image area, Indicates infrared light images. Represents the pixel elements in the infrared light image neighborhood, represents the regional variance corresponding to infrared light, represents the regional variance corresponding to visible light, Indicates After layer decomposition, Localized order of directions; The selection rules of fusion coefficients under the regional average energy contrast combined with regional variance weighted adaptive strategy are as follows: and is the weight coefficient and satisfies: in, Represents the fused image The high frequency coefficients.

7. A multi-source image fusion system based on adaptive cross window weighted link strength, characterized in that: include: An image acquisition module is used to acquire the registered visible light and infrared light images to form a multi-source image data set; A data decomposition module, used for decomposing image data in a multi-source image data set into high-frequency sub-band images and low-frequency sub-band images by a multi-scale transformation algorithm; The low-frequency sub-segment fusion module is used to input the low-frequency sub-band image into In the algorithm model, the low-frequency sub-segment fusion weight is obtained, and then the low-frequency fusion image is obtained; A high-frequency sub-segment fusion module is used to fuse high-frequency sub-segments of high-frequency sub-band images using a regional average energy contrast combined with a regional variance weighted adaptive strategy to obtain a high-frequency fused image; The inverse transformation module is used to perform inverse transformation on the low-frequency fusion image and the high-frequency fusion image by adopting a multi-scale transformation algorithm to achieve image fusion.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product includes computer instructions. A processor of a computer device reads the computer instructions, and the processor of the computer device executes the computer instructions to implement the steps of the method according to any one of claims 1 to 6.

Images