Infrared and visible light image fusion method based on multi-scale fractional gradient model

Abstract

The invention relates to an infrared and visible light image fusion method based on a multi-scale fractional gradient model. The method comprises the following steps: decomposing a source image into a base layer and a detail layer by applying multistage decomposition, fusing the detail layer by adopting a designed fractional order differential fusion rule, and fusing the base layer by adopting an average strategy. And adding the fused detail layer and the fused base layer to obtain a final fused image. Fractional order differential has global correlation and can well reflect the dependency relationship between image changes, and based on the characteristic of the fractional order differential, multi-level image features can be more comprehensively extracted and good texture details can be reserved, so that the overall quality of the fused image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for fusing infrared and visible light images based on a multi-scale fractional gradient model. Background Art

[0002] At present, many image fusion methods have been proposed in academia, which can be roughly divided into two categories. The first category is non-deep learning methods, which usually use multi-scale transformation methods or representation methods to achieve fusion operations. Although these methods have achieved good performance, the choice of transformation methods and fusion rules affects their fusion performance. The second type is a fusion method based on deep learning, which usually constructs a loss function to train an attention-based or Transformer-based network so that the fused image has the desired distribution characteristics. This type of method benefits from the strong robustness and nonlinear fitting ability of neural networks, so it can usually achieve better fusion performance. The disadvantage of the second type of method is that network training is difficult when there is insufficient training data, especially in the task of infrared and visible light image fusion.

[0003] Based on the problems of the above fusion technology, the academic community proposed a multi-level decomposition method as a framework for infrared and visible light image fusion to extract important features from the input data. Deep learning in the field of image fusion usually requires a large amount of labeled data for training to learn the features and fusion rules of the image. Without sufficient data, the performance of the deep learning model may be limited. The multi-level decomposition method uses a method based on algebraic feature extraction, which relies more on the intrinsic low-rank characteristics of the image rather than a large amount of training data. Therefore, it can perform well in infrared and visible light image fusion tasks even when the training data is limited.

[0004] However, the existing multi-level decomposition methods are based on the fusion rule of the nuclear norm, which is found in practice to lose texture details in the image, which is not conducive to the quality of the final fused image. Summary of the invention

[0005] Based on this, it is necessary to provide an infrared and visible light image fusion method based on a multi-scale fractional gradient model that can improve the image fusion performance in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for fusion of infrared and visible light images based on a multi-scale fractional gradient model. The method comprises:

[0007] A multi-level decomposition model is used to decompose the source image pair to obtain a base layer image pair and a detail layer feature pair; the source image pair includes an infrared image and a visible light image of the same scene;

[0008] The base layer image pairs are fused to obtain the first fusion layer features, and the detail layer feature pairs are fused based on fractional differentials to obtain the second fusion layer features;

[0009] Based on the first fusion layer features and the second fusion layer features, a fused image of the source image pair is obtained.

[0010] In one embodiment, obtaining second fusion layer features based on fractional differential fusion of detail layer features includes:

[0011] Construct fractional differential mask;

[0012] According to the fractional differential mask and the detail layer feature pair, a fractional gradient energy function is obtained;

[0013] The detail layer feature pairs are fused based on the fractional gradient energy function to obtain the second fusion layer features.

[0014] In one embodiment, constructing a fractional differential mask includes:

[0015] Construct fractional differential masks based on corrected spatial frequency and average gradient.

[0016] In one embodiment, the detail layer feature pair includes multiple levels of feature pairs, and each level of feature pair includes a first feature corresponding to the infrared image and a second feature corresponding to the visible light image;

[0017] According to the fractional differential mask and the detail layer feature pair, the fractional gradient energy function is obtained including:

[0018] For each level of feature pairs, a first function corresponding to the first feature and a second function corresponding to the second feature are obtained according to the fractional differential mask and the detail layer feature pair; the first function and the second function are both fractional gradient energy functions.

[0019] In one embodiment, fusing the detail layer feature pairs based on the fractional gradient energy function to obtain the second fusion layer features includes:

[0020] For each level of feature pair, according to the first function and the second function, obtain the weight coefficients corresponding to the first feature and the second feature respectively;

[0021] The first feature and the second feature are fused according to the weight coefficient to obtain the second fusion layer features corresponding to the feature pairs at each level.

[0022] In one embodiment, based on the first fusion layer feature and the second fusion layer feature, obtaining a fused image of the source image pair includes:

[0023] The second fusion layer features corresponding to the feature pairs at each level are reconstructed and combined with the first fusion layer features to obtain a fused image.

[0024] In a second aspect, the present application also provides an infrared and visible light image fusion device based on a multi-scale fractional gradient model. The device comprises:

[0025] A multi-level decomposition module is used to decompose a source image pair using a multi-level decomposition model to obtain a base layer image pair and a detail layer feature pair; the source image pair includes an infrared image and a visible light image of the same scene;

[0026] The first fusion module is used to fuse the base layer image pair to obtain the first fusion layer features, and to fuse the detail layer feature pair based on fractional differential to obtain the second fusion layer features;

[0027] The second fusion module is used to obtain a fused image of the source image pair based on the first fusion layer features and the second fusion layer features.

[0028] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the infrared and visible light image fusion method based on the multi-scale fractional gradient model when executing the computer program.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above-mentioned infrared and visible light image fusion method based on a multi-scale fractional gradient model when executed by a processor.

[0030] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps in the above-mentioned infrared and visible light image fusion method based on a multi-scale fractional gradient model when executed by a processor.

[0031] The above-mentioned infrared and visible light image fusion method based on the multi-scale fractional gradient model uses multi-level decomposition to decompose the source image into a base layer and a detail layer, and then uses the designed fractional differential fusion rule to fuse the detail layer, and uses the average strategy to fuse the base layer. The fused detail layer and the fused base layer are added to obtain the final fused image. Fractional differentials have global correlation and can well reflect the dependencies between image changes. Based on this feature of fractional differentials, the present invention can more comprehensively extract multi-level image features and retain good texture details, thereby improving the overall quality of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 1 is a flow chart of a method for fusion of infrared and visible light images based on a multi-scale fractional gradient model in one embodiment;

[0033] Figure 2is a schematic diagram of an image data conversion process in one embodiment;

[0034] Figure 3 The figure is a structural block diagram of an infrared and visible light image fusion device based on a multi-scale fractional gradient model in one embodiment. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] The present application embodiment provides a method for fusion of infrared and visible light images based on a multi-scale fractional gradient model. Figure 1 As shown, the following steps are included:

[0037] Step 102, using a multi-level decomposition model to decompose a source image pair to obtain a base layer image pair and a detail layer feature pair; the source image pair includes an infrared image and a visible light image of the same scene.

[0038] Infrared images are not affected by lighting conditions, but lack scene detail information; visible light images contain rich scene structure and detail texture information. Infrared images and visible light images have many complementary characteristics. Therefore, fusing the targets of infrared images and the scene details of visible light images into one image helps to obtain highlight target information and scene texture detail information, achieving good visual effects.

[0039] The infrared image is processed using a multi-level decomposition model, and the infrared image is decomposed into a primary base layer and multiple levels of detail layers. The same method is used to process the visible light image of the same scene, and the visible light image is also decomposed into a primary base layer and multiple levels of detail layers. The base layer of the infrared image and the base layer of the visible light image constitute a base layer image pair, and the multi-level detail layers of the infrared image and the multi-level detail layers of the visible light image constitute a detail layer feature pair.

[0040] Among them, the decomposition methods used in common multi-level decomposition models include wavelet transform, derivative filter, frequency domain filter, etc. This embodiment uses MDLatLRR (Multi-Dimensional Latent Low-Rank Representation) for multi-level decomposition.

[0041] Step 104: The base layer image pair is fused to obtain first fusion layer features, and the detail layer feature pair is fused based on fractional differentials to obtain second fusion layer features.

[0042] In this embodiment, the fusion of the base layer image adopts an average strategy. Given a pair of images (I1, I2) in the same scene, the fusion calculation is shown as follows:

[0043]

[0044] Where (m, n) represents the pixel position; I bf represents the first fusion layer features; represents the base layer image pair, ω b1 and ω b2 They are and Corresponding weights. In this embodiment, both weights are set to 0.5. In other embodiments, the weights can also be configured as needed.

[0045] The fractional differential operator can enhance the high-frequency signal while retaining the low-frequency signal. Due to the characteristics of the fractional differential operator, in the image fusion task, this characteristic can be used to generate a high-quality fused image with more detail information. Therefore, this embodiment adopts the fractional differential fusion strategy to fuse the detail layer features and obtain the second fusion layer features.

[0046] Step 106: Obtain a fused image of the source image pair based on the first fusion layer features and the second fusion layer features.

[0047] After obtaining the fused base layer (ie, the first fusion layer features) and the fused detail layer (ie, the second fusion layer features), adding them together can obtain the final fused image.

[0048] This embodiment applies multi-level decomposition to decompose the source image into a base layer and a detail layer, and then uses the designed adaptive fractional differential fusion rule to fuse the detail layer, and uses the average strategy to fuse the base layer. The fused detail layer and the fused base layer are added to obtain the final fused image. Compared with the existing image fusion method, the infrared and visible light image fusion method based on the multi-scale fractional gradient model proposed in this embodiment can better extract image details and reduce the noise effect, thereby more completely retaining the complementary information of the source infrared and visible light images, and the fused image has a higher fidelity, which improves the quality of the fused image.

[0049] In one embodiment, obtaining the second fusion layer features based on fractional differential fusion of detail layer feature pairs in step 104 includes: constructing a fractional differential mask; obtaining a fractional gradient energy function based on the fractional differential mask and the detail layer feature pairs; and fusing the detail layer feature pairs based on the fractional gradient energy function to obtain the second fusion layer features.

[0050] In image processing, the fractional differential mask is a matrix used to approximate the derivative of an image. The determination of the differential order is very important for fractional differentials, which determines the depth and nature of the differential operation, and the differential mask is a specific tool for implementing this operation. Therefore, when constructing a differential mask, it is necessary to consider the required differential order in order to obtain the desired image features.

[0051] After the fractional differential mask is determined, detail features are extracted according to the mask and detail layer features, thereby obtaining a fractional gradient energy function containing detail features.

[0052] Finally, based on the fractional gradient energy function, the feature pairs of detail layers at different levels are weightedly fused to obtain the features of the second fusion layer.

[0053] In one embodiment, constructing the fractional differential mask includes: constructing the fractional differential mask based on the modified spatial frequency and the average gradient.

[0054] Construct fractional differential masks G1 and G2 in the x- and y-directions. G1 is expressed as:

[0055]

[0056] G2 is expressed as:

[0057]

[0058] The calculation formula of parameter v in G1 and G2 is as follows:

[0059]

[0060] in,

[0061]

[0062] Among them, a and b are the orders of fractional differentiation; MSF (Modified Spatial Frequency) represents the modified spatial frequency, which reflects the rate of change of image grayscale; AG (Average Gradient) represents the average gradient, which describes the edge strength and degree of change of the image.

[0063] The calculation formula of MSF is:

[0064]

[0065] Among them, RF, CF and DF are the row frequency, column frequency and angular frequency of the image respectively.

[0066] The calculation formula of AG is:

[0067]

[0068] Where, ΔF x (m, n) and ΔF y (m, n) represent the image gradients in the x and y directions respectively.

[0069] Quantitative experiments show that when the order of fractional differential is 0.5 to 0.6, the texture feature can be effectively controlled. In this embodiment, a=0.6 and b=0.5 are set.

[0070] It can be seen that this embodiment proposes an adaptive fractional order calculation method, which can better extract image details and thus improve image fusion effect compared with MDLatLRR and fixed fractional order algorithms.

[0071] In one embodiment, the detail layer feature pair includes multiple levels of feature pairs, and each level of feature pair includes a first feature corresponding to the infrared image and a second feature corresponding to the visible light image. According to the fractional differential mask and the detail layer feature pair, obtaining the fractional gradient energy function includes: for each level of feature pair, according to the fractional differential mask and the detail layer feature pair, obtaining a first function corresponding to the first feature and a second function corresponding to the second feature; the first function and the second function are both fractional gradient energy functions.

[0072] This embodiment proposes a method for extracting detail features based on a fractional gradient energy function. The fractional gradient energy function is defined as follows:

[0073]

[0074] in, represents the fractional gradient energy function; G1 and G2 are the corresponding fractional differential masks; represents the first feature or the second feature, k is 1 or 2, i is an integer of [1, L], L is the decomposition level, that is, the level of feature pairs; (m, n) represents the pixel position.

[0075] The fractional differential mask and the detail layer feature pair are convolved to obtain the fractional gradient energy function.

[0076] In one embodiment, the detail layer feature pairs are fused based on the fractional gradient energy function to obtain the second fusion layer features, including: for each level of feature pairs, according to the first function and the second function, obtaining the weight coefficients corresponding to the first feature and the second feature respectively; according to the weight coefficients, the first feature and the second feature are fused to obtain the second fusion layer features corresponding to the feature pairs at each level.

[0077] This embodiment uses a fractional gradient energy function to weight the importance of the corresponding detail features, and then performs weighted summation on the detail features to obtain fused detail features. In particular, when the decomposed detail part contains more texture and structure information, the corresponding weight is larger.

[0078] The calculation formula of the weight coefficient is:

[0079]

[0080] in, represents the weight coefficient; represents the fractional gradient energy function; i is the decomposition level; K∈{1,2} is used to refer to the first feature and the second feature; (m,n) represents the pixel position.

[0081] The calculation formula of the second fusion layer is:

[0082]

[0083] It can be seen from the above formula that each level of feature pair corresponds to the second fusion layer.

[0084] In one embodiment, step 106 includes: reconstructing the second fusion layer features corresponding to each level of feature pairs, combining the first fusion layer features, and obtaining a fused image.

[0085] The calculation formula of the fused image is:

[0086]

[0087] Among them, I bf represents the first fusion layer feature; L represents the decomposition level; represents the reconstructed second fusion layer features, The calculation formula is: i=[1,2,…,L], R(·) represents the reconstruction operation.

[0088] In order to verify the superiority of the method proposed in the present invention, algorithm comparison experiments were carried out on two commonly used public datasets, the TNO dataset and the RoadScene dataset. The compared algorithms include: DCHWT, CVT, MDLatLRR, DenseFuse, FusionGAN, U2Fusion, SwinFusion, and LRNet. The decomposition levels for comparison are set to 2 and 4, and the fusion performance is compared using qualitative and quantitative evaluation methods. The specific quantitative evaluation indicators used include: entropy, standard deviation, mutual information, visual information fidelity, edge-based similarity, and corrected structural similarity. The larger the values ​​of these indicators, the better the quality of the fused image. The experimental results show that the method proposed in the present invention is superior to the comparison algorithm in both objective and subjective evaluation.

[0089] like Figure 2 As shown, the present invention proposes a method for fusion of infrared and visible light images based on a multi-scale fractional gradient model, and applies multi-level decomposition to decompose the source image into a base layer and a detail layer. Then, the designed adaptive fractional differential fusion rule is used to fuse the detail layer, and the average strategy is used to fuse the base layer. The fused detail layer and the fused base layer are added to obtain the final fused image. The method designs a new fractional calculation method based on modifying spatial frequency and average gradient. On this basis, an adaptive fractional differential model is designed to fuse the detail layer, and a fractional gradient energy function is proposed to weigh the importance of the corresponding detail features, so as to fully extract the detail features of the image and retain good texture details, and finally show superior performance in the image fusion result.

[0090] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0091] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the infrared and visible light image fusion method based on the multi-scale fractional gradient model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more infrared and visible light image fusion devices based on the multi-scale fractional gradient model provided below can refer to the limitations of the infrared and visible light image fusion method based on the multi-scale fractional gradient model above, and will not be repeated here.

[0092] In one embodiment, Figure 3 As shown, an infrared and visible light image fusion device based on a multi-scale fractional gradient model is provided, comprising: a multi-level decomposition module 302, a first fusion module 304 and a second fusion module 306, wherein:

[0093] A multi-level decomposition module 302 is used to decompose a source image pair using a multi-level decomposition model to obtain a base layer image pair and a detail layer feature pair; the source image pair includes an infrared image and a visible light image of the same scene;

[0094] The first fusion module 304 is used to fuse the base layer image pair to obtain the first fusion layer features, and to fuse the detail layer feature pair based on fractional differential to obtain the second fusion layer features;

[0095] The second fusion module 306 is used to obtain a fused image of the source image pair based on the first fusion layer features and the second fusion layer features.

[0096] In one embodiment, the first fusion module 304 is also used to construct a fractional differential mask; obtain a fractional gradient energy function based on the fractional differential mask and the detail layer feature pair; and fuse the detail layer feature pair based on the fractional gradient energy function to obtain a second fusion layer feature.

[0097] In one embodiment, the first fusion module 304 is further configured to construct a fractional differential mask based on the modified spatial frequency and the average gradient.

[0098] In one embodiment, the detail layer feature pair includes multiple levels of feature pairs, and each level of feature pair includes a first feature corresponding to the infrared image and a second feature corresponding to the visible light image. The first fusion module 304 is further used to obtain, for each level of feature pair, a first function corresponding to the first feature and a second function corresponding to the second feature according to the fractional differential mask and the detail layer feature pair; the first function and the second function are both fractional gradient energy functions.

[0099] In one embodiment, the first fusion module 304 is also used to obtain the weight coefficients corresponding to the first feature and the second feature respectively for each level of feature pair according to the first function and the second function; and fuse the first feature and the second feature according to the weight coefficient to obtain the second fusion layer features corresponding to the feature pairs at each level.

[0100] In one embodiment, the second fusion module 306 is further used to reconstruct the second fusion layer features corresponding to the feature pairs at each level, and combine them with the first fusion layer features to obtain a fused image.

[0101] Each module in the infrared and visible light image fusion device based on the multi-scale fractional gradient model can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0102] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in all the above method embodiments when executing the computer program.

[0103] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in all the above method embodiments are implemented.

[0104] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in all the above method embodiments when executed by a processor.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0106] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processors involved in each embodiment provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., but are not limited thereto.

[0107] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for fusion of infrared and visible light images based on a multi-scale fractional gradient model, characterized in that: The method comprises: Decomposing a source image pair using a multi-level decomposition model to obtain a base layer image pair and a detail layer feature pair; the source image pair includes an infrared image and a visible light image of the same scene; The base layer image pair is fused to obtain first fusion layer features, and the detail layer feature pair is fused based on fractional differentials to obtain second fusion layer features; Based on the first fusion layer features and the second fusion layer features, a fused image of the source image pair is acquired.

2. The method according to claim 1, characterized in that The acquiring the second fusion layer features based on fractional differential fusion of the detail layer features comprises: Construct fractional differential mask; Acquire a fractional gradient energy function according to the fractional differential mask and the detail layer feature pair; The detail layer feature pairs are fused based on the fractional gradient energy function to obtain second fusion layer features.

3. The method according to claim 2, characterized in that The constructing of the fractional differential mask comprises: Construct fractional differential masks based on corrected spatial frequency and average gradient.

4. The method according to claim 2, characterized in that: The detail layer feature pair includes multiple levels of feature pairs, and each level of feature pair includes a first feature corresponding to the infrared image and a second feature corresponding to the visible light image; The obtaining of the fractional gradient energy function according to the fractional differential mask and the detail layer feature pair comprises: For each level of feature pairs, a first function corresponding to the first feature and a second function corresponding to the second feature are obtained according to the fractional differential mask and the detail layer feature pair; the first function and the second function are both the fractional gradient energy functions.

5. The method according to claim 4, characterized in that The fusing the detail layer feature pairs based on the fractional gradient energy function to obtain the second fusion layer features comprises: For each level of feature pair, obtaining weight coefficients corresponding to the first feature and the second feature respectively according to the first function and the second function; The first feature and the second feature are fused according to the weight coefficient to obtain the second fusion layer features corresponding to each level of feature pairs.

6. The method according to claim 5, characterized in that: The acquiring the fused image of the source image pair based on the first fusion layer feature and the second fusion layer feature comprises: The second fusion layer features corresponding to each level of feature pairs are reconstructed and combined with the first fusion layer features to obtain the fused image.

7. An infrared and visible light image fusion device based on a multi-scale fractional gradient model, characterized in that: The device comprises: A multi-level decomposition module, used to decompose a source image pair using a multi-level decomposition model to obtain a base layer image pair and a detail layer feature pair; the source image pair includes an infrared image and a visible light image of the same scene; A first fusion module is used to fuse the base layer image pair to obtain first fusion layer features, and to fuse the detail layer feature pair based on fractional differentials to obtain second fusion layer features; The second fusion module is used to obtain a fused image of the source image pair based on the first fusion layer features and the second fusion layer features.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images