Image processing method and system based on feature fusion

Abstract

The present invention discloses an image processing method and system based on feature fusion, which is applied to the field of image intelligent processing technology. The method includes: forming a preprocessed image; extracting a GLCM feature map; extracting a first feature and a second feature; forming a first cross-projection feature and a second cross-projection feature; and performing feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fusion features. The present invention effectively alleviates the problems of small data sets and complex DRR imaging conditions faced in deep learning-based medical image processing, improves the local information perception ability of traditional convolutional neural networks when extracting features, smoothes the loss landscape of the model, improves the feature extraction ability and noise resistance of the model, better copes with the problems of unbalanced samples and small samples in medical image data sets, and has good generalization ability.

Description

Technical Field

[0001] The present invention relates to an image intelligent processing technology, and in particular to an image processing method and system based on feature fusion. Background Art

[0002] Image features refer to information in an image that can characterize the content of the image. This information can be local or global attributes of the image, such as color distribution, texture pattern, shape outline, etc. The purpose of image feature extraction is to express this information in mathematical form to form feature vectors or feature descriptors for computer processing and analysis. When extracting features from medical images, the reliable data set is small, and there are requirements for the model structure and optimization algorithm selection. At the same time, the collected medical images are often accompanied by noise, making it difficult to capture textures and easily causing model overfitting. Summary of the invention

[0003] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide an image processing method and system based on feature fusion.

[0004] In a first aspect, the present application provides an image processing method based on feature fusion, comprising:

[0005] Preprocessing the target image to form a preprocessed image;

[0006] Extracting a plurality of GLCM feature maps from the preprocessed image;

[0007] Extracting texture features of the GLCM feature map to form a first feature, and extracting CNN features of the preprocessed image as a second feature;

[0008] Cross-projecting the first feature to form a first cross-projected feature, and cross-projecting the second feature to form a second cross-projected feature;

[0009] The first cross-projection feature and the second cross-projection feature are subjected to feature fusion to form an image having fused features.

[0010] In a possible implementation manner, extracting a plurality of GLCM feature maps from the preprocessed image includes:

[0011] The feature parameters corresponding to each GLCM feature map are constructed using multiple direction parameters and multiple distance parameters;

[0012] The GLCM feature map C is generated according to the following formula:

[0013]

[0014] In the formula, and is the pixel value, and is the spatial position in the image, and is the defined spatial offset, For The pixel value at position, m is the maximum value in the y direction, and n is the maximum value in the x direction.

[0015] In a possible implementation, generating the first feature and the second feature includes:

[0016] The image to be processed is processed in multiple rounds in sequence, and each round of processing includes residual calculation and multi-head multi-scale self-attention calculation in sequence, and the feature data generated by each round of multi-head multi-scale self-attention calculation is used as the output feature;

[0017] When the image to be processed is the GLCM feature map, the output feature is the first feature; when the image to be processed is the pre-processed image, the output feature is the second feature.

[0018] In one possible implementation, the multi-head multi-scale self-attention calculation includes multiple multi-scale self-attention calculations;

[0019] The multi-scale self-attention calculation includes:

[0020] Generate three types of convolution kernels of 3×3, 5×5 and 7×7 as multi-scale kernels;

[0021] The query value of the attention mechanism of the input data and the key value of the attention mechanism are processed separately by the multi-scale kernel, fused and normalized, and the value of the attention mechanism of the input data is superimposed to form the output feature.

[0022] In one possible implementation, the multi-scale self-attention calculation is performed according to the following formula:

[0023]

[0024] Where Q is the query value of the attention mechanism, K is the key value of the attention mechanism, is the value of the attention mechanism, d is the dimension of K, is the jth output, is the normalization function.

[0025] In a possible implementation manner, generating the first cross-projection feature and the second cross-projection feature includes:

[0026] The output feature of the current round of processing is obtained as the first depth feature, and the output feature of the previous round of processing is obtained as the second depth feature;

[0027] Downsampling the second depth feature to form a feature with a size of 0.75 times as large as the second processing feature, and performing 1×1 convolution processing on the first depth feature and then upsampling it to form a feature with a size of 1.5 times as large as the first processing feature;

[0028] After fusing the first processing feature and the second processing feature, learning the spatial weight through 3×3 convolution;

[0029] Upsampling the spatial weights to form spatial guided weights, and integrating the spatial guided weights with the second depth features to form guided depth features;

[0030] splicing the guide depth feature and the first depth feature to form a cross-projection feature;

[0031] When the output feature is the first feature, the cross-projection feature is the first cross-projection feature; when the output feature is the second feature, the cross-projection feature is the second cross-projection feature.

[0032] In a possible implementation, the cross-projection feature is generated by the following formula:

[0033]

[0034] In the formula, Representation layer normalization, Represents a splicing operation, Indicates that the convolution kernel size is The convolution of is the first depth feature, is the second depth feature, is the cross-projection feature.

[0035] In a possible implementation manner, performing feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features includes:

[0036] Feature fusion is performed according to the following formula:

[0037]

[0038]

[0039] In the formula, X is the second cross-projection feature, Y is the first cross-projection feature, and Z is the fusion feature. is element-wise addition, is element-by-element multiplication, SE is the weight function, σ is the activation function, β is the batch normalization function, δ is the linear rectification function, is the global average pooling function, and is the weight of the fully connected layer.

[0040] In a possible implementation, the method further includes:

[0041] The image with fusion features is classified by a classifier to form an image recognition result;

[0042] The classifier adopts the following formula:

[0043]

[0044] In the formula, To fusion features, is the vector of label weights, is the weight of the fully connected layer, is the global average pooling function, σ is the activation function, and β is the batch normalization function.

[0045] In a second aspect, the present application also provides an image processing system based on feature fusion, comprising:

[0046] A preprocessing module is configured to preprocess the target image to form a preprocessed image;

[0047] An extraction module is configured to extract a plurality of GLCM feature maps from the preprocessed image;

[0048] A multi-head multi-scale self-attention residual network module is configured to extract texture features of the GLCM feature map to form a first feature, and extract CNN features of the preprocessed image as a second feature;

[0049] a cross-projection module, configured to perform cross-projection on the first feature to form a first cross-projection feature, and to perform cross-projection on the second feature to form a second cross-projection feature;

[0050] The feature fusion module is configured to perform feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features.

[0051] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0052] The present invention effectively alleviates the problems of small data set and complex DRR imaging in medical image processing based on deep learning by introducing a priori empirical texture features GLCM; by introducing multi-head multi-scale self-attention MMSA to construct MMSA-ResNet module, it helps to improve the local information perception ability of traditional convolutional neural networks when extracting features, smooth the loss landscape of the model, and improve the feature extraction ability and noise resistance of the model; based on the attention mechanism, the fusion weights between different features and different channels are calculated, which can better deal with the problems of unbalanced samples and small samples in medical image data sets, and has better generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0054] Figure 1 This is a schematic diagram of the steps of the method of the embodiment of the present application;

[0055] Figure 2 This is a flow chart of the feature fusion strategy according to an embodiment of the present application;

[0056] Figure 3 This is a schematic diagram of the MMSA-ResNet network structure of an embodiment of the present application;

[0057] Figure 4 This is a schematic diagram of multi-scale self-attention in an embodiment of the present application;

[0058] Figure 5 This is a cross-projection schematic diagram of an embodiment of the present application;

[0059] Figure 6 This is a schematic diagram of the fusion of the embodiments of the present application. DETAILED DESCRIPTION

[0060] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the embodiment of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0061] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0062] Please refer to Figure 1 , which is a flow chart of an image processing method based on feature fusion provided in an embodiment of the present invention. Furthermore, the image processing method based on feature fusion may specifically include the contents described in the following steps S1 to S5.

[0063] S1: preprocessing the target image to form a preprocessed image;

[0064] S2: extracting a plurality of GLCM feature maps from the preprocessed image;

[0065] S3: extracting texture features of the GLCM feature map to form a first feature, and extracting CNN features of the preprocessed image as a second feature;

[0066] S4: performing cross-projection on the first feature to form a first cross-projection feature, and performing cross-projection on the second feature to form a second cross-projection feature;

[0067] S5: Perform feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features.

[0068] When implementing the embodiment of this application, a method for image processing and analysis using a feature fusion strategy is proposed. Figure 2, shows a specific flow chart, in which the target image needs to be preprocessed first, and the preprocessing means include preprocessing means such as image size and grayscale normalization, which belongs to the prior art and is not limited in the embodiments of the present application. In the extraction of the GLCM feature map, GLCM is a grayscale co-occurrence matrix, which can be achieved by extracting the GLCM feature. For the GLCM feature map and the preprocessed image, further feature extraction can be performed to obtain the first feature and the second feature, respectively, wherein the extraction of the first feature and the second feature can be implemented based on a multi-head multi-scale self-attention residual network (MMSA-ResNet), CNN is a convolutional neural network, and MMSA is a multi-head multi-scale self-attention. For the processing of the first feature and the second feature, in the multi-feature fusion work, the features to be input into the feature fusion module are often integrated in the form of element-by-element addition. This is called initial integration. The quality of the initial integration has a key impact on the quality of feature fusion. However, for features of different sizes extracted by convolutional neural networks, shallow feature maps have larger sizes and fewer channels while focusing on local details, while high-level feature maps have smaller sizes and more channels while focusing on abstract information. As the network deepens, the semantic information differences represented by different features also expand. At this time, directly using simple element-by-element addition to mix feature maps of different levels may cause the ordered information of these original level features to be destroyed, thereby causing information distortion and blurring. In practical applications, the features to be fused are often composed of multi-scale features of different sizes and channels, and the method of directly using element-by-element addition for integration is not suitable. In order to improve the quality of initial integration, the embodiment of the present application uses cross-projection to perform initial integration of features of different scales. The first cross-projection feature and the second cross-projection feature after initial integration can form an image with fusion features after feature fusion. The present invention introduces a multi-head and multi-scale self-attention module (MMSA) in ResNet to improve the network's attention to low-frequency information, improve the model's ability to perceive local textures, and realize convolutional network (CNN) feature extraction. At the same time, a feature fusion module (Squeeze Excitation ResNet, SERNet) is proposed. This module realizes initial feature fusion by paying attention to the basic channel and adds the spatial attention module in an iterative form. At the same time, the residual attention structure is introduced to solve the problem of reduced fusion quality caused by the weighted summation of convolutional features and fusion features.

[0069] In a possible implementation manner, extracting a plurality of GLCM feature maps from the preprocessed image includes:

[0070] The feature parameters corresponding to each GLCM feature map are constructed using multiple direction parameters and multiple distance parameters;

[0071] The GLCM feature map C is generated according to the following formula:

[0072]

[0073] In the formula, and is the pixel value, and is the spatial position in the image, and is the defined spatial offset, For The pixel value at position, m is the maximum value in the y direction, and n is the maximum value in the x direction.

[0074] When the embodiment of the present application is implemented, the target image used is a DRR image, which exists in the form of a DICOM file, with a grayscale depth of 16 bits and a grayscale image size of 3000*3000 pixels. In the process of generating the GLCM feature map, for example, the GLCM feature map of the DR image is constructed, and 20 GLCM matrices are generated, which are divided into four directions of 0°, 45°, 90°, and 135°, and five distances of 1, 2, 3, 4, and 5; the combination of four directions and five distances can form 20 situations, and the size of each GLCM feature map is 256×256.

[0075] In a possible implementation, generating the first feature and the second feature includes:

[0076] The image to be processed is processed in multiple rounds in sequence, and each round of processing includes residual calculation and multi-head multi-scale self-attention calculation in sequence, and the feature data generated by each round of multi-head multi-scale self-attention calculation is used as the output feature;

[0077] When the image to be processed is the GLCM feature map, the output feature is the first feature; when the image to be processed is the pre-processed image, the output feature is the second feature.

[0078] When implementing the embodiments of this application, please refer to Figure 3, shows a schematic diagram of the module for multiple rounds of processing, in which 4 rounds introduce a multi-head multi-scale self-attention mechanism. The MMSA module can improve the model's attention to global information, smooth the model's loss landscape, and achieve better training results; Stage represents a round of residual calculation in ResNet, Stage1, Stage2, Stage3, and Stage4 represent the first, second, third, and fourth rounds of residual calculation, respectively, and the calculation results will be processed by the MMSA Block, i.e., the multi-head multi-scale self-attention module; The data calculated by the MMSA Block is the output feature, which will be collected by the Projection Block, i.e., the projection module, and used for subsequent cross-projection processing. In the multiple rounds of processing of the image to be processed, each round of processing needs to go through residual calculation and multi-head multi-scale self-attention calculation. The residual calculation is implemented by ResNet, and the multi-head multi-scale self-attention calculation is implemented by MMSA. Introducing the MMSA module into the convolutional neural network can avoid the problem that the model cannot fully obtain detailed features on a small data set, while improving the convolutional network's ability to perceive local information, smoothing the model's loss perception, and achieving better training results.

[0079] In one possible implementation, the multi-head multi-scale self-attention calculation includes multiple multi-scale self-attention calculations;

[0080] The multi-scale self-attention calculation includes:

[0081] Generate three types of convolution kernels of 3×3, 5×5 and 7×7 as multi-scale kernels;

[0082] The query value of the attention mechanism of the input data and the key value of the attention mechanism are processed separately by the multi-scale kernel, fused and normalized, and the value of the attention mechanism of the input data is superimposed to form the output feature.

[0083] When the embodiment of the present application is implemented, a more specific multi-scale self-attention calculation scheme is provided. The multi-head multi-scale self-attention is composed of multiple multi-scale self-attention processes, wherein a single multi-scale self-attention structure is as follows: Figure 4 As shown in the figure, the multi-scale kernels in the module are 3x3, 5x5 and 7x7 convolution kernels respectively.

[0084] In one possible implementation, the multi-scale self-attention calculation is performed according to the following formula:

[0085]

[0086] Where Q is the query value of the attention mechanism, K is the key value of the attention mechanism, is the value of the attention mechanism, d is the dimension of K, is the jth output, is the normalization function.

[0087] When the embodiment of the present application is implemented, each multi-head multi-scale self-attention calculation includes 8 groups of multi-scale self-attention calculations, and each multi-scale self-attention calculation includes three parameter values, namely query, key and value, which are represented as Q, K and V respectively, for a total of 24 parameters.

[0088] In a possible implementation manner, generating the first cross-projection feature and the second cross-projection feature includes:

[0089] The output feature of the current round of processing is obtained as the first depth feature, and the output feature of the previous round of processing is obtained as the second depth feature;

[0090] Downsampling the second depth feature to form a feature with a size of 0.75 times as large as the second processing feature, and performing 1×1 convolution processing on the first depth feature and then upsampling it to form a feature with a size of 1.5 times as large as the first processing feature;

[0091] After fusing the first processing feature and the second processing feature, learning the spatial weight through 3×3 convolution;

[0092] Upsampling the spatial weights to form spatial guided weights, and integrating the spatial guided weights with the second depth features to form guided depth features;

[0093] splicing the guide depth feature and the first depth feature to form a cross-projection feature;

[0094] When the output feature is the first feature, the cross-projection feature is the first cross-projection feature; when the output feature is the second feature, the cross-projection feature is the second cross-projection feature.

[0095] In a possible implementation, the cross-projection feature is generated by the following formula:

[0096]

[0097] In the formula, Representation layer normalization, Represents a splicing operation, Indicates that the convolution kernel size is The convolution of is the first depth feature, is the second depth feature, is the cross-projection feature.

[0098] When implementing the embodiments of this application, please refer to Figure 5, shows a cross-projection model structure, in the cross-projection module, is the deep feature of the fused i-th round MMSA output, that is, the first deep feature, is the depth feature output by the previous round of MSSA, i.e., the second depth feature. is the integrated feature after initial integration. In the cross-stage projection layer, the size of the fused feature is downsampled to 0.75 times the original size, and the size of the deep feature is upsampled to 1.5 times the original size through a linear sampling method. This operation is to convert the two features to an intermediate scale, thereby reducing the offset of feature information at different levels, so as to facilitate learning the contextual information between different features at an intermediate scale. A convolutional layer with a convolution kernel size of 3×3 is used to learn the spatial weights, and the learned spatial weight information is upsampled as the spatial guide weight and the fused feature. Finally, the deep features It is then spliced ​​with the fusion features enhanced by the spatial attention mechanism to complete the initial integration for further learning by the fusion layer, which is beneficial for the extraction of image context information.

[0099] In a possible implementation manner, performing feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features includes:

[0100] Feature fusion is performed according to the following formula:

[0101]

[0102]

[0103] In the formula, X is the second cross-projection feature, Y is the first cross-projection feature, and Z is the fusion feature. is element-wise addition, is element-by-element multiplication, SE is the weight function, σ is the activation function, β is the batch normalization function, δ is the linear rectification function, is the global average pooling function, and is the weight of the fully connected layer.

[0104] When implementing the embodiments of this application, please refer to Figure 6The structure of feature fusion is shown, where GAP is the global average pooling function, BN is the batch normalization function, FC is the fully connected layer, Sigmoid is the activation function, Scale is the weight function, C is the number of channels of the feature map, H is the height of the feature map, and W is the width of the feature map. It is an embedded channel-related attention module (SERNet) to calculate the weights of different channels of the initial weighted features. For the output initial fusion features, they are multiplied element by element with the original GLCM features, and the original GLCM features are emphasized using the information obtained from the initial fusion features. Compared with the general channel attention method, this method introduces weights between different feature maps, provides more attention information, and realizes the fusion of different features on the spatial channel. In order to solve the quality degradation problem that may be caused by multiple fusions in this module, the residual structure is introduced, that is, the CNN features of the image are weighted with the fusion features at the end to reduce the impact. This model provides the ability to learn the weights of different features and channels in the case of multiple feature fusions, and can better adapt to the channel differences and semantic inconsistencies that exist when multiple features are fused.

[0105] In a possible implementation, the method further includes:

[0106] The image with fusion features is classified by a classifier to form an image recognition result;

[0107] The classifier adopts the following formula:

[0108]

[0109] In the formula, To fusion features, is the vector of label weights, is the weight of the fully connected layer, is the global average pooling function, σ is the activation function, and β is the batch normalization function.

[0110] When the embodiment of the present application is implemented, the purpose of the classifier is to perform relevant recognition on the image, such as identifying whether there is a texture that needs to be marked in the image.

[0111] Based on the same inventive concept, the present application also provides an image processing system based on feature fusion, comprising:

[0112] A preprocessing module is configured to preprocess the target image to form a preprocessed image;

[0113] An extraction module is configured to extract a plurality of GLCM feature maps from the preprocessed image;

[0114] A multi-head multi-scale self-attention residual network module is configured to extract texture features of the GLCM feature map to form a first feature, and extract CNN features of the preprocessed image as a second feature;

[0115] a cross-projection module, configured to perform cross-projection on the first feature to form a first cross-projection feature, and to perform cross-projection on the second feature to form a second cross-projection feature;

[0116] The feature fusion module is configured to perform feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0119] The units described as separate components may or may not be physically separated. As units, it is obvious that a person of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0120] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0122] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method based on feature fusion, characterized in that: include: Preprocessing the target image to form a preprocessed image; Extracting a plurality of GLCM feature maps from the preprocessed image; Extracting texture features of the GLCM feature map to form a first feature, and extracting CNN features of the preprocessed image as a second feature; Cross-projecting the first feature to form a first cross-projected feature, and cross-projecting the second feature to form a second cross-projected feature; Performing feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features; The generation of the first feature and the second feature includes: The image to be processed is processed in multiple rounds in sequence, and each round of processing includes residual calculation and multi-head multi-scale self-attention calculation in sequence, and the feature data generated by each round of multi-head multi-scale self-attention calculation is used as the output feature; When the image to be processed is the GLCM feature map, the output feature is the first feature; when the image to be processed is the pre-processed image, the output feature is the second feature; The generation of the first cross-projection feature and the second cross-projection feature includes: The output feature of the current round of processing is obtained as the first depth feature, and the output feature of the previous round of processing is obtained as the second depth feature; Downsampling the second depth feature to form a feature with a size of 0.75 times as large as the second processing feature, and performing 1×1 convolution processing on the first depth feature and then upsampling it to form a feature with a size of 1.5 times as large as the first processing feature; After fusing the first processing feature and the second processing feature, learning the spatial weight through 3×3 convolution; Upsampling the spatial weights to form spatial guided weights, and integrating the spatial guided weights with the second depth features to form guided depth features; splicing the guide depth feature and the first depth feature to form a cross-projection feature; When the output feature is the first feature, the cross-projection feature is the first cross-projection feature; when the output feature is the second feature, the cross-projection feature is the second cross-projection feature; Performing feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features includes: Feature fusion is performed according to the following formula: ； ； In the formula, X is the second cross-projection feature, Y is the first cross-projection feature, and Z is the fusion feature. is element-wise addition, is element-by-element multiplication, SE is the weight function, σ is the activation function, β is the batch normalization function, δ is the linear rectification function, is the global average pooling function, and is the weight of the fully connected layer.

2. The image processing method based on feature fusion according to claim 1, characterized in that: Extracting a plurality of GLCM feature maps from the preprocessed image comprises: The feature parameters corresponding to each GLCM feature map are constructed using multiple direction parameters and multiple distance parameters; The GLCM feature map C is generated according to the following formula: ； In the formula, and is the pixel value, and is the spatial position in the image, and is the defined spatial offset, For The pixel value at position, m is the maximum value in the y direction, and n is the maximum value in the x direction.

3. The image processing method based on feature fusion according to claim 1, characterized in that: The multi-head multi-scale self-attention calculation includes multiple multi-scale self-attention calculations; The multi-scale self-attention calculation includes: Generate three types of convolution kernels of 3×3, 5×5 and 7×7 as multi-scale kernels; The query value of the attention mechanism of the input data and the key value of the attention mechanism are processed separately by the multi-scale kernel, fused and normalized, and the value of the attention mechanism of the input data is superimposed to form the output feature.

4. The image processing method based on feature fusion according to claim 2, characterized in that: The multi-scale self-attention calculation is performed according to the following formula: ； Where Q is the query value of the attention mechanism, K is the key value of the attention mechanism, is the value of the attention mechanism, d is the dimension of K, is the jth output, is the normalization function.

5. The image processing method based on feature fusion according to claim 1, characterized in that: The cross-projection feature is generated by the following formula: ； In the formula, Representation layer normalization, Represents a splicing operation, Indicates that the convolution kernel size is The convolution of is the first depth feature, is the second depth feature, is the cross-projection feature.

6. The image processing method based on feature fusion according to claim 1, characterized in that: Also includes: The image with fusion features is classified by a classifier to form an image recognition result; The classifier adopts the following formula: ； In the formula, To fusion features, is the vector of label weights, is the weight of the fully connected layer, is the global average pooling function, σ is the activation function, and β is the batch normalization function.

7. An image processing system based on feature fusion using the method according to any one of claims 1 to 6, characterized in that: include: A preprocessing module is configured to preprocess the target image to form a preprocessed image; An extraction module is configured to extract a plurality of GLCM feature maps from the preprocessed image; A multi-head multi-scale self-attention residual network module is configured to extract texture features of the GLCM feature map to form a first feature, and extract CNN features of the preprocessed image as a second feature; a cross-projection module, configured to perform cross-projection on the first feature to form a first cross-projection feature, and to perform cross-projection on the second feature to form a second cross-projection feature; The feature fusion module is configured to perform feature fusion on the first cross-projection feature and the second cross-projection feature to form an image with fused features.

Images