Image processing method, device, equipment and storage medium based on multi-level feature extraction

Abstract

The present application discloses an image processing method, device, equipment and storage medium based on multi-level feature extraction, relates to the field of image processing technology, and discloses an image processing method based on multi-level feature extraction, including: performing feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information; performing edge feature extraction according to the convolutional layer features of the basic feature information, determining the target edge feature map, and determining the channel statistical information in combination with the target channel attention mechanism; determining the target output features according to the channel statistical information; and performing image processing tasks according to the target output features. Through the above-mentioned method, different levels of information from details to abstraction are effectively captured, and the key visual clues of the object contour and spatial position are captured at the same time. By fusing multimodal features, the representation ability of features in complex scenes is improved, laying the foundation for subsequent image task processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, equipment and storage medium based on multi-level feature extraction. Background Art

[0002] In recent years, the processing technology of satellite remote sensing images has developed rapidly. However, for ground scenes, factors such as complex background, diverse targets, and clutter interference seriously affect the accuracy of target detection and recognition. In addition, in satellite images, the target size is usually small, accounts for a small proportion of the entire image, and is easily densely distributed in a small space, which brings great challenges to feature extraction. At present, in feature detection, traditional methods have designed feature detection methods and feature descriptors through a lot of experience. These features have insufficient robustness under conditions such as drastic changes in illumination and large rotation amplitudes. Deep learning methods have shown excellent performance in feature detection, but there is still no method that can perform very well on different tasks. Therefore, it is necessary to design corresponding solutions according to specific tasks. For most detection tasks, there are difficulties and challenges mainly in the following aspects: small target scale, large changes, complex background, unbalanced samples, and difficulty in balancing algorithm speed and accuracy. In the field of communications, especially in satellite image analysis, there are problems such as large target scale changes, complex postures, high density, high similarity, complex background, and low contrast. In summary, there is an urgent need for a method for multi-level feature extraction of satellite images.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide an image processing method, device, equipment and storage medium based on multi-level feature extraction, aiming to solve the technical problem that the existing technology cannot efficiently and accurately extract multi-level features from satellite images.

[0005] To achieve the above objectives, the present application proposes an image processing method based on multi-level feature extraction, the method comprising:

[0006] Performing feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information of the image to be processed;

[0007] Extract edge features based on the convolutional layer features of the basic feature information, and determine a target edge feature map based on the extraction results;

[0008] Determine channel statistics based on the target edge feature map and the target channel attention mechanism;

[0009] Determining target output features of the image to be processed according to the channel statistical information;

[0010] An image processing task corresponding to the image to be processed is performed according to the target output feature.

[0011] In one embodiment, the step of performing feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information of the image to be processed includes:

[0012] The image to be processed is divided into blocks according to the target self-attention neural network to obtain multiple segmented image blocks;

[0013] Convert each segmented image block according to the linear embedding layer of the target self-attention neural network to obtain an embedded feature vector of each segmented image block;

[0014] Feature processing is performed according to the embedded feature vectors of each segmented image block to obtain basic feature information of the image to be processed.

[0015] In one embodiment, the feature processing is performed according to the embedded feature vectors of each segmented image block to obtain basic feature information of the image to be processed, including:

[0016] Performing feature processing on the embedded feature vectors of each segmented image block according to the continuous Swin transformer blocks in the target self-attention neural network, to obtain a segmentation feature map corresponding to each segmented image block;

[0017] Performing image fusion on each segmentation feature map according to the Patch merging module in the target self-attention neural network to obtain a target feature map;

[0018] A dimensionality reduction operation is performed on the target feature map, and basic feature information of the image to be processed is obtained according to the dimensionality reduction result.

[0019] In one embodiment, performing edge feature extraction based on the convolutional layer features of the basic feature information and determining a target edge feature map based on the extraction result includes:

[0020] Downsampling is performed according to the basic feature information, and convolution layer features are determined according to the downsampling result;

[0021] Determine the edge response map of each channel of the convolution layer feature according to the target depth Sobel convolution kernel;

[0022] The target edge feature map is determined according to the edge response map of each channel and the convolution layer features.

[0023] In one embodiment, determining the target edge feature map according to the edge response map of each channel and the convolutional layer features includes:

[0024] Calculate the mean value of the edge response map of each channel to determine the initial edge feature map corresponding to the convolution layer feature;

[0025] Performing feature calculation according to the initial edge feature map and the convolutional layer features to determine edge guide features;

[0026] A target edge feature map is generated according to the edge guide feature.

[0027] In one embodiment, determining channel statistical information according to the target edge feature map and the target channel attention mechanism includes:

[0028] Performing an average pooling operation on the target edge feature map according to the target channel attention mechanism to obtain an average pooling feature;

[0029] Performing a maximum pooling operation on the target edge feature map according to the target channel attention mechanism to obtain a maximum pooling feature;

[0030] Channel statistical information is determined according to the maximum pooling feature and the average pooling feature.

[0031] In one embodiment, determining the target output feature of the image to be processed according to the channel statistical information includes:

[0032] Get the target activation function and target channel weight;

[0033] Generate an image according to the target activation function, the target channel weight, the target edge feature map and the channel statistical information to obtain a target channel attention image;

[0034] Feature extraction is performed based on the target channel attention image to determine the target output features of the image to be processed.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes an image processing device based on multi-level feature extraction, and the image processing device based on multi-level feature extraction includes: a processing module, which is used to perform feature processing on the image to be processed according to the target self-attention neural network to obtain the basic feature information of the image to be processed; an extraction module, which is used to extract edge features according to the convolutional layer features of the basic feature information, and determine the target edge feature map according to the extraction results; the processing module is also used to determine channel statistical information according to the target edge feature map and the target channel attention mechanism; the processing module is also used to determine the target output features of the image to be processed according to the channel statistical information; an execution module is used to execute the image processing task corresponding to the image to be processed according to the target output features.

[0036] In addition, to achieve the above-mentioned objectives, the present application also proposes an image processing device based on multi-level feature extraction, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the image processing method based on multi-level feature extraction as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the image processing method based on multi-level feature extraction as described above are implemented.

[0038] The present application provides an image processing method based on multi-level feature extraction, which performs feature processing on the image to be processed according to the target self-attention neural network to obtain the basic feature information of the image to be processed; performs edge feature extraction according to the convolutional layer features of the basic feature information, and determines the target edge feature map according to the extraction result; determines channel statistics according to the target edge feature map and the target channel attention mechanism; determines the target output feature of the image to be processed according to the channel statistics; and performs the image processing task corresponding to the image to be processed according to the target output feature. Through the above method, the self-attention neural network is used to perform feature processing on the image to be processed, and the edge feature extraction is performed on the convolutional layer features of the obtained basic feature information to obtain the corresponding target edge feature map, and the target edge feature map and the target channel attention mechanism are used for processing to obtain the target output feature of the image to be processed, which effectively captures different levels of information from details to abstraction, and at the same time uses the edge attention mechanism to extract the edge information of the image, so as to realize the capture of the key visual clues of the object contour and spatial position, and improves the representation ability of features in complex scenes by fusing multimodal features, and can use limited attention resources to quickly filter out high-value information from a large amount of information, laying the foundation for subsequent image task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1A flowchart of the first embodiment of the image processing method based on multi-level feature extraction provided in the present application;

[0042] Figure 2 A schematic diagram of the channel attention mechanism architecture flow of the image processing method based on multi-level feature extraction provided in Example 2 of the present application;

[0043] Figure 3 A flowchart diagram of the second embodiment of the image processing method based on multi-level feature extraction of the present application;

[0044] Figure 4 A schematic diagram of the framework of the edge-guided attention mechanism of the image processing method based on multi-level feature extraction provided in the second embodiment of the present application;

[0045] Figure 5 A schematic diagram of the overall framework of the image processing method based on multi-level feature extraction provided in Example 2 of the present application;

[0046] Figure 6 This is a schematic diagram of the module structure of an image processing device based on multi-level feature extraction according to an embodiment of the present application;

[0047] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the image processing method based on multi-level feature extraction in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] The main solution of the embodiment of the present application is: perform feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information of the image to be processed; perform edge feature extraction according to the convolutional layer features of the basic feature information, and determine the target edge feature map according to the extraction results; determine channel statistical information according to the target edge feature map and the target channel attention mechanism; determine the target output features of the image to be processed according to the channel statistical information; and execute the image processing task corresponding to the image to be processed according to the target output features.

[0052] In satellite images, the target size is usually small, accounts for a small proportion of the entire image, and is easily densely distributed in a small space, which brings great challenges to feature extraction. In feature detection, traditional methods have designed feature detection methods and descriptors through a lot of experience. These features have insufficient robustness in cases of drastic changes in illumination and large rotation amplitudes. Deep learning methods have shown excellent performance in feature detection, but there is still no method that can perform very well on different tasks. Therefore, it is necessary to design corresponding solutions according to specific tasks. For most detection tasks, there are difficulties and challenges mainly in the small target scale, large changes, complex background, unbalanced samples, and difficulty in balancing algorithm speed and accuracy. In the field of communications, especially in satellite image analysis, there are problems such as large changes in target scale, complex posture, high density, high similarity, complex background, and low contrast. In signal detection, appropriate features can be extracted from signal images to detect and identify signals and communication signal targets, thereby further improving the reliability and stability of signal detection.

[0053] This application aims to solve the image feature extraction technology for the field of wireless communications, and can provide a technical basis for the fields of space target detection, remote sensing image analysis, synaesthesia fusion, etc. It proposes a multi-level fusion feature extraction method, which places the end point on the complementarity between edge information and target information, which can enhance the feature representation capability.

[0054] This application uses a self-attention neural network to perform feature processing on the image to be processed, and extracts edge features from the convolutional layer features of the obtained basic feature information to obtain a corresponding target edge feature map, and uses the target edge feature map and the target channel attention mechanism for processing to obtain the target output features of the image to be processed, effectively capturing different levels of information from details to abstractions, and at the same time using the edge attention mechanism to extract image edge information, thereby realizing the capture of key visual clues of object contours and spatial positions, and improving the representation capability of features in complex scenes by fusing multimodal features, and being able to use limited attention resources to quickly filter out high-value information from a large amount of information, laying the foundation for subsequent image task processing.

[0055] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an image processing device based on multi-level feature extraction, etc. The following takes an image processing device based on multi-level feature extraction as an example to illustrate this embodiment and the following embodiments.

[0056] Based on this, the embodiment of the present application provides an image processing method based on multi-level feature extraction, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the image processing method based on multi-level feature extraction of the present application.

[0057] In this embodiment, the image processing method based on multi-level feature extraction includes steps S10 to S50:

[0058] Step S10, performing feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information of the image to be processed;

[0059] It should be noted that the target self-attention neural network refers to Swin Transformer, which is a convolutional neural network based on the Transformer architecture and performs well in computer vision tasks. The image to be processed refers to a picture that needs to be subjected to multi-level feature extraction, which is used to perform tasks corresponding to subsequent images to be processed, including but not limited to image recognition, target detection, semantic segmentation and other computer vision tasks. In this embodiment, Swin Transformer adopts a hierarchical structure, and in each layer, the input features are processed by a series of Swin Transformer blocks. Each Swin Transformer block contains local attention calculation, Shifted Window mechanism, multi-head self-attention mechanism and multi-layer perceptron (MLP) module. The local attention calculation is performed within a fixed-size window, which effectively reduces the computational complexity. The Shifted Window mechanism captures a wider range of contextual information by translating the window between adjacent Transformer blocks, thereby improving the feature expression ability of the model. The multi-head self-attention mechanism is performed within the window, and multiple attention heads capture different feature dimensions, enriching the feature representation. Finally, the features after self-attention calculation are further nonlinearly transformed and feature extracted by the MLP module. At the end of each layer, the Swin Transformer performs a downsampling operation to generate feature representations of different resolutions. This is usually done through a window merging or pooling operation, which reduces the spatial resolution of the feature map by half while increasing the number of channels.

[0060] It can be understood that the image to be processed is input into the target self-attention neural network to calculate the basic features, thereby obtaining the basic feature information of the image to be processed. The self-attention neural network is used to perform image segmentation and basic feature extraction, and then feature merging and dimensionality reduction are performed, and the obtained features are the basic feature information.

[0061] In a feasible implementation, step S10 may include steps A11 to A13:

[0062] Step A11, dividing the image to be processed into blocks according to the target self-attention neural network to obtain multiple segmented image blocks;

[0063] It should be noted that when the image to be processed is input into the target self-attention neural network, the image to be processed is divided into multiple non-overlapping patches of fixed sizes. The multiple non-overlapping patches of fixed sizes corresponding to the image to be processed are multiple segmented image blocks of the image to be processed.

[0064] Step A12, transforming each segmented image block according to the linear embedding layer of the target self-attention neural network to obtain an embedded feature vector of each segmented image block;

[0065] It should be noted that each segmented image block is processed through a linear embedding layer in the target self-attention neural network, and each segmented image block is converted into a low-dimensional feature representation. The converted low-dimensional feature representation is the embedded feature vector of each segmented image block.

[0066] Step A13, performing feature processing according to the embedded feature vectors of each segmented image block to obtain basic feature information of the image to be processed.

[0067] It should be noted that feature processing is performed on the embedded feature vectors of each segmented image block, and feature merging and dimension reduction are performed to obtain basic feature information of the image to be processed.

[0068] In this embodiment, the image to be processed is divided into blocks according to the target self-attention neural network to obtain multiple segmented image blocks; each segmented image block is converted according to the linear embedding layer of the target self-attention neural network to obtain the embedded feature vector of each segmented image block; feature processing is performed according to the embedded feature vector of each segmented image block to obtain the basic feature information of the image to be processed. In the above manner, the accuracy of the first-level feature generation is guaranteed, laying the foundation for subsequent feature extraction.

[0069] The above is only one feasible implementation of step S10 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S10.

[0070] In a feasible implementation, step A13 may include steps B11 to B13:

[0071] Step B11, performing feature processing on the embedded feature vectors of each segmented image block according to the continuous Swin transformer blocks in the target self-attention neural network, to obtain a segmentation feature map corresponding to each segmented image block;

[0072] It should be noted that the Swin transformer block refers to the Swin Transformer Block in the target self-attention neural network, which is used to further extract features from the embedded feature vector, and send the embedded feature vector of each segmented image block to two consecutive Swin Transformer Blocks for feature processing, so as to obtain the local and context features extracted from each segmented image block, and determine the segmentation feature map corresponding to each segmented image block based on the local and context features extracted from each segmented image block. In this embodiment, the processing of block segmentation-linear embedding layer-continuous Swin Transformer Block helps the model capture fine-grained features in the image and retain spatial information, providing a basis for subsequent feature merging. The feature generated at this stage is C1, which has a higher spatial resolution and a lower semantic abstraction level.

[0073] Step B12, performing image fusion on each segmentation feature map according to the Patch merging module in the target self-attention neural network to obtain a target feature map;

[0074] It should be noted that the Patch merging module refers to the Patch Merging module in the target self-attention neural network, which is used to fuse the segmentation feature maps to obtain a larger feature map, that is, the target feature map.

[0075] It is understandable that before C1 is input into the Patch Merging module, edge features can be extracted from each segmentation feature map through the edge-guided attention mechanism, and the feature map containing edge features can be input into the Patch Merging module for feature merging and dimensionality reduction. In this embodiment, the edge-guided attention mechanism is implemented by deep Sobel convolution.

[0076] Step B13, performing a dimensionality reduction operation on the target feature map, and obtaining basic feature information of the image to be processed according to the dimensionality reduction result.

[0077] It should be noted that the target feature map is reduced in dimension through the target self-attention neural network. This process can reduce the computational complexity and improve the global perception ability of information. The feature information obtained after the dimensionality reduction operation is the basic feature of the image to be processed.

[0078] In this embodiment, the embedded feature vectors of each segmented image block are processed according to the continuous Swin transformer block in the target self-attention neural network to obtain the segmentation feature map corresponding to each segmented image block; the segmentation feature map is fused according to the Patch merging module in the target self-attention neural network to obtain the target feature map; the target feature map is reduced in dimension, and the basic feature information of the image to be processed is obtained according to the result of the dimension reduction. In the above manner, the computational complexity in the feature extraction process can be reduced and the global perception ability of information can be improved.

[0079] The above is only one feasible implementation of step A13 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step A13.

[0080] Step S20, performing edge feature extraction based on the convolutional layer features of the basic feature information, and determining a target edge feature map based on the extraction results;

[0081] It should be noted that by downsampling the basic feature information, the convolutional layer feature C2 corresponding to the basic feature information is generated. At this time, the spatial resolution is low but the semantic abstraction level is improved. The edge feature is extracted according to the edge self-attention mechanism, and the target edge feature map containing edge information corresponding to the second stage is obtained based on the extracted edge guidance feature.

[0082] Step S30, determining channel statistical information according to the target edge feature map and the target channel attention mechanism;

[0083] It should be noted that in the hierarchical structure adopted by Swin Transformer, a target channel attention mechanism is introduced to aggregate and infer more accurate spatial information. For the target edge feature map, global average pooling and maximum pooling are combined to generate channel-level statistical information corresponding to the target edge feature map. In this embodiment, the channel-level statistical information corresponding to the target edge feature map is the channel statistical information, and the channel statistical information includes the average pooling features and the maximum pooling features of the feature map.

[0084] In a feasible implementation, step S30 may include steps C11 to C13:

[0085] Step C11, performing an average pooling operation on the target edge feature map according to the target channel attention mechanism to obtain an average pooling feature;

[0086] It should be noted that in order to aggregate and infer fine spatial information, global average pooling and maximum pooling are combined to generate channel-level statistical information. Channel-level attention is a computational unit, specifically defined as: It can be built on a transformation , this conversion will input Mapping to feature map , through the spatial dimension Downscaling To generate a new feature map In this embodiment, the architecture of the target channel attention mechanism is as follows Figure 2 shown.

[0087] It can be understood that the average pooling operation is used through the target channel attention mechanism to aggregate the spatial information of the target edge feature map and generate the spatial context descriptor corresponding to the target edge feature map. , the spatial context descriptor corresponding to the target edge feature map That is the average pooling feature of the target edge feature map.

[0088] Step C12, performing a maximum pooling operation on the target edge feature map according to the target channel attention mechanism to obtain a maximum pooling feature;

[0089] It should be noted that the maximum pooling operation is used through the target channel attention mechanism to aggregate the spatial information of the target edge feature map and generate the spatial context descriptor corresponding to the target edge feature map. , the spatial context descriptor corresponding to the target edge feature map That is the maximum pooling feature of the target edge feature map.

[0090] Step C13: determining channel statistical information according to the maximum pooling feature and the average pooling feature.

[0091] It should be noted that the maximum pooling features and average pooling features of the target edge feature map constitute the channel statistical information of the target edge feature map.

[0092] In this implementation, an average pooling operation is performed on the target edge feature map according to the target channel attention mechanism to obtain an average pooling feature; a maximum pooling operation is performed on the target edge feature map according to the target channel attention mechanism to obtain a maximum pooling feature; channel statistical information is determined according to the maximum pooling feature and the average pooling feature. In the above manner, accurate acquisition of spatial information is achieved by using the target channel attention mechanism on the target edge feature map.

[0093] The above is only one feasible implementation of step S30 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S30.

[0094] Step S40, determining a target output feature of the image to be processed according to the channel statistical information;

[0095] It should be noted that the obtained channel statistical information is input into a shared network to generate a new channel attention map corresponding to the target edge feature map, and the third and fourth stages of feature extraction are performed based on the new channel attention map. The model processes higher-level merged features through more Swin Transformer Blocks to help the model understand the broader semantic content and structural relationships of the image, thereby outputting multi-level features of the image to be processed. The multi-level features finally output have the lowest spatial resolution and the highest level of semantic abstraction. The multi-level features obtained here are the target output features.

[0096] In a feasible implementation, step S40 may include steps D11 to D13:

[0097] Step D11, obtaining a target activation function and a target channel weight;

[0098] It should be noted that the target activation function refers to the sigmoid function and the ReLU function, and the target channel weight and .

[0099] Step D12, generating an image according to the target activation function, the target channel weight, the target edge feature map and the channel statistical information to obtain a target channel attention image;

[0100] It should be noted that the channel statistics are input into a shared network to generate a new channel attention map corresponding to the target edge feature map. The new channel attention map is .in, represents the sigmoid function, ReLU function, target channel weight and The input and ReLU function share parameters. Here, represents element-wise addition, represents element-wise multiplication, and F is the target edge feature map.

[0101] Step D13, performing feature extraction based on the target channel attention image to determine the target output features of the image to be processed.

[0102] It should be noted that in the third and fourth stages of feature extraction based on the target channel attention image, the model processes higher-level merged features through more Swin Transformer Blocks, thereby outputting multi-level features of the image to be processed.

[0103] It is understandable that the target output features help the model understand the broader semantic content and structural relationships of the image, which is crucial for tasks such as recognizing scenes, objects, and actions, and enhances the model's generalization ability and prediction accuracy.

[0104] In this embodiment, by obtaining the target activation function and the target channel weight; generating an image according to the target activation function, the target channel weight, the target edge feature map and the channel statistical information, a target channel attention image is obtained; and feature extraction is performed according to the target channel attention image to determine the target output feature of the image to be processed. In this way, it is ensured that the feature contains more accurate spatial information.

[0105] The above is only one feasible implementation of step S40 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S40.

[0106] Step S50: executing an image processing task corresponding to the image to be processed according to the target output feature.

[0107] It should be noted that the image processing task corresponding to the image to be processed is performed based on the target output features. The image processing task includes but is not limited to various computer vision tasks such as image recognition, target detection, and semantic segmentation.

[0108] This embodiment provides an image processing method based on multi-level feature extraction. This embodiment performs feature processing on the image to be processed according to the target self-attention neural network to obtain the basic feature information of the image to be processed; performs edge feature extraction according to the convolutional layer features of the basic feature information, and determines the target edge feature map according to the extraction result; determines channel statistics according to the target edge feature map and the target channel attention mechanism; determines the target output feature of the image to be processed according to the channel statistics; and performs the image processing task corresponding to the image to be processed according to the target output feature. In the above manner, the self-attention neural network is used to perform feature processing on the image to be processed, and the edge feature extraction is performed on the convolutional layer features of the obtained basic feature information to obtain the corresponding target edge feature map, and the target edge feature map and the target channel attention mechanism are used for processing to obtain the target output feature of the image to be processed, which effectively captures different levels of information from details to abstraction, and at the same time uses the edge attention mechanism to extract the edge information of the image, so as to realize the capture of the key visual clues of the object contour and spatial position, and improves the representation ability of features in complex scenes by fusing multimodal features, and can use limited attention resources to quickly filter out high-value information from a large amount of information, laying a foundation for subsequent image task processing.

[0109] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S20, the image processing method based on multi-level feature extraction also includes steps S21~S23:

[0110] Step S21, downsampling is performed according to the basic feature information, and convolution layer features are determined according to the downsampling result;

[0111] It should be noted that the basic feature information is downsampled to generate low-level features C2 of the basic feature information, and the spatial resolution is reduced, but the semantic abstraction level is improved. In this embodiment, the convolutional layer features refer to the low-level features after downsampling.

[0112] Step S22, determining the edge response map of each channel of the convolution layer feature according to the target depth Sobel convolution kernel;

[0113] It should be noted that, considering that the low-level features contain enough edge features, in this embodiment, an edge-guided attention mechanism (EGA) is introduced, and its specific structure is as follows: Figure 4 As shown in the figure, this mechanism is implemented by the target depth Sobel convolution kernel. F1 shown in the figure is C1, and F2 shown in the figure is C2. The designed target depth Sobel convolution kernel (Gx, Gy) is applied to the shadow feature map of each channel, where Gx and Gy are two Sobel kernels, one for horizontal change and one for vertical change. The edge response map of each channel of the convolution layer feature is determined by two target depth Sobel convolution kernels.

[0114] Step S23, determining a target edge feature map according to the edge response map of each channel and the convolutional layer features.

[0115] It should be noted that by performing feature calculation based on the edge response kernel convolution layer features of each channel, the edge guide features corresponding to the basic feature information can be obtained, thereby obtaining a target edge feature map containing the edge guide features.

[0116] In a feasible implementation manner, step S23 may further include steps E11 to E13:

[0117] Step E11, performing mean calculation according to the edge response map of each channel to determine the initial edge feature map corresponding to the convolution layer feature;

[0118] Step E12, performing feature calculation according to the initial edge feature map and the convolution layer features to determine edge guide features;

[0119] Step E13, generating a target edge feature map according to the edge guide feature.

[0120] It should be noted that the initial edge feature map corresponding to the convolutional layer feature can be obtained by calculating the mean of the edge response map of each channel. ,in, and is the edge response map of each channel, containing approximate values ​​of the horizontal and vertical derivatives at each point, Represents the low-level features of each feature extraction stage, , as well as Respectively represent the channel size, width and height of the feature map. Finally, based on the convolutional layer feature C2 and the initial edge feature map Perform element-wise multiplication , get the edge-guided feature , image generation is performed based on the edge-guided features, and a target edge feature map containing the edge-guided features is obtained.

[0121] In this embodiment, the initial edge feature map corresponding to the convolutional layer feature is determined by performing mean calculation based on the edge response map of each channel; the edge guide feature is determined based on the initial edge feature map and the convolutional layer feature; and the target edge feature map is generated based on the edge guide feature. In the above manner, the efficiency and accuracy of the target edge feature map generation are guaranteed.

[0122] The above is only one feasible implementation of step S23 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S23.

[0123] This embodiment provides an image processing method based on multi-level feature extraction. This embodiment performs downsampling according to the basic feature information and determines the convolution layer features according to the downsampling results; determines the edge response map of each channel of the convolution layer features according to the target depth Sobel convolution kernel; and determines the target edge feature map according to the edge response map of each channel and the convolution layer features. In the above manner, the local edge information and the global position information are combined by using the edge-guided attention mechanism to obtain accurate edge features.

[0124] For example, in order to help understand the implementation process of the image processing method based on multi-level feature extraction obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 5 , Figure 5 A schematic diagram of the overall framework of an image processing method based on multi-level feature extraction is provided, specifically:

[0125] In this embodiment, Figure 5ST refers to the Transformer-based visual model Swin-Transformer, which is used to extract image features; WR refers to the feature pyramid network WA-RFB, which is used to process targets of different scales. It consists of multiple RFs (Receptive Field Blocks), and each RF block has different rate parameters; DH refers to the Detection Head, a part of the target detection model, which is used to predict bounding boxes and category labels; region extraction refers to proposing candidate regions through the RegionProposal Network, which may contain objects of interest; region conversion refers to converting candidate regions of different sizes into feature maps of fixed size through Region of Interest Pooling for subsequent classification and regression; the category prediction branch cls is used to predict the object category within the candidate region; the bounding box regression branch bbox is used to adjust the position of the candidate region to make it closer to the actual target bounding box.

[0126] In this embodiment, Swin Transformer generates multi-level feature representations by processing the input image layer by layer, usually labeled C1, C2, C3 and C4. These feature representations correspond to different spatial resolutions and semantic abstraction levels, respectively. The steps of Swin Transformer generating these multi-level features will be described in detail below. First, the input image is divided into multiple non-overlapping patches of fixed size (such as 4x4 or 7x7). Each patch is flattened and mapped to a high-dimensional feature space to form an initial feature representation. These feature representations will serve as input to subsequent Transformer blocks. Swin Transformer adopts a hierarchical structure, including Stage 1 (generating C1), Stage 2 (generating C2), Stage 3 (generating C3) and Stage 4 (generating C4). In each layer, the input features are processed by a series of Swin Transformer blocks. Each Swin Transformer block contains local attention calculation, Shifted Window mechanism, multi-head self-attention mechanism and multi-layer perceptron (MLP) module. Local attention calculation is performed within a fixed-size window, which effectively reduces the computational complexity. The Shifted Window mechanism captures a wider range of contextual information by shifting the window between adjacent Transformer blocks, thereby improving the model's feature expression ability. The multi-head self-attention mechanism is performed within the window, and multiple attention heads capture different feature dimensions, enriching the feature representation. Finally, the features calculated by self-attention are further nonlinearly transformed and feature extracted through the MLP module. At the end of each layer, the Swin Transformer performs a downsampling operation to generate feature representations of different resolutions. This is usually done through window merging or pooling operations, thereby reducing the spatial resolution of the feature map by half while increasing the number of channels. Specifically, after Stage 1 is completed, the generated feature representation is called C1, which has a higher spatial resolution and a lower level of semantic abstraction. After Stage 2 is completed, C2 is generated after downsampling, with a lower spatial resolution but an increased level of semantic abstraction. After Stage 3 is completed, C3 is further downsampled, with a further reduced spatial resolution and a further increased level of semantic abstraction. After Stage 4 is completed, C4 is finally generated, with the lowest spatial resolution and the highest level of semantic abstraction. It is passed to subsequent task-specific modules (such as detection head, segmentation head, etc.) to complete specific visual tasks. In summary, Swin Transformer processes the input image layer by layer through a hierarchical structure and local attention mechanism to generate multi-level feature representations. These features capture rich information at different spatial resolutions and semantic abstraction levels, providing a strong foundation for various computer vision tasks.

[0127] In this embodiment, an edge-guided attention mechanism (EGA) is proposed, which is implemented through deep Sobel convolution. Considering that the low-level features contain enough edge features, the first stage convolutional layer features is used to extract the edge attention map. Specifically, by transforming the low-level features and , and attention map , Represents element-wise multiplication (⊙) to obtain the edge-guided feature and To calculate the edge attention map , a deep Sobel kernel (Gx, Gy) is designed and applied to the shadow feature map of each channel. The edge feature formula is defined as: , , , , , where Gx and Gy are two Sobel kernels, one for horizontal change and one for vertical change. Here, i∈{1,2}, represents low-level features. In addition, and is the edge response map, containing approximations of the horizontal and vertical derivatives at each point. Then, the average response map of all filters is used to obtain the overall edge feature map Finally, we can get the edge-guided feature and . , as well as They represent the channel size, width and height of the feature map respectively.

[0128] In addition, in this embodiment, by explicitly modeling the interdependencies between feature channels, we can adaptively recalibrate the feature responses in the channel direction and then pay attention to "what is meaningful". In order to aggregate and infer fine spatial information, we combine global average pooling and maximum pooling to generate channel-level statistics. Channel-level attention is a computational unit that can be built on a transformation This conversion will input Mapping to feature map . By Downscaling To generate a new feature map First, we use average pooling and max pooling operations to aggregate the spatial information of feature maps and generate two different spatial context descriptors: and , representing the average pooling feature and the maximum pooling feature respectively. These two descriptors are then fed into a shared network to generate our new channel attention map , the calculation formula is as follows: ,in represents the sigmoid function, ReLU function, weight and The input and ReLU function share parameters. Here, represents element-wise addition, Represents element-wise multiplication.

[0129] Through the method of this embodiment, the edge-guided attention mechanism is used to combine local edge information and global position information to obtain edge features, and the focus is placed on the complementarity between edge information and target information. These two complementary information are modeled simultaneously in one network. This improves the hierarchical nature of feature expression and the ability to capture details, and significantly improves the performance of the model on these tasks.

[0130] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the image processing method based on multi-level feature extraction of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0131] This application also provides an image processing device based on multi-level feature extraction, please refer to Figure 6 , the image processing device based on multi-level feature extraction comprises:

[0132] A processing module 10 is used to perform feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information of the image to be processed;

[0133] An extraction module 20, configured to extract edge features based on the convolutional layer features of the basic feature information, and determine a target edge feature map based on the extraction results;

[0134] The processing module 10 is further used to determine channel statistical information according to the target edge feature map and the target channel attention mechanism;

[0135] The processing module 10 is further used to determine the target output feature of the image to be processed according to the channel statistical information;

[0136] The execution module 30 is used to execute the image processing task corresponding to the image to be processed according to the target output feature.

[0137] Optionally, the processing module 10 is further used for:

[0138] The image to be processed is divided into blocks according to the target self-attention neural network to obtain a plurality of segmented image blocks; each segmented image block is transformed according to the linear embedding layer of the target self-attention neural network to obtain an embedded feature vector of each segmented image block; feature processing is performed according to the embedded feature vector of each segmented image block to obtain basic feature information of the image to be processed.

[0139] Optionally, the processing module 10 is further used for:

[0140] According to the continuous Swin transformer block in the target self-attention neural network, feature processing is performed on the embedded feature vectors of each segmented image block to obtain a segmentation feature map corresponding to each segmented image block; according to the Patch merging module in the target self-attention neural network, image fusion is performed on each segmentation feature map to obtain a target feature map; a dimensionality reduction operation is performed on the target feature map, and basic feature information of the image to be processed is obtained according to the dimensionality reduction result.

[0141] Optionally, the extraction module 20 is further used for:

[0142] Downsampling is performed according to the basic feature information, and convolution layer features are determined according to the downsampling results; edge response maps of each channel of the convolution layer features are determined according to the target depth Sobel convolution kernel; and target edge feature maps are determined according to the edge response maps of each channel and the convolution layer features.

[0143] Optionally, the extraction module 20 is further used for:

[0144] The method comprises the following steps: performing mean calculation according to the edge response map of each channel to determine the initial edge feature map corresponding to the convolution layer feature; performing feature calculation according to the initial edge feature map and the convolution layer feature to determine the edge guide feature; and generating the target edge feature map according to the edge guide feature.

[0145] Optionally, the processing module 10 is further used for:

[0146] According to the target channel attention mechanism, an average pooling operation is performed on the target edge feature map to obtain an average pooling feature; according to the target channel attention mechanism, a maximum pooling operation is performed on the target edge feature map to obtain a maximum pooling feature; and channel statistical information is determined according to the maximum pooling feature and the average pooling feature.

[0147] Optionally, the processing module 10 is further used for:

[0148] Obtain a target activation function and a target channel weight; generate an image according to the target activation function, the target channel weight, the target edge feature map, and the channel statistical information to obtain a target channel attention image; perform feature extraction according to the target channel attention image to determine the target output features of the image to be processed.

[0149] The image processing device based on multi-level feature extraction provided by the present application adopts the image processing method based on multi-level feature extraction in the above-mentioned embodiment, and can solve the technical problem of image processing based on multi-level feature extraction. Compared with the prior art, the beneficial effects of the image processing device based on multi-level feature extraction provided by the present application are the same as the beneficial effects of the image processing method based on multi-level feature extraction provided by the above-mentioned embodiment, and other technical features in the image processing device based on multi-level feature extraction are the same as the features disclosed in the above-mentioned embodiment method, which will not be described in detail here.

[0150] The present application provides an image processing device based on multi-level feature extraction, and the image processing device based on multi-level feature extraction includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing method based on multi-level feature extraction in the above-mentioned embodiment one.

[0151] Reference below Figure 7 , which shows a schematic diagram of the structure of an image processing device based on multi-level feature extraction suitable for implementing the embodiment of the present application. The image processing device based on multi-level feature extraction in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The image processing device based on multi-level feature extraction shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0152] like Figure 7As shown, the image processing device based on multi-level feature extraction may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the image processing device based on multi-level feature extraction are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the image processing device based on multi-level feature extraction to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an image processing device based on multi-level feature extraction with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0153] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0154] The image processing device based on multi-level feature extraction provided by the present application adopts the image processing method based on multi-level feature extraction in the above-mentioned embodiment, and can solve the technical problem of image processing based on multi-level feature extraction. Compared with the prior art, the beneficial effects of the image processing device based on multi-level feature extraction provided by the present application are the same as the beneficial effects of the image processing method based on multi-level feature extraction provided by the above-mentioned embodiment, and other technical features in the image processing device based on multi-level feature extraction are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0155] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0156] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0157] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the image processing method based on multi-level feature extraction in the above-mentioned embodiment.

[0158] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0159] The computer-readable storage medium may be included in the image processing device based on multi-level feature extraction; or may exist independently without being assembled into the image processing device based on multi-level feature extraction.

[0160] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an image processing device based on multi-level feature extraction, the image processing device based on multi-level feature extraction performs image processing based on multi-level feature extraction.

[0161] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0164] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned image processing method based on multi-level feature extraction, and can solve the technical problems of image processing based on multi-level feature extraction. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the image processing method based on multi-level feature extraction provided by the above-mentioned embodiment, and will not be described in detail here.

[0165] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned image processing method based on multi-level feature extraction when executed by a processor.

[0166] The computer program product provided by the present application can solve the technical problem of image processing based on multi-level feature extraction. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the image processing method based on multi-level feature extraction provided by the above embodiment, which will not be repeated here.

[0167] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An image processing method based on multi-level feature extraction, characterized in that: The image processing method based on multi-level feature extraction comprises: Perform feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information of the image to be processed, wherein the target self-attention neural network is a convolutional neural network SwinTransformer based on the Transformer architecture, and the Swin Transformer adopts a hierarchical structure. In each layer, the input features are processed by multiple SwinTransformer blocks, and each Swin Transformer block includes local attention calculation, Shifted Window mechanism, multi-head self-attention mechanism and multi-layer perceptron module; Extract edge features based on the convolutional layer features of the basic feature information, and determine a target edge feature map based on the extraction results; Determine channel statistics based on the target edge feature map and the target channel attention mechanism; Determining target output features of the image to be processed according to the channel statistical information; Execute the image processing task corresponding to the image to be processed according to the target output feature; The step of performing feature processing on the image to be processed according to the target self-attention neural network to obtain basic feature information of the image to be processed includes: The image to be processed is divided into blocks according to the target self-attention neural network to obtain multiple segmented image blocks; Convert each segmented image block according to the linear embedding layer of the target self-attention neural network to obtain an embedded feature vector of each segmented image block; Performing feature processing according to the embedded feature vectors of each segmented image block to obtain basic feature information of the image to be processed; The step of performing feature processing according to the embedded feature vectors of each segmented image block to obtain basic feature information of the image to be processed includes: Performing feature processing on the embedded feature vectors of each segmented image block according to the continuous Swin transformer blocks in the target self-attention neural network, to obtain a segmentation feature map corresponding to each segmented image block; Performing image fusion on each segmentation feature map according to the Patch merging module in the target self-attention neural network to obtain a target feature map; A dimensionality reduction operation is performed on the target feature map, and basic feature information of the image to be processed is obtained according to the dimensionality reduction result.

2. The method according to claim 1, characterized in that The step of extracting edge features based on the convolutional layer features of the basic feature information and determining a target edge feature map based on the extraction result includes: Downsampling is performed according to the basic feature information, and convolution layer features are determined according to the downsampling result; Determine the edge response map of each channel of the convolution layer feature according to the target depth Sobel convolution kernel; The target edge feature map is determined according to the edge response map of each channel and the convolution layer features.

3. The method according to claim 2, characterized in that The step of determining a target edge feature map according to the edge response map of each channel and the convolutional layer features includes: Calculate the mean value of the edge response map of each channel to determine the initial edge feature map corresponding to the convolution layer feature; Performing feature calculation according to the initial edge feature map and the convolutional layer features to determine edge guide features; A target edge feature map is generated according to the edge guide feature.

4. The method according to claim 1, characterized in that The determining channel statistical information according to the target edge feature map and the target channel attention mechanism includes: Performing an average pooling operation on the target edge feature map according to the target channel attention mechanism to obtain an average pooling feature; Performing a maximum pooling operation on the target edge feature map according to the target channel attention mechanism to obtain a maximum pooling feature; Channel statistical information is determined according to the maximum pooling feature and the average pooling feature.

5. The method according to claim 1, characterized in that The step of determining the target output feature of the image to be processed according to the channel statistical information includes: Get the target activation function and target channel weight; Generate an image according to the target activation function, the target channel weight, the target edge feature map and the channel statistical information to obtain a target channel attention image; Feature extraction is performed based on the target channel attention image to determine the target output features of the image to be processed.

6. An image processing device based on multi-level feature extraction, characterized in that: The image processing device based on multi-level feature extraction comprises: A processing module is used to perform feature processing on the image to be processed according to a target self-attention neural network to obtain basic feature information of the image to be processed, wherein the target self-attention neural network is a convolutional neural network Swin Transformer based on a Transformer architecture, and the Swin Transformer adopts a hierarchical structure. In each layer, input features are processed by multiple Swin Transformer blocks, and each Swin Transformer block includes local attention calculation, ShiftedWindow mechanism, multi-head self-attention mechanism, and multi-layer perceptron module; An extraction module, used to extract edge features according to the convolutional layer features of the basic feature information, and determine a target edge feature map according to the extraction results; The processing module is further used to determine channel statistical information according to the target edge feature map and the target channel attention mechanism; The processing module is further used to determine the target output feature of the image to be processed according to the channel statistical information; An execution module, used for executing the image processing task corresponding to the image to be processed according to the target output feature; The processing module is further used to divide the image to be processed into blocks according to the target self-attention neural network to obtain a plurality of segmented image blocks; transform each segmented image block according to the linear embedding layer of the target self-attention neural network to obtain an embedded feature vector of each segmented image block; perform feature processing according to the embedded feature vector of each segmented image block to obtain basic feature information of the image to be processed; The processing module is also used to perform feature processing on the embedded feature vectors of each segmented image block according to the continuous Swin transformer block in the target self-attention neural network to obtain a segmentation feature map corresponding to each segmented image block; perform image fusion on each segmentation feature map according to the Patch merging module in the target self-attention neural network to obtain a target feature map; perform dimensionality reduction operation on the target feature map, and obtain basic feature information of the image to be processed according to the dimensionality reduction result.

7. An image processing device based on multi-level feature extraction, characterized in that: The image processing device based on multi-level feature extraction includes: a memory, a processor, and an image processing program based on multi-level feature extraction stored in the memory and executable on the processor, wherein the image processing program based on multi-level feature extraction is configured to implement the image processing method based on multi-level feature extraction as described in any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores an image processing program based on multi-level feature extraction, and when the image processing program based on multi-level feature extraction is executed by a processor, the image processing method based on multi-level feature extraction as described in any one of claims 1 to 5 is implemented.

Images