Image processing method and device, electronic equipment and readable storage medium

Abstract

The application provides an image processing method and device, electronic equipment and readable storage medium. The method comprises the following steps: preprocessing a to-be-detected image to obtain an input feature map; performing multi-channel processing on the input feature map to obtain a channel attention feature map; processing spatial domain information in the channel attention feature map to obtain a spatial attention weight; and determining an output feature map according to the spatial attention weight and the channel attention feature map. The features expressed by the input feature map are enhanced in the channel and spatial dimensions, and the most discriminative visual features in the input feature map are highlighted. The output feature map after processing can be more accurate, the accuracy of image classification is improved, the accuracy in target detection is ensured, and application in the field of machine vision is facilitated.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] When humans view an image, they can quickly identify the target area that requires focused attention, thereby acquiring more detailed information about that target—this is the visual attention mechanism. This mechanism can greatly improve the efficiency and accuracy of human information processing.

[0003] In recent years, researchers have proposed attention modules that can be applied to deep learning by mimicking the human visual attention mechanism. However, these attention modules have poor compatibility with objects of different sizes and distances in the input image, resulting in inaccurate output feature maps. Summary of the Invention

[0004] This application provides an image processing method, apparatus, electronic device, and readable storage medium.

[0005] This application provides an image processing method, which includes: preprocessing an image to be detected to obtain an input feature map; performing multi-channel processing on the input feature map to obtain a channel attention feature map; processing the spatial domain information in the channel attention feature map to obtain spatial attention weights; and determining an output feature map based on the spatial attention weights and the channel attention feature map.

[0006] This application provides an image processing apparatus, comprising: a preprocessing module for preprocessing an image to be detected to obtain an input feature map; a channel attention processing module for performing multi-channel processing on the input feature map to obtain a channel attention feature map; a spatial weight determination module for processing spatial domain information in the channel attention feature map to obtain spatial attention weights; and a spatial attention processing module for determining an output feature map based on the spatial attention weights and the channel attention feature map.

[0007] This application provides an electronic device, including: one or more processors; and a memory storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the image processing methods in this application.

[0008] This application provides a readable storage medium storing a computer program that, when executed by a processor, implements any of the image processing methods described in this application.

[0009] According to the image processing method, apparatus, electronic device, and readable storage medium of the present application embodiments, multi-channel processing is performed on the input feature map to obtain a channel attention feature map, and spatial domain information in the channel attention feature map is processed to obtain spatial attention weights. This enhances the features to be expressed by the input feature map in both channel and spatial dimensions, highlighting the most discriminative visual features in the input feature map. Based on the spatial attention weights and the channel attention feature map, an output feature map is determined, making the processed output feature map more accurate, improving the accuracy of image classification, ensuring accuracy in target detection, and facilitating applications in the field of machine vision.

[0010] Further details regarding the above embodiments and other aspects of this application, as well as their implementations, are provided in the accompanying drawings, detailed description, and claims. Attached Figure Description

[0011] Figure 1 A schematic flowchart of an image processing method according to an embodiment of this application is shown.

[0012] Figure 2 This is a schematic flowchart illustrating an image processing method according to another embodiment of this application.

[0013] Figure 3 This is a schematic diagram of the structure of the image processing apparatus provided in an embodiment of this application.

[0014] Figure 4 This is a schematic diagram of the structure of an image processing apparatus provided in another embodiment of this application.

[0015] Figure 5 This diagram illustrates the process of processing the input feature map through the channel attention module in an embodiment of this application.

[0016] Figure 6 This illustration shows a flowchart of the process of processing the channel attention feature map through the spatial attention module in an embodiment of this application.

[0017] Figure 7 This diagram illustrates a flowchart of a face image processing method based on a channel attention module and a spatial attention module, as described in an embodiment of this application.

[0018] Figure 8 A structural diagram of an exemplary hardware architecture of a computing device capable of implementing the image processing method and apparatus according to embodiments of this application is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be arbitrarily combined with each other.

[0020] In recent years, researchers have proposed attention modules applicable to deep learning by mimicking human visual attention mechanisms. Attention modules can select information more beneficial to the current task from the input image and suppress the influence of information from interfering regions. Attention modules include channel domain attention modules, spatial domain attention modules, and hybrid domain attention modules. Hybrid domain attention modules can simultaneously obtain attention weights for the input image in both the spatial and channel domains. Currently, the widely used Convolutional Block Attention Module (CBAM) is a type of hybrid domain attention module. CBAM uses a single convolutional kernel to extract a set of channel feature maps, which has poor compatibility with objects of different sizes and distances, resulting in inaccurate weight maps. Furthermore, the spatial attention module of CBAM requires first processing the input feature map using global max pooling and global average pooling separately. After obtaining the two processed feature maps, they are combined before performing the convolution operation, leading to a large computational load and making it difficult to implement.

[0021] Figure 1 This diagram illustrates a flowchart of an image processing method according to an embodiment of this application. This image processing method can be applied to an image processing apparatus, which can be used in a face recognition network. Figure 1 As shown, the image processing method in this application embodiment may include the following steps.

[0022] Step S110: Preprocess the image to be detected to obtain the input feature map.

[0023] The images to be detected include face images and / or object images. The images to be detected are subjected to operations such as feature extraction, image segmentation, matching and recognition to eliminate unnecessary information in the images to be detected, restore useful real information, enhance the detectability of information, simplify the data to the maximum extent, and improve the reliability of the obtained input feature maps.

[0024] For example, when the image to be detected is a face image, multiple face images in the image to be detected can be aligned to make the distance between images of the same type closer and the distance between images of different types farther, so as to facilitate the recognition of face images and quickly distinguish the same face images.

[0025] Step S120: Perform multi-channel processing on the input feature map to obtain the channel attention feature map.

[0026] In this context, a channel can be understood as a mapping of selected regions. The pixel color in each channel is composed of a set of brightness values ​​of primary colors. For example, the pixel color in the R channel is red, the pixel color in the G channel is green, and the pixel color in the B channel is blue. For CMYK images, channels can also be divided into various channels such as C, M, Y, and K. The pixel color in the C channel is cyan, the pixel color in the M channel is magenta, the pixel color in the Y channel is yellow, and the pixel color in the K channel is black (taking the last letter of 'blacK'). The above examples of channel types are merely illustrative and can be customized according to specific implementations. Other undescribed channel types are also within the scope of this application and will not be elaborated upon here. The channel attention feature map is obtained by processing the input feature map through multiple channels, thus identifying the feature map corresponding to the channel with the highest importance among the various channels.

[0027] For example, the input feature map can be input into any one or more of the R, G, and B channels. The input feature map is processed through multiple different channels, and each channel processes the input feature map in a different dimension. This allows the obtained channel attention feature map to reflect more and more comprehensive features, ensuring the accuracy of feature extraction from the input feature map.

[0028] Step S130: Process the spatial domain information in the channel attention feature map to obtain spatial attention weights.

[0029] Spatial domain information can include any one or more of spectral domain information, spatial domain neighborhood information, and edge information. For example, traditional support vector machines can be used to classify hyperspectral images based on spectral domain information; neighborhood information from the spatial domain can be combined to optimize the classification results; and edge information can be used to classify targets in the channel attention feature map. Spatial domain information can also be the height and / or width information of the feature map, enabling the rapid extraction of information from different spatial dimensions in the channel attention feature map and reflecting spatial attention weights. The above examples of spatial domain information are merely illustrative and can be customized according to specific implementations. Other undescribed spatial domain information is also within the scope of this application and will not be elaborated upon further.

[0030] Spatial attention weight is the proportion of each target to be detected (e.g., images of faces, trees, animals, etc.) in the two-dimensional space of the channel attention feature map. This proportion can reflect the importance of the channel attention feature map in the two-dimensional space, thereby determining which target the user's main attention is focused on, increasing the detection weight of the most important target, so that the target to be detected is more prominent, which facilitates subsequent processing and quickly obtains the output feature map.

[0031] In some specific implementations, the spatial domain information in the channel attention feature map is processed to obtain spatial attention weights, including: performing max pooling on the spatial domain information in the channel attention feature map on a channel-by-channel basis to obtain a pooled feature map, which includes a two-dimensional feature vector; and performing convolution processing on the pooled feature maps corresponding to each channel to determine the spatial attention weights.

[0032] The convolution process for the pooled feature maps corresponding to each channel can be achieved by using a 1*1 convolution kernel to perform convolution operations on the pooled feature maps corresponding to each channel to obtain spatial attention weights.

[0033] For example, a two-dimensional feature vector can be an H*W feature vector, where H represents the height of the pooled feature map and W represents its width. This two-dimensional feature vector can reflect the importance of spatial attention in different channels of the pooled feature map. By comparing the spatial attention in different channels, the most important spatial attention in that channel can be determined, thus establishing the spatial attention weights. The two-dimensional feature vector effectively represents the spatial features of the feature map, ensuring the comprehensiveness and completeness of both its channel and spatial features, thus facilitating subsequent processing of the feature map.

[0034] Step S140: Determine the output feature map based on the spatial attention weights and channel attention feature maps.

[0035] The output feature map is used to represent the most prominent features of attention, that is, features that can be reflected by both channel attention and spatial attention. For example, the spatial attention weights and channel attention feature maps can be multiplied by a dot product to obtain the output feature map, thereby reducing the computational load and enabling the rapid extraction of spatial features from the channel attention feature map. This allows the output feature map to more comprehensively reflect both spatial and channel features, ensuring the comprehensiveness and accuracy of the features.

[0036] In some specific implementations, the output feature map is determined based on the spatial attention weights and the channel attention feature maps, including: performing a dot product operation on the spatial attention weights and the channel attention feature maps to obtain the output feature map.

[0037] By performing a dot product between the channel attention feature map and the spatial attention weights, we can consider the spatial domain information of the feature map based on the channel attention feature map. By combining channel features and spatial features, the output feature map can be more comprehensive and accurate.

[0038] In this embodiment, multi-channel processing of the input feature map is performed to obtain a channel attention feature map. The spatial domain information in the channel attention feature map is then processed to obtain spatial attention weights. This enhances the features to be expressed by the input feature map in both channel and spatial dimensions, highlighting the most discriminative visual features in the input feature map. Based on the spatial attention weights and the channel attention feature map, the output feature map is determined, making the processed output feature map more accurate, improving the accuracy of image classification, ensuring accuracy in target detection, and facilitating applications in the field of machine vision.

[0039] In some specific implementations, step S120, which involves multi-channel processing of the input feature map to obtain channel attention feature maps, includes: performing global average pooling on the input feature map to obtain the feature map to be detected; and determining the channel attention feature map based on the N channel convolution kernels and the feature map to be detected, wherein the N channel convolution kernels have different scales and N is an integer greater than or equal to 1.

[0040] For example, using three different channel convolution kernel sizes—1*1, 3*3, and 5*5—convolved with the feature map to be detected, three different channel attention feature maps are obtained. Because different kernel sizes correspond to different user perceptual fields of view, users will have different viewing effects on targets at different scales (e.g., different distances or different sizes). This expands the feature range of the feature map to be detected, allowing its features to be represented more quickly and comprehensively, facilitating image processing within the feature map and accelerating the acquisition of channel attention feature maps from different angles. This enables the features of the channel attention feature maps to more comprehensively and accurately reflect the user's attention.

[0041] In some specific implementations, the channel attention feature map is determined based on N channel convolution kernels and the feature map to be detected, including: performing operations with the feature map to be detected using N channel convolution kernels respectively to obtain N channel feature maps; performing image equalization processing on the N channel feature maps to determine the equalized channel feature map, which includes a one-dimensional feature vector; and determining the channel attention feature map based on the equalized channel feature map and the input feature map.

[0042] For example, a one-dimensional feature vector can be a 1*1*C feature vector, where C represents the number of feature channels to reflect the channel characteristics of the equalized channel feature map. Then, based on the equalized channel feature map and the input feature map (e.g., by performing feature fusion on the equalized channel feature map and the input feature map), a channel attention feature map is obtained, which makes the channel characteristics of the feature map to be detected more prominent and improves channel attention.

[0043] In some specific implementations, the channel attention feature map is determined based on the equalized channel feature map and the input feature map, including: performing a dot product operation on the equalized channel feature map and the feature map to be detected to obtain the channel attention feature map.

[0044] By performing a dot product operation between the equalized channel feature map and the feature map to be detected, the computational load is significantly reduced, the channel attention feature map is obtained more quickly, and the subsequent processing of the input feature map is facilitated.

[0045] Figure 2 This diagram illustrates a flowchart of an image processing method according to another embodiment of this application. This image processing method can be applied to an image processing apparatus, which can be used in a face recognition network. Figure 2 As shown, the image processing method in this application embodiment may include the following steps.

[0046] Step S210: Perform detection alignment on each image to be detected in the set of input face images to obtain a set of face feature maps.

[0047] The set of face images includes a first image to be detected and a second image to be detected, and the set of face feature maps includes a first face feature map and a second face feature map.

[0048] For example, the first and second facial feature maps can both be used as input feature maps. Detecting and aligning these two facial feature maps involves calibrating five points in the face—the eyes, the tip of the nose, and the corners of the mouth—to the same position to eliminate the influence of head angle and face size on recognition. This allows for clearer feature filtering of the two facial feature maps, enabling quick differentiation between them.

[0049] Step S220: Perform multi-channel processing on the input feature map to obtain the channel attention feature map.

[0050] The input feature map can be the first face feature map in the face feature map set, or the second face feature map in the face feature map set. In some specific implementations, the face feature map set can also include N face feature maps, where N is an integer greater than or equal to 2.

[0051] Processing the input feature map using multiple channels yields multi-dimensional image features, which is beneficial for feature extraction. Because each channel processes the input feature map with different dimensions, the resulting channel attention feature map can reflect more and more comprehensive features, ensuring the accuracy of feature extraction from the input feature map.

[0052] Step S230: Process the spatial domain information in the channel attention feature map to obtain spatial attention weights.

[0053] Step S240: Determine the output feature map based on the spatial attention weights and channel attention feature maps.

[0054] It should be noted that steps S230 to S240 in this embodiment are the same as steps S130 to S140 in the previous embodiment, and will not be described again here.

[0055] In this embodiment, the detection alignment of each image to be detected in the input set of face images is performed, that is, the five points of the eyes, nose tip, and mouth corners in each face feature map are aligned to the same position (for example, the two points of the eyes (left and right eyes), one point of the nose tip, and two points of the left and right mouth corners in the face feature map are aligned to the same position) to eliminate the influence of head angle and face size on image recognition, which is beneficial for extracting face features. Then, the feature maps in the obtained face feature map set are processed in multiple channels to obtain channel attention feature maps to extract the channel features of each feature map. Then, the spatial domain information in the channel attention feature maps is processed to obtain spatial attention weights to reflect the spatial features of each feature map. Through channel features and spatial features, the most discriminative visual features in the input feature map are highlighted. Based on the spatial attention weights and channel attention feature maps, the output feature map is determined so that the processed output feature map is more accurate, improving the accuracy of image classification and ensuring the accuracy of target detection.

[0056] In some specific implementations, after determining the output feature map based on the spatial attention weight and the channel attention feature map, the method further includes: calculating the matching similarity between the first output feature map corresponding to the first face feature map and the second output feature map corresponding to the second face feature map; and determining whether the first image to be detected and the second image to be detected are the same based on the matching similarity and a preset similarity threshold.

[0057] For example, by setting a preset similarity threshold of 0.5, if the matching similarity between the first output feature map and the second output feature map is less than 0.5, the first and second images to be detected are determined to be different; otherwise, if the matching similarity between the first and second output feature maps is greater than or equal to 0.5, the first and second images to be detected are determined to be the same. This method can quickly distinguish between the first and second images to be detected, accelerating image processing and improving user experience.

[0058] In some specific implementations, calculating the matching similarity between the first output feature map corresponding to the first face feature map and the second output feature map corresponding to the second face feature map includes: calculating the cosine similarity between the first output feature map and the second output feature map based on n feature vectors in the first output feature map and n feature vectors in the second output feature map; where n is an integer greater than or equal to 1.

[0059] For example, the cosine similarity can be calculated using the following formula:

[0060]

[0061] Where S represents the cosine similarity, i represents the index of the feature vector in the output feature map, i = {1, 2, ..., n}, x i Let y represent the i-th feature vector in the first output feature map. i This represents the i-th feature vector in the second output feature map, where n represents the dimension of the feature and is an integer greater than or equal to 1.

[0062] By using cosine similarity to judge the first and second output feature maps, it is determined whether the first and second output feature maps are the same, and thus whether the first face feature map is the same as the second face feature map. This speeds up the differentiation of faces, enabling different face features to be quickly identified, thus accelerating the differentiation of face images and facilitating applications in the field of machine vision.

[0063] The image processing apparatus according to embodiments of this application will now be described in detail with reference to the accompanying drawings. Figure 3 This diagram illustrates the structure of an image processing apparatus provided in an embodiment of this application. Figure 3 As shown, the device may include the following modules.

[0064] The preprocessing module 301 is used to preprocess the image to be detected to obtain the input feature map; the channel attention processing module 302 is used to process the input feature map in multiple channels to obtain the channel attention feature map; the spatial weight determination module 303 is used to process the spatial domain information in the channel attention feature map to obtain the spatial attention weight; and the spatial attention processing module 304 is used to determine the output feature map based on the spatial attention weight and the channel attention feature map.

[0065] In this embodiment, the input feature map is processed through a multi-channel channel attention processing module to obtain a channel attention feature map. A spatial weight determination module then processes the spatial domain information within the channel attention feature map to obtain spatial attention weights. This enhances the features expressed by the input feature map in both channel and spatial dimensions, highlighting the most discriminative visual features. Finally, the spatial attention processing module determines the output feature map based on the spatial attention weights and the channel attention feature map, making the processed output feature map more accurate, improving the precision of image classification, ensuring accuracy in target detection, and facilitating applications in the field of machine vision.

[0066] Figure 4 This diagram illustrates the structure of an image processing apparatus according to yet another embodiment of this application. Figure 4 As shown, the image processing device is a Multiple Kernel Attention (MKA) module, which may include a channel attention module 410 and a spatial attention module 420. Figure 4 Both the input feature map 401 and the output feature map 402 are multi-dimensional feature maps. For example, the input feature map 401 is a three-dimensional feature map with dimensions of H*W*C, and the output feature map 402 is also a three-dimensional feature map with dimensions of H*W*C.

[0067] For example, the input feature map 401 is input into the channel attention module 410 for processing. The input feature map 401 is processed through multiple channels (e.g., R channel, G channel, B channel, etc.) to filter and obtain channel attention weights. These channel attention weights are the weights of the most important channels among all channels, suppressing unimportant channels. The channel attention weights are then multiplied by the input feature map 401 to obtain the channel attention feature map. Then, the channel attention feature map is input into the spatial attention module 420 for processing. For example, the spatial domain information in the channel attention feature map is subjected to corresponding spatial transformations to obtain spatial attention weights. Finally, the spatial attention weights are multiplied by the channel attention feature map to obtain the output feature map 402.

[0068] The input feature map 401 is processed sequentially by the channel attention module 410 and the spatial attention module 420, which enhances the features to be expressed by the input feature map 401 in both channel and spatial dimensions, highlighting the most discriminative visual features in the input feature map 401, so that the processed output feature map can be more accurate and improve the accuracy of image classification.

[0069] Figure 5 This diagram illustrates the process of processing the input feature map using a channel attention module in an embodiment of this application. Figure 5 As shown, the input feature map 401 is processed through the following steps to obtain the channel attention feature map 540.

[0070] First, the channel attention module 410 performs a global average pooling operation on the input feature map 401 to obtain a feature map 510 to be detected. The size of the feature map 510 to be detected is 1*1*C, where C represents the number of feature channels. By performing a global average pooling operation on the input feature map 401, the computational cost can be reduced.

[0071] Then, the first convolutional processing module 511, the second convolutional processing module 512, ..., the Kth convolutional processing module 51K are used to process the feature map 510 to be detected. Each convolutional processing module has a different kernel size, and K is an integer greater than or equal to 1. For example, selecting three different kernel sizes—1*1, 3*3, and 5*5—to process the feature map 510 to be detected (e.g., performing convolution operations between each kernel and the feature map 510 to be detected) yields three channel attention feature maps. Because different kernel sizes correspond to different user perception fields, users will have different viewing effects on targets at different scales (e.g., different distances or different sizes). Therefore, using multiple kernels to process the feature map 510 to be detected ensures that the extracted features are more comprehensive and accurate. In practical applications, the number and size of the kernels in each convolutional processing module can be flexibly adjusted.

[0072] The obtained N channel attention feature maps are then input into the averaging module 520 for processing, generating an equalized channel feature map 530, which is the final channel attention feature map 530. This equalized channel feature map 530 includes a one-dimensional feature vector (for example, the size of the equalized channel feature map 530 is 1*1*C). The equalized channel feature map 530 can represent the importance of information in each channel. Finally, the input feature map 401 and the equalized channel feature map 530 are subjected to a dot product operation to generate the channel attention feature map 540.

[0073] In this embodiment, the input feature map is transformed into a feature map to be detected through global average pooling. The feature map to be detected is processed using n different sizes of convolution kernels to expand the user's viewing field of vision and make the extracted features more comprehensive and accurate. Then, the N channel attention feature maps are balanced to highlight the most discriminative visual features in the input feature map, so that the obtained channel attention feature maps can better highlight the characteristics of the input feature map and ensure that the user can quickly capture the visual features of the input feature map.

[0074] Figure 6 This diagram illustrates a flowchart of how the spatial attention module processes the channel attention feature map in an embodiment of this application. Figure 6 As shown, the channel attention feature map 540 is processed through the following steps to obtain the output feature map 402.

[0075] First, the channel attention feature map 540 is input into the max pooling processing module 610, where max pooling is performed to obtain the pooled feature map. The size of the pooled feature map is H*W*1, where H represents the height of the pooled feature map and W represents the width of the pooled feature map. By performing max pooling on the channel attention feature map 540, the amount of subsequent computation can be reduced.

[0076] Then, the pooled feature map is output to the convolution processing module 620. Through the processing of the convolution processing module 620, spatial attention weights can be obtained. For example, a 1*1 convolution can be performed on the pooled feature map to keep the dimension of the pooled feature map unchanged at H*W*1, so as to reflect the spatial features of the feature map, i.e., the spatial attention weights. Then, the spatial attention weights are multiplied by the channel attention feature map 540 to generate the output feature map 402.

[0077] In this embodiment, max pooling is performed on the channel attention feature map, and the pooled feature map is output to the convolution processing module for processing to obtain spatial attention weights that can reflect the spatial features of the feature map. Then, the spatial attention weights are multiplied by the channel attention feature map to generate the output feature map, highlighting the most discriminative visual features in the input feature map, so that the processed output feature map can be more accurate, improve the accuracy of image classification, and ensure the accuracy in target detection.

[0078] Figure 7 This diagram illustrates a flowchart of a face image processing method based on a channel attention module and a spatial attention module, as described in an embodiment of this application. This image processing method can be applied to an image processing device, which can be used in a face recognition network. Figure 7As shown, the face image processing method in this application embodiment may include the following steps.

[0079] Step S710: Perform detection alignment on each image to be detected in the set of input face images to obtain the training set and test set of face feature maps.

[0080] For example, a deep learning-based face detection method is used to detect each image to be detected in the input set of face images (e.g., using Retinaface to detect faces in each image to be detected, or using Multi-Task Convolutional Neural Networks (MTCNN) to detect faces in each image to be detected), thus obtaining a training set and a test set. The training set includes face training feature maps, and the test set includes face test feature maps.

[0081] Then, the face training feature maps in the training set are aligned. For example, a fixed formula is used to map the face images so that the five points of the eyes, nose tip and mouth corners in the face image are aligned to the same position. This eliminates the influence of head angle and face size on recognition, and allows for clearer feature filtering of the face training feature maps to quickly distinguish different face feature maps.

[0082] Step S720: Train the face feature maps in the training set to obtain a face recognition network.

[0083] Deep learning (DL) methods are used to train facial feature maps in a training set to obtain a face recognition network. DL learns the inherent patterns in sample data, and its ultimate goal is to enable machines to have analytical and learning capabilities like humans, and to recognize data such as text, images, and sounds.

[0084] The face recognition network includes a feature extraction module and a classifier. The feature extraction module includes an attention module, which may include channel attention and spatial attention modules, to extract information useful for face recognition from the input face feature map, thereby improving the accuracy of face recognition. The classifier is a face recognition model-based classifier (e.g., a classifier determined based on a face recognition loss function). This classifier improves the classification ability of the face recognition network, making images of the same type closer together and images of different types farther apart, thus facilitating image differentiation.

[0085] Step S730: Input the first face test feature map and the second face test feature map from the test set into the face recognition network for testing.

[0086] The face recognition network may include Figure 4 The MKA module is shown. For example, the MKA module can be added to the anti-residual module of a face recognition network to improve the network's ability to represent facial features. By inputting the first and second face test feature maps from the test set into the face recognition network for testing, the most desired features in the first and second face test feature maps can be obtained. This makes the distinguishing features between the first and second face test feature maps more prominent, facilitating subsequent image comparison and rapid image recognition.

[0087] Step S740: Calculate the cosine similarity between the first face test feature map and the second face test feature map.

[0088] For example, the cosine similarity can be calculated using the following formula:

[0089]

[0090] Where S represents the cosine similarity, i represents the index of the feature vector in the output feature map, i = {1, 2, ..., n}, x i Let y represent the i-th feature vector in the first face test feature map. i Let represent the i-th feature vector in the second face test feature map, where n represents the dimension of the feature and is an integer greater than or equal to 1, for example, n = 512.

[0091] Using cosine similarity to characterize the distinguishing features of the first and second face test feature maps allows for the parameterization of feature distinction points, which is beneficial for judging the degree of similarity of distinguishing features and quickly determining whether the two images are the same.

[0092] Step S750: Determine whether the cosine similarity is greater than or equal to the preset similarity threshold.

[0093] For example, if the preset similarity threshold is set to 0.5, if the cosine similarity is greater than or equal to 0.5, step S760 is executed; otherwise, step S770 is executed.

[0094] Step S760: Determine that the first face image corresponding to the first face test feature map and the second face image corresponding to the second face test feature map are the same person.

[0095] Step S770: Determine that the first face image corresponding to the first face test feature map and the second face image corresponding to the second face test feature map are not the same person.

[0096] In this embodiment, by aligning the individual images to be detected in the set of input face images, a training set and a test set of face feature maps are obtained. Then, the face feature maps in the training set are trained to obtain a face recognition network. This face recognition network includes an MKA module, which enhances the features to be expressed by the face feature maps in both channel and spatial dimensions, highlighting the most discriminative visual features in the face feature maps. The first face test feature map and the second face test feature map are input into the face recognition network for testing to obtain the face recognition network. The cosine similarity between the first face test feature map and the second face test feature map is calculated. By determining whether the cosine similarity is greater than or equal to a preset similarity threshold, it is determined whether the first face image corresponding to the first face test feature map and the second face image corresponding to the second face test feature map are the same person. This makes the output feature map of the face recognition network more accurate, improves the accuracy of image classification, and ensures the accuracy of face recognition.

[0097] It should be clarified that this application is not limited to the specific configurations and processes described in the above embodiments and shown in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here, and the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0098] Figure 8 A structural diagram of an exemplary hardware architecture of a computing device capable of implementing the image processing method and apparatus according to embodiments of this application is shown.

[0099] like Figure 8 As shown, the computing device 800 includes an input device 801, an input interface 802, a central processing unit 803, a memory 804, an output interface 805, an output device 806, and a bus 807. The input interface 802, the central processing unit 803, the memory 804, and the output interface 805 are interconnected via the bus 807. The input device 801 and the output device 806 are connected to the bus 807 via the input interface 802 and the output interface 805, respectively, and are thus connected to other components of the computing device 800.

[0100] Specifically, input device 801 receives input information from the outside and transmits the input information to central processing unit 803 through input interface 802; central processing unit 803 processes the input information based on computer-executable instructions stored in memory 804 to generate output information, temporarily or permanently stores the output information in memory 804, and then transmits the output information to output device 806 through output interface 805; output device 806 outputs the output information to the outside of computing device 800 for user use.

[0101] In one embodiment, Figure 8 The computing device shown can be implemented as an electronic device, which may include: a memory configured to store a program; and a processor configured to run the program stored in the memory to perform the image processing method described in the above embodiments.

[0102] In one embodiment, Figure 8 The computing device shown can be implemented as an image processing system, which may include: a memory configured to store a program; and a processor configured to run the program stored in the memory to perform the image processing method described in the above embodiments.

[0103] The above description is merely an exemplary embodiment of this application and is not intended to limit the scope of protection of this application. Generally, various embodiments of this application can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. For example, some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device, although this application is not limited thereto.

[0104] Embodiments of this application can be implemented by executing computer program instructions through the data processor of a mobile device, for example, in a processor entity, or through hardware, or through a combination of software and hardware. The computer program instructions can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages.

[0105] Any block diagram of logical flow in the accompanying drawings of this application may represent program steps, or may represent interconnected logic circuits, modules, and functions, or may represent a combination of program steps and logic circuits, modules, and functions. The computer program may be stored in memory. The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as, but not limited to, read-only memory (ROM), random access memory (RAM), optical storage devices and systems (Digital Multifunction Discs, DVDs, or CDs), etc. Computer-readable media may include non-transitory storage media. The data processor may be of any type suitable to the local technical environment, such as, but not limited to, general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable logic devices (FGPAs), and processors based on multi-core processor architectures.

[0106] A detailed description of exemplary embodiments of this application has been provided above through exemplary and non-limiting examples. However, various modifications and adjustments to the above embodiments will be apparent to those skilled in the art when considered in conjunction with the accompanying drawings and claims, without departing from the scope of this application. Therefore, the proper scope of this application will be determined by the claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: performing preprocessing on a to-be-detected image to obtain an input feature map; performing multi-channel processing on the input feature map to obtain a channel attention feature map; performing maximum pooling processing on spatial domain information in the channel attention feature map in a channel unit to obtain a pooled feature map, wherein the pooled feature map comprises a two-dimensional feature vector; performing convolution processing on the pooled feature map corresponding to each channel to determine a spatial attention weight; determining an output feature map according to the spatial attention weight and the channel attention feature map; the method of performing multi-channel processing on the input feature map to obtain a channel attention feature map comprises the following steps: performing global average pooling processing on the input feature map to obtain a to-be-detected feature map; determining the channel attention feature map according to N channel convolution kernels and the to-be-detected feature map, wherein the N channel convolution kernels have different scales, and N is an integer greater than or equal to 1.

2. The method of claim 1, wherein, the method of determining the channel attention feature map according to the N channel convolution kernels and the to-be-detected feature map comprises the following steps: performing operation on the N channel convolution kernels and the to-be-detected feature map respectively to obtain N channel feature maps; performing image equalization processing on the N channel feature maps to determine equalized channel feature maps, wherein the equalized channel feature maps comprise one-dimensional feature vectors; determining the channel attention feature map according to the equalized channel feature maps and the input feature map.

3. The method of claim 2, wherein, the method of determining the channel attention feature map according to the equalized channel feature maps and the input feature map comprises the following steps: performing dot product operation on the equalized channel feature maps and the to-be-detected feature map to obtain the channel attention feature map.

4. The method of claim 1, wherein, the method of determining an output feature map according to the spatial attention weight and the channel attention feature map comprises the following steps: performing dot product operation on the spatial attention weight and the channel attention feature map to obtain the output feature map.

5. The method according to any one of claims 1 to 4, characterized in that, the to-be-detected image comprises a face image; the method of performing preprocessing on a to-be-detected image to obtain an input feature map comprises the following steps: performing detection alignment on each to-be-detected image in a set of input face images to obtain a set of face feature maps, wherein the set of face images comprises a first to-be-detected image and a second to-be-detected image, and the set of face feature maps comprises a first face feature map and a second face feature map.

6. The method of claim 5, wherein, after determining an output feature map according to the spatial attention weight and the channel attention feature map, the method further comprises the following steps: calculating a matching similarity between a first output feature map corresponding to the first face feature map and a second output feature map corresponding to the second face feature map; determining whether the first to-be-detected image and the second to-be-detected image are the same according to the matching similarity and a preset similarity threshold.

7. The method of claim 6, wherein, the method of calculating a matching similarity between a first output feature map corresponding to the first face feature map and a second output feature map corresponding to the second face feature map comprises the following steps: According to the n feature vectors in the first output feature map and the n feature vectors in the second output feature map, a cosine similarity of the first output feature map and the second output feature map is calculated, where n is an integer greater than or equal to 1.

8. An image processing apparatus characterized by comprising: Comprise: a pre-processing module configured to pre-process a to-be-detected image to obtain an input feature map; a channel attention processing module configured to perform global average pooling processing on the input feature map to obtain a to-be-detected feature map, and determine a channel attention feature map according to N channel convolution kernels and the to-be-detected feature map, wherein the N channel convolution kernels have different scales, and N is an integer greater than or equal to 1; a spatial weight determination module configured to perform maximum pooling processing on spatial domain information in the channel attention feature map in a channel unit to obtain a pooled feature map, the pooled feature map comprising two-dimensional feature vectors, and perform convolution processing on the pooled feature map corresponding to each channel to determine a spatial attention weight; a spatial attention processing module configured to determine an output feature map according to the spatial attention weight and the channel attention feature map. 9.An electronic device, comprising: one or more processors; a memory having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1-7.

10. A readable storage medium, characterized by, The readable storage medium stores a computer program, and the computer program is executed by a processor to implement the image processing method according to any one of claims 1-7.

Images