Image processing method and device, storage medium and electronic device

Abstract

The present disclosure provides an image processing method and device, a storage medium and an electronic device, and relates to the technical field of computers. In the method, a pre-trained image classification model is used to obtain initial features extracted from target image features, the initial features are pooled according to a preset pooling mode to obtain a global feature vector corresponding to the target image, a pre-trained local attention model is used to calculate the global feature vector to generate a local feature vector corresponding to the target image, a pre-trained global attention model is used to calculate the global feature vector and the local feature vector to determine a deep feature vector corresponding to the target image, and the category to which the target image belongs is determined based on the deep feature vector. In this way, the local attention model and the global attention model are added on the basis of the image classification model, and the sensitivity to key regions of an image during image processing can be improved without modifying the model architecture.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of computer technology, and more specifically, the embodiments of this disclosure relate to image processing methods and apparatus, storage media, and electronic devices. Background Technology

[0002] With the rapid development of the Internet, cloud computing, and other related technologies, multimedia data such as images, audio, and video are growing at an unprecedented rate. As the visual foundation for human perception of the world, image data helps people understand, express, and transmit information. How to accurately and quickly retrieve images relevant to user queries from massive amounts of image data is a problem of great concern to researchers in the field of multimedia retrieval. Due to its advantages in solving problems such as the curse of dimensionality, search efficiency, and storage overhead, hashing methods are widely used in approximate nearest neighbor queries for large-scale multimedia data. In hashing methods, the original image needs to be converted into a vector feature representation first, and then the final image features represented by hash codes are obtained through hash projection and quantization encoding. Therefore, the accuracy of the generated hash code representation largely depends on the accuracy of the representation of the input image feature vector.

[0003] In image feature extraction, key parts of an image often occupy only a portion of the image area. For example, in an image of a "running horse," the horse is usually located in the center of the image, with the majority being background irrelevant to semantics. Existing image feature extraction methods typically utilize a single deep learning model to extract image features, which easily leads to significant information loss during image quantization. Furthermore, image processing often considers the image as a whole, neglecting the main descriptive object, resulting in low accuracy in image feature extraction.

[0004] This section is intended to provide background or context for the embodiments of this disclosure set forth in the claims, and the description herein is not acknowledged as prior art simply because it is included in this section. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides an image processing method and apparatus, a storage medium, and an electronic device.

[0006] According to a first aspect of this disclosure, an image processing method is provided, the method comprising:

[0007] Using a pre-trained image classification model, the initial features obtained from feature extraction of the target image are acquired;

[0008] The initial features are pooled according to a preset pooling method to obtain the global feature vector corresponding to the target image;

[0009] The global feature vector is calculated by a pre-trained local attention model to generate a local feature vector corresponding to the target image;

[0010] A deep feature vector corresponding to the target image is determined by calculating the global feature vector and the local feature vector according to a pre-trained global attention model, so that a category to which the target image belongs is determined based on the deep feature vector.

[0011] Optionally, the pre-trained local attention model includes a pre-trained full connection function, a pre-trained activation function and a pre-trained convolution kernel, and the global feature vector is calculated by the pre-trained local attention model to generate a local feature vector corresponding to the target image, including:

[0012] The global feature vector is calculated by the pre-trained full connection function and the pre-trained activation function to obtain a local weight corresponding to the global feature vector;

[0013] A local attention feature corresponding to the local weight is calculated according to the local weight and the initial feature;

[0014] The target image corresponding local feature vector is obtained by performing convolution operation on the local attention feature by the pre-trained convolution kernel.

[0015] Optionally, the local attention feature corresponding to the local weight is calculated according to the local weight and the initial feature, including:

[0016] The local weight is expanded according to the dimension of the initial feature to obtain an expanded local weight;

[0017] The local attention feature corresponding to the local weight is obtained by multiplying the expanded local weight with the initial feature.

[0018] Optionally, the deep feature vector corresponding to the target image is determined by calculating the global feature vector and the local feature vector according to the pre-trained global attention model, including:

[0019] The global feature vector and the local feature vector are respectively converted by using pre-trained mapping parameters to obtain a first mapping vector corresponding to the local feature vector, a second mapping vector corresponding to the global feature vector and a third mapping vector;

[0020] The deep feature vector corresponding to the target image is obtained by calculating the first mapping vector, the second mapping vector and the third mapping vector according to a preset global attention formula.

[0021] Optionally, after the deep feature vector corresponding to the target image is determined, the method further comprises:

[0022] The deep feature vector is converted through a preset symbol formula to obtain a feature vector representing the target image using a hash code.

[0023] Optionally, the method further comprises:

[0024] A sample image and a real category corresponding to the sample image are obtained.

[0025] The sample image and the real category are taken as a training sample pair.

[0026] The initial image classification model, the initial local attention model and the initial global attention model are iteratively trained using the training sample pair to obtain the pre-trained image classification model, the pre-trained local attention model and the pre-trained global attention model, wherein the output sample deep feature vector conforms to the real category.

[0027] Optionally, the initial image classification model, the initial local attention model and the initial global attention model are iteratively trained using the training sample pair to obtain the pre-trained image classification model, the pre-trained local attention model and the pre-trained global attention model, wherein the output sample deep feature vector conforms to the real category, and the method comprises:

[0028] The sample image is processed for features using the initial image classification model, the initial local attention model and the initial global attention model to obtain a sample deep feature vector corresponding to the sample image.

[0029] A training category corresponding to the sample deep feature vector is determined through a preset classification full connection layer.

[0030] If the training category conforms to the real category, the image classification model obtained through training is determined as the pre-trained image classification model, the local attention model obtained through training is determined as the pre-trained local attention model, and the global attention model obtained through training is determined as the pre-trained global attention model.

[0031] If the training category does not conform to the real category, the training parameters in the initial image classification model, the initial local attention model and the initial global attention model are adjusted, and the operation of processing the sample image for features to obtain a sample deep feature vector corresponding to the sample image is performed again until the training category conforms to the real category.

[0032] Optionally, the feature processing of the sample image by using the initial image classification model, the initial local attention model and the initial global attention model to obtain a sample deep feature vector corresponding to the sample image comprises:

[0033] The initial image classification model is used to obtain a sample initial feature extracted from the sample image feature;

[0034] The sample initial feature is pooled according to the preset pooling mode to obtain a sample global feature vector corresponding to the sample image;

[0035] The sample global feature vector is calculated by the initial local attention model to generate a sample local feature vector corresponding to the sample image;

[0036] According to the initial global attention model, the sample global feature vector and the sample local feature vector are calculated to determine the sample deep feature vector corresponding to the sample image.

[0037] According to a second aspect of the present disclosure, an image processing device is provided, and the device comprises:

[0038] The first acquisition module is configured to use a pre-trained image classification model to obtain an initial feature extracted from a target image feature;

[0039] The pooling module is configured to pool the initial feature according to a preset pooling mode to obtain a global feature vector corresponding to the target image;

[0040] The first calculation module is configured to calculate the global feature vector by using a pre-trained local attention model to generate a local feature vector corresponding to the target image;

[0041] The second calculation module is configured to calculate the global feature vector and the local feature vector according to a pre-trained global attention model to determine a deep feature vector corresponding to the target image, so as to determine a category to which the target image belongs based on the deep feature vector.

[0042] Optionally, the pre-trained local attention model comprises a pre-trained full connection function, a pre-trained activation function and a pre-trained convolution kernel, and the first calculation module is further configured to:

[0043] The global feature vector is calculated by using the pre-trained full connection function and the pre-trained activation function to obtain a local weight corresponding to the global feature vector;

[0044] According to the local weight and the initial feature, a local attention feature corresponding to the local weight is calculated.

[0045] Convolving the local attention feature by the pre-trained convolution kernel, a local feature vector corresponding to the target image is obtained.

[0046] Optionally, the first calculation module is further configured to:

[0047] According to the dimension of the initial feature, the local weight is expanded to obtain an expanded local weight;

[0048] According to the multiplication of the expanded local weight and the initial feature, a local attention feature corresponding to the local weight is obtained.

[0049] Optionally, the second calculation module is further configured to:

[0050] The global feature vector and the local feature vector are respectively converted by using pre-trained mapping parameters to obtain a first mapping vector corresponding to the local feature vector and a second mapping vector and a third mapping vector corresponding to the global feature vector;

[0051] According to a preset global attention model, the first mapping vector, the second mapping vector and the third mapping vector are calculated to obtain a deep feature vector corresponding to the target image.

[0052] Optionally, the device further comprises:

[0053] The conversion module is configured to convert the deep feature vector by using a preset symbol formula to obtain a feature vector representing the target image by using a hash code.

[0054] Optionally, the device further comprises:

[0055] The second acquisition module is configured to acquire a sample image and a real category corresponding to the sample image.

[0056] The determination module is configured to take the sample image and the real category as a training sample pair.

[0057] The training module is configured to use the training sample pair to iteratively train an initial image classification model, an initial local attention model and an initial global attention model, so as to obtain the pre-trained image classification model, the pre-trained local attention model and the pre-trained global attention model, in which a sample deep feature vector output by the pre-trained image classification model conforms to the real category.

[0058] Optionally, the training module is further configured to:

[0059] The initial image classification model, the initial local attention model and the initial global attention model are used for feature processing of the sample image, to obtain a sample deep feature vector corresponding to the sample image.

[0060] A preset classification full connection layer is used to determine a training category corresponding to the sample deep feature vector.

[0061] If the training category is consistent with the real category, the trained image classification model is determined as the pre-trained image classification model, the trained local attention model is determined as the pre-trained local attention model, and the trained global attention model is determined as the pre-trained global attention model.

[0062] If the training category is not consistent with the real category, the training parameters in the initial image classification model, the initial local attention model and the initial global attention model are adjusted, and the operation of performing feature processing on the sample image to obtain a sample deep feature vector corresponding to the sample image is performed again until the training category is consistent with the real category.

[0063] Optionally, the training module is further configured to:

[0064] The initial image classification model is used to obtain a sample initial feature extracted from the sample image.

[0065] The sample initial feature is subjected to pooling processing according to the preset pooling mode, to obtain a sample global feature vector corresponding to the sample image.

[0066] The initial local attention model is used to calculate the sample global feature vector, to generate a sample local feature vector corresponding to the sample image.

[0067] The initial global attention model is used to calculate the sample global feature vector and the sample local feature vector, to determine a sample deep feature vector corresponding to the sample image.

[0068] According to an aspect of the present disclosure, there is provided a storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the image processing method.

[0069] According to an aspect of the present disclosure, there is provided an electronic device, comprising:

[0070] a processor; and

[0071] a memory configured to store executable instructions of the processor;

[0072] The processor is configured to execute the image processing method described in any one of the preceding methods by executing the executable instructions.

[0073] In summary, the image processing method provided by the embodiments of the present disclosure can first use a pre-trained image classification model to obtain initial features extracted from a target image, perform pooling processing on the initial features according to a preset pooling manner to obtain a global feature vector corresponding to the target image, generate a local feature vector corresponding to the target image by calculating the global feature vector through a pre-trained local attention model, and determine a deep feature vector corresponding to the target image by calculating the global feature vector and the local feature vector through a pre-trained global attention model, so as to determine a category to which the target image belongs based on the deep feature vector. In this way, on the one hand, the model output can be made to be more in line with the needs of users by training the model, and the accuracy of image processing can be improved. On the other hand, by adding a local attention model and a global attention model on the basis of an image classification model, the sensitivity to key regions of an image during image processing can be improved without modifying the model architecture, and to some extent, the problem of excessive loss of quantized information of a hash code during image processing by using a deep learning model is alleviated, so that the model can balance the consideration of subject details and background semantics in the image, and thus the information represented by the image processing result can be enriched. BRIEF DESCRIPTION OF DRAWINGS

[0074] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0075] Figure 1 A step flowchart of an image processing method provided by an embodiment of the present disclosure is schematically shown;

[0076] Figure 2 A flowchart of generating a local feature vector provided by an embodiment of the present disclosure is schematically shown;

[0077] Figure 3 A flowchart of determining a deep feature vector provided by an embodiment of the present disclosure is schematically shown;

[0078] Figure 4 A flowchart of training a model provided by an embodiment of the present disclosure is schematically shown;

[0079] Figure 5 A flowchart of iterative training of a model provided by an embodiment of the present disclosure is schematically shown;

[0080] Figure 6 A flowchart of image processing provided by an embodiment of the present disclosure is schematically shown;

[0081] Figure 7 a block diagram of an image processing apparatus provided by an embodiment of the present disclosure is shown schematically;

[0082] Figure 8 a schematic diagram of a storage medium provided by an embodiment of the present disclosure is shown schematically; and

[0083] Figure 9 a block diagram of an electronic device provided by an embodiment of the present disclosure is shown schematically.

[0084] In the drawings, identical or corresponding reference signs indicate identical or corresponding parts. DETAILED DESCRIPTION

[0085] The principles and spirits of the present disclosure will be described below with reference to several exemplary embodiments. It should be appreciated that these embodiments are given only to enable those skilled in the art to better understand and implement the present disclosure, and in no way limit the scope of the present disclosure. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0086] Those skilled in the art understand that the embodiments of the present disclosure can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure can be embodied in the form of a complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, and the data involved in the present disclosure can be data authorized by the user or data authorized by all parties.

[0087] In this document, any number of elements in the drawings are used to illustrate examples and not to limit, and any name is only used for distinction and does not have any limiting meaning.

[0088] The principles and spirits of the present disclosure will be described below with reference to several representative embodiments of the present disclosure.

[0089] Figure 1 a step flowchart of an image processing method provided by an embodiment of the present disclosure is shown schematically, as shown in Figure 1 The method can include:

[0090] In step S101, an initial feature obtained by extracting a feature of a target image is acquired by using a pre-trained image classification model.

[0091] In the embodiments of the present disclosure, the target image can be an image that needs to be recognized and classified. The target image can be downloaded online or uploaded by a user. The image classification model can be pre-trained according to sample images. Specifically, the image classification model can be obtained by iteratively training an initial image classification model. Through continuous iterative training, the image classification model can learn the ability to correctly extract initial features of an image. Therefore, in the embodiments of the present disclosure, the image classification model obtained by training can be used to extract features of the target image to obtain initial features contained in the target image.

[0092] In the embodiments of the present disclosure, the initial features obtained by extracting features of the target image can be obtained by filtering and extracting the target image using the pre-trained image classification model to obtain initial features that can be used to represent the image category. Specifically, the target image can be first converted into a grayscale image, and then the image can be subjected to feature extraction processing based on the pre-trained deep feature extractor in the pre-trained image classification model. Finally, the initial features that can be used to represent the image category in the target image can be obtained. For example, the deep feature extractor can be a ResNet-50 feature extraction network. The ResNet-50 feature extraction network can be used to extract deep features of the target image. The output features can have a feature channel number c = 2048, a feature width w = 7, and a feature height h = 7. The size of the initial features of the target image can be represented as h x w x c, i.e., the size of the initial features of the target image can be represented as 7 x 7 x 2048.

[0093] In step S102, the initial features are subjected to pooling processing according to a preset pooling manner to obtain a global feature vector corresponding to the target image.

[0094] In the embodiments of the present disclosure, the preset pooling manner can be a pooling manner set in advance according to actual operation. The preset pooling manner can be global pooling of the initial features, maximum pooling of the initial features, average pooling of the initial features, or the like. The initial features extracted from the target image can be subjected to aggregation and pooling statistics according to the preset pooling manner to only reduce the height and width of the initial feature matrix without changing the depth of the initial feature matrix. The result of the pooling processing can be used as the global feature vector corresponding to the target image. For example, the size of the initial features can be h x w x c. The global feature vector corresponding to the initial features can be obtained by subjecting the initial features to global pooling processing, and the size of the global feature vector can be 1 x 1 x c.

[0095] Step S103, calculating the global feature vector by the pre-trained local attention model to generate the local feature vector corresponding to the target image.

[0096] In the embodiments of the present disclosure, the local attention model can be obtained by pre-training an initial local attention model using sample images. Specifically, the local attention model can be iteratively trained using sample images to learn the ability to correctly generate a local feature vector corresponding to an image according to an input image feature vector. Therefore, in the embodiments of the present disclosure, the local attention model trained can be used to calculate the global feature vector to obtain the local feature vector contained in the global feature vector. The local attention model can be a feature space local attention (FSLA), an image space local attention (ISLA), or a bilateral local attention (BOAT), which is not limited in the present disclosure.

[0097] Step S104, calculating the global feature vector and the local feature vector according to the pre-trained global attention model to determine the deep feature vector corresponding to the target image, so as to determine the category corresponding to the target image based on the deep feature vector.

[0098] In the embodiments of the present disclosure, the global attention model can be obtained by pre-training an initial global attention model using sample images. The global attention model can be iteratively trained to learn the ability to correctly calculate a mapping vector according to an input feature vector. The global attention model can be based on an improved recurrent attention model (RAM). Specifically, the global feature vector and the local feature vector can be respectively input into the pre-trained global attention model for calculation. When the global attention model calculates the context vector of the feature vector decoder of the target image at each time step, it needs to consider all hidden states of the feature vector encoder of the target image to obtain a mapping vector corresponding to the global feature vector and a mapping vector corresponding to the local feature vector, respectively.

[0099] In the embodiments of the present disclosure, the pre-trained global attention model is used to calculate the global feature vector and the local feature vector to determine the deep feature vector corresponding to the target image. The pre-trained global attention model can be used to calculate the mapping vector corresponding to the global feature vector and the mapping vector corresponding to the local feature vector, and then the mapping vector corresponding to the global feature vector and the mapping vector corresponding to the local feature vector are fused to obtain the deep feature vector corresponding to the target image. In this way, the deep feature vector obtained by calculation can avoid excessive loss of information caused by deep learning calculation, and can pay attention to the subject details and background semantics on the target image at the same time, thereby enriching the information represented by the deep feature vector.

[0100] In summary, the image processing method provided by the embodiments of the present disclosure can first use a pre-trained image classification model to obtain initial features extracted from a target image, pool the initial features according to a preset pooling manner to obtain a global feature vector corresponding to the target image, use a pre-trained local attention model to calculate the global feature vector to generate a local feature vector corresponding to the target image, and use a pre-trained global attention model to calculate the global feature vector and the local feature vector to determine a deep feature vector corresponding to the target image. The category to which the target image belongs is determined based on the deep feature vector. In this way, on the one hand, the trained model can make the model output more in line with the user's needs, and can improve the accuracy of image processing. On the other hand, by adding a local attention model and a global attention model based on an image classification model, the sensitivity to key regions of an image during image processing can be improved without modifying the model architecture. To some extent, the problem of excessive loss of quantized information in the hash code when using a deep learning model to process an image is alleviated, so that the model can balance the consideration of subject details and background semantics in the image, thereby enriching the information represented by the image processing result.

[0101] Optionally, the pre-trained local attention model in the embodiments of the present disclosure can include a pre-trained full connection function, a pre-trained activation function, and a pre-trained convolution kernel. The operation of calculating the global feature vector by the pre-trained local attention model to generate the local feature vector corresponding to the target image can specifically include the following steps. Figure 2

[0102] Step S1031, calculating the global feature vector by the pre-trained full connection function and the pre-trained activation function to obtain the local weight corresponding to the global feature vector.

[0103] ​In the embodiments of the present disclosure, the pre-trained full connection function and the pre-trained activation function can be obtained after training the local attention model. The local weight corresponding to the global feature vector can be obtained by calculating the global feature vector through the pre-trained full connection function and the pre-trained activation function. For example, the local attention model can include a first layer full connection function, an activation function (relu), a second layer full connection function, and an activation function (sigmoid). The global feature vector can be input into the pre-trained local attention model, and the output calculation result can be taken as the local weight corresponding to the global feature vector by sequentially calculating the pre-trained first layer full connection function, the pre-trained activation function (relu), the pre-trained second layer full connection function, and the pre-trained activation function (sigmoid). For example, the size of the input global feature vector is 1×1×c, and the size of the local weight corresponding to the global feature vector obtained by the above calculation is 1×1×c, that is, the size of the local weight obtained by the above calculation is the same as that of the global feature vector.

[0104] In step S1032, the local attention feature corresponding to the local weight is calculated according to the local weight and the initial feature.

[0105] In the embodiments of the present disclosure, since the local weight and the initial feature can be represented as a matrix, the local attention feature corresponding to the local weight can be obtained by multiplying the local weight and the initial feature as a matrix.

[0106] In step S1033, the local attention feature is convoluted by using the pre-trained convolution kernel to obtain the local feature vector corresponding to the target image.

[0107] In the embodiments of the present disclosure, the pre-trained convolution kernel can be obtained after training the local attention model. The local feature vector corresponding to the target image can be obtained by convoluting the local attention feature through the pre-trained convolution kernel. For example, the calculation result of convoluting the local attention feature by using N pre-trained convolution kernels can be taken as the local feature vector corresponding to the target image, and the size of the local feature vector is 1×1×n. The number N of the pre-trained convolution kernels can be set according to actual operation, and the size of the pre-trained convolution kernel can be h×w.

[0108] Optionally, the operation of calculating the local attention feature corresponding to the local weight according to the local weight and the initial feature in the embodiments of the present disclosure can specifically include:

[0109] According to the dimension of the initial feature, the local weight is expanded to obtain an expanded local weight; and a position matrix is multiplied between the expanded local weight and the initial feature to obtain a local attention feature corresponding to the local weight.

[0110] In the embodiments of the present disclosure, the local weight can be expanded according to the dimension of the initial feature to obtain an expanded local weight. For example, the dimension of the initial feature can be determined as 7x7xc, and the dimension of the local weight can be determined as 1x1xc. In this case, the local weight can be expanded by copying 7 times to obtain an expanded local weight with a dimension of 7x7xc.

[0111] In the embodiments of the present disclosure, the local attention feature corresponding to the local weight can be obtained by multiplying a position matrix between the expanded local weight and the initial feature. For example, the expanded local weight and the initial feature can be multiplied according to a three-dimensional matrix based on the same dimension to obtain the local attention feature corresponding to the local weight.

[0112] Optionally, the operation of calculating and determining the deep feature vector corresponding to the target image according to the pre-trained global attention model, the global feature vector and the local feature vector can specifically include the following operations. Figure 3

[0113] In step S1041, the global feature vector and the local feature vector are respectively converted by using the pre-trained mapping parameter to obtain a first mapping vector corresponding to the local feature vector and a second mapping vector and a third mapping vector corresponding to the global feature vector.

[0114] In the embodiments of the present disclosure, the pre-trained mapping parameter can be determined according to the pre-trained global attention model. The pre-trained mapping parameter can include a local mapping parameter, a retrieval mapping parameter and a key mapping parameter. Specifically, the global feature vector and the local feature vector are respectively converted by using the pre-trained mapping parameter to obtain a first mapping vector corresponding to the local feature vector and a second mapping vector and a third mapping vector corresponding to the global feature vector. For example, the global feature vector and the local feature vector can be input into a preset mapping formula, the local feature vector can be multiplied by the local mapping parameter to obtain the first mapping vector corresponding to the local feature vector, the global feature vector can be multiplied by the retrieval mapping parameter to obtain the second mapping vector corresponding to the global feature vector, and the global feature vector can be multiplied by the key mapping parameter to obtain the third mapping vector corresponding to the global feature vector. The preset mapping formula can be represented as follows:

[0115] ​f lv = f local · W v

[0116] wherein f lv may be represented as a first mapping vector, a vector for representing a local mapping value, f local may be represented as a local feature vector, W v may be represented as a pre-trained local mapping parameter.

[0117] f gq = f global · W q

[0118] wherein f gq may be represented as a second mapping vector, a vector for representing a mapping retrieval value, f glocal may be represented as a global feature vector, W q may be represented as a pre-trained retrieval mapping parameter.

[0119] f gk = f global · W k

[0120] wherein f gk may be represented as a third mapping vector, a vector for representing a mapping key value, W k may be represented as a pre-trained key mapping parameter.

[0121] Step S1042, calculating the first mapping vector, the second mapping vector and the third mapping vector according to a preset global attention formula, to obtain a deep feature vector corresponding to the target image.

[0122] In the embodiments of the present disclosure, the first mapping vector f lv , the second mapping vector f gq and the third mapping vector f gk may be input into a preset global attention formula for normalization calculation, and the calculation result is taken as the deep feature vector corresponding to the target image. The preset global attention formula can be represented as:

[0123]

[0124] wherein f out may be represented as a deep feature vector, f gk T may be represented as the third mapping vector f gkThe transpose of c can be represented as the number of feature channels in the initial features, and the softmax can represent a normalized exponential function.

[0125] Optionally, after the operation of determining the deep feature vector corresponding to the target image, the embodiment of the present disclosure can further specifically include:

[0126] The deep feature vector is converted by a preset symbol formula to obtain a feature vector represented by a hash code.

[0127] In the embodiment of the present disclosure, each value in the deep feature vector can be traversed, and each value can be converted into a hash code by using a preset symbol formula, so as to obtain a feature vector represented by a hash code. The preset symbol formula can be represented as follows:

[0128]

[0129] wherein h i may be represented as an n-dimensional feature vector represented by a hash code 0 / 1, may be represented as any value in the n-dimensional deep feature vector. Any value in the n-dimensional deep feature vector determines whether the value meets the condition if the value meets the condition if the value does not meet the condition (otherwise), the value

[0130] Optionally, as shown in FIG. 1, the image processing method in the embodiment of the present disclosure can further include: Figure 4

[0131] Step S21, obtaining a sample image and a real category corresponding to the sample image.

[0132] In the embodiment of the present disclosure, the sample image can be an image downloaded online or an image uploaded by a user, and the real category corresponding to the sample image can be a category to which the sample image is manually labeled.

[0133] Step S22, taking the sample image and the real category as a training sample pair.

[0134] For example, the real category corresponding to the sample image 1 can be a desk, and the sample image 1 and the real category “desk” can be taken as a training sample pair; the real category corresponding to the sample image 2 can be a car, and the sample image 2 and the real category “car” can be taken as a training sample pair.

[0135] ​​Step S23, iteratively training the initial image classification model, the initial local attention model and the initial global attention model using the training sample pair to obtain the pre-trained image classification model, the pre-trained local attention model and the pre-trained global attention model whose output sample deep feature vector conforms to the real category.

[0136] In the embodiments of the present disclosure, the sample image in the training sample pair can be input into the initial image classification model, the initial local attention model and the initial global attention model for calculation and processing to obtain the deep feature vector corresponding to the sample image, and then the training category to which the sample image belongs can be determined according to the deep feature vector corresponding to the sample image. By calculating the loss function between the training category and the real category, if the loss function result does not conform to the preset threshold, the training parameters in the image classification model, the local attention model and the global attention model can be adjusted, and the operation of inputting the sample image in the training sample pair into the initial image classification model, the initial local attention model and the initial global attention model for calculation and processing can be re-executed until the loss function result conforms to the preset threshold. If the loss function result conforms to the preset threshold, that is, the training category is approximately equal to the real category, the trained image classification model can be used as the pre-trained image classification model, the trained local attention model can be used as the pre-trained local attention model, and the trained global attention model can be used as the pre-trained global attention model. In this way, by continuously training the sample image and adjusting the training parameters of the model, the ability of the model to learn to correctly calculate and process the sample image to obtain the real category can be improved, so that the trained model is more suitable for the classification task of the hash data set.

[0137] Optionally, the operation of iteratively training the initial image classification model, the initial local attention model and the initial global attention model using the training sample pair to obtain the pre-trained image classification model, the pre-trained local attention model and the pre-trained global attention model whose output sample deep feature vector conforms to the real category, as shown in FIG. 5, can specifically include:

[0138] Step S231, performing feature processing on the sample image using the initial image classification model, the initial local attention model and the initial global attention model to obtain a sample deep feature vector corresponding to the sample image.

[0139] In the embodiments of the present disclosure, the sample image can be sequentially input into the initial image classification model, the initial local attention model and the initial global attention model for feature processing, and finally a sample deep feature vector corresponding to the sample image can be obtained.

[0140] Step S232, determining a training category corresponding to the sample deep feature vector through the preset classification full connection layer.

[0141] In the embodiments of the present disclosure, the preset classification full connection layer can be a full connection layer for determining the category to which the feature vector belongs, and the preset classification full connection layer can be a two-layer classification full connection layer. Specifically, the category matched with the sample deep feature vector can be selected as the training category corresponding to the sample deep feature vector through the preset classification full connection layer.

[0142] Step S233, if the training category is consistent with the real category, determining the trained image classification model as the pre-trained image classification model, determining the trained local attention model as the pre-trained local attention model, and determining the trained global attention model as the pre-trained global attention model.

[0143] In the embodiments of the present disclosure, the training category being consistent with the real category can mean that the training category is the real category, or the difference between the training category and the real category is less than a threshold. When the training category is consistent with the real category, it can be determined that the model training task is completed, so that the trained image classification model can be determined as the pre-trained image classification model, the trained local attention model can be determined as the pre-trained local attention model, and the trained global attention model can be determined as the pre-trained global attention model.

[0144] Step S234, if the training category is not consistent with the real category, adjusting the training parameters in the initial image classification model, the initial local attention model and the initial global attention model, and re-executing the operation of performing feature processing on the sample image to obtain the sample deep feature vector corresponding to the sample image until the training category is consistent with the real category.

[0145] In the embodiments of the present disclosure, if the training category is different from the real category, or the difference between the training category and the real category is greater than a threshold, it can be determined that the model training task is not completed, and then the training parameters in the initial image classification model, the initial local attention model and the initial global attention model can be adjusted respectively, and the operation of performing feature processing on the sample image to obtain the sample deep feature vector corresponding to the sample image can be re-executed until the training category is consistent with the real category.

[0146] Optionally, the operation of performing feature processing on the sample image to obtain the sample deep feature vector corresponding to the sample image through the initial image classification model, the initial local attention model and the initial global attention model in the embodiments of the present disclosure can specifically include:

[0147] The initial image classification model is used to obtain sample initial features extracted from the sample image features, the sample initial features are processed by the preset pooling manner to obtain a sample global feature vector corresponding to the sample image, the initial local attention model is used to calculate the sample global feature vector to generate a sample local feature vector corresponding to the sample image, and the initial global attention model is used to calculate the sample global feature vector and the sample local feature vector to determine a sample deep feature vector corresponding to the sample image.

[0148] In the embodiments of the present disclosure, the initial image classification model can be used to extract features of a sample image to obtain sample initial features corresponding to the sample image, the sample initial features are processed by the preset pooling manner to obtain a sample global feature vector corresponding to the sample image, the initial local attention model is used to calculate the sample global feature vector to generate a sample local feature vector corresponding to the sample image, and finally, the initial global attention model is used to calculate the sample global feature vector and the sample local feature vector to determine a sample deep feature vector corresponding to the sample image.

[0149] In an example, Figure 6 An example of a flowchart of an image processing method provided by the present disclosure is shown as follows, Figure 6 As shown in the figure, ① the pre-trained image classification model is used to extract features of a target image to obtain initial features corresponding to the target image; ② the initial features are processed by the global pooling manner to obtain a global feature vector f glocal corresponding to the target image; ③ the pre-trained global feature vector f glocal is calculated by using the pre-trained full connection function and the pre-trained activation function in the pre-trained local attention model to obtain a local weight corresponding to the global feature vector f glocal ; ④ the local attention feature corresponding to the local weight is calculated according to the local weight and the initial feature; ⑤ the local attention feature is operated by the pre-trained convolution kernel in the pre-trained local attention model to obtain a local feature vector f local corresponding to the target image; ⑥ based on the pre-trained global attention model, the pre-trained local mapping parameter is used to convert the local feature vector f local to obtain a corresponding first mapping vector f lv , the pre-trained retrieval mapping parameter is used to convert the global feature vector f glocal to obtain a corresponding second mapping vector f gq , and the pre-trained key mapping parameter is used to convert the global feature vector f glocal to obtain a corresponding third mapping vector f gk⑦ Apply the preset global attention formula to the first mapping vector f lv The second mapping vector f gq Calculate f with the third mapping vector gk The deep feature vector f corresponding to the target image is obtained. out ⑧ By using a pre-defined classification fully connected layer to process the deep feature vector f out The process involves determining the category to which the target image belongs.

[0150] It should be noted that the image processing method provided in this disclosure can be executed by an image processing device, or a control module within that image processing device for executing the loading image processing method. This disclosure uses an image processing device executing the loading image processing method as an example to illustrate the image processing method provided in this disclosure. Next, refer to... Figure 6 An image processing apparatus according to exemplary embodiments of the present disclosure will be described.

[0151] Figure 7 A block diagram illustrating an image processing apparatus provided in an embodiment of this disclosure is shown, such as Figure 7 As shown, the image processing device 30 may include:

[0152] The first acquisition module 301 is used to acquire the initial features obtained by feature extraction of the target image using a pre-trained image classification model;

[0153] Pooling module 302 is used to perform pooling processing on the initial features according to a preset pooling method to obtain the global feature vector corresponding to the target image;

[0154] The first calculation module 303 is used to calculate the global feature vector using a pre-trained local attention model to generate a local feature vector corresponding to the target image.

[0155] The second calculation module 304 is used to calculate the deep feature vector corresponding to the target image based on the global feature vector and the local feature vector according to the pre-trained global attention model, so as to determine the category to which the target image belongs based on the deep feature vector.

[0156] In summary, the image processing apparatus provided by the embodiments of the present disclosure can first use the pre-trained image classification model to obtain initial features extracted from a target image, perform pooling processing on the initial features according to a preset pooling manner to obtain a global feature vector corresponding to the target image, generate a local feature vector corresponding to the target image by calculating the global feature vector through the pre-trained local attention model, determine a deep feature vector corresponding to the target image by calculating the global feature vector and the local feature vector through the pre-trained global attention model, and determine a category to which the target image belongs based on the deep feature vector. In this way, on the one hand, the model output can be made to meet the needs of users by training the model, and the accuracy of image processing can be improved. On the other hand, by adding the local attention model and the global attention model on the basis of the image classification model, the sensitivity to key regions of an image can be improved without modifying the model architecture, to a certain extent, the problem of excessive loss of quantized information of a hash code when an image is processed by using a deep learning model is alleviated, so that the model can balance the consideration of subject details and background semantics in the image, and the information represented by the image processing result can be enriched.

[0157] Optionally, the pre-trained local attention model includes a pre-trained full connection function, a pre-trained activation function, and a pre-trained convolution kernel, and the first calculation module 303 is further configured to:

[0158] The global feature vector is calculated through the pre-trained full connection function and the pre-trained activation function to obtain a local weight corresponding to the global feature vector.

[0159] According to the local weight and the initial feature, a local attention feature corresponding to the local weight is calculated.

[0160] The local attention feature is convoluted through the pre-trained convolution kernel to obtain a local feature vector corresponding to the target image.

[0161] Optionally, the first calculation module 303 is further configured to:

[0162] The local weight is expanded according to the dimension of the initial feature to obtain an expanded local weight.

[0163] According to the multiplication of the positional matrix between the expanded local weight and the initial feature, a local attention feature corresponding to the local weight is obtained.

[0164] Optionally, the second calculation module 304 is further configured to:

[0165] The global feature vector and the local feature vector are respectively converted by using pre-training mapping parameters, to obtain a first mapping vector corresponding to the local feature vector, and a second mapping vector and a third mapping vector corresponding to the global feature vector.

[0166] According to a preset global attention model, the first mapping vector, the second mapping vector, and the third mapping vector are calculated to obtain a deep feature vector corresponding to the target image.

[0167] Optionally, the device 30 further comprises:

[0168] The conversion module is configured to convert the deep feature vector by using a preset symbol formula to obtain a feature vector represented by a hash code of the target image.

[0169] Optionally, the device 30 further comprises:

[0170] The second acquisition module is configured to acquire a sample image and a real category corresponding to the sample image.

[0171] The determination module is configured to take the sample image and the real category as a training sample pair.

[0172] The training module is configured to iteratively train an initial image classification model, an initial local attention model, and an initial global attention model by using the training sample pair, to obtain the pre-trained image classification model, the pre-trained local attention model, and the pre-trained global attention model, which make the output sample deep feature vector conform to the real category.

[0173] Optionally, the training module is further configured to:

[0174] The initial image classification model, the initial local attention model, and the initial global attention model are used to perform feature processing on the sample image to obtain a sample deep feature vector corresponding to the sample image.

[0175] A training category corresponding to the sample deep feature vector is determined by using a preset classification full connection layer.

[0176] If the training category conforms to the real category, the training image classification model is determined as the pre-trained image classification model, the training local attention model is determined as the pre-trained local attention model, and the training global attention model is determined as the pre-trained global attention model.

[0177] If the training category does not conform to the true category, training parameters in the initial image classification model, the initial local attention model and the initial global attention model are adjusted, and the operation of performing feature processing on the sample image to obtain the sample deep feature vector corresponding to the sample image is performed again until the training category conforms to the true category.

[0178] Optionally, the training module is further configured to:

[0179] An initial image classification model is used to obtain a sample initial feature obtained by performing feature extraction on the sample image.

[0180] The sample initial feature is subjected to pooling processing according to the preset pooling manner to obtain a sample global feature vector corresponding to the sample image.

[0181] The sample global feature vector is calculated by the initial local attention model to generate a sample local feature vector corresponding to the sample image.

[0182] The sample global feature vector and the sample local feature vector are calculated according to the initial global attention model to determine a sample deep feature vector corresponding to the sample image.

[0183] After the image processing method and device of the example embodiment of the present disclosure are introduced, next, reference is made to Figure 8 The storage medium of the example embodiment of the present disclosure is described.

[0184] Reference is made to Figure 8 As shown in FIG. 4, a storage medium 400 for implementing the above method according to the embodiment of the present disclosure is described, which can adopt a portable compact disc read-only memory (CD-ROM) and include program codes, and can run on a device such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, the readable storage medium can be any tangible medium containing or storing a program which can be used by or in conjunction with an instruction execution system, device or apparatus.

[0185] The program product can employ any combination of one or more computer- readable media or storage media. The computer-readable media or storage media can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0186] The computer-readable signal medium can include a computer-readable data signal embodied in a carrier wave, or a propagated signal, that is based, at least in part, on the computer-readable program code. Such a propagated signal can take a wide variety of forms including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport program code thereon for use by or in connection with an instruction execution system, apparatus, or device.

[0187] Program code embodied on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0188] Program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a program product, partly on the remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0189] Having introduced the storage medium of the exemplary embodiments of the present disclosure, next, the electronic device of the exemplary embodiments of the present disclosure is described with reference to FIG. 5. Figure 9 An electronic device of the exemplary embodiments of the present disclosure is described.

[0190] Figure 9 The electronic device 500 shown in FIG. 5 is merely an example, and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0191] As shown in Figure 9 The electronic device 500 can be in the form of a general computing device. Components of the electronic device 500 can include, but are not limited to, the at least one processing unit 510 described above, the at least one storage unit 520 described above, a bus 530 connecting different system components, including the storage unit 520 and the processing unit 510, a display unit 540.

[0192] The storage unit stores program codes which can be executed by the processing unit 510, so that the processing unit 510 performs the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" part of the present specification. For example, the processing unit 510 can execute the step S101 of obtaining initial features extracted from a target image feature by using a pre-trained image classification model; the step S102 of performing a pooling processing on the initial features according to a preset pooling manner to obtain a global feature vector corresponding to the target image; the step S103 of generating a local feature vector corresponding to the target image by calculating the global feature vector through a pre-trained local attention model; the step S104 of determining a deep feature vector corresponding to the target image according to the calculation of the global feature vector and the local feature vector by using a pre-trained global attention model, so as to determine the category to which the target image belongs based on the deep feature vector.

[0193] The storage unit 520 can include a volatile storage unit, such as a random access memory (RAM) 5201 and / or a cache memory 5202, and further can include a non-volatile storage unit, such as a read-only memory (ROM) 5203.

[0194] The storage unit 520 can further include program / utility 5204 having a set of programs / modules 5205, each of which performs one or more functions of the electronic device 500, such as the example functions described herein. In this example, these programs / modules include operating system 5206, one or more application programs 5207, other programs 5208 and program data 5209.

[0195] The bus 530 can include a data bus, an address bus, and a control bus.

[0196] The electronic device 500 can also communicate with one or more external devices 60 such as a keyboard, a pointing device, a Bluetooth device, etc. through input / output (I / O) interface 550. The electronic device 500 further includes a display unit 540 connected to the input / output (I / O) interface 550 for displaying. Also, the electronic device 500 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet, through network adapter 560. As depicted, network adapter 560 communicates with the other modules of electronic device 500 through bus 530. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with electronic device 500 including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0197] It should be noted that although several modules or sub-modules of the audio playback device and the audio sharing device are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into embodied by multiple units / modules.

[0198] From the above description of the embodiments, it can be clear to those skilled in the art that the method of the embodiments can be realized by means of software plus necessary general hardware platforms, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product in essence or in the form of a part of the prior art that makes a contribution. The computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk), and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the method described in the various embodiments of the present disclosure.

[0199] The embodiments of the present disclosure are described above in conjunction with the drawings, but the present disclosure is not limited to the specific embodiments described above, which are merely illustrative rather than restrictive, and those of ordinary skill in the art can make many forms without departing from the spirit of the present disclosure and the scope of protection of the claims under the inspiration of the present disclosure.

Claims

1. An image processing method, characterized by, The method comprises: acquiring initial features obtained by feature extraction on a target image by using a pre-trained image classification model; pooling the initial features according to a preset pooling mode to obtain a global feature vector corresponding to the target image; generating a local feature vector corresponding to the target image by calculating the global feature vector through a pre-trained local attention model; determining a deep feature vector corresponding to the target image by calculating the global feature vector and the local feature vector according to a pre-trained global attention model, so as to determine a category to which the target image belongs based on the deep feature vector; wherein the pre-trained local attention model comprises a pre-trained full connection function, a pre-trained activation function and a pre-trained convolution kernel, and the generation of the local feature vector corresponding to the target image by calculating the global feature vector through the pre-trained local attention model comprises: calculating the global feature vector through the pre-trained full connection function and the pre-trained activation function to obtain a local weight corresponding to the global feature vector; calculating a local attention feature corresponding to the local weight according to the local weight and the initial feature; obtaining the local feature vector corresponding to the target image by performing convolution operation on the local attention feature through the pre-trained convolution kernel.

2. The method of claim 1, wherein, The calculation of the local attention feature corresponding to the local weight according to the local weight and the initial feature comprises: expanding the local weight according to the dimension of the initial feature to obtain an expanded local weight; multiplying a position matrix between the expanded local weight and the initial feature to obtain the local attention feature corresponding to the local weight.

3. The method of claim 1, wherein, The determination of the deep feature vector corresponding to the target image by calculating the global feature vector and the local feature vector according to the pre-trained global attention model comprises: transforming the global feature vector and the local feature vector respectively by using pre-trained mapping parameters to obtain a first mapping vector corresponding to the local feature vector, a second mapping vector corresponding to the global feature vector and a third mapping vector; calculating the first mapping vector, the second mapping vector and the third mapping vector according to a preset global attention formula to obtain the deep feature vector corresponding to the target image.

4. The method of claim 1, wherein, After determining the deep feature vector corresponding to the target image, the method further comprises: transforming the deep feature vector by using a preset symbol formula to obtain a feature vector representing the target image by using a hash code.

5. The method according to any one of claims 1-4, characterized in that, The method further comprises: acquiring a sample image and a real category corresponding to the sample image; taking the sample image and the real category as a training sample pair; iteratively training an initial image classification model, an initial local attention model and an initial global attention model by using the training sample pair to obtain the pre-trained image classification model, the pre-trained local attention model and the pre-trained global attention model, so that an output sample deep feature vector conforms to the real category.

6. The method of claim 5, wherein, The initial image classification model, the initial local attention model and the initial global attention model are iteratively trained by using the training sample pair, to obtain the pre-trained image classification model, the pre-trained local attention model and the pre-trained global attention model, wherein the output sample deep feature vector conforms to the true category, and the pre-training comprises the following steps: The sample image is subjected to feature processing by using the initial image classification model, the initial local attention model and the initial global attention model, to obtain a sample deep feature vector corresponding to the sample image; A training category corresponding to the sample deep feature vector is determined through a preset classification full connection layer; If the training category conforms to the true category, the training image classification model is determined as the pre-trained image classification model, the training local attention model is determined as the pre-trained local attention model, and the training global attention model is determined as the pre-trained global attention model; If the training category does not conform to the true category, the training parameters in the initial image classification model, the initial local attention model and the initial global attention model are adjusted, and the operation of the sample image is subjected to feature processing to obtain a sample deep feature vector corresponding to the sample image is re-executed until the training category conforms to the true category.

7. The method of claim 6, wherein, The sample image is subjected to feature processing by using the initial image classification model, the initial local attention model and the initial global attention model, to obtain a sample deep feature vector corresponding to the sample image, and the pre-training comprises the following steps: An initial feature of the sample image extracted by feature extraction is obtained by using the initial image classification model; A sample global feature vector corresponding to the sample image is obtained by performing pooling processing on the initial feature according to a preset pooling mode; A sample local feature vector corresponding to the sample image is generated by calculating the sample global feature vector through the initial local attention model; A sample deep feature vector corresponding to the sample image is determined by calculating the sample global feature vector and the sample local feature vector according to the initial global attention model.

8. An image processing apparatus characterized by comprising: The device comprises: A first obtaining module is configured to obtain an initial feature of a target image extracted by feature extraction by using a pre-trained image classification model; A pooling module is configured to perform pooling processing on the initial feature according to a preset pooling mode, to obtain a global feature vector corresponding to the target image; A first calculating module is configured to calculate the global feature vector through a pre-trained local attention model, to generate a local feature vector corresponding to the target image; A second calculating module is configured to calculate the global feature vector and the local feature vector according to a pre-trained global attention model, to determine a deep feature vector corresponding to the target image, so as to determine a category to which the target image belongs based on the deep feature vector; The pre-trained local attention model comprises a pre-trained full connection function, a pre-trained activation function and a pre-trained convolution kernel, and the first calculating module is further configured to The global feature vector is calculated by the pre-training full connection function and the pre-training activation function, to obtain a local weight corresponding to the global feature vector; According to the local weight and the initial feature, a local attention feature corresponding to the local weight is calculated; The local attention feature is convoluted by the pre-training convolution kernel to obtain a local feature vector corresponding to the target image.

9. The apparatus of claim 8, wherein, The first calculation module is further configured to: According to the dimension of the initial feature, the local weight is expanded to obtain an expanded local weight; According to the multiplication of the expanded local weight and the initial feature, a local attention feature corresponding to the local weight is obtained.

10. The apparatus of claim 8, wherein, The second calculation module is further configured to: The global feature vector and the local feature vector are respectively converted by using pre-training mapping parameters to obtain a first mapping vector corresponding to the local feature vector, a second mapping vector corresponding to the global feature vector, and a third mapping vector; According to a preset global attention model, the first mapping vector, the second mapping vector, and the third mapping vector are calculated to obtain a deep feature vector corresponding to the target image.

11. The apparatus of claim 8, wherein, The device further comprises: A conversion module is configured to convert the deep feature vector by using a preset symbol formula to obtain a feature vector representing the target image by using a hash code.

12. The apparatus of any one of claims 8-11, wherein, The device further comprises: A second acquisition module is configured to acquire a sample image and a real category corresponding to the sample image; A determination module is configured to take the sample image and the real category as a training sample pair; A training module is configured to iteratively train an initial image classification model, an initial local attention model, and an initial global attention model by using the training sample pair, so as to obtain the pre-trained image classification model, the pre-trained local attention model, and the pre-trained global attention model, in which the output sample deep feature vector conforms to the real category.

13. The apparatus of claim 12, wherein, The training module is further configured to: The sample image is processed by using the initial image classification model, the initial local attention model, and the initial global attention model to obtain a sample deep feature vector corresponding to the sample image; A training category corresponding to the sample deep feature vector is determined by using a preset classification full connection layer; If the training category conforms to the real category, the image classification model obtained by training is determined as the pre-trained image classification model, the local attention model obtained by training is determined as the pre-trained local attention model, and the global attention model obtained by training is determined as the pre-trained global attention model; If the training category does not conform to the real category, the training parameters in the initial image classification model, the initial local attention model, and the initial global attention model are adjusted, and the operation of processing the sample image to obtain the sample deep feature vector is re-executed until the training category conforms to the real category.

14. The apparatus of claim 13, wherein, The training module is further configured to: The initial image classification model is used to obtain sample initial features extracted from the sample image features; The sample initial features are pooled according to the preset pooling mode to obtain a sample global feature vector corresponding to the sample image; The sample global feature vector is calculated by the initial local attention model to generate a sample local feature vector corresponding to the sample image; The sample global feature vector and the sample local feature vector are calculated according to the initial global attention model to determine a sample deep feature vector corresponding to the sample image.

15. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by a processor to implement the image processing method of any one of claims 1-7.

16. An electronic device, comprising: Comprise: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the image processing method of any one of claims 1-7 by executing the executable instructions.

Images