Image feature selection method, device, equipment and storage medium

Abstract

The embodiments of the present application provide an image feature selection method, device, equipment and storage medium, which belongs to the field of artificial intelligence technology. The method includes: acquiring a target image; performing multi-level feature extraction processing on the target image to obtain at least two levels of initial features, wherein the initial features of any two levels are different; performing feature matching processing on the initial features to obtain at least two levels of matching features, wherein the matching features of any two levels are different; fusing the matching features to obtain the global features of the target image; obtaining the target spatial position features of the target image based on the global features, and obtaining the target channel features of the target image based on the global features; fusing the target spatial position features and the target channel features to obtain the target features of the target image. The embodiments of the present application can improve the comprehensiveness of image feature selection to meet the accuracy requirements of image representation.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image feature selection method, device, equipment and storage medium. Background Art

[0002] Currently, image recognition has become a hot topic in the field of computer vision. As the representation of an image, the accuracy of image features plays a decisive role in the accuracy of image recognition.

[0003] In related technologies, convolutional neural networks (CNNs) are usually used to select image features. However, CNNs can only extract local features of images, and selecting the extracted local features as image features is difficult to meet the accuracy requirements of image representation.

[0004] Therefore, how to improve the comprehensiveness of image feature selection to meet the accuracy requirements of image representation has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose an image feature selection method, device, equipment and storage medium, aiming to improve the comprehensiveness of image feature selection to meet the accuracy requirements of image representation.

[0006] To achieve the above object, a first aspect of an embodiment of the present application proposes an image feature selection method, the method comprising:

[0007] Get the target image;

[0008] Performing multi-level feature extraction processing on the target image to obtain initial features of at least two levels, wherein the initial features of any two levels are different;

[0009] Performing feature matching processing on the initial features to obtain matching features at least at two levels, wherein the matching features at any two levels are different;

[0010] Performing fusion processing on the matching features to obtain global features of the target image;

[0011] Acquire a target spatial position feature of the target image according to the global feature, and acquire a target channel feature of the target image according to the global feature;

[0012] The target spatial position feature and the target channel feature are fused to obtain the target feature of the target image.

[0013] In some embodiments, acquiring the target spatial position feature of the target image according to the global feature includes:

[0014] Performing channel dimension compression processing on the global features to obtain a spatial descriptor;

[0015] Acquire a first global compression feature of the target image according to the spatial descriptor;

[0016] Mapping the first global compression feature to at least two levels for embedding processing, respectively, to obtain at least two levels of subspace embedding features, wherein the subspace embedding features at any two levels are different;

[0017] Acquire at least two levels of spatial attention weights according to the first global compression feature and the subspace embedding feature;

[0018] The target spatial position feature is obtained according to the spatial attention weight and the matching feature.

[0019] In some embodiments, obtaining at least two levels of spatial attention weights respectively according to the first global compression feature and the subspace embedding feature includes:

[0020] Calculating a first correlation coefficient between the first global compression feature and the subspace embedding feature;

[0021] According to the first correlation coefficient, the spatial attention weight is obtained.

[0022] In some embodiments, acquiring the target spatial position feature of the target image according to the global feature includes:

[0023] Performing spatial dimension compression processing on the global features to obtain a channel descriptor;

[0024] Acquire a second global compression feature of the target image according to the channel descriptor;

[0025] Mapping the second global compression feature to at least two levels for embedding processing, respectively, to obtain at least two levels of channel feature maps, wherein the channel feature maps of any two levels are different;

[0026] Obtain at least two levels of channel attention weights according to the second global compression feature and the channel feature map;

[0027] The target channel feature is obtained according to the channel attention weight and the matching feature.

[0028] In some embodiments, obtaining at least two levels of channel attention weights according to the second global compression feature and the channel feature map includes:

[0029] Calculating a second correlation coefficient between the second global compression feature and the channel feature map;

[0030] According to the second correlation coefficient, the channel attention weight is obtained.

[0031] In some embodiments, performing feature matching processing on the initial features to obtain at least two levels of matching features includes:

[0032] Performing upsampling processing on the initial features to obtain at least two levels of enlarged features, wherein the enlarged features at any two levels are different;

[0033] Perform feature channel matching processing on the amplified feature to obtain the matching feature.

[0034] The multi-level feature extraction process is performed on the target image to obtain at least two levels of initial features, including:

[0035] The target image is input into a preset residual neural network, and a multi-level feature extraction process is performed on the target image through at least two convolutional layers of the residual neural network to obtain the initial features.

[0036] In some embodiments, after fusing the target spatial position feature and the target channel feature to obtain the target feature of the target image, the method further comprises:

[0037] The target image is classified according to the target feature.

[0038] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an image feature selection device, the device comprising:

[0039] An image acquisition module, used for acquiring a target image;

[0040] An extraction module, used for performing multi-level feature extraction processing on the target image to obtain initial features of at least two levels;

[0041] A matching module, used for performing feature matching processing on the initial features to obtain matching features of at least two levels;

[0042] A fusion module, used for fusing the matching features to obtain the global features of the target image;

[0043] A feature acquisition module, used to acquire a target spatial position feature of the target image according to the global feature, and to acquire a target channel feature of the target image according to the global feature;

[0044] The feature fusion module is used to fuse the target spatial position feature and the target channel feature to obtain the target feature of the target image.

[0045] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0046] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0047] The present application proposes an image feature selection method, device, equipment and storage medium. The image feature selection method obtains a target image; performs multi-level feature extraction processing on the target image to obtain at least two levels of initial features, wherein the initial features of any two levels are different; performs feature matching processing on the initial features to obtain at least two levels of matching features, wherein the matching features of any two levels are different; performs fusion processing on the matching features to obtain global features of the target image; obtains target spatial position features of the target image based on the global features, and obtains target channel features of the target image based on the global features; performs fusion processing on the target spatial position features and the target channel features to obtain the target features of the target image. In this way, by performing feature matching and cross-layer fusion processing on the initial features of different levels, the global features containing important information of the target image can be obtained more effectively. Then, guided by the global features, the target spatial position features and target channel features of the target image are selected from the spatial dimension and channel dimension respectively, thereby enhancing the representation ability of the target features from the spatial dimension and channel dimension. Then, the spatial position features and channel features are fused to obtain the target features, and the global dependencies between positions and channels at different levels are adaptively captured, so that the target features can better reflect the globally important characteristics of the target image, and provide a more comprehensive and richer image representation for the target image, thereby meeting the accuracy requirements of the image representation and improving the convenience and efficiency of image feature selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the image feature selection method provided in an embodiment of the present application;

[0049] Figure 2 It is a schematic diagram of the architecture of the preset MFSRNet provided in the embodiment of the present application;

[0050] Figure 3 yes Figure 1 A detailed flowchart of step S105 in ;

[0051] Figure 4 yes Figure 1 Another detailed flow chart of step S105 in FIG.

[0052] Figure 5 is a schematic diagram of the structure of an image feature selection device provided in an embodiment of the present application;

[0053] Figure 6 It is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] First, some nouns involved in this application are analyzed:

[0058] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0059] Based on this, the embodiments of the present application provide an image feature selection method, apparatus, device and storage medium, aiming to improve the comprehensiveness of image feature selection to meet the accuracy requirements of image representation.

[0060] The image feature selection method, device, equipment and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the image feature selection method in the embodiments of the present application is described.

[0061] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0062] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0063] The image feature selection method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The image feature selection method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the image feature selection method, etc., but is not limited to the above forms.

[0064] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0065] Figure 1 is an optional flow chart of the image feature selection method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0066] Step S101, acquiring a target image.

[0067] The target image refers to the image to be subjected to feature selection, and can be a picture or a video.

[0068] Exemplarily, the target image may be obtained by obtaining the target image from a local pre-stored image library, or by obtaining the target image uploaded by a user, or by receiving the target image sent by other devices.

[0069] Step S102, performing multi-level feature extraction processing on the target image to obtain at least two levels of initial features, wherein the initial features of any two levels are different.

[0070] Among them, the feature selection of the target image can be performed by presetting a multi-level feature selection recognition network (MFSRNet), thereby improving the accuracy, convenience and efficiency of feature selection.

[0071] See also Figure 2 , Figure 2 This is a schematic diagram of the architecture of the preset MFSRNet. The preset MFSRNet mainly includes a residual neural network (ResNet), a feature matcher, a position feature selector, and a channel position feature selector.

[0072] The technical solution of the embodiment of the present application calls the preset MFSRNet, performs multi-level feature extraction processing on the target image through ResNet, and obtains at least two levels of initial features; performs feature matching processing on the initial features through the feature matcher, and obtains at least two levels of matching features; performs fusion processing on the matching features through the preset MFSRNet to obtain the global features of the target image; obtains the target spatial position features of the target image according to the global features through the position feature selector, and obtains the target channel features of the target image according to the global features through the channel position feature selector; finally, the target spatial position features and the target channel features are fused through the preset MFSRNet to obtain the target features of the target image.

[0073] In this way, by using the preset MFSRNe network, the initial features of different levels are subjected to feature matching processing and cross-layer fusion processing, so as to more effectively obtain the global features containing important information of the target image, and then guided by the global features, the target spatial position features and target channel features of the target image are selected from the spatial dimension and channel dimension respectively, so as to enhance the representation ability of the target features from the spatial dimension and channel dimension, and then the spatial position features and channel features are fused to obtain the target features, so as to adaptively capture the global dependencies between positions and channels at different levels, so that the target features can better reflect the globally important characteristics of the target image, and provide a more comprehensive and richer image representation for the target image, thereby meeting the accuracy requirements of image representation and improving the convenience and efficiency of feature selection.

[0074] Specifically, in order to improve the comprehensiveness of feature selection, ResNet is used to perform multi-level feature extraction processing on the target image to obtain at least two levels of initial features. It is understandable that the initial features of any two levels are not the same.

[0075] In step S102 of some embodiments, in order to improve the granularity of multi-level feature extraction, the target image can be input into a preset residual neural network, and multi-level feature extraction processing is performed on the target image through at least two convolutional layers of the residual neural network to obtain initial features.

[0076] Among them, ResNet includes multiple convolutional layers, and the target image can be input into ResNet, and multi-level feature extraction processing can be performed on the target image through at least two convolutional layers of ResNet.

[0077] For example, please refer to Figure 2 , the last three convolutional layers of ResNet can be used to extract the initial features of the target image at three levels respectively, such as extracting the low-level initial features of the target image through ResNet's Conv3-X, extracting the mid-level initial features of the target image through ResNet's Conv4-X, and extracting the high-level initial features of the target image through ResNet's Conv5-X, thereby improving the granularity and comprehensiveness of multi-level feature extraction.

[0078] Step S103, performing feature matching processing on the initial features to obtain matching features at least at two levels, wherein the matching features at any two levels are different.

[0079] In order to facilitate feature selection, the initial features are processed by a feature matcher to obtain at least two levels of matching features. It is understandable that the matching features of any two levels are not the same.

[0080] In step S103 of some embodiments, the initial features may be upsampled to obtain at least two levels of magnified features; the magnified features may be subjected to feature channel matching to obtain matching features, wherein any two levels of magnified features are different.

[0081] The feature dimensions of initial features at different levels are different, and the feature dimensions include the number of feature channels, feature height, and feature width.

[0082] In order to adjust the matching features at different levels to the same feature dimension, the feature matcher first adjusts the initial features at different levels to the same feature height and the same feature width. Specifically, the initial features at different levels can be upsampled to expand the spatial resolution of the initial features at different levels to obtain magnified features at different levels. In this way, while adjusting the initial features at different levels to the same feature height and the same feature width, the original important information contained therein can also be retained.

[0083] Afterwards, the feature matcher adjusts the amplified features at different levels to the same number of feature channels. Specifically, a convolutional layer (e.g., The convolutional layer of the image processing unit is used to perform feature channel matching processing on the magnified features at different levels, so as to match the feature channel numbers of the magnified features at different levels to the same, thereby obtaining matching features at different levels. In this way, the matching features at different levels have the same feature dimension.

[0084] For example, Indicates the number of feature channels, Indicates the feature height, using represents the feature width, * represents the level, and the initial features of the corresponding level are expressed as , then, if the low-level initial features , the initial features of the middle level and high-level initial features , after feature matching processing by the feature matcher, low-level matching features can be obtained , the initial features of the middle level and high-level initial features ,in, In this way, the matching features at different levels They have the same number of feature channels, feature height and feature width, that is, matching features at different levels have the same feature dimension.

[0085] Step S104, performing fusion processing on the matching features to obtain the global features of the target image.

[0086] After obtaining the matching features at different levels, the matching features are fused through the preset MFSRNet to obtain the global features of the target image.

[0087] Exemplarily, the preset MFSRNet can use matrix addition to fuse at least two levels of matching features to obtain the global features of the target image. The formula of matrix addition is, for example:

[0088] ;

[0089] in, Represents global features, .

[0090] Step S105, acquiring target spatial position features of the target image according to the global features, and acquiring target channel features of the target image according to the global features.

[0091] The target spatial position features of the target image are obtained according to the global features through the position feature selector, and the target channel features of the target image are obtained according to the global features through the channel position feature selector.

[0092] See also Figure 3 In step S105 of some embodiments, the target spatial position features of the target image are obtained according to the global features, which may include but is not limited to sub-steps S1051 to S1055.

[0093] Sub-step S1051, compressing the global features in the channel dimension to obtain a spatial descriptor.

[0094] Among them, the position feature selector is used to adaptively select and fuse the position features of the same feature subspace in matching features of different levels using spatial attention under the guidance of global information, so as to enhance the representation ability of the target features of the target image from the channel dimension.

[0095] The position feature selector mainly consists of a feature embedding branch and a feature fusion branch. The process of selecting the target spatial position feature by the position feature selector mainly includes feature compression processing, feature embedding processing and feature operation processing.

[0096] For the feature compression process of the position feature selector, configure an adaptive feature compression operator for the position feature selector , the global features are compressed in the channel dimension to sufficiently retain the important information of the spatial position in the global features.

[0097] Specifically, the global features of the target image are input into the position feature selector so that the feature compression operator The global features are compressed in channel dimension to obtain a spatial descriptor with a global receptive field.

[0098] Feature compression operator for position feature selector The calculation formula is as follows:

[0099] ;

[0100] in, Represents global features middle The eigenvalue of the position; represents a spatial descriptor, such as ; is the identifier of the position feature selector; Represents global pooling.

[0101] Thus, a spatial descriptor is obtained that can represent the spatial position statistical information in the global feature.

[0102] Sub-step S1052, obtaining a first global compression feature of the target image according to the spatial descriptor.

[0103] Then, according to the spatial descriptor, the first global compression feature of the target image is obtained.

[0104] Specifically, since the spatial descriptor It is two-dimensional, and the spatial descriptor needs to be Flatten from two dimensions to one dimension to obtain the flattened space descriptor , ( Indicates the number of feature subspaces contained in the global feature, ,For example The value can be 784).

[0105] In order to improve computational efficiency, the spatial descriptor is flattened The input is further compressed into the fully connected layer to obtain the first low-dimensional global compression feature. , the calculation formula of the compression process is as follows:

[0106] ;

[0107] in, Represents the activation function ReLU; represents the fully connected layer parameter matrix; represents the flattened space descriptor; represents the first global compression feature, ; Represents the feature dimension reduction ratio parameter of the position feature selector, such as according to actual experiments, The possible value is 49.

[0108] In this way, after the compression processing of the position feature selector in the channel dimension, it can ensure that the first global compression feature Reserve enough space for globally important information.

[0109] Sub-step S1053, mapping the first global compression feature to at least two levels for embedding processing, to obtain at least two levels of subspace embedding features, wherein the subspace embedding features of any two levels are different.

[0110] It can be understood that the first global compression feature includes feature subspaces, each feature subspace represents the spatial position information of each position feature in the first global compression feature.

[0111] For the feature embedding process of the position feature selector, the first global compression feature is mapped to The feature subspaces are embedded to obtain the embedded features corresponding to each feature subspace at each level (defined as subspace embedding features).

[0112] Among them, the position feature selector has good scalability and can expand multiple feature embedding branches according to needs.

[0113] In this way, the first global compression feature can be embedded in the convolutional layers of different feature branches (such as The convolutional layer maps to The feature subspaces are embedded to obtain the embedding features of each subspace corresponding to each level.

[0114] The calculation formula of the embedding process is as follows:

[0115] ;

[0116] in, Indicates the first level of The subspace embedding features of feature subspaces, ; , Indicates the first level of feature subspaces; represents the first global compression feature; Indicates that Map to The parameter matrix of the corresponding convolutional layer in the feature subspace.

[0117] In this way, after the embedding processing of the position feature selector, not only the subsequent calculation complexity can be reduced, but also more comprehensive feature subspace information can be captured.

[0118] Step S1054, obtaining at least two levels of spatial attention weights according to the first global compression feature and the subspace embedding feature.

[0119] For the feature operation processing of the position feature selector, the spatial attention weight of each level is first calculated based on the first global compression feature and the subspace embedding feature.

[0120] In sub-step S1054 of some embodiments, a first correlation coefficient between the first global compression feature and the subspace embedding feature may be calculated; and based on the first correlation coefficient, the spatial attention weight may be obtained.

[0121] Exemplarily, the preset MFSRNet can use a feature concatenation method to calculate the correlation between the first global compression feature and each subspace embedding feature corresponding to each level, and a scalar is used to represent the size of the correlation.

[0122] Specifically, the first global compression feature and the first correlation coefficient between each subspace embedding feature corresponding to each level can be calculated. The formula for calculating the first correlation coefficient is as follows:

[0123] ;

[0124] in, Represents the first global compression feature With the * level Subspace embedding features The first correlation coefficient between .

[0125] After that, the preset MFSRNet calculates the importance of each subspace embedding feature corresponding to each level (defined as spatial attention weight, using express).

[0126] For example, the spatial attention weights of each subspace embedding feature corresponding to the low, medium and high levels of the target image are calculated according to the following formula:

[0127] ;

[0128] ;

[0129] ;

[0130] in, Represents exponential operation; Represents the low level The spatial attention weights of the subspace embedding features of the feature subspace; Indicates the middle level The spatial attention weights of the subspace embedding features of the feature subspace; Indicates high level The spatial attention weights of the subspace embedding features of the feature subspace.

[0131] Therefore, the preset MFSRNet, guided by the global information in the spatial dimension, adaptively determines the importance of each subspace embedding feature corresponding to each feature subspace at different levels through the spatial attention mechanism, providing a comprehensive and accurate basis for the subsequent positional selection and fusion of positional features of the same feature subspace from matching features at different levels.

[0132] Step S1055, obtaining the target spatial position features according to the spatial attention weight and matching features.

[0133] It can be understood that the matching features at each level include position features of multiple feature subspaces.

[0134] According to the spatial attention weight of each feature subspace at each level, the position features of each feature subspace of the matching features at each level are weighted and summed to obtain the spatial position features corresponding to each feature subspace of the target image.

[0135] For example, if the target image contains three levels: low, medium, and high, then the spatial position feature of the target image is calculated by the following formula:

[0136] ;

[0137] in, The target image The spatial position characteristics of feature subspaces; Represents low-level matching features Middle The position features of feature subspaces; Represents the mid-level matching features Middle The position features of feature subspaces; Represents high-level matching features Middle The position features of the feature subspace.

[0138] Finally, the spatial position features corresponding to all feature subspaces of the target image are fused through the feature fusion branch of the position feature selector to obtain the target spatial position features of the target image.

[0139] In this way, after the feature operation processing of the position feature selector, it can be ensured that the target spatial position feature contains important global spatial information.

[0140] As a result, it is possible to retain the target spatial position features that can reflect important global spatial information, thereby enhancing the characterization capability of the target features of the target image.

[0141] See also Figure 4 In step S105 of some embodiments, the target channel features of the target image are obtained according to the global features, which may include but is not limited to sub-steps S2051 to S2055.

[0142] Sub-step S2051, compressing the global features in spatial dimension to obtain a channel descriptor.

[0143] Among them, the channel feature selector is used to adaptively select important channel feature maps using channel attention under the guidance of global information, so as to enhance the representation capability of target features of the target image from the spatial dimension.

[0144] The channel feature selector mainly consists of a feature embedding branch and a feature fusion branch. The process of selecting channel features by the channel feature selector mainly includes feature compression processing, feature embedding processing and feature operation processing.

[0145] For the feature compression processing of the channel feature selector, an adaptive feature compression operator is also configured for the channel feature selector. , compress the spatial dimension of the global features to sufficiently retain the channel information in the global features.

[0146] Specifically, the global features of the target image are input into the channel feature selector so that the feature compression operator The global features are compressed in spatial dimension to obtain channel descriptors.

[0147] Exemplary, feature compression operator of channel feature selector For example:

[0148] ;

[0149] in, Represents global features middle The eigenvalue of the position; represents the channel descriptor, ; is the identifier of the channel feature selector; Represents global pooling.

[0150] Sub-step S2052, obtaining a second global compression feature of the target image according to the channel descriptor.

[0151] According to the channel descriptor, a second global compression feature of the target image in the channel dimension is obtained.

[0152] Specifically, the channel descriptor is mapped to a low-dimensional feature space, and further compression processing is performed to obtain the second global compression feature , the calculation formula of the compression process is as follows:

[0153] ;

[0154] in, Represents the activation function ReLU; represents the mapping parameter matrix; represents the second global compression feature, ; Represents the feature dimension reduction ratio parameter of the channel feature selector. According to actual experiments, The possible value is 49.

[0155] In this way, after the compression processing of the channel feature selector in the spatial dimension, the second global compression feature can be guaranteed Keep enough important information about the channel.

[0156] Sub-step S2053, mapping the second global compression feature to at least two levels for embedding processing, to obtain at least two levels of channel feature maps, wherein the channel feature maps of any two levels are different.

[0157] It can be understood that the second global compression feature includes feature channels.

[0158] For the feature embedding process of the channel feature selector, the second global compression feature can be mapped to The feature channels are embedded to obtain the channel embedding features (defined as channel feature maps) corresponding to each feature channel at each level.

[0159] Among them, the calculation formula for the embedding processing of the channel feature selector is as follows:

[0160] ;

[0161] in, Indicates the first level of The channel feature map corresponding to the feature channels is ; represents the second global compression feature; Indicates that Map to The weight matrix of feature channels; Indicates feature channels.

[0162] In this way, after the embedding processing of the channel feature selector, not only the subsequent calculation complexity can be reduced, but also more comprehensive feature channel information can be captured.

[0163] Sub-step S2054, obtaining at least two levels of channel attention weights based on the second global compression feature and the channel feature map.

[0164] For the feature operation processing of the channel feature selector, the channel attention weight of each level is first calculated based on the second global compression feature and the channel feature map.

[0165] In sub-step S2054 of some embodiments, a second correlation coefficient between the second global compression feature and the channel feature map may be calculated; and the channel attention weight may be obtained based on the second correlation coefficient.

[0166] Furthermore, the preset MFSRNet can use a feature concatenation method to calculate the second global compression feature and the second correlation coefficient between the channel feature maps corresponding to each feature channel of each level. The formula for calculating the second correlation coefficient is as follows:

[0167] ;

[0168] in, Represents the second correlation coefficient, which is the second global compression feature With the * level The channel feature map corresponding to the feature channel The correlation coefficient between .

[0169] Afterwards, according to the second global compression characteristic, the preset MFSRNet uses the softmax function to calculate the second correlation coefficient Perform channel-by-channel normalization to obtain the importance of each channel feature map corresponding to each feature channel at each level (defined as channel attention weight, using express).

[0170] For example, the spatial attention weight of each channel feature map corresponding to the low, medium and high levels of the target image is calculated according to the following formula:

[0171] ;

[0172] ;

[0173] ;

[0174] in, Represents exponential operation; Represents the low level The channel attention weight of the channel feature map of the feature channel; Indicates the middle level The channel attention weight of the channel feature map of the feature channel; Indicates high level The channel attention weights of the channel feature maps of feature channels.

[0175] Therefore, the preset MFSRNet, guided by the global information in the channel dimension, adaptively determines the importance of each channel feature map corresponding to each feature channel at different levels through the channel attention mechanism, providing a comprehensive and accurate basis for the subsequent selection and fusion of channel features from matching features at different levels.

[0176] Sub-step S2055, obtaining channel features according to the channel attention weights and matching features.

[0177] According to the channel attention weight of each channel feature map at each level, the channel features corresponding to each feature channel of the matching features at each level are weighted summed to obtain the candidate channel features corresponding to each feature channel of the target image.

[0178] For example, if the target image contains three levels: low, medium, and high, then the candidate channel features corresponding to each channel of the target image are calculated by the following formula:

[0179] ;

[0180] in, The target image Candidate channel features of feature channels; Represents low-level matching features Middle Channel features of feature channels; Represents the mid-level matching features Middle Channel features of feature channels; Represents high-level matching features Middle The channel characteristics of the feature channels.

[0181] Finally, the candidate channel features corresponding to all feature channels of the target image are fused through the feature fusion branch of the channel feature selector to obtain the target channel features of the target image.

[0182] In this way, after the feature operation processing of the channel feature selector, it can be ensured that the target channel feature contains the global important information of the channel.

[0183] As a result, it is possible to retain the target channel features that can reflect the global important information of the channel, thereby enhancing the characterization capability of the target features of the target image.

[0184] Step S106, fusing the target spatial position feature and the target channel feature to obtain the target feature of the target image.

[0185] Finally, the target spatial position features and target channel features of the target image are fused by the preset MFSRNet to obtain the target features of the target image.

[0186] Specifically, the preset MFSRNet can substitute the target spatial position features and target channel features of the target image into the preset fusion calculation formula , the target features of the target image are calculated, where represents the target feature, represents the target spatial location feature, represents the target channel characteristics, represents the feature height, Indicates the feature width.

[0187] This enables more comprehensive and richer target features to be determined for the target image.

[0188] In some embodiments, after step S102, the target image may be classified according to the target features.

[0189] The target features of the target image can be classified using a pre-trained image classification model to obtain the category of the target image.

[0190] Exemplarily, the trained image classification model is, for example, a pre-trained YOLO (You Only Look Once) network. The target feature is input into the pre-trained YOLO network, and the pre-trained YOLO network divides the target feature into multiple grids. For each grid, the pre-trained YOLO network predicts the results of B borders, and the predicted output is the confidence of the border, the coordinates of the border, and the probability of each border in multiple categories. Afterwards, the confidence of each border is compared with the preset confidence threshold, and the borders less than the preset confidence threshold are filtered out, wherein the preset confidence threshold is determined during the training of the YOLO network. For each remaining border whose confidence is greater than the preset confidence threshold, its confidence is multiplied by the probability of each category to obtain the probability score of each category of each remaining border, and the remaining border with the highest probability score is retained by non-maximum suppression. The category with the highest probability score is the category of the retained remaining border, that is, the category of the target image.

[0191] Therefore, the target image is classified based on more comprehensive and richer target features, so that efficient classification of the target image is achieved and the classification accuracy of the target image is improved.

[0192] The image feature selection method provided in the embodiment of the present application obtains a target image; performs multi-level feature extraction processing on the target image to obtain at least two levels of initial features, wherein the initial features of any two levels are different; performs feature matching processing on the initial features to obtain at least two levels of matching features, wherein the matching features of any two levels are different; performs fusion processing on the matching features to obtain global features of the target image; obtains target spatial position features of the target image based on the global features, and obtains target channel features of the target image based on the global features; performs fusion processing on the target spatial position features and the target channel features to obtain target features of the target image. By performing feature matching and cross-layer fusion processing on the initial features of different levels, the global features containing important information of the target image can be obtained more effectively. Then, guided by the global features, the target spatial position features and target channel features of the target image are selected from the spatial dimension and channel dimension respectively, so as to enhance the representation ability of the target features from the spatial dimension and channel dimension. Then, the spatial position features and channel features are fused to obtain the target features, so as to adaptively capture the global dependencies between positions and channels at different levels, so that the target features can better reflect the globally important characteristics of the target image, and provide a more comprehensive and richer image representation for the target image, thereby meeting the accuracy requirements of image representation and improving the convenience and efficiency of feature selection.

[0193] See also Figure 5 The embodiment of the present application further provides an image feature selection device, which can implement the above-mentioned image feature selection method, and the device includes:

[0194] An image acquisition module, used for acquiring a target image;

[0195] An extraction module is used to perform multi-level feature extraction processing on the target image to obtain at least two levels of initial features;

[0196] A matching module, used for performing feature matching processing on the initial features to obtain matching features of at least two levels;

[0197] A fusion module is used to fuse the matching features to obtain the global features of the target image;

[0198] A feature acquisition module, used to acquire target spatial position features of a target image according to global features, and to acquire target channel features of the target image according to global features;

[0199] The feature fusion module is used to fuse the target spatial position features and the target channel features to obtain the target features of the target image.

[0200] The specific implementation of the image feature selection device is basically the same as the specific implementation of the above-mentioned image feature selection method, and will not be repeated here.

[0201] The embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned image feature selection method when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0202] See also Figure 6 , Figure 6 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0203] The processor 601 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0204] The memory 602 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 602, and the processor 601 calls and executes the image feature selection method of the embodiment of this application;

[0205] Input / output interface 603, used to implement information input and output;

[0206] Communication interface 604, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0207] A bus 605 that transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0208] The processor 601 , the memory 602 , the input / output interface 603 and the communication interface 604 are connected to each other in communication within the device via a bus 605 .

[0209] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image feature selection method is implemented.

[0210] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0211] The image feature selection method, apparatus, device and storage medium provided in the embodiments of the present application perform feature matching processing and cross-layer fusion processing on initial features of different levels, which can more effectively obtain global features containing important information of the target image, and then use the global features as a guide to select target spatial position features and target channel features of the target image from the spatial dimension and channel dimension respectively, thereby enhancing the representation capability of the target features from the spatial dimension and channel dimension, and then fuse the spatial position features and channel features to obtain the target features, so as to adaptively capture the global dependencies between positions and channels at different levels, so that the target features can better reflect the global important characteristics of the target image, and provide a more comprehensive and richer image representation for the target image, thereby meeting the accuracy requirements of the image representation, and improving the convenience and efficiency of feature selection.

[0212] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0213] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0214] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0215] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0216] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0217] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0218] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0219] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0220] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0221] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0222] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for selecting image features, characterized in that: The method comprises: Get the target image; Performing multi-level feature extraction processing on the target image to obtain initial features of at least two levels, wherein the initial features of any two levels are different; Performing feature matching processing on the initial features to obtain matching features at least at two levels, wherein the matching features at any two levels are different; Performing fusion processing on the matching features to obtain global features of the target image; Acquire the target spatial position feature of the target image according to the global feature, and acquire the target channel feature of the target image according to the global feature; wherein acquiring the target spatial position feature of the target image according to the global feature comprises: Performing channel dimension compression processing on the global features to obtain a spatial descriptor; Acquire a first global compression feature of the target image according to the spatial descriptor; Mapping the first global compression feature to at least two levels for embedding processing, respectively, to obtain at least two levels of subspace embedding features, wherein the subspace embedding features at any two levels are different; Acquire at least two levels of spatial attention weights according to the first global compression feature and the subspace embedding feature; Acquire the target spatial position feature according to the spatial attention weight and the matching feature; The target spatial position feature and the target channel feature are fused to obtain the target feature of the target image.

2. The method according to claim 1, characterized in that The obtaining at least two levels of spatial attention weights respectively according to the first global compression feature and the subspace embedding feature includes: Calculating a first correlation coefficient between the first global compression feature and the subspace embedding feature; According to the first correlation coefficient, the spatial attention weight is obtained.

3. The method according to claim 1, characterized in that The step of acquiring the target spatial position feature of the target image according to the global feature comprises: Performing spatial dimension compression processing on the global features to obtain a channel descriptor; Acquire a second global compression feature of the target image according to the channel descriptor; Mapping the second global compression feature to at least two levels for embedding processing, respectively, to obtain at least two levels of channel feature maps, wherein the channel feature maps of any two levels are different; Obtain at least two levels of channel attention weights according to the second global compression feature and the channel feature map; The target channel feature is obtained according to the channel attention weight and the matching feature.

4. The method according to claim 3, characterized in that The acquiring at least two levels of channel attention weights according to the second global compression feature and the channel feature map comprises: Calculating a second correlation coefficient between the second global compression feature and the channel feature map; According to the second correlation coefficient, the channel attention weight is obtained.

5. The method according to claim 1, characterized in that The multi-level feature extraction process is performed on the target image to obtain at least two levels of initial features, including: Inputting the target image into a preset residual neural network, performing multi-level feature extraction processing on the target image through at least two convolutional layers of the residual neural network to obtain the initial features; The performing feature matching processing on the initial features to obtain at least two levels of matching features includes: Performing upsampling processing on the initial features to obtain at least two levels of enlarged features, wherein the enlarged features at any two levels are different; Perform feature channel matching processing on the amplified feature to obtain the matching feature.

6. The method according to claim 1, characterized in that After fusing the target spatial position feature and the target channel feature to obtain the target feature of the target image, the method further comprises: The target image is classified according to the target feature.

7. An image feature selection device, characterized in that: The device comprises: An image acquisition module, used for acquiring a target image; An extraction module, used for performing multi-level feature extraction processing on the target image to obtain initial features of at least two levels; A matching module, used for performing feature matching processing on the initial features to obtain matching features of at least two levels; A fusion module, used for fusing the matching features to obtain the global features of the target image; A feature acquisition module is used to acquire the target spatial position feature of the target image according to the global feature, and to acquire the target channel feature of the target image according to the global feature; wherein acquiring the target spatial position feature of the target image according to the global feature includes: Performing channel dimension compression processing on the global features to obtain a spatial descriptor; Acquire a first global compression feature of the target image according to the spatial descriptor; Mapping the first global compression feature to at least two levels for embedding processing, respectively, to obtain at least two levels of subspace embedding features, wherein the subspace embedding features at any two levels are different; Acquire at least two levels of spatial attention weights according to the first global compression feature and the subspace embedding feature; Acquire the target spatial position feature according to the spatial attention weight and the matching feature; The feature fusion module is used to fuse the target spatial position feature and the target channel feature to obtain the target feature of the target image.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the image feature selection method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image feature selection method according to any one of claims 1 to 6 is implemented.

Images