Feature map processing method, image recognition method and related device

Abstract

The application provides a feature map processing method, an image recognition method and related devices, and relates to the technical field of image processing. After obtaining a global feature map of a to-be-recognized image, low-order image information and high-order image information of the global feature map are extracted through calling a target channel attention model for deep learning, to obtain a low-order channel attention vector corresponding to the low-order image information and a high-order channel attention vector corresponding to the high-order image information. Then, the global feature map of the to-be-recognized image is subjected to attention vector fusion and weighting processing according to the low-order channel attention vector and the high-order channel attention vector, to obtain an expected feature map of the to-be-recognized image. Thus, in the visual recognition process, the channel attention mechanism is introduced to fuse the low-order image information and the high-order image information of the to-be-recognized image for image feature extraction, so as to improve the quality of the image features extracted from the to-be-recognized image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a feature map processing method, an image recognition method, and related apparatus. Background Technology

[0002] With the continuous development of science and technology, image processing technology is increasingly widely used in fields such as video surveillance and public security, typically employing it to achieve computer vision recognition functions such as face recognition, pedestrian recognition, vehicle recognition, and object recognition. In implementing these various computer vision recognition functions, the accuracy of each function largely depends on the quality of the image features extracted from the image to be recognized; the better the quality of the extracted image features, the better the accuracy of the corresponding visual recognition function. Therefore, effectively improving the quality of image features extracted from the image to be recognized during the visual recognition process is a crucial technical challenge in contemporary computer vision recognition technology. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a feature map processing method and apparatus, an image recognition method and apparatus, a computer device and a readable storage medium, which can introduce a channel attention mechanism in the visual recognition process to fuse low-order image information and high-order image information of the image to be recognized for image feature extraction, so as to improve the quality of the image features extracted from the image to be recognized.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0005] Firstly, this application provides a feature map processing method, the method comprising:

[0006] Obtain the global feature map of the image to be identified;

[0007] The target channel attention model is invoked to extract low-order and high-order image information from the global feature map and perform deep learning to obtain a low-order channel attention vector corresponding to the low-order image information and a high-order channel attention vector corresponding to the high-order image information.

[0008] Based on the low-order channel attention vector and the high-order channel attention vector, the global feature map is subjected to attention vector fusion and weighting processing to obtain the expected feature map of the image to be identified.

[0009] In an optional implementation, the target channel attention model includes a low-order information learning branch model and a high-order information learning branch model. The step of calling the target channel attention model to extract low-order and high-order image information from the global feature map and performing deep learning to obtain a low-order channel attention vector corresponding to the low-order image information and a high-order channel attention vector corresponding to the high-order image information includes:

[0010] The global feature map is synchronously input into the low-order information learning branch model and the high-order information learning branch model;

[0011] The low-order information learning branch model is driven to extract first-order image information from the input global feature map and learn it to obtain the low-order channel attention vector;

[0012] The higher-order information learning branch model is driven to extract second-order image information from the input global feature map and learn it to obtain the higher-order channel attention vector.

[0013] In an optional implementation, the low-order information learning branch model includes a global average pooling layer, a first fully connected layer, and a second fully connected layer connected in sequence. The first fully connected layer and the second fully connected layer use a ReLU function as the activation function, and a Sigmoid function is used at the output of the second fully connected layer. The step of driving the low-order information learning branch model to extract first-order image information from the input global feature map and learn the low-order channel attention vector includes:

[0014] The global average pooling layer is invoked to perform feature map compression processing on the global feature map at the spatial dimension level to obtain the initial channel attention vector of the global feature map;

[0015] The first fully connected layer and the ReLU function are invoked to perform vector upsizing on the initial channel attention vector at the channel dimension level to obtain the intermediate channel attention vector.

[0016] The second fully connected layer and the Sigmoid function are invoked to perform vector dimensionality reduction on the intermediate channel attention vector at the channel dimension level to obtain the low-order channel attention vector.

[0017] In an optional implementation, the higher-order information learning branch model includes a feature map unfolding module, a similarity matrix construction module, and an attention vector extraction module connected in sequence. The step of driving the higher-order information learning branch model to extract second-order image information from the input global feature map for learning, and obtaining the higher-order channel attention vector, includes:

[0018] The feature map unfolding module is invoked to perform feature map unfolding processing on the global feature map along the channel dimension direction to obtain a two-dimensional feature matrix corresponding to the global feature map at the channel dimension level, wherein each row of the two-dimensional feature matrix is ​​used to represent the image features of the global feature map at the corresponding channel;

[0019] The similarity matrix construction module is invoked to calculate the vector similarity between the matrix vectors in each row of the two-dimensional feature matrix, and a matrix is ​​constructed based on all the calculated vector similarities to obtain the feature similarity matrix of the global feature map at the channel dimension level.

[0020] The attention vector extraction module is invoked to perform element-wise mean operation on each column of the feature similarity matrix to obtain the higher-order channel attention vector composed of the actual element-wise mean of each column of the matrix vector.

[0021] In an optional implementation, the step of performing attention vector fusion and weighting processing on the global feature map based on the low-order channel attention vector and the high-order channel attention vector to obtain the desired feature map of the image to be recognized includes:

[0022] Calculate the average channel attention vector between the low-order channel attention vector and the high-order channel attention vector;

[0023] The desired feature map is obtained by performing channel-by-channel multiplication on the global feature map and the average channel attention vector.

[0024] Secondly, this application provides a feature map processing apparatus, the apparatus comprising:

[0025] The global feature acquisition module is used to acquire the global feature map of the image to be recognized;

[0026] The channel vector extraction module is used to call the target channel attention model to extract low-order and high-order image information of the global feature map for deep learning, and obtain low-order channel attention vectors corresponding to the low-order image information and high-order channel attention vectors corresponding to the high-order image information.

[0027] The vector fusion weighting module is used to perform attention vector fusion weighting processing on the global feature map based on the low-order channel attention vector and the high-order channel attention vector to obtain the expected feature map of the image to be recognized.

[0028] Thirdly, this application provides an image recognition method, the method comprising:

[0029] Global feature extraction is performed on the image to be identified to obtain the global feature map of the image to be identified;

[0030] The target channel attention model is invoked to process the global feature map to obtain a desired feature map corresponding to the global feature map, wherein the desired feature map is obtained by processing the feature map processing method described in any of the foregoing embodiments.

[0031] Extract the object features of the target object at the location in the image to be identified from the desired feature map.

[0032] Fourthly, this application provides an image recognition device, the device comprising:

[0033] The global feature extraction module is used to extract global features from the image to be recognized, thereby obtaining a global feature map of the image to be recognized.

[0034] The image feature processing module is used to call the target channel attention model to perform feature map processing on the global feature map to obtain a desired feature map corresponding to the global feature map, wherein the desired feature map is obtained by processing the feature map processing method described in any of the foregoing embodiments.

[0035] The object feature extraction module is used to extract the object features of the target object in the image to be identified from the desired feature map.

[0036] Fifthly, this application provides a computer device including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the feature map processing method described in any of the foregoing embodiments, or to implement the image recognition method described in the foregoing embodiments.

[0037] Sixthly, this application provides a readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the feature map processing method described in any one of the foregoing embodiments, or implements the image recognition method described in the foregoing embodiments.

[0038] In this case, the beneficial effects of the embodiments of this application may include the following:

[0039] After obtaining the global feature map of the image to be recognized, this application extracts low-order and high-order image information from the global feature map by calling the target channel attention model and performs deep learning to obtain low-order channel attention vectors corresponding to the low-order image information and high-order channel attention vectors corresponding to the high-order image information. Then, based on the low-order and high-order channel attention vectors, the global feature map of the image to be recognized is subjected to attention vector fusion and weighting processing to obtain the expected feature map of the image to be recognized. Thus, a channel attention mechanism is introduced into the visual recognition process to fuse the low-order and high-order image information of the image to be recognized for image feature extraction, thereby improving the quality of the image features extracted from the image to be recognized.

[0040] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A simplified diagram illustrating the extraction of image features using an existing channel attention model;

[0043] Figure 2 A schematic diagram of the composition of a computer device provided in the embodiments of this application;

[0044] Figure 3 A schematic flowchart illustrating the feature map processing method provided in this application embodiment;

[0045] Figure 4 A schematic diagram of the target channel attention model provided in the embodiments of this application;

[0046] Figure 5 for Figure 3 A flowchart illustrating the sub-steps included in step S320;

[0047] Figure 6 for Figure 3 A flowchart illustrating the sub-steps included in step S330;

[0048] Figure 7 A schematic flowchart illustrating the image recognition method provided in this application embodiment;

[0049] Figure 8This is a schematic diagram of the composition of the feature map processing apparatus provided in the embodiments of this application;

[0050] Figure 9 This is a schematic diagram illustrating the composition of the image recognition device provided in the embodiments of this application.

[0051] Icons: 10-Computer equipment; 11-Memory; 12-Processor; 13-Communication unit; 100-Feature map processing device; 200-Image recognition device; 110-Global feature acquisition module; 120-Channel vector extraction module; 130-Vector fusion weighting module; 210-Global feature extraction module; 220-Image feature processing module; 230-Object feature extraction module. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0054] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0055] In the description of this application, it should be understood that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art will understand the specific meaning of the above terms in this application based on the specific circumstances.

[0056] Through diligent research, the applicant discovered that existing solutions for improving the quality of extracted image features from an image to be identified typically employ channel attention mechanisms to acquire low-level image information for feature extraction. Specifically, please refer to... Figure 1 The existing channel attention model shown extracts image features. Current implementations require using a deep learning model to extract the global feature map of the image to be recognized (i.e.,...). Figure 1 Global feature map F∈R C×H×W Where C, H, and W represent the number of channels, height, and width of the global feature map, respectively. Then, the global feature map of the image to be recognized is input into an existing channel attention model. The squeze module in the existing channel attention model performs a "compression" operation on the global feature map at the spatial dimension level (i.e., by performing global average pooling on the global feature map; this process can use F...). sq (This is represented) to obtain the corresponding initial channel attention vector f∈R C Then, the excitation module in the existing channel attention model performs a "purification" operation on the initial channel attention vector at the channel dimension level (that is, using two fully connected layers with their respective activation functions to perform continuous "dimensionality increase" and "dimensionality reduction" on the initial channel attention vector; this process can use F... ex (This process involves) representing the low-order image information (i.e., first-order image information) of the image to be recognized, resulting in a low-order channel attention vector. Next, the low-order channel attention vector is multiplied channel by channel by channel with the global feature map (i.e., Figure 1 F in scale (operation), ultimately obtaining a feature map of the image to be identified that carries low-level image information (i.e., Figure 1 Feature map in This is to improve the quality of the extracted image features in the image to be identified.

[0057] In this process, the actual calculation formula for the initial channel attention vector under the "purification" operation is expressed as follows:

[0058]

[0059] The actual calculation formula for the aforementioned low-order channel attention vector under the "purification" operation is expressed as follows:

[0060]

[0061] Where f represents the initial channel attention vector, F h,w Image features used to represent the global feature map of the image to be identified at the spatial dimension H×W level. W1 is used to represent the low-order channel attention vector, δ is used to represent the fully connected layer that achieves "dimensionality increase", δ is used to represent the activation function (i.e., the ReLU function) that achieves "dimensionality increase", W2 is used to represent the fully connected layer that achieves "dimensionality reduction", and σ is used to represent the activation function (i.e., the Sigmoid function) that achieves "dimensionality reduction".

[0062] It is worth noting that this implementation scheme can only ensure that the extracted feature map of the image to be recognized carries low-level image information, resulting in insufficient image feature information in the corresponding extracted feature map. This cannot effectively improve the quality of the image features extracted from the image to be recognized, let alone improve the recognition accuracy of the image recognition function.

[0063] In this context, to address the aforementioned issues, embodiments of this application provide a feature map processing method and apparatus, an image recognition method and apparatus, a computer device, and a readable storage medium. These methods introduce a channel attention mechanism during visual recognition to fuse low-order and high-order image information of the image to be recognized for image feature extraction. This improves the quality of the extracted image features, ensures that the extracted image features have good robustness and recognizability, and simultaneously enhances the recognition accuracy of the visual recognition function.

[0064] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features described herein can be combined with each other.

[0065] Please refer to Figure 2 , Figure 2This is a schematic diagram illustrating the composition of the computer device 10 provided in this embodiment. In this embodiment, the computer device 10 may be equipped with a deep neural network model, and visual recognition functions may be implemented through this deep neural network model. The visual recognition functions may be any one of image recognition functions such as face recognition, pedestrian recognition, vehicle recognition, and object recognition. The computer device 10 may be, but is not limited to, a smartphone, robot, laptop computer, personal computer, server, etc.

[0066] In this embodiment, the computer device 10 may include a memory 11, a processor 12, and a communication unit 13. The memory 11, the processor 12, and the communication unit 13 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0067] In this embodiment, the memory 11 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The memory 11 is used to store computer programs, and the processor 12 can execute the computer programs accordingly after receiving execution instructions.

[0068] In this embodiment, the processor 12 can be an integrated circuit chip with signal processing capabilities. The processor 12 can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0069] In this embodiment, the communication unit 13 is used to establish a communication connection between the computer device 10 and other electronic devices via a network, and to send and receive data through the network, wherein the network includes wired communication networks and wireless communication networks. For example, the computer device 10 can obtain images to be identified uploaded by other electronic devices through the communication unit 13, wherein the electronic devices may be, but are not limited to, monitoring devices, cameras, etc.

[0070] Optionally, in this embodiment, the deep neural network model mounted on the computer device 10 may embed a target channel attention model, and the computer device 10 may further include a feature map processing device 100. The feature map processing device 100 may include at least one software function module that can be stored in the memory 11 in the form of software or firmware or embedded in the operating system of the computer device 10. The processor 12 can be used to execute executable modules stored in the memory 11, such as the software function modules and computer programs included in the feature map processing device 100. The computer device 10 can use the feature map processing device 100 to introduce a channel attention mechanism during visual recognition to fuse low-order and high-order image information of the image to be recognized for image feature extraction, thereby improving the quality of the extracted image features, ensuring that the extracted image features have good robustness and recognizability, and simultaneously improving the recognition accuracy of the visual recognition function.

[0071] Optionally, in this embodiment, the computer device 10 may further include an image recognition device 200. The image recognition device 200 may include at least one software functional module that can be stored in the memory 11 in the form of software or firmware or embedded in the operating system of the computer device 10. The processor 12 can be used to execute executable modules stored in the memory 11, such as the software functional modules and computer programs included in the image recognition device 200. The computer device 10 can improve the recognition accuracy of visual recognition functions by utilizing a target channel attention model through the image recognition device 200.

[0072] Understandable, Figure 2 The block diagram shown is only a schematic diagram of one configuration of the computer device 10. The computer device 10 may also include components such as... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown. Figure 2 The components shown can be implemented using hardware, software, or a combination thereof.

[0073] In this application, to ensure that the computer device 10 can introduce a channel attention mechanism to fuse low-order and high-order image information of the image to be recognized during the visual recognition process for image feature extraction, thereby improving the quality of the extracted image features and ensuring that the extracted image features have good robustness and recognizability, and simultaneously improving the recognition accuracy of the visual recognition function, this application provides a feature map processing method to achieve the aforementioned objectives. The feature map processing method provided in this application will be described in detail below.

[0074] Please refer to Figure 3 , Figure 3 This is a schematic flowchart of a feature map processing method provided in an embodiment of this application. In this embodiment, the feature map processing method may include steps S310 to S330.

[0075] Step S310: Obtain the global feature map of the image to be identified.

[0076] In this embodiment, the computer device 10 can input the image to be identified into its own deep neural network model, so as to extract the global feature map of the image to be identified through the deep neural network model.

[0077] Step S320: Call the target channel attention model to extract low-order and high-order image information of the global feature map and perform deep learning to obtain the low-order channel attention vector corresponding to the low-order image information and the high-order channel attention vector corresponding to the high-order image information.

[0078] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the target channel attention model provided in this embodiment. In this embodiment, the target channel attention model may include a low-order information learning branch model and a high-order information learning branch model. The low-order information learning branch model is used to extract low-order image information from the global feature map of the image to be recognized and perform deep learning to output a low-order channel attention vector corresponding to the low-order image information. The high-order information learning branch model is used to extract high-order image information from the global feature map of the image to be recognized and perform deep learning to output a high-order channel attention vector corresponding to the low-order image information.

[0079] The low-order information learning branch model may include a global average pooling layer, a first fully connected layer, and a second fully connected layer connected in sequence. The first fully connected layer and the second fully connected layer use the ReLU function as the activation function, and the output of the second fully connected layer uses the Sigmoid function as the activation function to ensure that the low-order information learning branch model can extract the first-order image information of the global feature map of the image to be recognized as low-order image information for deep learning.

[0080] The higher-order information learning branch model may include a feature map unfolding module, a similarity matrix construction module, and an attention vector extraction module connected in sequence, to ensure that the higher-order information learning branch model can extract the second-order image information of the global feature map of the image to be identified as higher-order image information for deep learning.

[0081] In this case, after obtaining the global feature map of the image to be recognized, the computer device 10 can synchronously input the global feature map into the low-order information learning branch model and the high-order information learning branch model of the target channel attention model, and drive the low-order information learning branch model and the high-order information learning branch model to perform deep learning respectively, so as to obtain the low-order channel attention vector carrying the low-order image information of the image to be recognized and the high-order channel attention vector carrying the high-order image information of the image to be recognized, wherein the target channel attention model is embedded in the neural learning network model carried by the computer device 10.

[0082] Alternatively, please refer to Figure 5 , Figure 5 yes Figure 3 The flowchart of step S320 is shown below. In this embodiment, step S320 may include sub-steps S321 to S323 to ensure that the target channel attention model can extract low-order image information and high-order image information of the image to be recognized to construct corresponding channel attention vectors respectively.

[0083] Sub-step S321: Synchronously input the global feature map into the low-order information learning branch model and the high-order information learning branch model.

[0084] Sub-step S322 drives the low-order information learning branch model to extract the first-order image information of the input global feature map for learning, and obtains the low-order channel attention vector.

[0085] In this embodiment, the step of the computer device 10 driving the low-order information learning branch model to extract first-order image information of the global feature map to obtain the low-order channel attention vector may include:

[0086] Sub-step a: Call the global average pooling layer to compress the global feature map in the spatial dimension, and obtain the initial channel attention vector of the global feature map.

[0087] The initial channel attention vector of the global feature map can be calculated using the following formula:

[0088]

[0089] Where f represents the initial channel attention vector, F h,w The global feature map used to represent the image features in the spatial dimension H×W of the image to be identified.

[0090] Sub-step b involves calling the first fully connected layer and the ReLU function to perform vector upsizing on the initial channel attention vector at the channel dimension level, resulting in the intermediate channel attention vector.

[0091] The formula for calculating the intermediate channel attention vector can be expressed as follows:

[0092]

[0093] Where f represents the initial channel attention vector. W1 is used to represent the attention vector of the intermediate channel, W1 is used to represent the first fully connected layer, and δ is used to represent the ReLU function.

[0094] Sub-step c involves calling the second fully connected layer and the Sigmoid function to perform dimensionality reduction on the intermediate channel attention vector at the channel dimension level, resulting in a low-order channel attention vector.

[0095] The calculation formula for the low-order channel attention vector can be expressed as follows:

[0096]

[0097] Among them, f lowerUsed to represent the low-order channel attention vector. W1 is used to represent the attention vector of the intermediate channel, W2 is used to represent the second fully connected layer, and σ is used to represent the Sigmoid function.

[0098] Therefore, by executing the above sub-steps a to c, this application ensures that the target channel attention model can use the low-order information learning branch model to extract low-order image information of the image to be recognized to construct the corresponding channel attention vector.

[0099] Sub-step S323 drives the high-order information learning branch model to extract second-order image information from the input global feature map for learning, and obtains the high-order channel attention vector.

[0100] In this embodiment, the step of the computer device 10 driving the higher-order information learning branch model to extract second-order image information of the global feature map to obtain the higher-order channel attention vector may include:

[0101] Sub-step e calls the feature map unrolling module to unroll the global feature map along the channel dimension, obtaining the two-dimensional feature matrix corresponding to the global feature map at the channel dimension level.

[0102] Wherein, if the global feature map adopts F∈R C×H×W If represented in this way, the two-dimensional feature matrix corresponding to this global feature map at the channel dimension level can be expressed as F. R ∈R C×(HW) Each row of the two-dimensional feature matrix is ​​used to represent the image features of the global feature map in the corresponding channel. For example, the matrix vector corresponding to the k-th row of the two-dimensional feature matrix is... The image feature length of each row of the matrix vector in the same two-dimensional feature matrix is ​​HW, which represents the image feature at the channel corresponding to the kth row of the global feature map.

[0103] Sub-step f calls the similarity matrix construction module to calculate the vector similarity between the matrix vectors in each row of the two-dimensional feature matrix, and constructs a matrix based on all the calculated vector similarities to obtain the feature similarity matrix of the global feature map at the channel dimension level.

[0104] The vector similarity between the matrix vectors in the i-th row and the j-th row of the two-dimensional feature matrix can be used as the matrix element value in the i-th row and j-th column of the feature similarity matrix. In this case, the feature similarity matrix is ​​a C×C matrix with diagonal elements of 1. The vector similarity between the matrix vectors in the i-th row and the j-th row can be calculated using the following formula:

[0105]

[0106] Among them, S i,j F is used to represent the matrix element value in the i-th row and j-th column of the feature similarity matrix. i R F is used to represent the matrix vector of the i-th row in the two-dimensional feature matrix. j R The vector representing the j-th row of the two-dimensional feature matrix is ​​used, and ||·|| is used to represent the 2-norm operation.

[0107] Sub-step g calls the attention vector extraction module to perform element-wise mean operation on each column of the feature similarity matrix to obtain a high-order channel attention vector composed of the actual element-wise mean of each column of the matrix vector.

[0108] Specifically, the mean value of all matrix element values ​​in the same column of the feature similarity matrix can be calculated, and the calculated actual element mean value is used as the mean value of that column of the matrix vector in the higher-order channel attention vector f. higher The corresponding vector element value in the middle, at this time the higher-order channel attention vector f higher The value of the i-th vector element is the actual mean of all element values ​​in the i-th column of the feature similarity matrix.

[0109] Therefore, by executing the above sub-steps e to g, this application ensures that the target channel attention model can use the higher-order information learning branch model to extract the higher-order image information of the image to be recognized to construct the corresponding channel attention vector.

[0110] This application can ensure that the target channel attention model can extract the low-order image information and high-order image information of the image to be identified and construct the corresponding channel attention vectors by executing the above sub-steps S321 to S323.

[0111] Step S330: Based on the low-order channel attention vector and the high-order channel attention vector, perform attention vector fusion and weighting processing on the global feature map to obtain the expected feature map of the image to be recognized.

[0112] In this embodiment, when the computer device 10 obtains a low-order channel attention vector carrying low-order image information of the image to be identified and a high-order channel attention vector carrying high-order image information of the image to be identified, it fuses the low-order channel attention vector and the high-order channel attention vector. Then, it uses the fused channel attention vector to perform weighted processing on the global feature map of the image to be identified, thereby obtaining a high-quality expected feature map, ensuring that the image features extracted from the corresponding image to be identified have good robustness and recognizability, and simultaneously improving the recognition accuracy of the visual recognition function.

[0113] Alternatively, please refer to Figure 6 , Figure 6 yes Figure 3 The flowchart of step S330 includes the sub-steps. In this embodiment, step S330 may include sub-steps S331 and S332 to ensure that the image features extracted from the image to be identified can fuse low-order and high-order image information, ensuring that the corresponding image features have good robustness and recognizability, and simultaneously improving the recognition accuracy of the visual recognition function.

[0114] Sub-step S331: Calculate the average channel attention vector between the low-order channel attention vector and the high-order channel attention vector.

[0115] The average channel attention vector can be expressed as 0.5 * (low-order channel attention vector f). lower +High-order channel attention vector f higher ).

[0116] Sub-step S332 involves performing channel-by-channel multiplication on the global feature map and the average channel attention vector to obtain the desired feature map.

[0117] Specifically, the channel-wise multiplication operation can be achieved by performing Kronecker product on the global feature map and the average channel attention vector, thereby ensuring that the image features of the desired feature map substantially integrate low-order and high-order image information, ensuring that the corresponding image features have good robustness and recognizability, and simultaneously improving the recognition accuracy of the visual recognition function.

[0118] Therefore, by executing the above sub-steps S331 to S333, this application can ensure that the image features extracted from the image to be identified can fuse low-order image information and high-order image information, ensure that the corresponding image features have good robustness and recognizability, and simultaneously improve the recognition accuracy of the visual recognition function.

[0119] This application can introduce a channel attention mechanism to fuse low-order and high-order image information of the image to be recognized during the visual recognition process by performing the above steps S310 to S330, so as to improve the quality of the image features extracted from the image to be recognized, ensure that the image features extracted from the corresponding image to be recognized have good robustness and recognizability, and simultaneously improve the recognition accuracy of the visual recognition function.

[0120] In this application, to ensure that the computer device 10 can effectively improve the accuracy of visual recognition while realizing the visual recognition function, this application provides an image recognition method to achieve the aforementioned objective. The image recognition method provided in this application will be described in detail below.

[0121] Please refer to Figure 7 , Figure 7 This is a schematic flowchart of the image recognition method provided in an embodiment of this application. In this embodiment, the image recognition method may include steps S410 to S430.

[0122] Step S410: Perform global feature extraction on the image to be recognized to obtain the global feature map of the image to be recognized.

[0123] In this embodiment, the computer device 10 can input the image to be identified into its own deep neural network model, so as to extract the global feature map of the image to be identified through the deep neural network model.

[0124] Step S420: Call the target channel attention model to process the global feature map to obtain the expected feature map corresponding to the global feature map.

[0125] The desired feature map is obtained by processing the feature map using the above-mentioned feature map processing method. In the visual recognition process, a channel attention mechanism is introduced to fuse low-order and high-order image information of the image to be recognized for image feature extraction, ensuring that the image feature quality of the desired feature map is high enough and that the image features of the desired feature map have good robustness and recognizability.

[0126] Step S430: Extract the object features of the target object in the image to be identified from the desired feature map.

[0127] The target object to be identified is the object targeted by the visual recognition function that the image recognition method expects to achieve. Taking the visual recognition function "vehicle recognition function" as an example, the target object to be identified by the "vehicle recognition function" is a vehicle such as a bicycle, motorcycle, electric vehicle, or car.

[0128] Therefore, by performing the above steps 410 to S430, this application can effectively improve the recognition accuracy of the visual recognition function while realizing the visual recognition function.

[0129] In this application, to ensure that the computer device 10 can effectively execute the aforementioned feature map processing method, the aforementioned function is achieved by dividing the feature map processing device 100 stored in the computer device 10 into functional modules. The specific composition of the feature map processing device 100 applied to the aforementioned computer device 10 provided in this application will be described below.

[0130] Please refer to Figure 8 , Figure 8This is a schematic diagram of the feature map processing device 100 provided in an embodiment of this application. In this embodiment, the feature map processing device 100 may include a global feature acquisition module 110, a channel vector extraction module 120, and a vector fusion weighting module 130.

[0131] The global feature acquisition module 110 is used to acquire the global feature map of the image to be recognized.

[0132] The channel vector extraction module 120 is used to call the target channel attention model to extract low-order image information and high-order image information of the global feature map for deep learning, so as to obtain a low-order channel attention vector corresponding to the low-order image information and a high-order channel attention vector corresponding to the high-order image information.

[0133] The vector fusion weighting module 130 is used to perform attention vector fusion weighting processing on the global feature map based on the low-order channel attention vector and the high-order channel attention vector to obtain the expected feature map of the image to be recognized.

[0134] It should be noted that the feature map processing apparatus 100 provided in this embodiment has the same basic principle and technical effects as the aforementioned feature map processing method. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the above description of the feature map processing method.

[0135] In this application, to ensure that the computer device 10 can effectively execute the above-described image recognition method, the aforementioned function is achieved by dividing the image recognition device 200 stored in the computer device 10 into functional modules. The specific composition of the image recognition device 200 applied to the above-described computer device 10 provided in this application is described below.

[0136] Please refer to Figure 9 , Figure 9 This is a schematic diagram of the composition of the image recognition device 200 provided in an embodiment of this application. In this embodiment, the image recognition device 200 may include a global feature extraction module 210, an image feature processing module 220, and an object feature extraction module 230.

[0137] The global feature extraction module 210 is used to extract global features from the image to be recognized, and obtain the global feature map of the image to be recognized.

[0138] The image feature processing module 220 is used to call the target channel attention model to perform feature map processing on the global feature map to obtain a desired feature map corresponding to the global feature map. Specifically, the image feature processing module 220 can obtain the desired feature map of the image to be recognized by calling the feature map processing device 100 to execute a feature map processing method.

[0139] The object feature extraction module 230 is used to extract the object features of the target object in the image to be identified from the desired feature map.

[0140] It should be noted that the image recognition device 200 provided in this embodiment has the same basic principle and technical effect as the aforementioned image recognition method. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the above description of the image recognition method.

[0141] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0142] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the various functions provided in this application are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned readable storage medium includes: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0143] In summary, in the feature map processing method and apparatus, image recognition method and apparatus, computer device, and readable storage medium provided in the embodiments of this application, after obtaining the global feature map of the image to be recognized, this application extracts the low-order image information and high-order image information of the global feature map by calling the target channel attention model and performs deep learning to obtain the low-order channel attention vector corresponding to the low-order image information and the high-order channel attention vector corresponding to the high-order image information. Then, based on the low-order channel attention vector and the high-order channel attention vector, the global feature map of the image to be recognized is subjected to attention vector fusion weighting processing to obtain the expected feature map of the image to be recognized. Thus, a channel attention mechanism is introduced into the visual recognition process to fuse the low-order image information and high-order image information of the image to be recognized for image feature extraction, so as to improve the quality of the image features extracted from the image to be recognized.

[0144] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A feature map processing method, characterized in that, The method includes: Obtain the global feature map of the image to be identified; The target channel attention model is invoked to extract low-order and high-order image information from the global feature map and perform deep learning to obtain a low-order channel attention vector corresponding to the low-order image information and a high-order channel attention vector corresponding to the high-order image information. Based on the low-order channel attention vector and the high-order channel attention vector, the global feature map is subjected to attention vector fusion and weighting processing to obtain the expected feature map of the image to be identified. The target channel attention model includes a higher-order information learning branch model, which comprises a feature map unfolding module, a similarity matrix construction module, and an attention vector extraction module connected in sequence, to extract the second-order image information of the global feature map as higher-order image information for deep learning. The step of driving the higher-order information learning branch model to extract the second-order image information of the input global feature map for learning and obtaining the higher-order channel attention vector includes: The feature map unfolding module is invoked to perform feature map unfolding processing on the global feature map along the channel dimension direction to obtain a two-dimensional feature matrix corresponding to the global feature map at the channel dimension level, wherein each row of the two-dimensional feature matrix is ​​used to represent the image features of the global feature map at the corresponding channel; The similarity matrix construction module is invoked to calculate the vector similarity between the matrix vectors in each row of the two-dimensional feature matrix, and a matrix is ​​constructed based on all the calculated vector similarities to obtain the feature similarity matrix of the global feature map at the channel dimension level. The attention vector extraction module is invoked to perform element-wise mean operation on each column of the feature similarity matrix to obtain the higher-order channel attention vector composed of the actual element-wise mean of each column of the matrix vector.

2. The method according to claim 1, characterized in that, The target channel attention model further includes a low-order information learning branch model. The step of calling the target channel attention model to extract low-order and high-order image information from the global feature map and performing deep learning to obtain a low-order channel attention vector corresponding to the low-order image information and a high-order channel attention vector corresponding to the high-order image information includes: The global feature map is synchronously input into the low-order information learning branch model and the high-order information learning branch model; The low-order information learning branch model is driven to extract first-order image information from the input global feature map and learn it to obtain the low-order channel attention vector; The higher-order information learning branch model is driven to extract second-order image information from the input global feature map and learn it to obtain the higher-order channel attention vector.

3. The method according to claim 2, characterized in that, The low-order information learning branch model includes a global average pooling layer, a first fully connected layer, and a second fully connected layer connected in sequence. The first and second fully connected layers use a ReLU function as the activation function, and the output of the second fully connected layer uses a Sigmoid function as the activation function. The step of driving the low-order information learning branch model to extract first-order image information from the input global feature map and learn the low-order channel attention vector includes: The global average pooling layer is invoked to perform feature map compression processing on the global feature map at the spatial dimension level to obtain the initial channel attention vector of the global feature map; The first fully connected layer and the ReLU function are invoked to perform vector upsizing on the initial channel attention vector at the channel dimension level to obtain the intermediate channel attention vector. The second fully connected layer and the Sigmoid function are invoked to perform vector dimensionality reduction on the intermediate channel attention vector at the channel dimension level to obtain the low-order channel attention vector.

4. The method according to any one of claims 1-3, characterized in that, The step of performing attention vector fusion and weighting processing on the global feature map based on the low-order channel attention vector and the high-order channel attention vector to obtain the desired feature map of the image to be recognized includes: Calculate the average channel attention vector between the low-order channel attention vector and the high-order channel attention vector; The desired feature map is obtained by performing channel-by-channel multiplication on the global feature map and the average channel attention vector.

5. A feature map processing apparatus, characterized in that, The device includes: The global feature acquisition module is used to acquire the global feature map of the image to be recognized; The channel vector extraction module is used to call the target channel attention model to extract low-order and high-order image information of the global feature map for deep learning, and obtain low-order channel attention vectors corresponding to the low-order image information and high-order channel attention vectors corresponding to the high-order image information. The vector fusion weighting module is used to perform attention vector fusion weighting processing on the global feature map based on the low-order channel attention vector and the high-order channel attention vector to obtain the expected feature map of the image to be recognized. The target channel attention model includes a higher-order information learning branch model, which comprises a feature map unfolding module, a similarity matrix construction module, and an attention vector extraction module connected in sequence. This module extracts second-order image information from the global feature map as higher-order image information for deep learning. The channel vector extraction module drives the higher-order information learning branch model to extract the second-order image information of the input global feature map for learning, obtaining the higher-order channel attention vector in the following ways: The feature map unfolding module is invoked to perform feature map unfolding processing on the global feature map along the channel dimension direction to obtain a two-dimensional feature matrix corresponding to the global feature map at the channel dimension level, wherein each row of the two-dimensional feature matrix is ​​used to represent the image features of the global feature map at the corresponding channel; The similarity matrix construction module is invoked to calculate the vector similarity between the matrix vectors in each row of the two-dimensional feature matrix, and a matrix is ​​constructed based on all the calculated vector similarities to obtain the feature similarity matrix of the global feature map at the channel dimension level. The attention vector extraction module is invoked to perform element-wise mean operation on each column of the feature similarity matrix to obtain the higher-order channel attention vector composed of the actual element-wise mean of each column of the matrix vector.

6. An image recognition method, characterized in that, The method includes: Global feature extraction is performed on the image to be identified to obtain the global feature map of the image to be identified; The target channel attention model is invoked to process the global feature map to obtain a desired feature map corresponding to the global feature map, wherein the desired feature map is obtained by the feature map processing method described in any one of claims 1-4. Extract the object features of the target object at the location in the image to be identified from the desired feature map.

7. An image recognition device, characterized in that, The device includes: The global feature extraction module is used to extract global features from the image to be recognized, thereby obtaining a global feature map of the image to be recognized. The image feature processing module is used to call the target channel attention model to perform feature map processing on the global feature map to obtain a desired feature map corresponding to the global feature map, wherein the desired feature map is obtained by the feature map processing method according to any one of claims 1-4. The object feature extraction module is used to extract the object features of the target object in the image to be identified from the desired feature map.

8. A computer device, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor, the processor executing the computer program to implement the feature map processing method according to any one of claims 1-4, or to implement the image recognition method according to claim 6.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the feature map processing method according to any one of claims 1-4, or the image recognition method according to claim 6.

Images