Image recognition method and device, computer device and storage medium

Abstract

The present disclosure provides an image recognition method and device, computer equipment and a storage medium, wherein the method comprises: obtaining a feature sequence corresponding to an image; converting a first initial position of each initial query feature in a query matrix into a first target position and converting a second initial position of each initial key feature in a key matrix into a second target position based on a target size of a feature map; determining a first weighted weight of each initial query feature based on the first target position and determining a second weighted weight of each initial key feature based on the second target position; for each image feature, determining an attention fusion feature corresponding to the image feature based on the first weighted weight of the initial query feature corresponding to the image feature, the second weighted weight of each initial key feature, and each target value feature in a value matrix; and determining an image recognition result corresponding to the image based on the attention fusion feature corresponding to each image feature.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image recognition, and in particular, to an image recognition method and device, a computer device, and a storage medium. BACKGROUND

[0002] As a model based on attention mechanism, the Transformer plays an important role in the field of image recognition. The Transformer model can be used to accurately identify the content of an image. However, when the Transformer model identifies an image, as the length of the feature sequence corresponding to the image increases, the computational complexity and the computation time of image recognition will also increase.

[0003] In this way, not only does the use of computing resources for image recognition increase, but the speed and accuracy of image recognition also decrease, resulting in the problem that the recognition effect is not ideal when the Transformer model identifies an image with a long feature sequence. Therefore, how to improve the image recognition effect of an image with a long feature sequence has become a problem to be solved. SUMMARY

[0004] The present disclosure provides at least an image recognition method and device, a computer device, and a storage medium.

[0005] In a first aspect, the present disclosure provides an image recognition method, comprising:

[0006] obtaining a feature sequence corresponding to an image to be identified, the feature sequence comprising image features corresponding to respective regions in a feature map of the image;

[0007] based on a target size of the feature map, converting each initial query feature in a query matrix corresponding to the feature sequence from a first initial position in a first feature dimension to a first target position in a second feature dimension, and converting each initial key feature in a key matrix corresponding to the feature sequence from a second initial position in the first feature dimension to a second target position in the second feature dimension;

[0008] based on the first target position, determining a first weighting weight corresponding to each initial query feature, and based on the second target position, determining a second weighting weight corresponding to each initial key feature;

[0009] for each image feature in the feature sequence, based on the first weighting weight of the initial query feature corresponding to the image feature, the second weighting weight of each initial key feature, and each target value feature in a value matrix corresponding to the feature sequence, determining an attention fusion feature corresponding to the image feature;

[0010] Determine the image recognition result corresponding to the image based on the attention fusion feature corresponding to each image feature.

[0011] In this embodiment, the first initial position of the initial query feature in the first feature dimension is converted into the first target position in the second feature dimension, and the second initial position of the initial key feature in the first feature dimension is converted into the second target position in the second feature dimension, so as to represent the Manhattan distance between the initial query feature and the initial key feature by using the first target position and the second target position. Then, the cosine weighting of the initial query feature and the initial key feature is determined according to the Manhattan distance, so as to accurately obtain the first weighting weight corresponding to each initial query feature and the second weighting weight corresponding to each initial key feature. Finally, the attention fusion feature corresponding to each image feature is calculated based on the first weighting weight, the second weighting weight, and each target value feature, which can improve the accuracy of determining the attention fusion feature, avoid the problem of excessive calculation amount caused by directly multiplying the query matrix and the key matrix, effectively reduce the calculation resources and the calculation time length required for determining the attention fusion feature, accurately identify the image corresponding to the long feature sequence based on this embodiment, effectively reduce the identification time, and finally improve the image recognition effect.

[0012] In a possible implementation, the first target position includes a first target row coordinate and a first target column coordinate.

[0013] The first initial position of each initial query feature in the query matrix corresponding to the feature sequence in the first feature dimension is converted into the first target position in the second feature dimension based on the target size of the feature map, and the method includes the following steps.

[0014] For each initial query feature, the first target row coordinate corresponding to the initial query feature is determined based on the first initial row coordinate indicated by the first initial position corresponding to the initial query feature and the height size indicated by the target size of the feature map.

[0015] The first target column coordinate corresponding to the initial query feature is determined based on the first initial column coordinate indicated by the first initial position corresponding to the initial query feature and the width size indicated by the target size of the feature map.

[0016] In the embodiment, the row and column coordinates are used to uniquely identify the position of the initial query feature, the first initial row coordinate corresponding to the initial query feature is converted by using the height indicated by the target size of the feature map, and the first initial column coordinate corresponding to the initial query feature is converted by using the width indicated by the target size of the feature map, so that the first initial position of the initial query feature in the first feature dimension can be accurately converted into the first target position in the second feature dimension.

[0017] In a possible implementation, the first weighting weight and the second weighting weight are respectively taken as a target weighting weight, and the first target position and the second target position are respectively taken as a target position, and then:

[0018] The target weighting weight includes a sub-weight obtained by performing a sine operation and a cosine operation on a row coordinate and a column coordinate in the target position corresponding to the target weighting weight.

[0019] In the embodiment, by performing a sine operation and a cosine operation on the first target position, each sub-weight corresponding to each initial query feature can be accurately obtained. By performing a sine operation and a cosine operation on the second target position, each sub-weight corresponding to each initial key feature can be accurately obtained. Finally, by using the sub-weights for weighting processing, the attention fusion feature corresponding to each image feature can be accurately determined.

[0020] In a possible implementation, the attention fusion feature corresponding to the image feature is determined based on the first weighting weight of the initial query feature corresponding to the image feature, the second weighting weight of each initial key feature, and each target value feature in the value matrix corresponding to the feature sequence, and the determination includes:

[0021] The initial query feature is converted into a target query feature and each initial key feature is converted into a target key feature by using a target kernel function;

[0022] Each target query feature corresponding to the initial query feature is weighted processed by using each sub-weight corresponding to the first weighting weight of the initial query feature, to obtain each first weighted result corresponding to the initial query feature.

[0023] For each initial key feature, each target key feature corresponding to the initial key feature is weighted processed by using each sub-weight corresponding to the second weighting weight of the initial key feature, to obtain each second weighted result corresponding to the initial key feature.

[0024] determine the attention fusion feature corresponding to the image feature based on each of the first weighted results corresponding to the initial query feature, a sum of each of the third weighted results corresponding to each of the initial key features, and the fourth weighted result.

[0025] In this embodiment, the target kernel function is used to convert the initial query feature into a non-negative target query feature and convert the initial key feature into a non-negative target key feature, thereby avoiding direct calculation using the originally negative initial query feature and / or the originally negative initial key feature, affecting the accuracy of the obtained attention fusion feature, and ensuring the accuracy of the calculated attention fusion feature.

[0026] In a possible implementation, the determining the attention fusion feature corresponding to the image feature based on each of the first weighted results corresponding to the initial query feature, each of the second weighted results corresponding to each of the initial key features, and each of the target value features comprises:

[0027] For each of the initial key features, based on the sum of the product of each of the second weighted results corresponding to the initial key feature and the target value feature corresponding to the same image feature as the initial key feature, a third weighted result corresponding to the initial key feature is obtained;

[0028] Adding each of the second weighted results corresponding to each of the initial key features to obtain a fourth weighted result;

[0029] determine the attention fusion feature corresponding to the image feature based on each of the first weighted results, the sum of each of the third weighted results corresponding to each of the initial key features, and the fourth weighted result.

[0030] In this embodiment, the target value feature and the initial key feature corresponding to the same image feature can be used as the key-value pair required for calculating the attention fusion feature. By multiplying each of the second weighted results corresponding to the initial key feature in the key-value pair and the target value feature in the key-value pair, the third weighted result corresponding to the initial key feature can be quickly and accurately obtained. Adding each of the second weighted results corresponding to each of the initial key features to obtain the fourth weighted result can avoid directly multiplying the key matrix and the query matrix, thereby reducing the amount of calculation and quickly obtaining the fourth weighted result. Finally, based on each of the first weighted results, the sum of the third weighted results, and the fourth weighted result, the attention fusion feature corresponding to the image feature can be quickly and accurately calculated.

[0031] In a possible implementation, the determining the image recognition result corresponding to the image based on the attention fusion feature corresponding to each of the image features comprises:

[0032] perform a linear full connection operation on the first attention matrix composed of the attention fusion features corresponding to the image features of each region to obtain a second attention matrix;

[0033] determine the image recognition result based on the second attention matrix.

[0034] In this embodiment, based on the linear full connection operation, the first attention matrix in the first feature dimension can be converted into the second attention matrix in the second feature dimension, and the feature dimension is restored.

[0035] In a possible implementation, the determining the image recognition result based on the second attention matrix comprises:

[0036] perform a pooling operation on the feature sequence to obtain a pooled feature sequence matrix;

[0037] multiply the pooled feature sequence matrix and the second attention matrix to obtain a third attention matrix;

[0038] add the third attention matrix and the second attention matrix to obtain a target attention matrix, and determine the image recognition result based on the target attention matrix.

[0039] In this embodiment, by multiplying the pooled feature sequence matrix and the second attention matrix, the feature information contained in the second attention matrix can be enriched, that is, the third attention matrix containing richer feature information is obtained. By adding the third attention matrix and the second attention matrix, the richness of the feature information contained in the obtained target attention matrix can be further improved. Finally, based on the target attention matrix, the image recognition result is determined, which can effectively improve the accuracy of the determined image recognition result.

[0040] In a possible implementation, the feature sequence corresponding to the image to be recognized is obtained, comprising:

[0041] obtain an image to be recognized;

[0042] generate a feature map with the target size based on the image to be recognized;

[0043] obtain the feature sequence according to the image features of each region in the feature map.

[0044] In this embodiment, by generating the feature map corresponding to the image, the RGB image to be recognized can be converted to the feature dimension. Then, based on the region division of the feature map, the accurate feature sequence can be obtained according to the image features of each region.

[0045] In a possible implementation, the feature sequence is obtained according to the image features of each region in the feature map, including:

[0046] According to the corresponding row and column coordinates of each region in the feature map, the position of the image feature corresponding to the region in the feature sequence is determined.

[0047] According to the corresponding position of each image feature, the image features are arranged to obtain the feature sequence.

[0048] In this implementation, the ordered and accurate feature sequence can be obtained according to the row and column coordinates of each region and the image features of each region.

[0049] In a possible implementation, the method further includes the steps of determining the query matrix, the key matrix, and the value matrix:

[0050] The feature sequence is processed by feature mapping using a target feature matrix to obtain the query matrix, the key matrix, and the value matrix corresponding to the feature sequence; wherein the query matrix, the key matrix, and the value matrix correspond to different target feature matrices, and the query matrix, the key matrix, and the value matrix correspond to the first feature dimension.

[0051] In this implementation, the query matrix, the key matrix, and the value matrix corresponding to the feature sequence can be accurately obtained by processing the feature sequence by feature mapping using a target feature matrix. Furthermore, based on the image recognition method provided in the embodiments of the present disclosure, the image can be quickly and accurately recognized by using the obtained query matrix, key matrix, and value matrix.

[0052] In a second aspect, the embodiments of the present disclosure further provide an image recognition device, including:

[0053] The acquisition module is configured to acquire a feature sequence corresponding to an image to be recognized, the feature sequence including image features corresponding to each region in a feature map of the image;

[0054] The conversion module is configured to convert each initial query feature in a query matrix corresponding to the feature sequence from a first initial position in a first feature dimension to a first target position in a second feature dimension based on a target size of the feature map, and convert each initial key feature in a key matrix corresponding to the feature sequence from a second initial position in the first feature dimension to a second target position in the second feature dimension.

[0055] The first determining module is configured to determine a first weighting weight corresponding to each of the initial query features based on the first target position, and determine a second weighting weight corresponding to each of the initial key features based on the second target position.

[0056] The second determining module is configured to determine, for each of the image features in the feature sequence, an attention fusion feature corresponding to the image feature based on the first weighting weight of the initial query feature corresponding to the image feature, the second weighting weight of each of the initial key features, and each of the target value features in the value matrix corresponding to the feature sequence.

[0057] The third determining module is configured to determine the image recognition result corresponding to the image based on the attention fusion feature corresponding to each of the image features.

[0058] In a possible implementation, the first target position includes a first target row coordinate and a first target column coordinate.

[0059] When the conversion module converts each of the initial query features in the query matrix corresponding to the feature sequence from a first initial position in a first feature dimension to a first target position in a second feature dimension based on the target size of the feature map, the conversion module is configured to determine, for each of the initial query features, a first target row coordinate corresponding to the initial query feature based on a first initial row coordinate indicated by the first initial position corresponding to the initial query feature and a height size indicated by the target size of the feature map.

[0060] The conversion module is configured to determine a first target column coordinate corresponding to the initial query feature based on a first initial column coordinate indicated by the first initial position corresponding to the initial query feature and a width size indicated by the target size of the feature map.

[0061] In a possible implementation, the first determining module is configured to take the first weighting weight and the second weighting weight as target weighting weights respectively, and take the first target position and the second target position as target positions, and the target weighting weight includes a sub-weight obtained by performing a sine operation and a cosine operation on a row coordinate and a column coordinate in the target position corresponding to the target weighting weight respectively.

[0062] The target weighting weight includes a sub-weight obtained by performing a sine operation and a cosine operation on a row coordinate and a column coordinate in the target position corresponding to the target weighting weight respectively.

[0063] In a possible implementation, the second determining module, when determining the attention fusion feature corresponding to the image feature based on the first weighted weight of the initial query feature corresponding to the image feature, the second weighted weight of each initial key feature, and each target value feature in the value matrix corresponding to the feature sequence, is configured to convert the initial query feature into a target query feature and convert each initial key feature into a target key feature by using a target kernel function.

[0064] The initial query feature corresponding to each of the first weighted results is weighted by using each of the sub-weights corresponding to the first weighted weight of the initial query feature, to obtain each of the first weighted results corresponding to the initial query feature.

[0065] For each of the initial key features, the target key feature corresponding to the initial key feature is weighted by using each of the sub-weights corresponding to the second weighted weight of the initial key feature, to obtain each of the second weighted results corresponding to the initial key feature.

[0066] The attention fusion feature corresponding to the image feature is determined based on each of the first weighted results corresponding to the initial query feature, each of the second weighted results corresponding to each of the initial key features, and each of the target value features.

[0067] In a possible implementation, the second determining module, when determining the attention fusion feature corresponding to the image feature based on each of the first weighted results corresponding to the initial query feature, each of the second weighted results corresponding to each of the initial key features, and each of the target value features, is configured to obtain, for each of the initial key features, a third weighted result based on a sum of products of each of the second weighted results corresponding to the initial key feature and a target value feature corresponding to the same image feature as the initial key feature.

[0068] Each of the second weighted results corresponding to each of the initial key features is added to obtain a fourth weighted result.

[0069] The attention fusion feature corresponding to the image feature is determined based on each of the first weighted results, a sum of the third weighted results corresponding to each of the initial key features, and the fourth weighted result.

[0070] In a possible implementation, the third determining module, when determining the image recognition result corresponding to the image based on the attention fusion feature corresponding to each of the image features, is configured to perform a linear full connection operation on a first attention matrix composed of the attention fusion features corresponding to the image features of each region to obtain a second attention matrix.

[0071] determine the image recognition result based on the second attention matrix.

[0072] In a possible implementation, the third determining module, when determining the image recognition result based on the second attention matrix, is configured to perform a pooling operation on the feature sequence to obtain a pooled feature sequence matrix.

[0073] multiply the pooled feature sequence matrix and the second attention matrix to obtain a third attention matrix.

[0074] add the third attention matrix and the second attention matrix to obtain a target attention matrix, and determine the image recognition result based on the target attention matrix.

[0075] In a possible implementation, the obtaining module, when obtaining the feature sequence corresponding to the image to be recognized, is configured to obtain the image to be recognized.

[0076] generate a feature map with the target size based on the image to be recognized.

[0077] obtain the feature sequence according to the image features of each region in the feature map.

[0078] In a possible implementation, the obtaining module, when obtaining the feature sequence according to the image features of each region in the feature map, is configured to determine the position of the image feature corresponding to each region in the feature sequence according to the row and column coordinates corresponding to the region in the feature map.

[0079] arrange each image feature according to the position corresponding to the image feature to obtain the feature sequence.

[0080] In a possible implementation, the apparatus further includes:

[0081] a mapping module configured to determine the query matrix, the key matrix, and the value matrix by performing feature mapping processing on the feature sequence using a target feature matrix, to obtain the query matrix, the key matrix, and the value matrix corresponding to the feature sequence; wherein the query matrix, the key matrix, and the value matrix correspond to different target feature matrices, and the query matrix, the key matrix, and the value matrix correspond to the first feature dimension.

[0082] In a third aspect, the optional implementation of the present disclosure further provides a computer device, a processor, and a memory. The memory stores machine readable instructions executable by the processor. The processor is configured to execute the machine readable instructions stored in the memory. When the machine readable instructions are executed by the processor, the machine readable instructions perform the steps of the first aspect or any possible implementation of the first aspect.

[0083] In a fourth aspect, the optional implementation of the present disclosure further provides a computer readable storage medium, which stores a computer program. When the computer program is executed, the steps of the first aspect or any possible implementation of the first aspect are performed.

[0084] For the effects of the image recognition device, the computer device, and the computer readable storage medium, refer to the description of the image recognition method. Details are not repeated here.

[0085] In order to make the above objectives, characteristics and advantages of the present disclosure more apparent, the following will describe a preferred embodiment in detail, and the accompanying drawings will be described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0086] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings are incorporated into the specification and form a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0087] Figure 1 A flow chart of an image recognition method provided by an embodiment of the present disclosure is shown;

[0088] Figure 2 A schematic diagram of a feature map corresponding to an image to be recognized provided by an embodiment of the present disclosure is shown;

[0089] Figure 3 A recognition schematic diagram of recognizing an image to be recognized by using an image recognition neural network provided by an embodiment of the present disclosure is shown;

[0090] Figure 4 A recognition comparison diagram of an image recognition method provided by an embodiment of the present disclosure and an existing image recognition method is shown;

[0091] Figure 5A schematic diagram of an image recognition device is shown.

[0092] Figure 6 A structural schematic diagram of a computer device is shown. DETAILED DESCRIPTION

[0093] To make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the following will be combined with the accompanying drawings of the embodiments of the present disclosure to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure and are not all the embodiments. The components of the embodiments of the present disclosure described and shown herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present disclosure.

[0094] In addition, the terms "first", "second", and the like in the specification and claims of the embodiments of the present disclosure and the above-described accompanying drawings are used to distinguish similar objects and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0095] "Multiple or several" mentioned herein refers to two or more. "And / or" describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents that the front and rear associated objects are in an "or" relationship.

[0096] It is found through research that when using a Transformer model to identify images, the transpose matrix of the query matrix and the key matrix corresponding to the feature sequence of the image is directly multiplied to obtain a weighted weight matrix, and the weighted weight matrix is used to weight the value matrix corresponding to the feature sequence, thereby obtaining the attention fusion feature corresponding to each image feature.

[0097] The amount of calculation required by the above operation of directly multiplying the transpose matrix of the query matrix and the key matrix will increase with the increase of the target size of the query matrix and the key matrix. For example, the query matrix and the key matrix are both N by d matrices, where N represents the number of features in the matrix, and d represents the feature dimension of a single feature in the matrix. The amount of calculation required by the direct multiplication of the transpose matrix of the query matrix and the key matrix is N 2Xd. On the basis that the target size of the query matrix and the key matrix is determined by the length of the feature sequence corresponding to the image, the longer the feature sequence, the greater the amount of calculation required, the greater the calculation pressure, the longer the calculation time, and the longer the image recognition time. Therefore, the above image recognition process seriously affects the recognition speed and recognition effect of the image corresponding to the long feature sequence.

[0098] Based on the above research, the present disclosure provides an image recognition method, device, computer equipment and storage medium. By converting the first initial position of the initial query feature in the first feature dimension into the first target position in the second feature dimension, and converting the second initial position of the initial key feature in the first feature dimension into the second target position in the second feature dimension, the Manhattan distance between the initial query feature and the initial key feature can be represented by the first target position and the second target position. Then, according to the Manhattan distance, the cosine weighting of the initial query feature and the initial key feature is determined, so that the first weighted weight corresponding to each initial query feature and the second weighted weight corresponding to each initial key feature can be accurately obtained. Finally, based on the first weighted weight, the second weighted weight, and each target value feature, the calculation of the attention fusion feature corresponding to each image feature is performed, which can not only improve the accuracy of determining the attention fusion feature, but also avoid the problem of excessive calculation amount caused by directly multiplying the query matrix and the key matrix, effectively reducing the calculation resources and calculation time required to determine the attention fusion feature. For the image corresponding to the long feature sequence, based on this embodiment, accurate image recognition can be achieved, the recognition time can be effectively reduced, and the recognition effect of the image can be ultimately improved.

[0099] The defects of the above scheme are the result of the inventors' practice and careful research, therefore, the discovery process of the above problems and the solutions proposed by the present disclosure to solve the above problems should be the contribution of the inventors to the present disclosure in the process of the present disclosure.

[0100] It should be noted that similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0101] To facilitate understanding of this embodiment, a detailed description of the image recognition method disclosed in this disclosure embodiment will be provided first. The execution subject of the image recognition method provided in this disclosure embodiment is generally a terminal device or other processing device with certain computing power. The terminal device can be a mobile device, a user terminal, a terminal, a computer device, a neural network deployed in a computer device, etc. In some possible implementations, the image recognition method can be implemented by the processor calling computer-readable instructions stored in the memory.

[0102] The image recognition method provided in this disclosure will be described below using an image recognition neural network as an example.

[0103] like Figure 1 The flowchart shown is a method for image recognition provided in an embodiment of this disclosure, which may include the following steps:

[0104] S101: Obtain the feature sequence corresponding to the image to be identified. The feature sequence is included in the feature map of the image and the image features corresponding to each region.

[0105] Here, the image to be identified can be an RGB image captured using any imaging device. The feature sequence can include multiple image features. The feature sequence corresponding to the image to be identified is generated based on the feature map of the image to be identified. The feature map of the image can be divided into multiple regions, and the image features corresponding to each region can be considered as one image feature in the feature sequence.

[0106] For example, the feature sequence X corresponding to the image to be identified can be an N×d feature matrix, where N represents the number of image features in feature sequence X, and d represents the feature dimension of each image feature in feature sequence X. In specific implementations, the specific value of d can be determined by the network parameters of the image recognition neural network executing the image recognition method provided in this embodiment of the disclosure, and is not limited here. The image recognition neural network is a transformer-based visual neural network.

[0107] In practice, for the image to be recognized, a pre-generated feature sequence corresponding to the image can be directly obtained, and this feature sequence can be input into the image recognition neural network for processing. The feature sequence corresponding to the image to be recognized can be generated in advance based on the image's feature map and obtained when needed.

[0108] In one embodiment, the feature sequence corresponding to the image to be identified can also be generated by the image recognition neural network based on the image to be identified when it is determined that image recognition needs to be performed. Specifically, the feature sequence can be obtained according to the following steps:

[0109] S101-1: Obtain an image to be recognized.

[0110] S101-2: Based on the image to be recognized, generate a feature map with a target size.

[0111] Here, the target size of the feature map can be determined according to the image size of the image input to the image recognition neural network, and the target size can be represented as m x n, where m represents the width and n represents the height. Specifically, the target size of the feature map of the image is determined by dividing the image to be recognized into the target in both the image width and the image length. For example, the input image to be recognized can be an image with an image size of 256 x 256, and the target size of the feature map corresponding to the image can be 64 x 64. That is, the image to be recognized is divided into 64 parts in both the image width direction and the image height direction, so that 64 x 64 sub-regions are obtained in the image to be recognized, and one sub-region of the image corresponds to one region in the feature map. As shown in FIG. 1, it is a schematic diagram of a feature map corresponding to an image to be recognized provided by an embodiment of the present disclosure, wherein the target size of the feature map is 11 x 11. That is, Figure 2 Figure 2 As shown in the feature map, it includes 11 rows and 11 columns, wherein the 11 rows are in turn the 0th row, the 1st row, …, the 10th row, and the 11 columns are in turn the 0th column, the 1st column, …, the 10th column. In Figure 2 In FIG. 1, the row and column information corresponding to part of the regions is shown, i.e., the region of the 0th row and the 0th column, the region of the 0th row and the 1st column, the region of the 1st row and the 0th column, and the region of the 10th row and the 1st column.

[0112] Alternatively, in another implementation, a default target size can also be set, and for any size of the input image, a feature map with the default target size corresponding to the image is generated. For example, the default target size is 32 x 32, for the input image 1 with an image size of 128 x 128, a feature map with a size of 32 x 32 corresponding to the image 1 is generated, and for the input image 2 with an image size of 256 x 256, a feature map with a size of 32 x 32 corresponding to the image 2 is also generated. In specific applications, the target size of the feature map can be determined according to the actual network parameters set, and the embodiments of the present disclosure are not limited.

[0113] In specific implementation, the image to be recognized can be input to the image recognition neural network, and the image recognition neural network is used to perform convolution processing on the image to obtain a feature map with a target size corresponding to the image. In this way, the three-dimensional RGB image is usually converted to the feature dimension, which is convenient for the network to perform subsequent feature recognition processing.

[0114] S101-3: Obtain a feature sequence according to the image features of each region in the feature map.​

[0115] In implementation, the image feature of each region in the feature map can be directly taken as an image feature in the sequence. The feature dimension of the image feature of each region in the feature map is d.

[0116] In an embodiment, for S101-3, the following steps can be implemented:

[0117] S101-3-1: According to the row and column coordinates of each region in the feature map, the position of the image feature corresponding to the region in the feature sequence is determined.

[0118] Here, each region in the feature map with the target size corresponds to a row and column coordinate (u, r). Wherein, u is the row coordinate, and r is the column coordinate. For example, in the feature map shown in Figure 2 , the row and column coordinate of the region located at the 0th row and the 0th column can be (0, 0), the row and column coordinate of the region located at the 1st row and the 2nd column can be (1, 2), and the row and column coordinate of the region located at the 10th row and the 3rd column can be (10, 3).

[0119] Each image feature included in the feature sequence can be arranged in order, and the position corresponding to each image feature can be determined according to the row and column coordinates of the region corresponding to the image feature in the feature map.

[0120] In implementation, for the image feature of each region in the feature map, the product of the row coordinate of the region and the width indicated by the target size can be determined first, and then the column coordinate corresponding to the region is added to obtain a numerical value as the position of the image feature of the region in the feature sequence. The width indicated by the target size can be the number of sub-regions in the image width direction. For example, if the row and column coordinates of a region are (3, 5) and the width indicated by the target size is 10, the image feature of the region can be the 35th image feature in the feature sequence, where 35 = 3 x 10 + 5.

[0121] For example, the image feature of the region located at the 0th row and the 0th column in the feature map shown in Figure 2 can be taken as the 1st image feature in the feature sequence, the image feature of the region located at the 0th row and the 1st column can be taken as the 2nd image feature in the feature sequence, the image feature of the region located at the 0th row and the 2nd column can be taken as the 3rd image feature in the feature sequence, …, the image feature of the region located at the 1st row and the 0th column can be taken as the 11th image feature in the feature sequence, and so on. The feature sequence corresponding to the feature map shown in Figure 2 including 121 image features can be obtained.

[0122] S101-3-2: arrange each image feature according to the position corresponding to the image feature to obtain a feature sequence.

[0123] In specific implementation, the image features of each region in the feature map can be sorted according to the positions corresponding to the image features of each region, so as to obtain the feature sequence.

[0124] S102: based on the target size of the feature map, converting each initial query feature in the query matrix corresponding to the feature sequence from a first initial position in a first feature dimension to a first target position in a second feature dimension, and converting each initial key feature in the key matrix corresponding to the feature sequence from a second initial position in the first feature dimension to a second target position in the second feature dimension.

[0125] Here, before S102 is performed, the feature sequence can also be subjected to feature mapping processing to obtain a query matrix, a key matrix and a value matrix corresponding to the feature sequence, and then the three matrices are used to determine the attention fusion feature.

[0126] In specific implementation, the query matrix, the key matrix and the value matrix can be obtained according to the following steps:

[0127] The feature sequence is subjected to feature mapping processing by using a target feature matrix, to obtain a query matrix, a key matrix and a value matrix corresponding to the feature sequence; wherein the query matrix, the key matrix and the value matrix correspond to different target feature matrices, and the query matrix, the key matrix and the value matrix correspond to a first feature dimension.

[0128] Here, the first feature dimension can be half of the feature dimension of the image feature. For example, in the case where the feature dimension of the image feature is d, the first feature dimension is

[0129] The query matrix is a matrix obtained by using a preset first target feature matrix corresponding to the matrix to perform feature mapping processing on the image features in the feature sequence, wherein the matrix corresponds to the first feature dimension. The initial query feature in the query matrix is used to represent the feature information of the image feature itself.

[0130] The key matrix is a matrix obtained by using a preset second target feature matrix corresponding to the matrix to perform feature mapping processing on the image features in the feature sequence, wherein the matrix corresponds to the first feature dimension. The initial key feature in the key matrix is used to represent the feature information after feature extraction of the image feature in a feature direction. The initial key feature in the key matrix can be used to match the initial query feature.

[0131] The value matrix is a matrix obtained by performing feature mapping processing on the image features in the feature sequence using a preset third target feature matrix corresponding to the value matrix, wherein the matrix corresponds to the first feature dimension. The target value feature in the value matrix is used to represent the feature information after feature extraction of the image features in another feature direction. The target value feature in the value matrix is used to determine the attention fusion feature corresponding to each image feature in the feature sequence.

[0132] The number of features included in each of the above query matrix, key matrix and value matrix is consistent with the number of image features in the feature sequence.

[0133] For example, after inputting the Nxd feature sequence X into the image recognition neural network, the first target feature matrix, the second target feature matrix and the third target feature matrix can be used to perform feature mapping processing on the feature sequence X respectively to obtain the query matrix , the key matrix and the value matrix . Wherein, represents the first feature dimension.

[0134] The second feature dimension can be twice the feature dimension corresponding to the image feature. For example, in the case where the feature dimension corresponding to the image feature is d, the second feature dimension can be 2d.

[0135] The first initial position is the initial query feature in the query matrix under the first feature dimension, and the second initial position is the initial key feature in the query matrix under the first feature dimension. For example, the first initial position can be the position of the initial query feature in the query matrix , and the second initial position is the position of the initial key feature in the key matrix .

[0136] Wherein, the first initial position and the second initial position can include initial row and column coordinates. Since the initial query feature is obtained by performing feature mapping processing on the image features in the feature sequence, there is a corresponding relationship between the initial query feature and the image features, and each image feature is the image feature of each region in the feature map. Therefore, for each initial query feature, the target image feature corresponding to the initial query feature can be determined from the plurality of image features in the feature sequence according to the corresponding relationship, and then the row and column coordinates of the region corresponding to the target image feature in the feature map can be taken as the first initial row and column coordinates of the initial query feature, that is, the first initial position of the initial query feature is obtained.

[0137] Similarly, since the initial key feature is obtained after the feature mapping processing of the image features in the feature sequence, there is a corresponding relationship between the initial key feature and the image features, and each image feature is the image feature of each region in the feature map, so for each initial key feature, according to the corresponding relationship, the target image feature corresponding to the initial key feature can be determined from the feature sequence including multiple image features, and then the row and column coordinates of the region corresponding to the target image feature in the feature map can be taken as the second initial row and column coordinates of the initial key feature, that is, the second initial position of the initial key feature is obtained.

[0138] For example, for the i-th initial query feature q i in the query matrix, or the j-th initial key feature k j in the key matrix, the value of i and the value of k can be determined according to the following formula (1):

[0139] i = u i m + r i , j = u j m + r j , 0 < r i < r j < m, (formula 1)

[0140] where q i and k j may correspond to the target image feature 1, the maximum value of i and j is equal to the number of image features in the feature sequence. u i represents the row coordinate of the region corresponding to the target image feature 1 in the feature map, r i represents the column coordinate of the region corresponding to the target image feature 1 in the feature map, and m represents the width of the feature map. u j represents the row coordinate of the region corresponding to the target image feature 1 in the feature map, r j represents the column coordinate of the region corresponding to the target image feature 1 in the feature map.

[0141] In specific implementation, for each initial query feature in the query matrix, the first initial position can be converted into the first target position according to the size of the feature map and the first initial position of the initial query feature in a preset conversion manner. For each initial key feature, the second initial position can also be converted into the second target position according to the size of the feature map and the second initial position of the initial key feature in a preset conversion manner.

[0142] In an embodiment, the first target position includes a first target row coordinate and a first target column coordinate. The second target position can also include a second target row coordinate and a second target column coordinate.

[0143] In specific implementation, the first target position of each initial query feature can be determined according to the following steps:

[0144] Step one, for each initial query feature, the first target row coordinate corresponding to the initial query feature is determined based on the first initial row coordinate indicated by the first initial position corresponding to the initial query feature and the height size indicated by the target size of the feature map.

[0145] Here, the first initial position includes the first initial row coordinate and the first initial column coordinate. The height size indicated by the target size is the height of the feature map.

[0146] In specific implementation, the first initial row coordinate can be converted into the first target row coordinate according to the following Formula Two:

[0147]

[0148] wherein a i represents the first target row coordinate corresponding to the i th initial query feature, n represents the height size indicated by the target size, u i represents the first initial row coordinate corresponding to the i th initial query feature. The maximum value of i is the number of image features in the feature sequence.

[0149] For each initial query feature in the query matrix, the height size indicated by the target size and the first initial row coordinate corresponding to the initial query feature can be substituted into the above Formula Two, so as to obtain the first target row coordinate corresponding to the initial query feature.

[0150] Step two, the first target column coordinate corresponding to the initial query feature is determined based on the first initial column coordinate indicated by the first initial position corresponding to the initial query feature and the width size indicated by the target size of the feature map.

[0151] Here, the width size indicated by the target size is the width of the feature map. There is no strict execution sequence between step one and step two, and they can be executed simultaneously.

[0152] In specific implementation, the first initial column coordinate can be converted into the first target column coordinate according to the following Formula Three:

[0153]

[0154] wherein b i represents the first target column coordinate corresponding to the i th initial query feature, m represents the width size indicated by the target size, r i represents the first initial column coordinate corresponding to the i th initial query feature.

[0155] For each initial query feature in the query matrix, the width size indicated by the target size and the first initial column coordinate corresponding to the initial query feature can be substituted into the above Formula Three, so as to obtain the first target column coordinate corresponding to the initial query feature.

[0156] For each initial query feature, the first target row coordinate and the first target column coordinate corresponding to the initial query feature are obtained, that is, the first target position corresponding to the initial query feature is obtained.

[0157] Similarly, the second initial row coordinate indicated by the second initial position corresponding to each initial key feature can be converted into the second target row coordinate according to the following Formula Four:

[0158]

[0159] wherein a j represents the second target row coordinate corresponding to the jth initial key feature, u j represents the second initial row coordinate corresponding to the jth initial key feature. The maximum value of j is the number of image features in the feature sequence.

[0160] For each initial key feature in the key matrix, the height size indicated by the target size and the second initial row coordinate corresponding to the initial key feature can be substituted into the above Formula Four, so as to obtain the second target row coordinate corresponding to the initial key feature.

[0161] Meanwhile, the second initial column coordinate indicated by the second initial position corresponding to each initial key feature can be converted into the second target column coordinate according to the following Formula Five:

[0162]

[0163] wherein b j represents the second target column coordinate corresponding to the jth initial key feature, r j represents the second initial column coordinate corresponding to the jth initial key feature.

[0164] For each initial key feature in the key matrix, the width size indicated by the target size and the second initial column coordinate corresponding to the initial key feature can be substituted into the above Formula Five, so as to obtain the second target column coordinate corresponding to the initial key feature.

[0165] For each initial key feature, the second target row coordinate and the second target column coordinate corresponding to the initial key feature are obtained, that is, the second target position corresponding to the initial key feature is obtained.

[0166] S103: Determine the first weighted weight corresponding to each initial query feature based on the first target position, and determine the second weighted weight corresponding to each initial key feature based on the second target position.

[0167] Here, the first weighting weight is used for performing cosine weighting processing on the target query feature corresponding to each initial query feature, and the second weighting weight is used for performing cosine weighting processing on the target key feature corresponding to each initial key feature. The target query feature is obtained by converting the initial query feature by using the target kernel function, and the target key feature is also obtained by converting the initial key feature by using the target kernel function.

[0168] In specific implementation, the first weighting weight and the second weighting weight can be respectively taken as a target weighting weight, and the first target position and the second target position can be respectively taken as a target position. The target weighting weight includes: sub-weights obtained by performing a sine operation and a cosine operation on a row coordinate and a column coordinate in the target position corresponding to the target weighting weight, respectively.

[0169] Here, for each initial query feature, the first weighting weight corresponding to the initial query feature can be taken as a target weighting weight of the initial query feature, and the first target position corresponding to the initial query feature can be taken as a target position of the initial query feature. The target position can include a first target row coordinate and a first target column coordinate.

[0170] Then, the first target row coordinate corresponding to the initial query feature can be respectively taken to perform a sine operation, so as to obtain two sine sub-weights corresponding to the initial query feature. Meanwhile, the first target column coordinate corresponding to the initial query feature can be respectively taken to perform a cosine operation, so as to obtain two cosine sub-weights corresponding to the initial query feature.

[0171] For example, the initial query feature q i corresponds to the first target row coordinate a i and the first target column coordinate b i . The two sine sub-weights corresponding to the initial query feature q i are sin(a i ) and sin(b i ), respectively. The two cosine sub-weights corresponding to the initial query feature q i are cos(a i ) and cos(b i ), respectively.

[0172] For each initial key feature, the second weighting weight corresponding to the initial key feature can be taken as a target weighting weight of the initial key feature, and the second target position corresponding to the initial key feature can be taken as a target position of the initial key feature. The target position can include a second target row coordinate and a second target column coordinate.

[0173] Then, the sine operation can be performed on the second target row coordinates corresponding to the initial key feature respectively to obtain two sine sub-weights corresponding to the initial key feature, and the cosine operation can be performed on the second target column coordinates corresponding to the initial key feature respectively to obtain two cosine sub-weights corresponding to the initial query feature.

[0174] For example, the initial key feature k j corresponds to the second target row coordinate a j and the second target column coordinate b j . Then, the two sine sub-weights corresponding to the initial key feature k j are sin(a j ) and sin(b j ) respectively, and the two cosine sub-weights corresponding to the initial key feature k j are cos(a j ) and cos(b j ) respectively.

[0175] That is, for any initial query feature q i , the first weighted weight corresponding thereto can include four sub-weights of sin(a i ), sin(b i ), cos(a i ) and cos(b i ), and for any initial key feature k j , the second weighted weight corresponding thereto can include four sub-weights of sin(a j ), sin(b j ), cos(a j ) and cos(b j ).

[0176] S104: For each image feature in the feature sequence, based on the first weighted weight of the initial query feature corresponding to the image feature, the second weighted weight of each initial key feature, and each target value feature in the value matrix corresponding to the feature sequence, determine the attention fusion feature corresponding to the image feature.

[0177] Here, one image feature corresponds to one attention fusion feature.

[0178] In practice, for each image feature, the target query feature corresponding to that initial query feature can be weighted using the first weighted weights of each term in the query matrix, yielding a weighted result. Then, the target key feature corresponding to each initial key feature is weighted using the second weighted weights, yielding a weighted result. Finally, based on the weighted results and each target value feature in the value matrix, the attention fusion feature corresponding to that image feature is determined.

[0179] In one embodiment, S104 can be implemented according to the following steps:

[0180] S104-1: Using the target kernel function, the initial query features are transformed into target query features, and each initial key feature is transformed into a target key feature.

[0181] Here, the target kernel function can be the ReLU function, which can be specifically represented by the following formula six:

[0182]

[0183] Where x can be an initial query feature or an initial key feature.

[0184] In practice, for each initial query feature in the initial query matrix, the initial query feature can be substituted into Formula 6 above to obtain the target query feature corresponding to that initial query feature. Similarly, for each initial key feature in the initial key matrix, the initial key feature can be substituted into Formula 6 above to obtain the target key feature corresponding to that initial key feature.

[0185] S104-2: Using the sub-weights corresponding to the first weighted weights of the initial query features, the target query features corresponding to the initial query features are weighted respectively to obtain the first weighted results corresponding to the initial query features.

[0186] In practice, for each initial query feature q i q can be used i The corresponding sub-weights sin(a) i ), sin(b i ), cos(a i ) and cos(b i ), respectively for q i Corresponding target query features After weighting, we get q. i The four corresponding first-weighted results: and

[0187] in,

[0188] S104-3: For each initial key feature, the target key feature corresponding to the initial key feature is respectively weighted processed by using each sub-weight corresponding to the second weighting weight of the initial key feature, to obtain each second weighting result corresponding to the initial key feature.

[0189] In specific implementation, for each initial key feature k j , the target key feature corresponding to k j may be respectively weighted processed by using each sub-weight sin(a j ), sin(b j ), cos(a j ) and cos(b j ) corresponding to k j , to obtain four second weighting results corresponding to k j : and

[0190] wherein, K j = k j ,

[0191] wherein, the first weighting result and the second weighting result both belong to the 2d feature dimension.

[0192] When the image recognition method provided by the embodiment of the present disclosure is used for image recognition, the query matrix under the first feature dimension can be converted into the query matrix under the second feature dimension and the key matrix under the first feature dimension can be converted into the key matrix under the second feature dimension based on the cosine re-weighting manner. For example, the query matrix of N x d can be converted into the query matrix of N x 2d, and the key matrix of N x d can be converted into the key matrix of N x 2d.

[0193] S104-4: Based on each first weighting result corresponding to the initial query feature, each second weighting result corresponding to each initial key feature, and each target value feature, the attention fusion feature corresponding to the image feature is determined.

[0194] In specific implementation, for each image feature, the attention fusion feature corresponding to the image feature can be determined by performing matrix multiplication operation based on each first weighting result of the initial query feature corresponding to the image feature, each second weighting result of each initial key feature, and each target value feature.

[0195] ​​​In an embodiment, the S104-4 above can be implemented according to the following steps:

[0196] S104-4-1: For each initial key feature, based on the sum of the product of each second weighted result corresponding to the initial key feature respectively and the target value feature corresponding to the same image feature as the initial key feature, obtain the third weighted result corresponding to the initial key feature.

[0197] Here, since the target value feature in the value matrix is obtained after the image feature in the feature sequence is subjected to feature mapping processing, there is also a corresponding relationship between the target value feature and the image feature. In the process of determining the attention fusion feature, for any target key feature used, the initial key feature corresponding to the target key feature and the target value feature corresponding to the same image feature need to be multiplied, so the target value feature corresponding to the same image feature as the initial key feature needs to be determined first.

[0198] For example, for the jth initial key feature k j corresponding to the same image feature, the target value feature is the jth target value feature V j in the value matrix. j

[0199] S104-4-2: Add each second weighted result corresponding to each initial key feature to obtain a fourth weighted result.

[0200] S104-4-3: Based on each first weighted result, the sum of the third weighted result corresponding to each initial key feature, and the fourth weighted result, determine the attention fusion feature corresponding to the image feature.

[0201] In specific implementation, the S104-4-1 to S104-4-3 above can be executed according to the following formula seven:

[0202]

[0203] Wherein, O i represents the attention fusion feature corresponding to the ith initial query feature, since the initial query feature has a corresponding relationship with the image feature in the feature sequence, the ith initial query feature can correspond to the ith image feature in the feature sequence, therefore, O i , that is, the attention fusion feature corresponding to the ith image feature. N is equal to the number of image features in the feature sequence. represents obtaining the attention fusion feature O i is the feature under the feature dimension.

[0204] ​

[0205] wherein Q i denotes the respective first weighting result corresponding to the i-th initial query feature, R 1×2d denotes Q i is a feature under 2 feature dimensions.

[0206] K j T denotes K j is the transpose of K j denotes the respective second weighting result corresponding to the j-th initial key feature, respectively,

[0207] wherein R 1×2d denotes K j is a feature under 2 feature dimensions. V j denotes the j-th target value feature in the value matrix. K j and V j correspond to the same image feature.

[0208] K j T V j denotes the third weighting result corresponding to the j-th initial key feature K j . denotes the sum of the third weighting results corresponding to the respective initial key features. denotes the fourth weighting result.

[0209] In a specific implementation, for each image feature, the respective first weighting result of the initial query feature corresponding to the image feature, the sum of the third weighting results corresponding to the respective initial key features, and the fourth weighting result described above can be substituted into the formula seven, so as to obtain the attention fusion feature corresponding to the image feature.

[0210] For the formula seven, the following process can be used to calculate:

[0211] Firstly, the similarity between the i-th initial query feature in the query matrix and the j-th initial key feature in the key matrix can be determined by using the following formula eight:

[0212]

[0213] wherein S(Q i , K j ) denotes the similarity between the i-th initial query feature and the j-th initial key feature in the key matrix.

[0214] Based on the formula eight, the calculation formula of the attention feature can be obtained, as shown in the following formula nine:

[0215]

[0216] Will Substituting into Formula 9 above and transforming it, we obtain Formula 10 as follows:

[0217]

[0218] The softmax operation plays a crucial role in the attention mechanism. It ensures that the query and key matrices are non-negative and provides a non-linear reweighting mechanism to describe the global contextual information corresponding to each image feature. To enable the attention features O obtained using the transformed Equation 10... i Approximating the similarity O determined by softmax i That is, to approximate the attention feature O obtained using the linear attention mechanism corresponding to Formula 9. i Q can be introduced i and K j The Manhattan distance between them is used, and Equation 8 above is transformed using cosine reweighting to obtain Equation 11 below:

[0219]

[0220] Among them, a i b i a j b j It can be determined according to formulas two through five above. And in formula eleven... Where y can be a i b i a j b j cos(a) i -a j )+cos(b i -b j ) represents the Manhattan distance between the i-th initial query feature and the j-th initial key feature.

[0221] Formula 11 above includes Q' i K' j Therefore, based on Formula 11 and Formula 10, we can obtain Formula 7 above.

[0222] S105: Determine the image recognition result corresponding to the image based on the attention fusion feature corresponding to each image feature.

[0223] In practice, an image recognition neural network can be used to recognize the attention fusion features corresponding to each image feature, thereby outputting the image recognition result corresponding to the image to be recognized.

[0224] For example, the output image recognition result can be that the image includes a puppy. For another example, the output image recognition result can be that the image includes a pedestrian. In actual implementation, the image recognition neural network can be used to recognize an image in any scene, and output an image recognition result of the image.

[0225] In this way, by converting the first initial position of the initial query feature in the first feature dimension into the first target position in the second feature dimension, and converting the second initial position of the initial key feature in the first feature dimension into the second target position in the second feature dimension, the Manhattan distance between the initial query feature and the initial key feature can be represented by the first target position and the second target position. Then, according to the Manhattan distance, the cosine weighting of the initial query feature and the initial key feature is determined, so that the first weighting weight corresponding to each initial query feature and the second weighting weight corresponding to each initial key feature can be accurately obtained. Finally, based on the first weighting weight, the second weighting weight, and each target value feature, the calculation of the attention fusion feature corresponding to each image feature can improve the accuracy of determining the attention fusion feature, and can avoid the problem of excessive calculation amount caused by directly multiplying the query matrix and the key matrix, thereby effectively reducing the calculation resources and the calculation time required for determining the attention fusion feature. For an image corresponding to a long feature sequence, based on the embodiment, accurate image recognition can be realized, the recognition time can be effectively reduced, and the image recognition effect can be finally improved.

[0226] The image recognition method provided by the embodiments of the present disclosure can be applied to various image recognition scenarios. For example, it can be applied to an automatic driving scenario. Specifically, an automatic driving vehicle can perform image recognition on a driving image captured in real time during driving, determine the positions of objects such as obstacles and pedestrians on the road based on the recognition result, and avoid the objects based on the positions of the objects, thereby improving driving safety. For another example, it can be applied to an access control scenario. Specifically, a portrait of an access control can be captured to determine whether a current object that wants to pass through the access control is an authorized object. If yes, the object is allowed to pass through the access control; otherwise, the object is not allowed to pass through the access control.

[0227] In an embodiment, for S105 described above, the following steps can be implemented:

[0228] S105-1: Perform a linear full connection operation on the first attention matrix composed of the attention fusion features corresponding to the image features of each region, to obtain a second attention matrix.

[0229] Here, since the attention fusion feature O i corresponding to the first feature dimension Therefore, it is also necessary to restore the attention fusion features located in the first feature dimension to the feature dimension d corresponding to the feature sequence. The above first attention matrix can be: The first attention matrix is ​​an N×d matrix, where N is the number of attention fusion features equal to the number of image features. The second attention matrix is ​​an N×d matrix containing multiple attention fusion features corresponding to the feature dimension d.

[0230] Therefore, in practical implementation, the target linear layer in the image recognition neural network can be used to process the attention fusion features corresponding to the image features of each region. The first attention matrix is ​​used to perform a linear fully connected operation to obtain the second attention matrix.

[0231] The target linear layer (Linear) can contain pre-defined configurations. The preset feature matrix. The target linear layer utilizes... The preset feature matrix, and Multiplying the first attention matrix by the second attention matrix (N×d) yields the second attention matrix (N×d), thus restoring the attention fusion features.

[0232] For example, the first attention matrix Where O1 represents the first attention fusion feature, O N This represents the Nth attention fusion feature, where N equals the number of image feature points. (Preset) The preset feature matrix is ​​W o This means that the first attention matrix O and the preset feature matrix W are used to represent... o After multiplication, we can obtain the N×d second attention matrix O.

[0233] S105-2: Determine the image recognition result based on the second attention matrix.

[0234] Here, an image recognition neural network can be used to identify and process each attention fusion feature in the second attention matrix to output the image recognition result corresponding to the image to be recognized.

[0235] In one embodiment, S105-2 can be implemented according to the following steps:

[0236] S105-2-1: Perform pooling operation on the feature sequence to obtain the pooled feature sequence matrix.

[0237] In implementation, the pooling layer in the image recognition neural network can be used to perform a pooling operation on the Nxd feature sequence to obtain a pooled feature sequence matrix. The pooled feature sequence matrix can be a 1xd matrix. That is, the pooling layer can be used to determine the mean feature corresponding to the feature sequence.

[0238] S105-2-2: multiply the pooled feature sequence matrix and the second attention matrix to obtain a third attention matrix.

[0239] In implementation, the 1xd pooled feature sequence matrix and the second attention matrix can be directly multiplied to obtain the third attention matrix.

[0240] Alternatively, after obtaining the pooled feature sequence matrix, the pooled feature sequence matrix can be input into the first linear layer Linear, the first linear layer can be used to perform linear full connection processing on the pooled feature sequence matrix to obtain a first processing result, the first processing result can be input into the nonlinear activation layer ReLU, the ReLU layer can be used to process the first processing result to obtain a second processing result, the second processing result can be input into the second linear layer Linear, the second linear layer can be used to perform linear full connection processing on the second processing result to obtain a third processing result, and finally the third processing result can be input into the full connection layer sigmoid, the sigmoid layer can be used to process the third processing result, and the result output by the sigmoid layer can be taken as the final pooled feature sequence matrix.

[0241] Then, the final pooled feature sequence matrix and the second attention matrix can be multiplied to obtain the third attention matrix.

[0242] S105-2-3: add the third attention matrix and the second attention matrix to obtain a target attention matrix, and determine an image recognition result based on the target attention matrix.

[0243] Here, the image recognition neural network can be used to perform matrix addition on the third attention matrix and the second attention matrix to obtain an Nxd target attention matrix. The number of attention fusion features in the target attention matrix is N, and the feature dimension is d.

[0244] As Figure 3The diagram illustrates an image recognition method using an image recognition neural network according to an embodiment of this disclosure. The input to the image recognition neural network can be an N×C feature sequence corresponding to the image to be recognized. Here, C represents the feature dimension d in the aforementioned embodiments. The N×C feature sequence is then subjected to feature mapping processing using a first target feature matrix, a second target feature matrix, and a third target feature matrix, respectively, to obtain... The query matrix Q, The key matrix K and The value matrix V. Then, the objective kernel function ReLU can be used to... The initial query features in the query matrix, and Each initial key feature in the key matrix is ​​transformed to obtain the target query feature corresponding to each initial query feature, and the target key feature corresponding to each initial key feature. Simultaneously, Q can be introduced... i and K j The Manhattan distance between them is used, and cosine reweighting is applied to each target query feature and each target key feature to achieve the desired result. The query matrix is ​​transformed into an N×2C query matrix, and the query matrix is ​​transformed into an N×2C query matrix. The key matrix is ​​transformed into an N×2C key matrix, where the N×2C query matrix includes N target query features. The key matrix includes N target key features. Simultaneously, based on the linear attention mechanism provided in this embodiment, the transpose of the N×2C key matrix and... Multiplying the value matrices yields... The target matrix. Then, the query matrix of N×2C and Multiplying the target matrix yields... The first attention matrix. Using the target linear layer, ... The first attention matrix is ​​used to perform a linear fully connected operation to obtain an N×C second attention matrix.

[0245] Meanwhile, the pooling layer in the image recognition neural network can be used to perform a pooling operation on the feature sequence of NxC to obtain a feature sequence matrix of 1xC after pooling. Then, the feature sequence matrix of 1xC after pooling is input into the first linear layer Linear, the output of the first linear layer Linear is input into the nonlinear activation layer ReLU layer to obtain the output of the ReLU layer, and then the output of the ReLU layer is input into the second linear layer Linear to obtain the output of the second linear layer Linear. Finally, the output of the second linear layer Linear is input into the full connection layer sigmoid layer, and the sigmoid layer outputs the final feature sequence matrix after pooling, which is a matrix of 1xC.

[0246] Further, the final feature sequence matrix after pooling and the second attention matrix can be multiplied to obtain a third attention matrix, and the third attention matrix and the second attention matrix are added to obtain a target attention matrix of NxC. Finally, the image recognition neural network can output the image recognition result of the image to be recognized based on the target attention matrix of NxC.

[0247] Using the image recognition neural network provided by the embodiments of the present disclosure, the image recognition method provided by the embodiments of the present disclosure is executed to realize the recognition of the image, which can effectively reduce the amount of calculation. As shown in Figure 4 The recognition comparison diagram of the image recognition method provided by the embodiments of the present disclosure and the existing image recognition method is shown in FIG. 1. In region 1, the existing image recognition method is used, and in region 2, the image recognition method provided by the embodiments of the present disclosure is used.

[0248] In region 1, the feature mapping processing is performed on the feature sequence of dN, and the query matrix Q of dN, the key matrix K of dN and the value matrix V of dN can be obtained. Wherein, d is much smaller than N. Then, based on the softmax mechanism, the query matrix Q of dN and the transpose matrix (matrix of Nxd) of the key matrix K are directly multiplied to obtain a matrix of NxN. Here, the query matrix Q of dN and the transpose matrix (matrix of Nxd) of the key matrix K are directly multiplied, and the corresponding calculation complexity is O(N 2 d). Then, the matrix of NxN and the value matrix V of dN are directly multiplied to obtain an attention matrix of dN. Here, the matrix of NxN and the value matrix V of dN are directly multiplied, and the corresponding calculation complexity is also O(N 2 d). Therefore, in the above process corresponding to region 1, the total calculation complexity is O(N 2 d+N 2 d), since d is much smaller than N, the total calculation complexity is O(N 2 d+N 2 d)≈O(N2 ).

[0249] In region 2, feature mapping is performed on the d×N feature sequence to obtain a d×N query matrix Q, a d×N key matrix K, and a d×N value matrix V. Then, based on the linear attention mechanism, the transpose of the key matrix K (an N×d matrix) and the d×N value matrix V are directly multiplied to obtain a d×d matrix. Here, the computational complexity of directly multiplying the transpose of the key matrix K (an N×d matrix) and the d×N value matrix V is O(Nd). 2 Then, the d×d matrix and the d×N query matrix are directly multiplied to obtain the d×N attention matrix. Here, the computational complexity of directly multiplying the d×d matrix and the d×N query matrix is ​​also O(Nd). 2 In the process described above, corresponding to region 2, the total computational complexity is O(Nd). 2 +Nd 2 Since d is much smaller than N, the total computational complexity is O(Nd). 2 +Nd 2 )≈O(N).

[0250] In conclusion, O(N) is much smaller than O(N). 2 Therefore, the image recognition method based on linear attention mechanism provided in this embodiment can effectively reduce the amount of computation, thereby effectively improving the image recognition effect for images with corresponding long feature sequences.

[0251] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0252] Based on the same inventive concept, this disclosure also provides an image recognition device corresponding to the image recognition method. Since the principle of the device in this disclosure for solving the problem is similar to that of the image recognition method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0253] like Figure 5 The diagram shown is a schematic representation of an image recognition device provided in an embodiment of this disclosure, comprising:

[0254] The acquisition module 501 is used to acquire the feature sequence corresponding to the image to be identified, wherein the feature sequence includes image features corresponding to each region in the feature map of the image;

[0255] The conversion module 502 is configured to convert, based on a target size of the feature map, each initial query feature in a query matrix corresponding to the feature sequence from a first initial position in a first feature dimension to a first target position in a second feature dimension, and convert each initial key feature in a key matrix corresponding to the feature sequence from a second initial position in the first feature dimension to a second target position in the second feature dimension.

[0256] The first determination module 503 is configured to determine, based on the first target position, a first weighting weight corresponding to each of the initial query features, and determine, based on the second target position, a second weighting weight corresponding to each of the initial key features.

[0257] The second determination module 504 is configured to determine, for each image feature in the feature sequence, an attention fusion feature corresponding to the image feature based on the first weighting weight of the initial query feature corresponding to the image feature, the second weighting weight of each of the initial key features, and each target value feature in a value matrix corresponding to the feature sequence.

[0258] The third determination module 505 is configured to determine, based on the attention fusion feature corresponding to each of the image features, an image recognition result corresponding to the image.

[0259] In a possible implementation, the first target position includes a first target row coordinate and a first target column coordinate.

[0260] The conversion module 502 is configured to, when converting, based on the target size of the feature map, each initial query feature in the query matrix corresponding to the feature sequence from the first initial position in the first feature dimension to the first target position in the second feature dimension, determine, for each of the initial query features, a first target row coordinate corresponding to the initial query feature based on a first initial row coordinate indicated by the first initial position corresponding to the initial query feature and a height size indicated by the target size of the feature map.

[0261] determine a first target column coordinate corresponding to the initial query feature based on a first initial column coordinate indicated by the first initial position corresponding to the initial query feature and a width size indicated by the target size of the feature map.

[0262] In a possible implementation, the first determination module 503 is configured to take the first weighting weight and the second weighting weight as target weighting weights respectively, and take the first target position and the second target position as target positions, and the method further includes:

[0263] The target weighting weight comprises: a sub-weight obtained by performing a sine operation and a cosine operation on a row coordinate and a column coordinate in a target position corresponding to the target weighting weight, respectively.

[0264] In a possible implementation, the second determining module 504, when determining the attention fusion feature corresponding to the image feature based on the first weighting weight of the initial query feature corresponding to the image feature, the second weighting weight of each initial key feature, and each target value feature in the value matrix corresponding to the feature sequence, is configured to convert the initial query feature into a target query feature and convert each initial key feature into a target key feature by using a target kernel function.

[0265] The initial query feature corresponding to each first weighting result is obtained by performing weighting processing on the target query feature corresponding to the initial query feature by using each sub-weight corresponding to the first weighting weight of the initial query feature.

[0266] For each initial key feature, the target key feature corresponding to the initial key feature is processed by using each sub-weight corresponding to the second weighting weight of the initial key feature, to obtain each second weighting result corresponding to the initial key feature.

[0267] The attention fusion feature corresponding to the image feature is determined based on each first weighting result corresponding to the initial query feature, each second weighting result corresponding to each initial key feature, and each target value feature.

[0268] In a possible implementation, the second determining module 504, when determining the attention fusion feature corresponding to the image feature based on each first weighting result corresponding to the initial query feature, each second weighting result corresponding to each initial key feature, and each target value feature, is configured to obtain, for each initial key feature, a third weighting result corresponding to the initial key feature based on a sum of products of each second weighting result corresponding to the initial key feature and a target value feature corresponding to the same image feature as the initial key feature.

[0269] Each second weighting result corresponding to each initial key feature is added to obtain a fourth weighting result.

[0270] The attention fusion feature corresponding to the image feature is determined based on each first weighting result, a sum of third weighting results corresponding to each initial key feature, and the fourth weighting result.

[0271] In a possible implementation, the third determining module 505, when determining the image recognition result corresponding to the image based on the attention fusion feature corresponding to each image feature, is configured to perform a linear full connection operation on a first attention matrix composed of the attention fusion features corresponding to the image features of each region to obtain a second attention matrix.

[0272] The image recognition result is determined based on the second attention matrix.

[0273] In a possible implementation, the third determining module 505, when determining the image recognition result based on the second attention matrix, is configured to perform a pooling operation on the feature sequence to obtain a pooled feature sequence matrix.

[0274] The pooled feature sequence matrix and the second attention matrix are multiplied to obtain a third attention matrix.

[0275] The third attention matrix and the second attention matrix are added to obtain a target attention matrix, and the image recognition result is determined based on the target attention matrix.

[0276] In a possible implementation, the obtaining module 501, when obtaining the feature sequence corresponding to the image to be recognized, is configured to obtain the image to be recognized.

[0277] Based on the image to be recognized, a feature map with the target size is generated.

[0278] According to the image feature of each region in the feature map, the feature sequence is obtained.

[0279] In a possible implementation, the obtaining module 501, when obtaining the feature sequence according to the image feature of each region in the feature map, is configured to determine the position of the image feature corresponding to each region in the feature sequence according to the row and column coordinates corresponding to the region in the feature map.

[0280] According to the position corresponding to each image feature, each image feature is arranged to obtain the feature sequence.

[0281] In a possible implementation, the apparatus further includes:

[0282] The mapping module 506 is used to determine the query matrix, the key matrix, and the value matrix according to the following steps: performing feature mapping processing on the feature sequence using the target feature matrix to obtain the query matrix, the key matrix, and the value matrix corresponding to the feature sequence; wherein the query matrix, the key matrix, and the value matrix correspond to different target feature matrices, and the query matrix, the key matrix, and the value matrix correspond to the first feature dimension.

[0283] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0284] Based on the same technical concept, embodiments of this application also provide a computer device. (Refer to...) Figure 6 The diagram shown is a structural schematic of a computer device provided in an embodiment of this application, comprising:

[0285] The system includes a processor 61, a memory 62, and a bus 63. The memory 62 stores machine-readable instructions executable by the processor 61. The processor 61 executes these machine-readable instructions, and when executed, it performs the following steps: S101: Obtaining a feature sequence corresponding to the image to be identified. The feature sequence includes image features corresponding to each region in the feature map of the image; S102: Based on the target size of the feature map, converting each initial query feature in the query matrix corresponding to the feature sequence from its first initial position in the first feature dimension to its first target position in the second feature dimension, and converting each initial key feature in the key matrix corresponding to the feature sequence from its first initial position in the second feature dimension to its first target position in the second feature dimension. S103: Based on the first target position, determine the first weighted weight corresponding to each initial query feature, and based on the second target position, determine the second weighted weight corresponding to each initial key feature; S104: For each image feature in the feature sequence, determine the attention fusion feature corresponding to the image feature based on the first weighted weight of the initial query feature corresponding to the image feature, the second weighted weight corresponding to each initial key feature, and each target value feature in the value matrix corresponding to the feature sequence; and S105: Based on the attention fusion feature corresponding to each image feature, determine the image recognition result corresponding to the image.

[0286] The memory 62 includes an internal memory 621 and an external memory 622; the internal memory 621 is also referred to as an internal storage, and is used for temporarily storing operation data in the processor 61 and exchanging data with the external memory 622 such as a hard disk. The processor 61 exchanges data with the external memory 622 through the internal memory 621. When the computer device is running, the processor 61 communicates with the memory 62 through the bus 63, so that the processor 61 executes the execution instructions mentioned in the above method embodiments.

[0287] The disclosure also provides a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the image recognition method described in the above method embodiments are executed. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0288] The computer program product of the image recognition method provided by the disclosure includes a computer readable storage medium storing program codes. The program codes include instructions for executing the steps of the image recognition method described in the above method embodiments. For details, refer to the above method embodiments, which will not be repeated here.

[0289] The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0290] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In several embodiments provided by the disclosure, it should be understood that the disclosed devices and methods can be implemented by other means. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and actual implementation can have another division manner. For another example, a plurality of units or components can be combined, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed elements can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0291] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0292] In addition, each functional unit in various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0293] If the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or say the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, and various media that can store program codes.

[0294] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, set up clear and obvious mark to inform that it has entered the personal information collection range, and will collect personal information, if the individual voluntarily enters the collection range, it is considered to agree to collect personal information; or on the device for processing personal information, through the pop-up information or by uploading personal information by the individual, the individual is authorized under the condition of using obvious mark / information to inform the personal information processing rules; wherein, the personal information processing rules can include personal information processor, personal information processing purpose, processing method, processing personal information type and other information.

[0295] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit the present disclosure. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easy changes to the technical solutions described in the foregoing embodiments, or easily think of changes or equivalent replacements for some of the technical features; and these modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. An image recognition method characterized by, The method comprises: obtaining a feature sequence corresponding to an image to be recognized, the feature sequence comprising image features corresponding to respective regions in a feature map of the image; based on a target size of the feature map, converting each initial query feature in a query matrix corresponding to the feature sequence from a first initial position in a first feature dimension to a first target position in a second feature dimension, and converting each initial key feature in a key matrix corresponding to the feature sequence from a second initial position in the first feature dimension to a second target position in the second feature dimension; the first target position comprises a first target row coordinate and a first target column coordinate; the target size comprises a height size and a width size of the feature map; based on the first target position, determining a first weighted weight corresponding to each of the initial query features, and based on the second target position, determining a second weighted weight corresponding to each of the initial key features; the first weighted weight comprises two sub-weights corresponding to the first target row coordinate and the first target column coordinate, respectively; wherein, taking the first weighted weight and the second weighted weight as target weighted weights, and taking the first target position and the second target position as target positions, the target weighted weight comprises sub-weights obtained by performing a sine operation and a cosine operation on a row coordinate and a column coordinate in the target position corresponding to the target weighted weight, respectively; for each image feature in the feature sequence, based on the first weighted weight of the initial query feature corresponding to the image feature, the second weighted weight of each of the initial key features, and each target value feature in a value matrix corresponding to the feature sequence, determining an attention fusion feature corresponding to the image feature; based on the attention fusion feature corresponding to each of the image features, determining an image recognition result corresponding to the image.

2. The method of claim 1, wherein, The method comprises: for each of the initial query features, based on a first initial row coordinate indicated by the first initial position corresponding to the initial query feature and a height size indicated by the target size of the feature map, determining a first target row coordinate corresponding to the initial query feature; based on a first initial column coordinate indicated by the first initial position corresponding to the initial query feature and a width size indicated by the target size of the feature map, determining a first target column coordinate corresponding to the initial query feature.

3. The method of claim 2, wherein, The method comprises: using a target kernel function, converting the initial query feature into a target query feature, and converting each of the initial key features into a target key feature; The target query features corresponding to the initial query features are respectively weighted by using the sub-weights corresponding to the first weighting weight of the initial query features, to obtain first weighted results corresponding to the initial query features; For each initial key feature, the target key features corresponding to the initial key feature are respectively weighted by using the sub-weights corresponding to the second weighting weight of the initial key feature, to obtain second weighted results corresponding to the initial key feature; Based on the first weighted results corresponding to the initial query features, the second weighted results corresponding to each initial key feature, and each target value feature, an attention fusion feature corresponding to the image feature is determined.

4. The method of claim 3, wherein, The determination of the attention fusion feature corresponding to the image feature based on the first weighted results corresponding to the initial query features, the second weighted results corresponding to each initial key feature, and each target value feature comprises: For each initial key feature, a third weighted result corresponding to the initial key feature is obtained based on the sum of the products of the second weighted results corresponding to the initial key feature and the target value features corresponding to the same image feature as the initial key feature; The second weighted results corresponding to each initial key feature are added to obtain a fourth weighted result; The attention fusion feature corresponding to the image feature is determined based on the first weighted results, the sum of the third weighted results corresponding to each initial key feature, and the fourth weighted result.

5. The method of claim 1, wherein, The determination of the image recognition result corresponding to the image based on the attention fusion feature corresponding to each image feature comprises: A linear full connection operation is performed on a first attention matrix composed of the attention fusion features corresponding to the image features of each region to obtain a second attention matrix; The image recognition result is determined based on the second attention matrix.

6. The method of claim 5, wherein, The determination of the image recognition result based on the second attention matrix comprises: A pooling operation is performed on the feature sequence to obtain a pooled feature sequence matrix; The pooled feature sequence matrix and the second attention matrix are multiplied to obtain a third attention matrix; The third attention matrix and the second attention matrix are added to obtain a target attention matrix, and the image recognition result is determined based on the target attention matrix.

7. The method of claim 1, wherein, Obtaining a feature sequence corresponding to an image to be recognized comprises: Obtaining an image to be recognized; Generating a feature map with the target size based on the image to be recognized; Obtaining the feature sequence according to the image features of each region in the feature map.

8. The method of claim 7, wherein, The obtaining of the feature sequence according to the image features of each region in the feature map comprises: Determining the position of the image feature corresponding to each region in the feature sequence according to the row and column coordinates corresponding to the region in the feature map; Arranging each image feature according to the position corresponding to the image feature to obtain the feature sequence.

9. The method according to any one of claims 1 to 8, characterized in that, The method further comprises the step of determining the query matrix, the key matrix and the value matrix: The feature sequence is respectively mapped by using a target feature matrix to obtain the query matrix, the key matrix and the value matrix corresponding to the feature sequence; wherein the query matrix, the key matrix and the value matrix correspond to different target feature matrices respectively, and the query matrix, the key matrix and the value matrix correspond to the first feature dimension respectively.

10. An image recognition apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire a feature sequence corresponding to an image to be recognized, the feature sequence comprising image features corresponding to respective regions in a feature map of the image; A conversion module is configured to convert each initial query feature in a query matrix corresponding to the feature sequence from a first initial position in a first feature dimension to a first target position in a second feature dimension based on a target size of the feature map, and convert each initial key feature in a key matrix corresponding to the feature sequence from a second initial position in the first feature dimension to a second target position in the second feature dimension; the first target position comprises a first target row coordinate and a first target column coordinate; the target size comprises a height size and a width size of the feature map; A first determination module is configured to determine a first weighting weight corresponding to each initial query feature based on the first target position, and determine a second weighting weight corresponding to each initial key feature based on the second target position; the first weighting weight comprises two sub-weights corresponding to the first target row coordinate and the first target column coordinate respectively; wherein, when the first weighting weight and the second weighting weight are taken as target weighting weights respectively, and the first target position and the second target position are taken as target positions respectively, the target weighting weight comprises sub-weights obtained by performing a sine operation and a cosine operation on a row coordinate and a column coordinate in the target position corresponding to the target weighting weight respectively; A second determination module is configured to determine, for each image feature in the feature sequence, an attention fusion feature corresponding to the image feature based on the first weighting weight of the initial query feature corresponding to the image feature, the second weighting weight of each initial key feature, and each target value feature in a value matrix corresponding to the feature sequence; A third determination module is configured to determine an image recognition result corresponding to the image based on the attention fusion feature corresponding to each image feature.

11. A computer device, comprising: Comprise: A processor and a memory, the memory storing machine-readable instructions executable by the processor, the processor being configured to execute the machine-readable instructions stored in the memory, and the machine-readable instructions, when executed by the processor, causing the processor to perform the steps of the image recognition method of any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program, when executed by a computer device, causes the computer device to perform the steps of the image recognition method of any one of claims 1 to 9.

Images