Image recognition method, electronic device, and storage medium

Abstract

This invention discloses an image recognition method, an electronic device, and a storage medium. The image recognition method includes: acquiring an image to be recognized; extracting features from the image to obtain a basic feature map; dividing the basic feature map into multiple sub-feature maps; performing sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map; and determining the image features of the basic feature map based on the features of each sub-feature map; and determining an approximate image of the image to be recognized from multiple candidate images based on the image features. Through the above methods, this invention can improve the accuracy of image recognition.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to image recognition methods, electronic devices, and storage media. Background Technology

[0002] With the rapid development of science and technology and the arrival of the big data era, the amount of information is becoming increasingly massive. Image recognition, as a safe, contactless, convenient, user-friendly, and efficient method of information retrieval, has been widely applied to all aspects of social life.

[0003] Image recognition technology allows users to retrieve target images based on the image information they provide, therefore, deep learning-based image recognition technology has received widespread attention and research.

[0004] However, due to the wide variety of network images currently available, existing algorithms suffer from numerous problems, such as low accuracy. Summary of the Invention

[0005] This invention provides an image recognition method, an electronic device, and a storage medium to solve the problem of low image recognition accuracy.

[0006] To address the aforementioned technical problems, this invention provides an image recognition method, comprising: acquiring an image to be recognized; extracting features from the image to be recognized to obtain a basic feature map of the image to be recognized; dividing the basic feature map into multiple sub-feature maps; performing sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map; and determining the image features of the basic feature map based on the features of each sub-feature map; and determining an approximate image of the image to be recognized from multiple candidate images based on the image features.

[0007] The process involves dividing the base feature map into multiple sub-feature maps, performing sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map, and determining the image features of the base feature map based on the features of each sub-feature map. This includes: dividing the base feature map into multiple sub-feature maps, performing sub-region mean pooling on each sub-feature map to obtain the features of each sub-feature map; determining the mean pooling features of the base feature map by combining the features of each sub-feature map; performing channel pooling on the base feature map to obtain the channel pooling features of the base feature map; and determining the image features of the base feature map based on the mean pooling features and the channel pooling features.

[0008] The process of determining the image features of the base feature map based on mean pooling features and channel pooling features includes: concatenating the mean pooling features and channel pooling features to obtain the image features of the base feature map.

[0009] The basic feature map is divided into multiple sub-feature maps, and sub-region mean pooling is performed on each sub-feature map to obtain the features of each sub-feature map. This includes: dividing the basic feature map into multiple sub-feature maps based on the width and height dimensions of the basic feature map; and performing sub-region mean pooling on each sub-feature map to obtain the one-dimensional features of each sub-feature map.

[0010] The process involves channel pooling of the basic feature maps to obtain the channel pooling features of each basic feature map. This includes converting the number of channels in the basic feature map to 1 through channel pooling and expanding the width and height dimensions of the basic feature map to obtain the channel pooling features of the basic feature map.

[0011] The process of acquiring the image to be identified and extracting features from the image to obtain the basic feature map of the image to be identified includes: acquiring the image to be identified, extracting semantic information from the image to be identified, and forming an initial feature map of the image to be identified; performing multiple convolution processes on the initial feature map with different kernel sizes to obtain multiple basic feature maps of different scales; and determining an approximate image of the image to be identified from multiple candidate images based on image features, which also includes: performing a weighted average of the image features of each basic feature map; and performing feature matching between the weighted averaged image features and the image features of each candidate image to determine an approximate image of the image to be identified from multiple candidate images.

[0012] The process involves matching the weighted average image features with the image features of each candidate image to determine an approximate image of the image to be identified from multiple candidate images. This includes: matching the weighted average image features of the image to be identified with the image features of each candidate image to determine the matching degree between the image to be identified and each candidate image; arranging the multiple candidate images in descending order of matching degree; and determining the first preset number of candidate images as approximate images of the image to be identified.

[0013] The image recognition model includes cascaded feature extraction networks, feature processing networks, and feature matching networks. The process involves: acquiring the image to be recognized; extracting features from the image to obtain a basic feature map; dividing the basic feature map into multiple sub-feature maps; performing sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map; and determining the image features of the basic feature map based on the features of each sub-feature map. The final step involves determining an approximate image of the image to be recognized from multiple candidate images based on the image features; and using the feature matching network of the image recognition model to determine an approximate image of the image to be recognized from multiple candidate images based on the image features.

[0014] Before acquiring the image to be recognized and extracting features from it to obtain the basic feature map, the process includes: acquiring a training image, extracting features from the training image to obtain the training basic feature map; dividing the training basic feature map into multiple training sub-feature maps, performing sub-region feature processing on each training sub-feature map to obtain the features of each training sub-feature map, and determining the image features of the training basic feature map based on the features of each training sub-feature map; and training the image recognition model in training based on the image features of the training basic feature map until a fully trained image recognition model is obtained.

[0015] The process of training the image recognition model based on the image features of the training base feature map until a fully trained image recognition model is obtained includes: training the image recognition model based on the differences between the features of each training sub-feature map in the image features and the features of the corresponding sub-feature map in the positive sample image, as well as the differences between the features of each training sub-feature map in the training base feature map and the features of the corresponding sub-feature map in the negative sample image, until a fully trained image recognition model is obtained.

[0016] The process of training the image recognition model based on the image features of the training base feature map until a fully trained image recognition model is obtained includes: predicting the classification type of the training image based on the image features of the training base feature map; and training the image recognition model based on the difference between the classification type of the training image and the classification type of the sample image until a fully trained image recognition model is obtained.

[0017] The process of acquiring an image to be identified and extracting features from the image to obtain a basic feature map of the image to be identified includes: acquiring an image of a product to be identified and extracting features from the image to obtain a basic feature map of the product to be identified; and determining an approximate image of the image to be identified from multiple candidate images based on image features, including: determining an approximate product image of the image to be identified from multiple candidate product images based on image features.

[0018] To address the aforementioned technical problems, the present invention also provides an electronic device comprising: a memory and a processor coupled to each other, wherein the processor is configured to execute program instructions stored in the memory to implement the image recognition method described above.

[0019] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing program data that can be executed to implement the image recognition method as described above.

[0020] The beneficial effects of this invention are as follows: Unlike existing technologies, this invention extracts features from the image to be identified to obtain a basic feature map of the image; divides the basic feature map into multiple sub-feature maps, performs sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map, and determines the image features of the basic feature map based on the features of each sub-feature map; and determines an approximate image of the image to be identified from multiple candidate images based on the image features. This can obtain image features of the basic feature map including features of multiple local sub-feature maps, thereby improving the accuracy of the image features of the basic feature map, and thus improving the accuracy of determining the approximate image of the image to be identified from multiple candidate images, and improving the accuracy of image recognition. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating an embodiment of the image recognition method provided by the present invention;

[0022] Figure 2 This is a schematic diagram of the structure of one embodiment of the image recognition model of the present invention;

[0023] Figure 3 This is a flowchart illustrating another embodiment of the image recognition method provided by the present invention;

[0024] Figure 4 yes Figure 3 A schematic diagram of the structure of one embodiment of the image recognition model in the example;

[0025] Figure 5 yes Figure 3 Sub-features in the embodiments Figure 1 A schematic diagram of the implementation method;

[0026] Figure 6 This is a schematic diagram of the training process of an embodiment of the image recognition model of the present invention;

[0027] Figure 7 This is a schematic diagram of the framework of an embodiment of the image recognition device of the present invention;

[0028] Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device provided by the present invention;

[0029] Figure 9 This is a schematic diagram of an embodiment of the computer-readable storage medium provided by the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0031] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the image recognition method provided by the present invention.

[0032] Step S11: Obtain the image to be recognized, extract features from the image to be recognized, and obtain the basic feature map of the image to be recognized.

[0033] The image to be identified is obtained, which may include any target object such as goods, objects, living organisms, plants, tools, etc.

[0034] Feature extraction is performed on the image to be recognized to obtain the basic feature map of the image.

[0035] In a specific application scenario, a trained feature extraction network can be used to extract features from the image to be recognized, thereby obtaining the basic feature map of the image. The feature extraction network can include ResNet-50, AlexNet, VGG (Visual Geometry Group), or residual networks, etc., and is not limited here.

[0036] In another specific application scenario, features can also be extracted from the image to be recognized using LBP (Local Binary Patterns) feature extraction algorithms, HOG (Histogram of Oriented Gradients) algorithms, SIFT (Scale-invariant feature transform) operators, or the Haar algorithm to obtain the basic feature map of the image to be recognized. No further limitations are specified here.

[0037] Step S12: Divide the basic feature map into multiple sub-feature maps, perform sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map, and determine the image features of the basic feature map based on the features of each sub-feature map.

[0038] The basic feature map is divided into multiple sub-feature maps. Sub-region feature processing is performed on each sub-feature map to obtain the features of each sub-feature map. Based on the features of each sub-feature map, the image features of the basic feature map are determined.

[0039] The basic feature map is formed by combining multiple sub-feature maps obtained from the basic feature map. The number of sub-feature maps can be set based on actual needs, such as 16, 25, or 20. The more sub-feature maps there are, the higher the accuracy of the image features in the basic feature map.

[0040] The sub-region feature processing refers to specific feature processing performed on each sub-region individually. This feature processing may specifically include feature refinement, pooling, feature filtering, and further feature extraction, etc., which are not limited in this embodiment.

[0041] In a specific application scenario, the basic feature map can be divided into 25 sub-feature maps. Sub-region feature processing is performed on each of the 25 sub-feature maps to obtain the features of the 25 sub-feature maps. Based on the features of the 25 sub-feature maps, the image features of the basic feature map are determined.

[0042] This step divides the basic feature map into multiple sub-feature maps, performs feature processing on each sub-feature map to obtain the features of each sub-feature map, and then obtains the image features of the basic feature map. This can improve the accuracy of the image features of the basic feature map, and thus improve the accuracy of subsequent image recognition.

[0043] Step S13: Based on image features, determine the approximate image of the image to be identified from multiple candidate images.

[0044] After obtaining the image features of the basic feature map, an approximate image of the image to be identified is determined from multiple candidate images based on the image features of the basic feature map. The approximate image may include candidate images from multiple candidate images whose similarity to the image to be identified exceeds a certain threshold, or it may include candidate images from multiple candidate images whose target object is the same as the target object in the image to be identified. There is no limitation here.

[0045] Multiple candidate images can be combined to form a candidate library for image recognition methods to identify and judge. The selection of multiple candidate images can be based on manual selection, image import from the platform, or import of third-party image data, etc., and there are no restrictions here.

[0046] In one specific application scenario, when image recognition methods are applied to product recognition, multiple candidate images can include product images of multiple products. In another specific application scenario, when image recognition methods are applied to human body recognition, multiple candidate images can include full-body images of multiple human bodies. No further limitations are imposed here.

[0047] In one specific application scenario, feature matching can be performed based on the similarity between the image features of the base feature map and the corresponding image features of each candidate image to determine an approximate image of the image to be identified. In another specific application scenario, one or more of the following feature matching networks, such as the SuperGlue feature matching network, BP (Back Propagation) neural network, RBF-Radial Basis Function neural network, linear neural network, convolutional neural network, and recurrent neural network, can be used to determine an approximate image of the image to be identified from multiple candidate images based on the image features of the base feature map.

[0048] Through the above steps, the image recognition method of this embodiment extracts features from the image to be recognized to obtain a basic feature map of the image to be recognized; divides the basic feature map into multiple sub-feature maps, performs sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map, and determines the image features of the basic feature map based on the features of each sub-feature map; and determines an approximate image of the image to be recognized from multiple candidate images based on the image features. This method can obtain image features of the basic feature map that include features of multiple local sub-feature maps, thereby improving the accuracy of the image features of the basic feature map, and thus improving the accuracy of determining the approximate image of the image to be recognized from multiple candidate images, and improving the accuracy of image recognition.

[0049] In other embodiments, the image recognition method can perform image recognition using an image recognition model, wherein the image recognition model includes a cascaded feature extraction network, a feature processing network, and a feature matching network.

[0050] Please see Figure 2 , Figure 2 This is a schematic diagram of one embodiment of the image recognition model of the present invention.

[0051] The image recognition model 20 of this embodiment includes a cascaded feature extraction network 21, a feature processing network 22, and a feature matching network 23. The feature extraction network 21 is responsible for extracting semantic information from the image to be recognized to form a basic feature map. The feature processing network 22 is used to extract image features from the basic feature map. The feature matching network 23 is used to perform feature matching based on the image features of the image to be recognized and the corresponding features of the candidate images to perform image recognition.

[0052] During image recognition, the image to be recognized is acquired, and the feature extraction network 21 of the image recognition model 20 is used to extract features from the image to be recognized to obtain the basic feature map of the image to be recognized.

[0053] The basic feature map is then divided into multiple sub-feature maps, and the feature processing network 22 of the image recognition model 20 performs sub-region feature processing on each sub-feature map of the basic feature map to obtain the features of each sub-feature map. Based on the features of each sub-feature map, the image features of the basic feature map are determined.

[0054] Finally, the feature matching network 23 of the image recognition model 20 determines an approximate image of the image to be recognized from multiple candidate images based on the image features of the basic feature map.

[0055] The image recognition model 20 of this embodiment can be applied to the image recognition method of any embodiment of this application.

[0056] Image recognition using image recognition models can improve the efficiency and accuracy of image recognition.

[0057] Please see Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of the image recognition method provided by the present invention. This embodiment uses feature processing including pooling as an example for explanation. When the feature processing is other, the specific method is similar to this embodiment and will not be described again.

[0058] Step S31: Obtain the image to be recognized, extract the semantic information of the image to be recognized, and form the initial feature map of the image to be recognized; perform multiple convolution processes on the initial feature map with different kernel sizes to obtain multiple basic feature maps of different scales.

[0059] The image to be identified is acquired, and its semantic information is extracted to form an initial feature map. Then, the initial feature map is subjected to multiple convolutional processes with different kernel sizes to obtain multiple base feature maps of different scales.

[0060] In a specific application scenario, the semantic information of the image to be recognized can be extracted by the feature extraction network of the image recognition model to form the initial feature map of the image to be recognized. Then, the initial feature map is subjected to multiple convolution processes with different kernel sizes to obtain multiple basic feature maps of different scales.

[0061] Please see Figure 4 , Figure 4 yes Figure 3 A schematic diagram of the structure of one embodiment of the image recognition model in the example.

[0062] The image recognition model in this embodiment includes a cascaded feature extraction network 41, a feature processing network 42, a concatenation layer 44, a normalization layer 45, and a feature matching network 43. The feature extraction network 41 extracts features from the image to be recognized, obtaining a basic feature map. The feature processing network 42 performs pooling on the basic feature map. The concatenation layer 44 concatenates the pooled features. The normalization layer 45 normalizes the concatenated features. The feature matching network 43 matches the normalized features with the features corresponding to the candidate images.

[0063] The feature extraction network 41 includes cascaded feature extraction layers 401 and multiple convolutional layers. The feature processing network 42 includes multiple pooling layers. The number of convolutional layers and pooling layers is the same, and can be 3, 5, 7, 8, etc., without limitation.

[0064] This step can extract semantic information of the image to be recognized through the feature extraction layer 401 of the feature extraction network 41 to form an initial feature map of the image to be recognized. Then, the initial feature map is subjected to multiple convolutional processes with different kernel sizes through multiple convolutional layers of the feature extraction network 41 to obtain multiple basic feature maps of different scales.

[0065] In a specific application scenario, the feature extraction layer 401 can use the ResNet-50 network. In order to retain more detailed information of the basic feature map, the stride of the last convolutional layer of the feature extraction layer 401 can be set to 1, and the size of the initial output feature map is 64*64 with 256 channels.

[0066] In other application scenarios, the feature extraction layer 401 can also use other types of networks to generate initial feature maps of other sizes and number of channels.

[0067] After generating the initial feature map of the image to be recognized, the basic feature map is convolved by multiple convolutional layers with kernels of different sizes to obtain multiple basic feature maps of different scales. The number of convolutional layers with different kernel sizes and the specific size of the kernels can be set according to actual needs and are not limited here.

[0068] In a specific application scenario, the convolutional layers in the feature extraction network 41 may include a first convolutional layer 411, a second convolutional layer 412, and a third convolutional layer 413. After the feature extraction layer 401 generates the initial feature map, the initial feature map is input into the first convolutional layer 411, the second convolutional layer 412, and the third convolutional layer 413 for different convolutional processing, resulting in three basic feature maps of different scales. Specifically, the kernel size in the first convolutional layer 411 can be 3*3, the kernel size in the second convolutional layer 412 can be 5*5, and the kernel size in the third convolutional layer 413 can be 7*7. This application scenario uses three convolutional layers in the feature extraction network 41 as an example. In other application scenarios, the number of convolutional layers in the feature extraction network 41 can be 2, 4, 5, 10, etc., and the kernel size of each convolutional layer can also be other sizes, such as 9*9 or 15*15, etc., without limitation here.

[0069] Since base feature maps of different scales contain different feature scales, subsequent processing using base feature maps of different scales can improve the accuracy of feature processing and the accuracy of subsequent image recognition.

[0070] Step S32: Divide the basic feature map into multiple sub-feature maps, perform sub-region mean pooling on each sub-feature map of the basic feature map to obtain the features of each sub-feature map; determine the set of features of each sub-feature map as the mean pooling feature of the basic feature map.

[0071] Specifically, when dividing the feature maps into sub-feature maps, the base feature maps can be divided into multiple sub-feature maps based on their width and height dimensions. Then, sub-region mean pooling is performed on each sub-feature map of each base feature map to obtain one-dimensional features of each sub-feature map. Finally, the set of one-dimensional features of each sub-feature map is determined as the mean pooling feature of the corresponding base feature map.

[0072] Since the width and height dimensions of the basic feature map correspond to the width and height of the image to be recognized, the basic feature map is divided into multiple sub-feature maps based on the width and height dimensions of each basic feature map. Sub-region mean pooling is then performed on each sub-feature map. This fully considers the contribution of features at different locations on the image to be recognized to image matching, thereby improving the robustness and accuracy of image recognition.

[0073] Please see Figure 5 , Figure 5 yes Figure 3 Sub-features in the embodiments Figure 1 A schematic diagram of the implementation method.

[0074] In this embodiment, when the size of the basic feature map is 64*64*256 (width, height, number of channels), the basic feature map is divided into 16 sub-feature maps based on the 64*64 width and height dimensions. Then, sub-region mean pooling is performed on each sub-feature map to obtain a 16*4*4 one-dimensional feature of each sub-feature map. Finally, the set of one-dimensional features of the 16 sub-feature maps, 16*16*4*4=4096, is determined as the mean pooling feature corresponding to the basic feature map.

[0075] The above partitioning of the basic feature map into spatial groups can help improve the spatial distribution of semantic features of different sub-feature maps, generate a large statistical variance, enhance the feature learning of semantic regions, compress noise and interference, and at the same time have very strong interpretability of semantic representation enhancement, thus improving the robustness of image recognition models.

[0076] In other implementations, the number of sub-feature maps obtained by dividing the basic feature map can also be other numbers, which are not limited here.

[0077] In a specific application scenario, after obtaining three basic feature maps of different scales from the first convolutional layer 411, the second convolutional layer 412, and the third convolutional layer 413, the basic feature map output by the first convolutional layer 411 is input into the first pooling layer 421 of the feature processing network 42, the basic feature map output by the second convolutional layer 412 is input into the second pooling layer 422 of the feature processing network 42, and the basic feature map output by the third convolutional layer 413 is input into the third pooling layer 432 of the feature processing network 42. The basic feature maps are then divided into multiple sub-feature maps, and mean pooling is performed on each sub-feature map to obtain one-dimensional features of each sub-feature map. Then, the mean pooling feature of the corresponding basic feature map is determined based on the set of one-dimensional features of each sub-feature map corresponding to the basic feature map.

[0078] In this network, the number of pooling layers in the feature processing network 42 is the same as the number of convolutional layers in the feature extraction network 41. When the number of convolutional layers in the feature extraction network 41 is other than the number of convolutional layers, the number of pooling layers in the feature processing network 42 is also other than the number of convolutional layers.

[0079] Step S33: Perform channel pooling on the basic feature map to obtain the channel pooled features of the basic feature map.

[0080] After obtaining multiple basic feature maps of different scales, when performing mean pooling on each sub-feature map of each basic feature map, channel pooling will also be performed on each basic feature map to obtain the channel pooled features of each basic feature map.

[0081] Specifically, the number of channels in the basic feature map is converted to 1 through channel pooling, and the width and height dimensions of the basic feature map are expanded to obtain the channel pooled features of the basic feature map.

[0082] In a specific application scenario, when the base feature map is 64*64*256 (width, height, number of channels), channel pooling is used to convert the number of channels in the base feature map to 1, and the width and height dimensions of the base feature map are expanded to obtain the channel pooling feature of the base feature map, that is, the channel pooling feature with a dimension of 64*64=4096.

[0083] In this step, the channel pooling process is independent of the mean pooling process in the previous step, and they do not interfere with each other and run in parallel.

[0084] In a specific application scenario, in addition to performing channel pooling on the corresponding basic feature maps, the first pooling layer 421, the second pooling layer 422, and the third pooling layer 432 also perform channel pooling on the corresponding basic feature maps to convert the number of channels of each basic feature map to 1, and expand the width and height dimensions of the basic feature maps to obtain the channel pooled features of each basic feature map.

[0085] Step S34: Determine the image features of the basic feature map based on mean pooling features and channel pooling features.

[0086] Image features of the base feature map are determined based on mean pooling and channel pooling features. Specifically, the mean pooling and channel pooling features can be concatenated to obtain the image features of the base feature map. The concatenation can be performed using methods such as end-to-end concatenation or Python matrices, and is not limited here. In other embodiments, feature fusion, feature filtering, etc., can also be used to determine the image features of the base feature map based on mean pooling and channel pooling features, and is not limited thereto.

[0087] In a specific application scenario, the first pooling layer 421, the second pooling layer 422, and the third pooling layer 432 respectively input the mean pooling features and channel pooling features of the corresponding basic feature maps with a dimension of 4096 into the concatenation layer 44, so that the mean pooling features and channel pooling features of each basic feature map are concatenated by the concatenation layer 44 to obtain image features of the three basic feature maps with a dimension of 6144.

[0088] Step S35: Perform weighted averaging on the image features of each basic feature map; perform feature matching between the weighted averaged image features and the image features of each candidate image to determine an approximate image of the image to be identified from multiple candidate images.

[0089] Based on the image features of the basic feature maps, an approximate image of the image to be identified is determined from multiple candidate images. Specifically, the image features of each basic feature map are subjected to weighted averaging to normalize the image features of each basic feature map, resulting in the final image features of the image to be identified. The weighted averaged image features are then matched with the image features of each candidate image to determine the approximate image of the image to be identified from multiple candidate images.

[0090] In a specific application scenario, after the stitching layer 44 stitches together the image features of the three basic feature maps with a dimension of 6144, the image features of the three basic feature maps are input into the normalization layer 45 for weighted averaging to obtain the final image features of the image to be identified. The image features of the image to be identified are then input into the feature matching network 43 to perform feature matching with the image features of each candidate image in order to determine the approximate image of the image to be identified from multiple candidate images.

[0091] The image features of candidate images can be obtained in the same way as the final image features of the image to be recognized, so that feature matching can reflect the similarity between the image to be recognized and each candidate image. Furthermore, the image features of each candidate image are obtained by pre-processing each candidate image before the image recognition method begins.

[0092] In one specific application scenario, the weighted average image features of the image to be identified can be matched with the image features of each candidate image to determine the matching degree between the image to be identified and each candidate image. Multiple candidate images are then arranged in descending order of matching degree, and the first preset number of candidate images are determined as approximate images of the image to be identified. In another specific application scenario, candidate images whose matching degree meets a preset threshold can also be determined as approximate images of the image to be identified based on the feature matching results. The preset number and preset threshold can be determined based on actual needs and are not limited here.

[0093] The image recognition in this embodiment can determine the approximate image of the image to be recognized from multiple candidate images, without requiring the candidate images to have classification labels themselves. Therefore, it can simplify the cumbersome data processing and label acquisition process and improve the efficiency of image recognition.

[0094] In one specific implementation, the image recognition method can be applied to product image recognition. First, an image of the product to be recognized is acquired. Features are extracted from the product image to obtain a basic feature map. This basic feature map is then divided into multiple sub-feature maps. Sub-region feature processing is performed on each sub-feature map to obtain its features. Based on the features of each sub-feature map, the image features of the basic feature map are determined. Based on these image features, an approximate product image is determined from multiple candidate product images, and this approximate product image is then displayed.

[0095] Specifically, the process begins by acquiring an image of the product to be identified, extracting its semantic information, and forming an initial feature map. This initial feature map is then subjected to multiple convolutional processes with different kernel sizes to obtain multiple base feature maps at different scales. These base feature maps are further divided into sub-feature maps. Mean pooling is then applied to each sub-feature map to obtain mean pooling features, and channel pooling is applied to obtain channel pooling features. Based on these mean pooling and channel pooling features, the image features of the base feature maps are determined. Finally, approximate product images are determined based on the image features of the base feature maps, and these approximate product images are then displayed together.

[0096] Through the above steps, after obtaining the initial feature map, this embodiment performs convolution processing on the initial feature map using multiple convolutional layers with different kernels to obtain multiple basic feature maps of different scales. This allows for feature extraction from the image to be recognized based on a multi-branch feature extraction network, effectively achieving multi-model integration and significantly reducing the size and training time of the entire image recognition model. Furthermore, this embodiment combines spatial group mean pooling and channel global pooling image features for feature matching. This takes into account the fact that information from different locations in the image to be recognized contributes differently to image matching. By adding channel global pooling, this limitation of spatial group pooling features is compensated for, enhancing the expressive and discriminative capabilities of the features for the target object in the image to be recognized, thus improving image recognition accuracy. Moreover, image recognition only requires determining an approximate image of the image to be recognized from multiple candidate images, without requiring the candidate images themselves to have classification labels. Therefore, it simplifies the cumbersome data processing and label acquisition process, improving image recognition efficiency.

[0097] Please see Figure 6 , Figure 6 This is a schematic diagram of the training process of an embodiment of the image recognition model of the present invention.

[0098] Step S61: Obtain the training image, extract features from the training image, and obtain the training base feature map of the training image.

[0099] Once the training images are obtained, features are extracted from them using the image recognition model in training to obtain the training base feature map of the training images.

[0100] The feature extraction method in this step is the same as the feature extraction method for the image to be identified in the previous embodiment. Please refer to the previous text for details, and it will not be repeated here.

[0101] Step S62: Divide the training base feature map into multiple training sub-feature maps, perform sub-region feature processing on each training sub-feature map to obtain the features of each training sub-feature map, and determine the image features of the training base feature map based on the features of each training sub-feature map.

[0102] The sub-region division and sub-region feature processing in this step are the same as those in the previous embodiment for the sub-region division and sub-region feature processing of the image to be identified. Please refer to the previous text for details, which will not be repeated here.

[0103] This embodiment performs sub-region feature processing on multiple training sub-feature maps of the training base feature map, which can fully consider the importance of the features of each training sub-feature map on the training base feature map, so that the image recognition model in training can learn each training sub-feature map and the corresponding printed background noise in a targeted manner, thereby improving the training effect of the image recognition model.

[0104] Step S63: Train the image recognition model in training based on the image features of the training base feature map until the trained image recognition model is obtained.

[0105] The image recognition model is trained based on the image features of the training base feature map until a fully trained image recognition model is obtained.

[0106] In a specific application scenario, the sample images include positive sample images and negative sample images. The image recognition model in training can be trained based on the differences between the features of each training sub-feature map in the image features and the features of the corresponding sub-feature map in the positive sample image, as well as the differences between the features of each training sub-feature map in the training base feature map and the features of the corresponding sub-feature map in the negative sample image, until the trained image recognition model is obtained.

[0107] The image features of positive and negative sample images during the training process can be obtained by extracting features from the positive and negative sample images using a mature feature extraction model, or by extracting features from the positive and negative sample images using an image recognition model during training. No limitation is made here.

[0108] The spatial group metric learning loss of each training sub-feature map of the training features can be modified and optimized based on the triplet loss to supervise the features of each spatial group output by the network, so that the image representation learned by the network has stronger discriminativeness and specificity.

[0109] The feature loss metric for this application scenario is as follows:

[0110]

[0111] Among them, I tri The feature loss is measured by I. q For training images, q i For the feature corresponding to the i-th sub-feature map, the basic feature map is divided into H sub-feature maps, where i is any sub-feature map in H; + For the relevant images of positive samples, For the features corresponding to the i-th sub-feature map; I - Unrelated images representing negative samples. Let M be the feature corresponding to the i-th sub-feature map. M is a parameter that controls the feature spacing margin. During training, this optimized loss will bring the feature distance of corresponding semantic regions of related images closer together and widen the feature distance of unrelated images, making the trained image recognition model more robust.

[0112] The aforementioned spatial feature metric loss can constrain the features of spatial sub-feature maps of the base feature map, enhance the spatial distribution of semantic features, and improve the robustness of the image recognition model.

[0113] In a specific application scenario, the classification type of a training image can be predicted based on the image features of the training base feature map. The image recognition model in training can then be trained based on the difference between the classification type of the training image and the classification type of the sample images, until a fully trained image recognition model is obtained. Specifically, the class prediction can consist of Softmax, outputting a normalized class confidence score, representing the probability that the training image belongs to the corresponding class.

[0114] The classification type of the sample images during the training process can be obtained by using a mature image classification model, or by receiving manual annotations of the sample images; no limitation is made here.

[0115] Specifically, the class prediction results of the network can be extracted using supervised feature extraction via Softmax loss. Let the feature of the i-th image be x. i ∈R d d represents the feature dimension, and y represents the target category of the image. i The number of categories is n, and the network predicts that the image is the yth category. iThe probability of class is:

[0116]

[0117] Among them, W j ∈R d W represents the weight matrix of the fully connected layer. j ∈R d×n The j-th column, B∈R n Let the bias of the fully connected layer be represented. Then, the Softmax loss formula for the classification network is as follows:

[0118]

[0119] Where m is the number of sample images.

[0120] In a specific application scenario, the overall loss function of the feature extraction network can combine the classification loss and feature metric loss mentioned above, as shown below:

[0121] L total =L cls +λL tri

[0122] Here, λ is the balance parameter between classification and metric supervision, and in this application scenario, λ can be set to 1. In other application scenarios, λ can also be set to other parameters without limitation.

[0123] Training the image recognition model using the classification loss and feature measurement loss described above can improve the feature extraction capability and image recognition accuracy of the model.

[0124] Furthermore, since the image recognition in this application only needs to determine the approximate image of the image to be recognized from multiple candidate images, it does not require the candidate images themselves to have classification labels. Therefore, only coarse-grained image category labels are needed during the training process. The entire image can be directly fed into the image for recognition, which simplifies the tedious data processing and label acquisition process and improves the training effect.

[0125] Please see Figure 7 , Figure 7 This is a schematic diagram of the framework of an embodiment of the image recognition device of the present invention. The image recognition device 70 includes an acquisition module 71, a feature processing module 72, and a recognition module 73. The acquisition module 71 is used to acquire the image to be recognized, extract features from the image to be recognized, and obtain a basic feature map of the image to be recognized; the feature processing module 72 is used to divide the basic feature map into multiple sub-feature maps, perform sub-region feature processing on each sub-feature map to obtain the features of each sub-feature map, and determine the image features of the basic feature map based on the features of each sub-feature map; the recognition module 73 is used to determine an approximate image of the image to be recognized from multiple candidate images based on the image features.

[0126] The feature processing module 72 is further configured to divide the basic feature map into multiple sub-feature maps, perform sub-region mean pooling on each sub-feature map of the basic feature map to obtain the features of each sub-feature map; determine the set of features of each sub-feature map as the mean pooling feature of the basic feature map; perform channel pooling on the basic feature map to obtain the channel pooling feature of the basic feature map; and determine the image features of the basic feature map based on the mean pooling feature and the channel pooling feature.

[0127] The feature processing module 72 is also used to concatenate the mean pooling features and channel pooling features to obtain the image features of the basic feature map.

[0128] The feature processing module 72 is also used to divide the basic feature map into multiple sub-feature maps based on the wide and high dimensions of the basic feature map; and to perform sub-region mean pooling on each sub-feature map to obtain one-dimensional features of each sub-feature map.

[0129] The feature processing module 72 is also used to convert the number of channels of the basic feature map to 1 through channel pooling, and to expand the width and height dimensions of the basic feature map to obtain the channel pooling features of the basic feature map.

[0130] The acquisition module 71 is also used to acquire the image to be recognized, extract the semantic information of the image to be recognized, and form an initial feature map of the image to be recognized; perform multiple convolution processes on the initial feature map with different kernel sizes to obtain multiple basic feature maps of different scales; perform weighted average processing on the image features of each basic feature map; and perform feature matching between the image features after weighted average processing and the image features of each candidate image to determine an approximate image of the image to be recognized from multiple candidate images.

[0131] The recognition module 73 is also used to perform feature matching between the image features of the image to be recognized after weighted average processing and the image features of each candidate image, so as to determine the matching degree between the image to be recognized and each candidate image; and to arrange multiple candidate images in descending order of matching degree, and to determine the first preset number of candidate images as approximate images of the image to be recognized.

[0132] The acquisition module 71 is also used to acquire the image to be recognized, and to extract features from the image to be recognized through the feature extraction network of the image recognition model to obtain the basic feature map of the image to be recognized; the feature processing module 72 is also used to divide the basic feature map into multiple sub-feature maps, and to perform sub-region feature processing on each sub-feature map through the feature processing network of the image recognition model to obtain the features of each sub-feature map, and to determine the image features of the basic feature map based on the features of each sub-feature map; the recognition module 73 is also used to determine the approximate image of the image to be recognized from multiple candidate images based on the image features through the feature matching network of the image recognition model.

[0133] The acquisition module 71 is also used to acquire training images, extract features from the training images to obtain training base feature maps of the training images; divide the training base feature maps into multiple training sub-feature maps, perform sub-region feature processing on each training sub-feature map to obtain the features of each training sub-feature map, and determine the image features of the training base feature map based on the features of each training sub-feature map; train the image recognition model in training based on the image features of the training base feature map until the trained image recognition model is obtained.

[0134] The acquisition module 71 is also used to train the image recognition model in training based on the differences between the features of each training sub-feature map in the image features and the features of the corresponding sub-feature map in the positive sample image, as well as the differences between the features of each training sub-feature map in the training base feature map and the features of the corresponding sub-feature map in the negative sample image, until the trained image recognition model is obtained.

[0135] The acquisition module 71 is also used to predict the classification type of the training image based on the image features of the training base feature map; and to train the image recognition model in training based on the difference between the classification type of the training image and the classification type of the sample image until the trained image recognition model is obtained.

[0136] The acquisition module 71 is also used to acquire the product image to be identified, extract features from the product image to be identified, and obtain the basic feature map of the product image to be identified; the recognition module 73 is also used to determine the approximate product image of the product image to be identified from multiple candidate product images based on image features.

[0137] The above solution can improve the accuracy of image recognition.

[0138] Based on the same inventive concept, the present invention also proposes an electronic device capable of executing the image recognition method of any of the above embodiments. Please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the electronic device provided by the present invention. The electronic device includes a processor 81 and a memory 82.

[0139] The processor 81 is used to execute the program instructions stored in the memory 82 to implement the steps of any of the above image recognition methods. In a specific implementation scenario, the electronic device may include, but is not limited to, a microcomputer or a server. In addition, the electronic device may also include mobile devices such as laptops and tablets, which are not limited here.

[0140] Specifically, processor 81 controls itself and memory 82 to implement the steps of any of the above embodiments. Processor 81 may also be referred to as a CPU (Central Processing Unit). Processor 81 may be an integrated circuit chip with signal processing capabilities. Processor 81 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 81 may be implemented using integrated circuit chips.

[0141] The above solution improves the accuracy of image recognition.

[0142] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, please refer to [link to relevant documentation]. Figure 9 , Figure 9 This is a schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. The computer-readable storage medium 90 stores at least one program data 91, which is used to implement any of the methods described above. In one embodiment, the computer-readable storage medium 90 includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0143] In the several embodiments provided by this invention, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0145] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0146] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium.

[0147] The above are merely embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An image recognition method characterized by, The method comprises: obtaining a to-be-recognized image, performing feature extraction on the to-be-recognized image, and obtaining a basic feature map of the to-be-recognized image; dividing the basic feature map into a plurality of sub-feature maps, performing sub-region feature processing on each of the sub-feature maps to obtain a feature of each of the sub-feature maps, and determining an image feature of the basic feature map based on the features of the sub-feature maps; wherein, sub-region mean pooling processing is performed on each of the sub-feature maps of the basic feature map to obtain the feature of each of the sub-feature maps; the combination of the features of each of the sub-feature maps is determined as a mean pooling feature of the basic feature map; and channel pooling processing is performed on the basic feature map to obtain a channel pooling feature of the basic feature map; the image feature of the basic feature map is determined based on the mean pooling feature and the channel pooling feature; determining an approximate image of the to-be-recognized image from a plurality of candidate images based on the image feature.

2. The image recognition method of claim 1, wherein, The method comprises: performing feature splicing on the mean pooling feature and the channel pooling feature to obtain the image feature of the basic feature map.

3. The image recognition method of claim 1, wherein, The method comprises: dividing the basic feature map into a plurality of sub-feature maps based on the width dimension and the height dimension of the basic feature map; performing sub-region mean pooling processing on each of the sub-feature maps to obtain a one-dimensional feature of each of the sub-feature maps.

4. The image recognition method of claim 1, wherein, The method comprises: converting the number of channels of the basic feature map to 1 through the channel pooling processing, and expanding the width dimension and the height dimension of the basic feature map to obtain the channel pooling feature of the basic feature map.

5. The image recognition method of claim 1, wherein, The method comprises: obtaining the to-be-recognized image, extracting semantic information of the to-be-recognized image, and forming an initial feature map of the to-be-recognized image; performing convolution processing with different kernel sizes on the initial feature map multiple times to obtain a plurality of basic feature maps with different scales; The method further comprises: performing weighted average processing on the image features of each of the basic feature maps; performing feature matching on the image features after the weighted average processing and the image features of each of the candidate images to determine the approximate image of the to-be-recognized image from the plurality of candidate images.

6. The image recognition method of claim 5, wherein, The method comprises: performing feature matching on the image features after the weighted average processing corresponding to the to-be-recognized image and the image features of each of the candidate images to determine the matching degree between the to-be-recognized image and each of the candidate images. arrange the plurality of candidate images in an order from large to small based on the matching degrees, and determine a preset number of candidate images as the approximate image of the to-be-identified image.

7. The image recognition method of any one of claims 1-6, wherein, the obtaining of the to-be-identified image and the feature extraction on the to-be-identified image to obtain the base feature map of the to-be-identified image comprises: obtaining the to-be-identified image and performing feature extraction on the to-be-identified image by a feature extraction network of the image recognition model to obtain the base feature map of the to-be-identified image; the dividing of the base feature map into a plurality of sub-feature maps and the sub-region feature processing on each of the sub-feature maps to obtain the feature of each of the sub-feature maps and the determination of the image feature of the base feature map based on the features of the sub-feature maps comprises: dividing the base feature map into a plurality of sub-feature maps and performing sub-region feature processing on each of the sub-feature maps by a feature processing network of the image recognition model to obtain the feature of each of the sub-feature maps and determine the image feature of the base feature map based on the features of the sub-feature maps; the determination of the approximate image of the to-be-identified image from the plurality of candidate images based on the image feature comprises: determining the approximate image of the to-be-identified image from the plurality of candidate images based on the image feature by a feature matching network of the image recognition model; wherein the image recognition model comprises the feature extraction network, the feature processing network, and the feature matching network which are cascaded with each other.

8. The image recognition method of claim 7, wherein, Before the obtaining of the to-be-identified image and the feature extraction on the to-be-identified image to obtain the base feature map of the to-be-identified image, the method further comprises: obtaining a training image, performing feature extraction on the training image to obtain a training base feature map of the training image; dividing the training base feature map into a plurality of training sub-feature maps, performing sub-region feature processing on each of the training sub-feature maps to obtain the feature of each of the training sub-feature maps, and determining the image feature of the training base feature map based on the features of the training sub-feature maps; training the image recognition model in training based on the image feature of the training base feature map until a trained image recognition model is obtained.

9. The image recognition method of claim 8, wherein, the training of the image recognition model in training based on the image feature of the training base feature map until the trained image recognition model is obtained comprises: training the image recognition model in training based on the difference between the feature of each of the training sub-feature maps in the image feature and the feature of the corresponding sub-feature map in the positive sample image and the difference between the feature of each of the training sub-feature maps of the training base feature map and the feature of the corresponding sub-feature map in the negative sample image until the trained image recognition model is obtained.

10. The image recognition method of claim 8 or 9, characterized in that, the training of the image recognition model in training based on the image feature of the training base feature map until the trained image recognition model is obtained further comprises: predicting the classification type of the training image based on the image feature of the training base feature map. The image recognition model in training is trained based on a difference between the classification type of the training image and the classification type of the sample image until a trained image recognition model is obtained.

11. The image recognition method of claim 1, wherein, The obtaining of the to-be-recognized image and the feature extraction on the to-be-recognized image to obtain a basic feature map of the to-be-recognized image include: The obtaining of the to-be-recognized image and the feature extraction on the to-be-recognized image to obtain a basic feature map of the to-be-recognized image include: The determining of the approximate image of the to-be-recognized image from the multiple candidate images based on the image feature includes: The determining of the approximate image of the to-be-recognized image from the multiple candidate images based on the image feature includes:

12. An electronic device, comprising: The electronic device includes a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory to implement the image recognition method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores program data, and the program data can be executed to implement the image recognition method according to any one of claims 1 to 11.

Images