Image recognition method, recognition network training method and related device

By using a second sample set with a large amount of data to train the recognition network, and combining the feature fusion technology of self-attention, the problem of low image recognition accuracy in the prior art is solved, and accurate recognition of the recognized images is achieved.

CN113673583BActive Publication Date: 2024-11-29ZHEJIANG DAHUA TECH CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202110874936.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-30
Publication Date
2024-11-29
Estimated Expiration
2041-07-30

AI Technical Summary

Technical Problem

In the prior art, images containing characters are difficult to accurately identify the model through a small number of images due to weather, defilement, occlusion, etc., resulting in low model accuracy and inability to effectively identify the image type.

Method used

After training the recognition network with a large amount of data, the first sample set with a small amount of data is input as the support set, and the feature map is fused through self-attention to extract and splice image features. Achieve accurate recognition of images to be recognized.

Benefits of technology

A recognition network with high recognition accuracy is built, which reduces the dependence on the first sample set data, can accurately identify the categories of images to be identified, and improves the accuracy of image recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113673583B_ABST
    Figure CN113673583B_ABST
Patent Text Reader

Abstract

The present application discloses an image recognition method, a recognition network training method and related devices, the image recognition method includes: obtaining an image to be recognized; inputting the image to be recognized and a first sample set into the recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is trained by a second sample set including multiple sample images of each attribute and another image labeled with attributes of each attribute that is different from the second sample set; the attribute category of the second sample set is not less than the attribute category of the first sample set; using the recognition network to extract features from the first sample set and the image to be recognized, obtaining feature maps of multiple images and feature maps of the image to be recognized; based on the feature map obtained above, performing attribute recognition on the image to be recognized, obtaining the attribute type of the image to be recognized. In the above manner, the present application can accurately identify the category of the image to be recognized through the first sample set with a small amount of data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image recognition method, a recognition network training method and related devices. Background Art

[0002] With the continuous development of image recognition technology, character recognition technology has been widely used in many fields.

[0003] In the prior art, images containing characters are often taken by surveillance cameras. Due to weather, stains, occlusions, and the small size of characters in some images, it is difficult to improve the accuracy of the model by training the model with only a small number of images, and it is also impossible to accurately identify the type of specific image through the model. Summary of the invention

[0004] The main technical problem solved by the present application is to provide an image recognition method, a recognition network training method and related devices. After the recognition network is trained with a second sample set with a larger data volume, a first sample set with a smaller data volume is input into the recognition network as a support set. The image to be recognized is recognized by the first sample set, and the category of the image to be recognized can be accurately identified.

[0005] To solve the above technical problems, the first technical solution adopted in the present application is to provide an image recognition method, including: obtaining an image to be recognized of an attribute category to be recognized; inputting the image to be recognized and a first sample set into a recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is trained by a second sample set including multiple sample images of each attribute and another image labeled with attributes of each attribute that is different from the second sample set; the categories of the first sample set and the second sample set are different, and the attribute category of the second sample set is not less than the attribute category of the first sample set; using the recognition network to extract features from the first sample set and the image to be recognized, and obtain feature maps of the multiple images and the feature map of the image to be recognized; based on the feature maps of the multiple images and the feature map of the image to be recognized, attribute recognition is performed on the image to be recognized to obtain the attribute type of the image to be recognized.

[0006] Among them, based on the feature maps of multiple images and the feature map of the image to be identified, the step of performing attribute identification on the image to be identified to obtain the attribute type of the image to be identified includes: fusing the feature maps of images with the same attribute to obtain a fused feature map of the image with the same attribute; splicing the fused feature map of the image with the same attribute with the feature map of the image to be identified to obtain the spliced ​​features of each attribute image; and classifying and predicting the image to be identified based on the spliced ​​features of each attribute image to obtain the type of the image to be identified.

[0007] Among them, the step of fusing the feature maps of images with the same attribute to obtain a fused feature map of images with the same attribute includes: using self-attention to obtain the weight value of the feature map of each image in the first sample set; performing weighted fusion on the feature maps of images with the same attribute based on their respective weight values ​​to obtain a fused feature map of images with the same attribute.

[0008] Among them, the step of using self-attention to obtain the weight value of the feature map of each image in the first sample set specifically includes: globally average pooling the feature maps of the images with the same attribute in the first sample set and the feature maps of the images to be identified according to the channel dimension one by one, and then performing dot multiplication processing on the feature maps of the images with the same attribute that have been subjected to global average pooling and the feature maps of the images to be identified that have been subjected to global average pooling, to obtain the unnormalized weight value of the feature map of each image with the same attribute; the step of weightedly fusing the feature maps of the images with the same attribute based on their respective weight values ​​to obtain the fused feature map of the images with the same attribute specifically includes: obtaining the normalized weight values ​​corresponding to the feature maps of the images with the same attribute based on the unnormalized weight values; and weighted fusing the feature maps of the images with the same attribute based on the normalized weight values ​​to obtain the fused feature map of the images with the same attribute.

[0009] Among them, the step of splicing the fused feature map of the same attribute image with the feature map of the image to be identified to obtain the splicing features of each attribute image includes: splicing the fused feature map corresponding to each attribute image with the feature map of the image to be identified, and performing feature extraction to obtain the splicing features of each attribute image.

[0010] Among them, the step of obtaining an image to be identified of a category of attribute to be identified specifically includes: obtaining an image to be processed; performing aspect ratio analysis on the image to be processed; in response to the aspect ratio of the image to be processed being greater than a set value, rotating the image to be processed counterclockwise by a first set angle; in response to the aspect ratio of the image to be processed being less than a set value, maintaining the angle of the image to be processed.

[0011] Among them, the step of obtaining the image to be identified of the attribute category to be identified also includes: using the direction classification model to determine whether the image to be processed is rotated by a second set angle; in response to the image to be processed being rotated by the second set angle, rotating the image to be processed by the second set angle; in response to the image to be processed not being rotated by the second set angle, maintaining the angle of the image to be processed.

[0012] Among them, the step of obtaining the image to be identified of the attribute category to be identified includes: obtaining the license plate image to be identified of the non-motor vehicle of the territory to be identified; inputting the image to be identified and the first sample set into the recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is trained by a second sample set including multiple sample images of each attribute and another image with annotated attributes of each attribute that is different from the second sample set; the categories of the first sample set and the second sample set are different, and the attribute categories of the second sample set are not less than the attribute categories of the first sample set, including: inputting the license plate image to be identified and the non-motor vehicle sample set into the license plate recognition network; wherein the non-motor vehicle sample set includes multiple license plate images of non-motor vehicles of each territory; the license plate recognition network is trained by a motor vehicle sample set including multiple license plate images of motor vehicles of each territory and another image with annotated attributes of each territory that is different from the motor vehicle sample set. The method is obtained by training with another motor vehicle license plate image with marked location in the motor vehicle sample set; the location category of the motor vehicle sample set is not less than the location category of the non-motor vehicle sample set; using the recognition network to extract features from the first sample set and the image to be recognized, and obtaining feature maps of multiple images and feature maps of the image to be recognized, including: using the license plate recognition network to extract features from multiple non-motor vehicle license plate images in the non-motor vehicle sample set and the license plate image to be recognized, and obtaining feature maps of multiple non-motor vehicle license plate images and the license plate image to be recognized; based on the feature maps of the multiple images and the feature maps of the image to be recognized, performing attribute recognition on the image to be recognized, and obtaining the attribute type of the image to be recognized, including: recognizing the location of the license plate image to be recognized based on the feature maps of the multiple non-motor vehicle license plate images and the license plate image to be recognized, and obtaining the location of the license plate image to be recognized.

[0013] To solve the above technical problems, the second technical solution adopted in the present application is to provide a training method for a recognition network, including: inputting a sample set including multiple sample images of various attributes and another reference image of each attribute that is different from the sample set and is labeled with the attribute into an initial deep learning model; using the initial deep learning model to extract features of the multiple sample images and the reference image, and obtaining feature maps of the multiple sample images and feature maps of the reference image; fusing the feature maps of images with the same attribute to obtain a fused feature map of the image with the same attribute; splicing the fused feature map of the image with the same attribute with the feature map of the image to be identified to obtain the spliced ​​features of each attribute image; predicting the attribute category of the reference image based on the spliced ​​features of each attribute image to obtain the predicted attribute; training the initial deep learning model using the labeled attributes and the predicted attributes of the reference image, and determining the trained model as a recognition network.

[0014] Among them, the step of fusing the feature maps of images with the same attribute to obtain a fused feature map of the images with the same attribute includes: using self-attention to obtain the weight value of the feature map of each image in the sample set; performing weighted fusion on the feature maps of images with the same attribute based on their respective weight values ​​to obtain a fused feature map of the images with the same attribute.

[0015] To solve the above technical problems, the third technical solution adopted in the present application is to provide an image recognition device, including: an image acquisition module, used to acquire an image to be recognized of a category of attribute to be recognized; an image input module, used to input the image to be recognized and a first sample set into a recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is obtained by training a second sample set including multiple sample images of each attribute and another image labeled with attributes of each attribute that is different from the second sample set; the categories of the first sample set and the second sample set are different, and the attribute category of the second sample set is not less than the attribute category of the first sample set; a feature extraction module, used to use the recognition network to extract features from the first sample set and the image to be recognized, and obtain feature maps of multiple images and feature maps of the image to be recognized; an image recognition module, used to perform attribute recognition on the image to be recognized based on the feature maps of multiple images and the feature map of the image to be recognized, and obtain the attribute type of the image to be recognized.

[0016] To solve the above technical problems, the fourth technical solution adopted in the present application is to provide a training device for a recognition network, comprising: an input module, used to input a sample set including multiple sample images of various attributes and another reference image of each attribute that is different from the sample set and is labeled with attributes into an initial deep learning model; a first extraction module, used to use the initial deep learning model to perform feature extraction on multiple sample images and reference images, and obtain feature maps of multiple sample images and feature maps of reference images; a fusion module, used to fuse the feature maps of images with the same attributes to obtain a fused feature map of images with the same attributes; a splicing module, used to splice the fused feature map of images with the same attributes with the feature map of the image to be identified to obtain spliced ​​features of images of each attribute; a prediction module, used to predict the attribute category of the reference image based on the spliced ​​features of the images of each attribute to obtain predicted attributes; an output module, used to train the initial deep learning model using the labeled attributes and predicted attributes of the reference image, and determine the trained model as a recognition network.

[0017] In order to solve the above technical problems, the fifth technical solution adopted in the present application is to provide an electronic device, including: a memory, used to store program data, and when the program data is executed, the steps in the image recognition method as described in any one of the above items or the steps in the training method of the recognition network as described in any one of the above items are implemented; a processor, used to execute the program instructions stored in the memory to implement the steps in the image recognition method as described in any one of the above items or the steps in the training method of the recognition network as described in any one of the above items.

[0018] In order to solve the above technical problems, the sixth technical solution adopted in the present application is to provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the image recognition method described in any one of the above items or the steps in the training method of the recognition network described in any one of the above items are implemented.

[0019] The beneficial effects of the present application are as follows: different from the prior art, the present application can construct a recognition network with higher recognition accuracy after training the recognition network with the second sample set with a larger data volume, so that when the recognition network migrates to the first sample set, the dependence on the data in the first sample set can be reduced; further, the first sample set with a smaller data volume is input into the recognition network as a support set, and the image to be recognized is recognized by the first sample set, so that the category of the image to be recognized can be accurately identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a flowchart of an implementation method of a training method for a recognition network of the present application;

[0022] Figure 2 It is a flowchart of an implementation method of the image recognition method of the present application;

[0023] Figure 3 is a flowchart of a specific implementation method of step S21;

[0024] Figure 4 yes Figure 2 A flowchart of a specific implementation method of step S23;

[0025] Figure 5 yes Figure 4 A flowchart of a specific implementation method of step S41;

[0026] Figure 6 It is a flowchart of a specific implementation method of the image recognition method of the present application;

[0027] Figure 7 It is a schematic diagram of some non-motor vehicle license plate images obtained in a traffic monitoring scenario;

[0028] Figure 8 is a schematic diagram comparing the captured non-motor vehicle license plate image and the motor vehicle license plate image; wherein, Figure 8 a is a schematic diagram of the captured non-motor vehicle license plate image. Figure 8 b is a schematic diagram of a captured motor vehicle license plate image;

[0029] Fig. 9 This is a schematic diagram of the process of classifying the license plate image to be recognized by the recognition network of this application;

[0030] Fig.10 It is a structural schematic diagram of an embodiment of the image recognition device of the present application;

[0031] Fig.11 It is a structural diagram of an implementation method of a training device for recognizing a network of the present application;

[0032] Fig.12 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0033] Fig.13 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0035] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless otherwise clearly indicated above, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0036] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0037] It should be understood that the terms "include", "comprises" or any other variations used herein are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "includes..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0038] In the prior art, images containing characters are often taken by surveillance cameras. Due to weather, stains, occlusions, and the small size of characters in some images, it is difficult to improve the accuracy of the model by training the model with only a small number of images, and it is also impossible to accurately identify the type of specific image through the model.

[0039] Based on the above situation, the present application provides an image recognition method, a recognition network training method and related devices. After the recognition network is trained using a second sample set with a larger data volume, the first sample set with a smaller data volume is input into the recognition network as a support set. The image to be recognized is recognized by the first sample set, and the category of the image to be recognized can be accurately identified.

[0040] This application first provides a training method for a recognition network.

[0041] Specifically, see Figure 1 , Figure 1 FIG. 1 is a flow chart of an implementation method of a training method for identifying a network in the present application. Figure 1 As shown, in this embodiment, the training method of the recognition network includes:

[0042] S11: Inputting a sample set including a plurality of sample images of each attribute and another reference image of each attribute labeled with the attribute different from the sample set into an initial deep learning model.

[0043] In this embodiment, the sample set includes a large number of sample images of various attributes, and all images in the sample set are images with attributes labeled.

[0044] In this embodiment, the recognition network is a model based on a deep learning network. Specifically, in a classification network, if the network is too shallow, no useful classification features can be learned, and if the network is too deep, too much training data and too long training time are required. In this embodiment, the initial deep learning model is used based on the abstract feature learning ability of the deep learning model, the existing training data and the training time.

[0045] Among them, the initial deep learning models include InceptionV3 structure model and InceptionV4 structure model.

[0046] S12: Using the initial deep learning model to perform feature extraction on the multiple sample images and the reference image, and obtaining feature maps of the multiple sample images and the reference image.

[0047] In this embodiment, feature extraction is performed on multiple sample images and reference images through a feature extractor of an initial deep learning model to obtain feature maps of multiple sample images of each attribute and feature maps of reference images of each attribute.

[0048] S13: Fusing the feature maps of images with the same attribute to obtain a fused feature map of the image with the same attribute.

[0049] In this embodiment, after obtaining the weight value of the feature map of each image in the sample set by using the self-attention method, the feature maps of images with the same attribute are weighted fused based on their respective weight values ​​to obtain a fused feature map of the image with the same attribute.

[0050] Among them, the self-attention method refers to an attention mechanism involving different positions in a single sequence, which reduces the dependence on external information and is better at capturing the internal correlation of data or features.

[0051] Specifically, one of the reference images of each attribute that is different from another attribute-labeled reference image in the sample set is randomly selected as the image to be classified, and the feature maps of the image with the same attribute in the sample set and the feature maps of the image to be classified are globally average pooled according to the channel dimension one by one, and then the feature maps of the image with the same attribute that have been globally average pooled are multiplied with the feature maps of the image to be classified that have been globally average pooled to obtain the unnormalized weight values ​​corresponding to the feature maps of the image with the same attribute. Then, based on the unnormalized weight values, the normalized weight values ​​corresponding to the feature maps of the image with the same attribute are obtained, and based on the normalized weight values, the feature maps of the image with the same attribute are weightedly fused to obtain the fused feature maps of the image with the same attribute.

[0052] It can be understood that obtaining the weight values ​​of the feature maps of different images of the same attribute in the sample set through self-attention can fully consider the importance of different images in the same attribute, which is conducive to improving the accuracy of image recognition.

[0053] S14: splicing the fused feature map of the same attribute image with the feature map of the image to be identified to obtain splicing features of each attribute image.

[0054] In this embodiment, the splicing features include the multi-dimensional features of the fusion feature map and the multi-dimensional features of the feature map of the image to be classified.

[0055] S15: Predicting the attribute category of the reference image based on the splicing features of each attribute image to obtain a predicted attribute.

[0056] In this embodiment, by comparing the multi-dimensional features, if multiple feature points of the fused feature map corresponding to a certain attribute are consistent with multiple feature points of the feature map of the image to be classified, it is predicted that the image to be classified belongs to the attribute category.

[0057] S16: Train the initial deep learning model using the annotation attributes and prediction attributes of the reference image, and determine the trained model as a recognition network.

[0058] Although the initial deep learning model has certain image recognition capabilities after training, the probability prediction of image categories is not necessarily accurate. Therefore, in this embodiment, the initial deep learning model is further iteratively optimized based on the prediction results.

[0059] In this embodiment, the initial deep learning model can be optimized by optimizing the loss function. Specifically, a set number of sample images are selected from the sample set for optimization in each iteration. When the loss value curve of the deep learning model is basically flat, the training is considered complete, and the trained model is determined as a recognition network.

[0060] Different from the existing technology, this implementation can build a recognition network by training the initial deep learning model with a sample set including a large number of sample images. Moreover, by directly training according to the images, the complex image recognition technology can be simplified into multi-classification tasks, which reduces the difficulty of training, saves R&D costs and labor costs, and has strong portability and a wider range of applications.

[0061] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of an embodiment of the image recognition method of the present application. Figure 2 As shown, in this embodiment, the image recognition method includes:

[0062] S21: Obtain an image to be identified with an attribute category to be identified.

[0063] Specifically, see Figure 3 , Figure 3 Schematic diagram of a specific implementation of step S21. Figure 3 As shown, in this embodiment, the step of acquiring the image to be recognized specifically includes:

[0064] S31: Obtain the image to be processed.

[0065] In this implementation manner, the image to be processed is an image captured by a surveillance camera and uploaded or an image uploaded by a user.

[0066] S32: Perform aspect ratio analysis on the image to be processed.

[0067] In this embodiment, the aspect ratio of the image to be processed is analyzed to determine whether the aspect ratio of the image to be processed is greater than a set value.

[0068] The formula of the aspect ratio is ratio=h / w, where h and w are the height and width of the image to be processed respectively.

[0069] In this implementation manner, the set value is 1.2, and the first set angle is 90°.

[0070] S33: In response to the aspect ratio of the image to be processed being greater than a set value, rotating the image to be processed counterclockwise by a first set angle; in response to the aspect ratio of the image to be processed being less than the set value, maintaining the angle of the image to be processed.

[0071] In a specific implementation, the aspect ratio of the image in the first sample set is less than 1, indicating that the width of the image is greater than the height. If the aspect ratio of the image to be processed is greater than 1.2, it indicates that the width of the image to be processed is less than the height. In order to reduce the difficulty of subsequent recognition, the image to be processed is rotated 90° counterclockwise so that the image to be processed is consistent with the image in the first sample set and keeps its width greater than its height.

[0072] In other implementations, the set value may also be 1.05, 1.1 or other values, which is not limited in this application.

[0073] Furthermore, the direction classification model can be used to determine whether the image to be processed is rotated by a second set angle. In response to the image to be processed being rotated by the second set angle, the image to be processed is rotated by the second set angle. In response to the image to be processed not being rotated by the second set angle, the angle of the image to be processed is maintained.

[0074] The second setting angle is 180°.

[0075] Specifically, the characters in the processed image to be processed may be upside down, indicating that the image is flipped 180° at this time, so it needs to be rotated 180° to facilitate subsequent recognition.

[0076] S22: Input the image to be recognized and the first sample set into the recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is trained by a second sample set including multiple sample images of each attribute and another image labeled with attributes of each attribute that is different from the second sample set; the categories of the first sample set and the second sample set are different, and the attribute categories of the second sample set are not less than the attribute categories of the first sample set.

[0077] In this embodiment, all images in the first sample set and the second sample set are images with attributes labeled.

[0078] Among them, all images in the first sample set and the second sample set are images with clear characters and correct angles.

[0079] In this embodiment, the number of sample images of each attribute in the second sample set is much larger than the number of sample images of the same attribute in the first sample set.

[0080] Specifically, the attribute categories of the second sample set are not less than the attribute categories of the first sample set, which means that the attribute categories in the second sample set may include other attribute categories after including all the attribute categories of the first sample set.

[0081] In other implementations, the attribute categories of the second sample set are not less than the attribute categories of the first sample set, which may simply mean that the attribute categories of the second sample set are more than the attribute categories of the first sample set, and there is no corresponding or subordinate relationship between the attribute categories included in the two, and this application does not limit this.

[0082] In this embodiment, the identification network is achieved by Figure 1 It is obtained by the training method of the recognition network.

[0083] It can be understood that the number of sample images used for training in the second sample set is much larger than the number of sample images in the first sample set. This is to train the recognition network with a large amount of data to improve the accuracy of the recognition network, so that when the recognition network is migrated to the first sample set, it can reduce its dependence on the data in the first sample set, and then accurately identify the image to be recognized through a small data sample.

[0084] S23: Using the recognition network to extract features from the first sample set and the image to be recognized, to obtain feature maps of the multiple images and a feature map of the image to be recognized.

[0085] In this embodiment, feature extraction is performed on the multiple images in the first sample set and the image to be recognized by the feature extractor of the recognition network to obtain feature maps of the multiple images and the feature map of the image to be recognized.

[0086] S24: Based on the feature maps of the multiple images and the feature map of the image to be identified, perform attribute identification on the image to be identified to obtain the attribute type of the image to be identified.

[0087] See also Figure 4 , Figure 4 yes Figure 2 Schematic diagram of a specific implementation of step S24 in FIG. Figure 4 As shown, in this embodiment, based on the feature maps of multiple images and the feature map of the image to be identified, the step of performing attribute identification on the image to be identified and obtaining the attribute type of the image to be identified specifically includes:

[0088] S41: Fusing the feature maps of images with the same attribute to obtain a fused feature map of the images with the same attribute.

[0089] Specifically, see Figure 5 , Figure 5 yes Figure 4 Schematic diagram of a specific implementation of step S41 in FIG. Figure 5 As shown, in this embodiment, the step of fusing the feature maps of images with the same attribute to obtain a fused feature map of the images with the same attribute specifically includes:

[0090] S51: Obtain the weight value of the feature map of each image in the first sample set by using self-attention.

[0091] In this implementation, feature maps of images with the same attribute in the first sample set and feature maps of images to be identified are globally average pooled according to the channel dimension one by one, and then dot product processing is performed on the feature maps of images with the same attribute that have undergone global average pooling and the feature maps of images to be identified that have undergone global average pooling to obtain unnormalized weight values ​​of the feature maps of each image with the same attribute.

[0092] Specifically, the unnormalized weight values ​​corresponding to the feature maps of images with the same attributes are obtained by the following formula:

[0093]

[0094] in, is the feature map of the i-th image in the j-th attribute, f q is the feature map of the image to be recognized, α jis the unnormalized weight value corresponding to the feature map of the i-th image in the j-th attribute, and score(·) is the dot product of the feature map under the L2 norm after global average pooling according to the channel dimension.

[0095] Among them, the L2 norm refers to the Euclidean norm, that is, the Euclidean length, which can also be understood as the L2 distance, which is defined as the square root of the sum of the squares of all elements of the vector, and can be considered as the distance between two points in space.

[0096] Specifically, α j The larger the value, the smaller the metric distance between the feature map of the i-th image in the j-th attribute and the feature map of the image to be identified, the closer the category of the image is to the image to be identified, and the more important the image is.

[0097] In this implementation, considering the differences in importance of different sample images in the same attribute, multiple sample images with different importance in the same attribute cannot be fused in the same proportion during subsequent fusion. Therefore, the importance of all samples in the same attribute is analyzed through self-attention, and when feature fusion is performed subsequently, more features of images with greater importance are fused to improve the accuracy of image recognition.

[0098] S52: Perform weighted fusion on the feature maps of the images with the same attribute based on their respective weight values ​​to obtain a fused feature map of the images with the same attribute.

[0099] In this embodiment, normalized weight values ​​corresponding to feature maps of images of the same attribute are obtained based on unnormalized weight values, and weighted fusion is performed on feature maps of images of the same attribute based on the normalized weight values ​​to obtain fused feature maps of images of the same attribute.

[0100] Specifically, the fusion feature map of the same attribute image is obtained by the following formula:

[0101]

[0102] in, is the feature map of the i-th image in the j-th attribute, f q is the feature map of the image to be identified, k is the total number of images in the jth attribute, α j is the unnormalized weight value corresponding to the feature map of the i-th image in the j-th attribute, e is a natural constant with a value of approximately 2.718281828, is the normalized weight value corresponding to the feature map of the i-th image in the j-th attribute, is the fusion feature map of the j-th attribute image.

[0103] It can be understood that the greater the importance of an image in the same attribute, the greater the corresponding normalized weight value, and the feature map corresponding to the image will occupy a larger proportion during feature fusion.

[0104] S42: splicing the fused feature map of the same attribute image with the feature map of the image to be identified to obtain splicing features of each attribute image.

[0105] In this implementation, the fused feature map corresponding to each attribute image is spliced ​​with the feature map of the image to be identified, and feature extraction is performed to obtain the spliced ​​features of each attribute image.

[0106] The splicing features include the multi-dimensional features of the fusion feature map and the multi-dimensional features of the feature map of the image to be identified.

[0107] S43: Classify and predict the image to be identified based on the splicing features of each attribute image to obtain the type of the image to be identified.

[0108] In this embodiment, by comparing the multi-dimensional features, if the multiple feature points of the fused feature map corresponding to a certain attribute are consistent with the multiple feature points of the feature map of the image to be identified, the attribute category of the image to be identified is predicted to be the attribute category. Furthermore, the attribute category label is mapped to the actual attribute value as the discriminated image attribute.

[0109] Different from the prior art, this embodiment uses the second sample set with a larger amount of data to train the recognition network, and can build a recognition network with a higher recognition accuracy, so that when the recognition network migrates to the first sample set, it can reduce the dependence on the data in the first sample set. Then the first sample set with a smaller amount of data is input into the recognition network as a support set, and the image to be recognized is recognized by the first sample set, so that the category of the image to be recognized can be accurately identified. In addition, the weight value of the feature map of each image in the first sample set is obtained by the self-attention method, and the importance analysis of all samples in the same attribute can be performed, so that the proportion of images with greater importance in the fusion feature map is greater, thereby further improving the accuracy of image recognition.

[0110] In a specific implementation, the recognition of a license plate image of a non-motor vehicle is used as an example. Figure 6 As shown, Figure 6 FIG. 1 is a flowchart of a specific implementation method of the image recognition method of the present application. Figure 6 As shown, in this embodiment, the image to be identified of the attribute category to be identified is a license plate image of a non-motor vehicle to be identified in the area to be identified, and the image recognition method specifically includes the following steps:

[0111] S61: Acquire the license plate image of the non-motor vehicle in the area to be identified.

[0112] In this embodiment, a license plate image of a non-motor vehicle to be identified in the area where the non-motor vehicle to be identified belongs is obtained from traffic monitoring or images uploaded by users.

[0113] With the widespread use of electric bicycles, the traffic control department has become more and more strict in controlling electric bicycles to ensure travel safety. Under the premise of "electronic" and "automatic" processing, the traffic control department is promoting non-motor vehicle electronic license plates. Therefore, the original non-electronic electric bicycle license plates or license plates that have reached their expiration date need to be replaced. Therefore, it is necessary to obtain the license plate image that needs to be replaced from traffic monitoring or images uploaded by users.

[0114] In the automatic license plate replacement application scenario (users upload license plate images, and the system recognizes and processes the images), in addition to paying attention to the number information on the bottom layer of the electric vehicle license plate, it is also necessary to analyze the regional attributes of the license plate. For details, please refer to Figure 7 , Figure 7 is a schematic diagram of some non-motor vehicle license plate images obtained in traffic monitoring scenarios. Figure 7 As shown, the upper layer of the non-motor vehicle license plate image generally contains information about the region to which it belongs, such as "Lin'an", "Hangzhou", "Yuhang", etc. This information is defined as the regional attribute of the non-motor vehicle license plate, that is, the place that needs to be identified in this implementation.

[0115] Depend on Figure 7 It can also be seen that some license plate images are flipped 180°. During the processing, a direction classification model needs to be used for identification and processing to facilitate subsequent applications.

[0116] S62: Input the license plate image to be identified and the non-motor vehicle sample set into the license plate recognition network; wherein the non-motor vehicle sample set includes multiple non-motor vehicle license plate images of each location; the license plate recognition network is trained by a motor vehicle sample set including multiple motor vehicle license plate images of each location and another motor vehicle license plate image of each location that is different from the motor vehicle sample set; the location category of the motor vehicle sample set is not less than the location category of the non-motor vehicle sample set.

[0117] In this embodiment, since the license plate image of the non-motor vehicle is double-layer license plate data, the license plate image of the motor vehicle used as the training sample in the motor vehicle sample set is also double-layer license plate data.

[0118] Among them, all images in the non-motor vehicle sample set and the motor vehicle sample set are images with clear characters and correct angles. Among them, the sample images in the non-motor vehicle sample set are all screened and processed images, while the sample images in the motor vehicle sample set can be directly obtained from the traffic monitoring scene.

[0119] Specifically, automatic license plate replacement requires that the regional attribute information on the upper layer of the acquired license plate image is clear and distinguishable by the human eye. However, due to the small size and small license plate image of non-motor vehicles, the license plate is stained or blocked, and the weather is bad when the photo is taken, the license plate images of non-motor vehicles that can be directly obtained in traffic monitoring scenarios are often difficult to use directly in license plate replacement scenarios. Figure 8 As shown in a. Since motor vehicles are large and the license plate images are large, it is easier to obtain double-layer license plate images of motor vehicles in traffic monitoring scenarios, where both upper and lower characters are clear and can be distinguished by the human eye. In traffic monitoring scenarios, it is easier to obtain and can meet a certain amount of data, such as Figure 8 As shown in b.

[0120] In this embodiment, the attribute categories of the license plate images of motor vehicles in the motor vehicle sample set are not less than the attribute categories of the non-motor vehicle sample set, but there is no corresponding relationship between the attribute categories in the two sample sets.

[0121] In a specific implementation scenario, the non-motor vehicle sample set consists of multiple non-motor vehicle license plate images from various cities in Zhejiang Province. After including multiple motor vehicle license plate images from various cities in Zhejiang Province, the motor vehicle sample set can also include license plate images of motor vehicles from other provinces.

[0122] In another specific implementation scenario, the non-motor vehicle sample set is composed of multiple non-motor vehicle license plate images from various cities in Zhejiang Province, and the motor vehicle sample set can be composed entirely of motor vehicle license plate images from other provinces. It is only necessary to ensure that the attribute categories of the areas in the motor vehicle sample set are not less than the attribute categories corresponding to the various cities in Zhejiang Province.

[0123] Furthermore, the number of sample images used for training in the motor vehicle sample set is much larger than the number of sample images in the non-motor vehicle sample set.

[0124] In a specific implementation scenario, the area where automatic license plate replacement is required for non-motor vehicles is called the target area, and the area where automatic license plate replacement is not required is called the non-target area. Assuming that there are n-1 target areas and b non-target areas, since the non-target areas do not need to be subdivided, just to increase the robustness during testing, all the b non-target areas can be set as 1 area, and together with the target area, form n areas. These n areas are regarded as all the area categories included in the non-motor vehicle sample set, and then k (k>1) images are selected from each area as multiple non-motor vehicle license plate images of each attribute included in the non-motor vehicle sample set (a total of n×k images), where the multiple non-motor vehicle license plate images of each attribute are recorded as c according to category. j (j=1,...,n), where c n Indicates license plates from non-target areas.

[0125] Among them, increasing the robustness during testing means that the images uploaded by users may be license plate images other than the n-1 attributes. Since these license plate images do not belong to the target area, there is no need to implement automatic license plate replacement. The recognition network does not need to classify these images when detecting them. Therefore, these images can be used as input error data to detect the robustness of the recognition network.

[0126] Next, m regions are selected from the motor vehicle license plate image database as all region categories included in the motor vehicle sample set, and then R images are selected from each region as multiple motor vehicle license plate images of various attributes included in the non-motor vehicle sample set (a total of m×R images), where m≥n, R>>k. The m regions are randomly selected regions, which may include target regions or all non-target regions, and there is no restriction on this.

[0127] Specifically, see Fig. 9 , Fig. 9 This is a schematic diagram of the training process of the license plate recognition network of this application.

[0128] Randomly select n categories from the motor vehicle sample set to form an iterative sample set, randomly select k motor vehicle license plate images for each category, and at the same time select a motor vehicle license plate image other than the iterative sample set from the motor vehicle sample set as the image to be classified x q , the iterative sample set and the single image to be classified x q Input into the preset deep learning model, and use the feature extractor of the preset deep learning model to iterate the sample set and the image to be identified x q Perform feature extraction to obtain feature maps of license plate images of multiple motor vehicle license plates And the image to be recognized x q The feature map f q ,in, is the feature map of the i-th image in the j-th attribute in the iterative sample set.

[0129] Next, the self-attention method is used to obtain the weight value of the feature map of each image in the iterative sample set. Specifically, the feature maps of images with the same attributes in the iterative sample set are added one by one. And the feature map f of the image to be recognized q Perform global average pooling according to the channel dimension, and then perform global average pooling on the feature map of the image with the same attribute The feature map f of the image to be recognized after global average pooling q Perform point multiplication to obtain the unnormalized weight value α of the feature map of each image with the same attribute j , where α jis the unnormalized weight value corresponding to the feature map of the i-th image in the j-th attribute.

[0130] Furthermore, the feature maps of images with the same attributes Based on their respective weight values ​​α j Perform weighted fusion to obtain the fusion feature map of the same attribute image in, is the fusion feature map of the j-th attribute image.

[0131] The fusion feature map of the same attribute image And the feature map f of the image to be recognized q The input is sent to the concatenation layer (concat) for concatenation, and after the convolutional layer (conv) obtains the concatenation features of each attribute image, the attribute category of the reference image is predicted based on the concatenation features of each attribute image to obtain the predicted attribute, and the predicted attribute is classified through the fully connected layer (FC).

[0132] Based on the classification results, determine whether the loss value is stable. If the loss value is not stable, that is, the loss value curve is not flat, continue to randomly select n categories from the motor vehicle sample set to form the iterative sample set after the iteration, randomly select k images from each category, and at the same time select an image from the motor vehicle sample set except the iterative sample set after the iteration as the image to be classified, and continue training until the loss value curve of the loss function is basically flat, then stop training to obtain the trained recognition network.

[0133] S63: Using the license plate recognition network, feature extraction is performed on a plurality of non-motor vehicle license plate images and the license plate images to be recognized in the non-motor vehicle sample set to obtain feature maps of the plurality of non-motor vehicle license plate images and the license plate images to be recognized.

[0134] S64: Identify the location of the license plate image to be identified based on the plurality of non-motor vehicle license plate images and the feature map of the license plate image to be identified, and obtain the location of the license plate image to be identified.

[0135] In this embodiment, through the above Figure 2 The image recognition method in is used to identify the license plate image to be identified. After calculating the category corresponding to the maximum value of the output of the last fully connected layer of the license plate recognition network, the category label is mapped to the actual attribute value according to the non-motor vehicle sample set used as the attribute of the license plate image to be identified.

[0136] In the prior art, in the automatic license plate replacement application scenario (users upload license plate images, and the system recognizes and processes the images), it is necessary to analyze the regional information and number information of the license plate to ensure the accuracy of the license plate replacement. However, due to weather, contamination, occlusion, and the small characters of some license plates, most of the specific types of license plate images that can be obtained in traffic monitoring scenarios are not clear enough, and it is difficult to accurately identify the regional information of the license plate. Only a small number of such license plate images can be used in the license plate replacement scenario, and the license plate image cannot be accurately recognized, which leads to a decrease in the accuracy of the license plate replacement.

[0137] Different from the prior art, this embodiment uses a motor vehicle sample set including a large number of motor vehicle license plate images to train the recognition network, and can build a recognition network with a high recognition accuracy, so that when the recognition network is transferred to a non-motor vehicle sample set including a small number of non-motor vehicle license plate images, it can reduce the dependence on the data in the non-motor vehicle sample set. Then, the non-motor vehicle sample set with a smaller amount of data is input into the recognition network as a support set, and the license plate image to be recognized is recognized through the non-motor vehicle sample set, so that the location of the license plate image to be recognized can be accurately obtained.

[0138] Correspondingly, the present application provides an image recognition device.

[0139] Specifically, see Fig.10 , Fig.10 Schematic diagram of the structure of an embodiment of the image recognition device of the present application. Fig.10 As shown, the image recognition device 100 includes an image acquisition module 1001 , an image input module 1002 , a feature extraction module 1003 and an image recognition module 1004 .

[0140] The image acquisition module 1001 is used to acquire an image to be identified with an attribute category to be identified.

[0141] The image input module 1002 is used to input the image to be identified and the first sample set into the recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is trained by a second sample set including multiple sample images of each attribute and another attribute-labeled image of each attribute that is different from the second sample set; the categories of the first sample set and the second sample set are different, and the attribute categories of the second sample set are not less than the attribute categories of the first sample set.

[0142] The feature extraction module 1003 is used to extract features from the first sample set and the image to be identified by using the recognition network to obtain feature maps of multiple images and a feature map of the image to be identified.

[0143] The image recognition module 1004 is used to perform attribute recognition on the image to be recognized based on the feature maps of multiple images and the feature map of the image to be recognized, so as to obtain the attribute type of the image to be recognized.

[0144] For the acquisition and identification process, please refer to the relevant text descriptions in steps S11 to S16, steps S21 to S24, steps S31 to S33, steps S41 to S43 and steps S51 to S52, which will not be repeated here.

[0145] Different from the prior art, after the image acquisition module 1001 acquires the image to be identified, the feature extraction module 1003 performs feature extraction on the first sample set and the image to be identified to obtain feature maps of multiple images and feature maps of the image to be identified, and then the image recognition module 1004 performs attribute recognition on the image to be identified based on the feature maps of multiple images and the feature maps of the image to be identified, and can accurately identify the category of the image to be identified. In addition, the weight value of the feature map of each image in the first sample set is obtained by the self-attention method, and the importance analysis can be performed on all samples in the same attribute, so that the image with greater importance in the fusion feature map has a greater proportion, thereby further improving the accuracy of image recognition.

[0146] Correspondingly, the present application provides a training device for a recognition network.

[0147] Specifically, see Fig.11 , Fig.11 Schematic diagram of the structure of an embodiment of a training device for recognizing a network in the present application. Fig.11 As shown, the training device 110 of the recognition network includes an input module 1101 , a first extraction module 1102 , a fusion module 1103 , a splicing module 1104 , a prediction module 1105 and an output module 1106 .

[0148] The input module 1101 is used to input a sample set including multiple sample images of each attribute and another reference image of each attribute that is different from the sample set and is labeled with the attribute into the initial deep learning model.

[0149] The first extraction module 1102 is used to perform feature extraction on multiple sample images and reference images using an initial deep learning model to obtain feature maps of the multiple sample images and feature maps of the reference images.

[0150] The fusion module 1103 is used to fuse the feature maps of images with the same attribute to obtain a fused feature map of the image with the same attribute.

[0151] The stitching module 1104 is used to stitch the fused feature map of the same attribute image with the feature map of the image to be identified to obtain the stitching features of each attribute image.

[0152] The prediction module 1105 is used to predict the attribute category of the reference image based on the splicing features of each attribute image to obtain the predicted attribute.

[0153] The output module 1106 is used to train the initial deep learning model using the annotation attributes and prediction attributes of the reference image, and determine the trained model as a recognition network.

[0154] For the specific training process, please refer to the relevant text description in steps S11 to S16, which will not be repeated here.

[0155] Different from the prior art, this embodiment trains the initial deep learning model through the input module 1101, the first extraction module 1102, the fusion module 1103, the splicing module 1104, the prediction module 1105 and the output module 1106, so as to build a recognition network. Moreover, by directly training according to the image, the complex image recognition technology can be simplified into a multi-classification task, which reduces the difficulty of training, saves R&D costs and labor costs, and has strong portability and a wider range of applications.

[0156] See also Fig.12 , Fig.12 Schematic diagram of the structure of an electronic device of the present application. Fig.12 As shown, the electronic device 120 includes a memory 1201 and a processor 1202 .

[0157] In this embodiment, the memory 1201 is used to store program data, and when the program data is executed, the steps in the image recognition method as described in any one of the above items or the steps in the training method of the recognition network as described in any one of the above items are implemented; the processor 1202 is used to execute the program instructions stored in the memory 1201 to implement the steps in the image recognition method as described in any one of the above items or the steps in the training method of the recognition network as described in any one of the above items.

[0158] Specifically, the processor 1202 is used to control itself and the memory 1201 to implement the steps in any of the above method embodiments. The processor 1202 can also be called a CPU (Central Processing Unit). The processor 1202 may be an integrated circuit chip with signal processing capabilities. The processor 1202 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 1202 can be implemented by multiple integrated circuit chips.

[0159] Different from the prior art, after the processor 1202 uses the second sample set with a larger amount of data to train the recognition network, it can build a recognition network with a higher recognition accuracy, so that when the recognition network migrates to the first sample set, it can reduce the dependence on the data in the first sample set. Then the first sample set with a smaller amount of data is input into the recognition network as a support set, and the image to be recognized is recognized by the first sample set, so that the category of the image to be recognized can be accurately identified. In addition, the weight value of the feature map of each image in the first sample set is obtained by the self-attention method, and the importance analysis of all samples in the same attribute can be performed, so that the image with greater importance in the fusion feature map accounts for a larger proportion, thereby further improving the accuracy of image recognition.

[0160] Correspondingly, the present application provides a computer-readable storage medium.

[0161] See also Fig.13 , Fig.13 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application.

[0162] The computer-readable storage medium 130 includes a computer program 1301 stored on the computer-readable storage medium 130 . When the computer program 1301 is executed by the processor, the computer program 1301 implements the steps in any of the above method embodiments or the steps correspondingly executed by the relevant apparatus in the above method embodiments.

[0163] Specifically, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 130. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a computer-readable storage medium 130, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 130 includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0164] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0166] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0167] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0168] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image recognition method, characterized in that: include: Obtaining an image to be identified with an attribute category to be identified; Input the image to be recognized and the first sample set into the recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is trained by a second sample set including multiple sample images of each attribute and another image of each attribute labeled with attributes different from the second sample set; the first sample set and the second sample set have different categories, and the attribute categories of the second sample set are not less than the attribute categories of the first sample set; Using the recognition network to perform feature extraction on the first sample set and the image to be recognized, to obtain feature maps of the multiple images and a feature map of the image to be recognized; Based on the feature maps of the multiple images and the feature map of the image to be identified, attribute identification is performed on the image to be identified to obtain the attribute type of the image to be identified; specifically including: fusing the feature maps of images with the same attribute to obtain a fused feature map of the image with the same attribute; splicing the fused feature map of the image with the same attribute with the feature map of the image to be identified to obtain splicing features of each attribute image; and classifying and predicting the image to be identified based on the splicing features of each attribute image to obtain the type of the image to be identified.

2. The image recognition method according to claim 1, characterized in that: The step of fusing the feature maps of images with the same attribute to obtain a fused feature map of images with the same attribute includes: Obtaining weight values ​​of feature maps of each image in the first sample set by using a self-attention method; The feature maps of the images with the same attribute are weightedly fused based on their respective weight values ​​to obtain a fused feature map of the images with the same attribute.

3. The image recognition method according to claim 2, characterized in that: The step of obtaining the weight value of the feature map of each image in the first sample set by using the self-attention method specifically includes: Performing global average pooling on the feature maps of the images with the same attribute in the first sample set and the feature maps of the images to be identified according to the channel dimension one by one, and then performing point multiplication processing on the feature maps of the images with the same attribute that have been subjected to the global average pooling and the feature maps of the images to be identified that have been subjected to the global average pooling, to obtain unnormalized weight values ​​of the feature maps of the images with the same attribute; The step of performing weighted fusion on the feature maps of the images with the same attribute based on their respective weight values ​​to obtain the fused feature map of the images with the same attribute specifically includes: Obtaining normalized weight values ​​corresponding to the feature maps of the images with the same attribute based on the unnormalized weight values; The feature maps of the images with the same attribute are weightedly fused based on the normalized weight value to obtain a fused feature map of the images with the same attribute.

4. The image recognition method according to claim 1, characterized in that: The step of splicing the fused feature map of the image with the same attribute with the feature map of the image to be identified to obtain the splicing features of each attribute image includes: The fused feature maps corresponding to each attribute image are spliced ​​with the feature map of the image to be identified respectively, and feature extraction is performed to obtain the splicing features of each attribute image.

5. The image recognition method according to claim 1, characterized in that: The step of obtaining the image to be identified of the attribute category to be identified specifically includes: Obtain the image to be processed; Performing aspect ratio analysis on the image to be processed; In response to the aspect ratio of the image to be processed being greater than a set value, the image to be processed is rotated counterclockwise by a first set angle; in response to the aspect ratio of the image to be processed being less than the set value, the angle of the image to be processed is maintained.

6. The image recognition method according to claim 5, characterized in that: The step of acquiring the image to be identified of the attribute category to be identified further includes: Using the direction classification model to determine whether the image to be processed is rotated by a second set angle; In response to the image to be processed rotating by the second set angle, the image to be processed is rotated by the second set angle; in response to the image to be processed not rotating by the second set angle, the angle of the image to be processed is maintained.

7. The image recognition method according to any one of claims 1 to 6, characterized in that: The step of obtaining the image to be identified of the attribute category to be identified comprises: Obtaining a license plate image of a non-motor vehicle of a location to be identified; The step of inputting the image to be identified and the first sample set into the recognition network; wherein the first sample set includes multiple images of each attribute; the recognition network is trained by a second sample set including multiple sample images of each attribute and another image labeled with attributes of each attribute different from the second sample set; the first sample set and the second sample set have different categories, and the attribute categories of the second sample set are not less than the attribute categories of the first sample set, comprises: Input the license plate image to be identified and the non-motor vehicle sample set into the license plate recognition network; wherein the non-motor vehicle sample set includes a plurality of non-motor vehicle license plate images of each location; the license plate recognition network is trained by a motor vehicle sample set including a plurality of motor vehicle license plate images of each location and another motor vehicle license plate image of each location that is different from the motor vehicle sample set and is labeled with the location; the location category of the motor vehicle sample set is not less than the location category of the non-motor vehicle sample set; The step of using the recognition network to extract features from the first sample set and the image to be recognized to obtain feature maps of the multiple images and the feature map of the image to be recognized includes: Using the license plate recognition network, feature extraction is performed on the plurality of non-motor vehicle license plate images and the license plate image to be recognized in the non-motor vehicle sample set to obtain feature maps of the plurality of non-motor vehicle license plate images and the license plate image to be recognized; The step of performing attribute recognition on the image to be recognized based on the feature maps of the multiple images and the feature map of the image to be recognized to obtain the attribute type of the image to be recognized includes: Based on the multiple non-motor vehicle license plate images and the feature map of the license plate image to be identified, the location of the license plate image to be identified is identified to obtain the location of the license plate image to be identified.

8. A method for training a recognition network, characterized in that: include: Inputting a sample set including a plurality of sample images of each attribute and another reference image of each attribute labeled with the attribute different from the sample set into an initial deep learning model; Using the initial deep learning model to perform feature extraction on the multiple sample images and the reference image to obtain feature maps of the multiple sample images and the reference image; The feature maps of images with the same attribute are fused to obtain a fused feature map of images with the same attribute; splicing the fused feature map of the image with the same attribute with the feature map of the image to be identified to obtain splicing features of each attribute image; Predicting the attribute category of the reference image based on the splicing features of the attribute images to obtain a predicted attribute; The initial deep learning model is trained using the annotation attributes of the reference image and the prediction attributes, and the trained model is determined as the recognition network.

9. The method for training a recognition network according to claim 8, characterized in that: The step of fusing the feature maps of images with the same attribute to obtain a fused feature map of images with the same attribute includes: Obtaining the weight value of the feature map of each image in the sample set by using a self-attention method; The feature maps of the images with the same attribute are weightedly fused based on their respective weight values ​​to obtain a fused feature map of the images with the same attribute.

10. An image recognition device, characterized in that: include: An image acquisition module is used to acquire an image to be identified with an attribute category to be identified; An image input module, used for inputting the image to be recognized and a first sample set into a recognition network; wherein the first sample set includes a plurality of images of each attribute; the recognition network is trained by a second sample set including a plurality of sample images of each attribute and another image of each attribute labeled with attributes different from the second sample set; the first sample set and the second sample set have different categories, and the attribute categories of the second sample set are not less than the attribute categories of the first sample set; A feature extraction module, used to extract features from the first sample set and the image to be identified using the recognition network to obtain feature maps of the multiple images and a feature map of the image to be identified; The image recognition module is used to perform attribute recognition on the image to be recognized based on the feature maps of the multiple images and the feature map of the image to be recognized, so as to obtain the attribute type of the image to be recognized; specifically, it is used to fuse the feature maps of images with the same attribute to obtain a fused feature map of the image with the same attribute; splice the fused feature map of the image with the same attribute with the feature map of the image to be recognized to obtain the spliced ​​features of each attribute image; and classify and predict the image to be recognized based on the spliced ​​features of each attribute image to obtain the type of the image to be recognized.

11. A training device for a recognition network, characterized in that: include: An input module, used to input a sample set including a plurality of sample images of each attribute and another reference image of each attribute labeled with the attribute different from the sample set into an initial deep learning model; A first extraction module, configured to perform feature extraction on the plurality of sample images and the reference image using the initial deep learning model to obtain feature maps of the plurality of sample images and the reference image; A fusion module is used to fuse the feature maps of images with the same attribute to obtain a fused feature map of the image with the same attribute; A splicing module, used for splicing the fused feature map of the image with the same attribute with the feature map of the image to be identified to obtain the splicing features of each attribute image; A prediction module, used for predicting the attribute category of the reference image based on the splicing features of the attribute images to obtain a predicted attribute; The output module is used to train the initial deep learning model using the annotation attributes of the reference image and the prediction attributes, and determine the trained model as the recognition network.

12. An electronic device, characterized in that: include: A memory for storing program data, wherein when the program data is executed, the steps of the image recognition method according to any one of claims 1 to 7 or the steps of the recognition network training method according to any one of claims 8 to 9 are implemented; A processor, configured to execute program instructions stored in the memory to implement the steps in the image recognition method according to any one of claims 1 to 7 or the steps in the recognition network training method according to any one of claims 8 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the image recognition method according to any one of claims 1 to 7 or the steps in the recognition network training method according to any one of claims 8 to 9 are implemented.

Citation Information

Patent Citations

  • Image processing method and device

    CN109902723A

  • Power equipment image recognition method and device based on transfer learning and neural network

    CN110378408A

  • Image recognition method, electronic equipment and storage medium

    CN110717554A

  • Brain midline recognition method and device, computer equipment and storage medium

    CN112241952A

  • Image classification method and system based on small sample learning, storage medium and terminal

    CN112434721A