Image recognition method, device and storage medium

Abstract

The embodiment of the present invention discloses an image recognition method, device and storage medium; the embodiment of the present invention can obtain an image to be recognized and an image recognition model, the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups; feature extraction is performed on the image to be recognized to obtain image features of the image to be recognized; the image features are grouped and analyzed using the fully connected first sublayer to obtain a grouping vector of the image to be recognized; the type grouping of the image to be recognized is determined based on the grouping vector; the grouping vector is weighted and summed based on the type grouping using the fully connected second sublayer to obtain a classification vector of the image to be recognized; the image type of the image to be recognized is determined according to the classification vector. This solution reduces the amount of calculation by first determining the grouping of the image and then calculating the type of the image based on the grouping of the image, thereby improving the efficiency of the image recognition method.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image recognition method, device and storage medium. Background Art

[0002] Computer vision technology (CV) refers to the use of artificial intelligence (AI) technology to enable computers to identify, track and measure target images instead of human eyes, and further process target images into images that are more suitable for human eye observation or transmission to instrument detection. At present, computer vision technology usually includes image processing, image recognition, image semantic understanding and other technologies.

[0003] However, current image recognition methods are inefficient. Summary of the invention

[0004] Embodiments of the present invention provide an image recognition method, an apparatus, and a storage medium, which can improve the efficiency of the image recognition method.

[0005] An embodiment of the present invention provides an image recognition method, including:

[0006] Obtaining an image to be recognized and an image recognition model, wherein the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups;

[0007] Extracting features of the image to be identified to obtain image features of the image to be identified;

[0008] Using the fully connected first sublayer to perform group analysis on the image features to obtain a group vector of the image to be identified;

[0009] Determining the type grouping of the image to be identified based on the grouping vector;

[0010] Using the fully connected second sublayer to perform weighted sum processing on the grouping vector based on the type grouping, to obtain a classification vector of the image to be identified;

[0011] The image type of the image to be identified is determined according to the classification vector.

[0012] The embodiment of the present invention further provides an image recognition device, comprising:

[0013] An acquisition unit, used to acquire an image to be recognized and an image recognition model, wherein the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups;

[0014] A feature unit, used for extracting features of the image to be identified to obtain image features of the image to be identified;

[0015] An analysis unit, configured to perform group analysis on the image features using the fully connected first sublayer to obtain a group vector of the image to be identified;

[0016] A grouping unit, configured to determine a type grouping of the image to be identified based on the grouping vector;

[0017] A classification unit, configured to use the fully connected second sublayer to perform weighted sum processing on the grouping vector based on the type grouping to obtain a classification vector of the image to be identified;

[0018] A determination unit is used to determine the image type of the image to be identified according to the classification vector.

[0019] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in the image recognition method of any one of the claims.

[0020] The embodiment of the present invention can obtain an image to be identified and an image recognition model, wherein the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups; feature extraction is performed on the image to be identified to obtain image features of the image to be identified; the image features are grouped and analyzed using the fully connected first sublayer to obtain a grouping vector of the image to be identified; the type grouping of the image to be identified is determined based on the grouping vector; the grouping vector is weighted and summed based on the type grouping using the fully connected second sublayer to obtain a classification vector of the image to be identified; and the image type of the image to be identified is determined according to the classification vector.

[0021] The embodiment of the present invention can perform image recognition through an image recognition model based on an artificial neural network, wherein when the image recognition model is used to calculate the classification vector of an image, the grouping of the image can be first determined in the fully connected first sublayer of the image recognition model, and then the type of the image can be calculated only for the grouping of the image in the fully connected second sublayer of the image recognition model, thereby reducing the amount of calculation consumed by calculating the image type. Thus, the solution can improve the efficiency of the image recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1a is a schematic diagram of a scene of an image recognition method provided by an embodiment of the present invention;

[0024] Figure 1b is a schematic diagram of a first flow chart of an image recognition method provided by an embodiment of the present invention;

[0025] Figure 1c is a schematic diagram of image type grouping in an image recognition method provided by an embodiment of the present invention;

[0026] Figure 2a is a second flow chart of the image recognition method provided by an embodiment of the present invention;

[0027] Figure 2b is a schematic diagram of node grouping of an image recognition method provided by an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of the structure of an image recognition device provided by an embodiment of the present invention;

[0029] Figure 4 It is a schematic diagram of the structure of a network device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] Embodiments of the present invention provide an image recognition method, device and storage medium.

[0032] Specifically, the image recognition device can be integrated into an electronic device, and the electronic device can be a terminal, a server, or other devices. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, or a personal computer (PC) or other devices; the server can be a single server or a server cluster composed of multiple servers. For example, in some embodiments, the image recognition device can be integrated into multiple servers, and the image recognition method of the present invention can be implemented by multiple servers.

[0033] Computer vision technology (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing to make computer processing into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0034] The solution provided in the embodiments of the present application involves artificial intelligence computer vision technology, which is specifically described by the following embodiments:

[0035] refer to Figure 1a , the electronic device can obtain an image to be recognized and an image recognition model, wherein the image recognition model can include a fully connected first sublayer and a fully connected second sublayer; extract features of the image to be recognized to obtain image features of the image to be recognized; use the fully connected first sublayer to group and analyze the image features to obtain a grouping vector of the image to be recognized; determine the type grouping of the image to be recognized based on the grouping vector; use the fully connected second sublayer to perform weighted summation processing on the grouping vector based on the type grouping to obtain a classification vector of the image to be recognized; determine the image type of the image to be recognized according to the classification vector.

[0036] It should be noted that the serial numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0037] In this embodiment, an image recognition method is provided, such as Figure 1b As shown, the specific process of the image recognition method can be as follows:

[0038] 101. Obtain an image to be recognized and an image recognition model, wherein the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups.

[0039] The image recognition model is an artificial neural network model (ANN), which may include a convolution layer, a downsampling pooling layer, a fully connected layer, an activation function, etc. The image recognition model may be specifically determined according to the needs of the actual application.

[0040] For example, the image recognition model can be various convolutional neural network (CNN) models such as ResNet-50, LeNet-5, AlexNet, and VGGNet, etc.

[0041] In some embodiments, the image recognition model may include a convolutional neural network including multiple convolutional layers, a fully connected layer, and an activation function, wherein the fully connected layer may include two cascaded sublayers, namely a fully connected first sublayer and a fully connected second sublayer.

[0042] Among them, each fully connected sublayer can include multiple network nodes (also called network nodes), the network nodes between the fully connected sublayers are connected to each other, and each network node of the topmost fully connected sublayer is fully connected with all network nodes of the previous layer (for example, the bottom convolutional layer of multiple convolutional layers).

[0043] Among them, after the output value of the bottom fully connected sublayer is normalized, the probability that the image to be identified belongs to different image types can be obtained. For example, using the softmax logistic regression function as the softmax layer can normalize the output value of the bottom fully connected sublayer to obtain the probability that the image to be identified belongs to different image types.

[0044] The image recognition model can be trained by a training image set labeled with image types and image type groups.

[0045] The image type is the image content type, for example, the image type may be a cat image type, a dog image type, a parrot image type, etc. The image type group refers to the type group to which the image content belongs, for example, the image type group may include a pet group, an animal group, a plant group, etc.

[0046] For example, a certain image is of a cat image type, and its image type grouping may include a pet group and an animal group.

[0047] In some embodiments, an image recognition model may be obtained through training, so the following steps may be included before step 101:

[0048] (1) obtaining a preset model and a training image set labeled with image types and image type groups, wherein the preset model includes a preset fully connected first sublayer;

[0049] (2) A training image set labeled with image types and image type groups is used to train the preset fully connected first sublayer until convergence to obtain an image recognition model.

[0050] The preset model may be a commonly used neural network model that can be used for image classification.

[0051] Specifically, in some embodiments, step (1) "obtaining a preset model, a training image set annotated with image types and image type groups, wherein the preset model includes a preset fully connected first sublayer" may include the following steps:

[0052] a. Obtain a training image set and an initial model with labeled image types and image type groups, where the initial model includes multiple network nodes;

[0053] b. Based on the training image set labeled with image types and image type groups, multiple network nodes of the initial model are grouped by type to obtain a preset model.

[0054] Among them, the initial model is a neural network model for image classification that needs to be trained before it can be used.

[0055] Based on a training image set labeled with image types and image type groups, multiple network nodes of the initial model are grouped by type. A method for obtaining a preset model is to group multiple network nodes in the initial model that can calculate all image types in the image type group.

[0056] For example, reference Figure 1c , the right side of the image shows multiple network nodes of the initial model, among which the gray network nodes can calculate image type a1, image type a2, and image type a3 in image type group a, and the gray network nodes are divided into a group, which can generate an association relationship with image type group a.

[0057] It should be noted that a network node can generate association relationships with multiple image type groups at the same time.

[0058] In some embodiments, in order to further reduce the amount of calculation and thus improve the efficiency of image recognition, the training image set annotated with image types and image type groups includes the association relationship between the image types and image type groups of the training images, and step a may specifically include the following steps:

[0059] According to the association relationship between the image types and the image type groups, multiple network nodes of the initial model are grouped to obtain a grouped network node set;

[0060] Merging the grouped network node sets to obtain a merged network node set;

[0061] According to the association relationship between the image type and the image type grouping, an association relationship between the type grouping and the network node after the merging process is generated;

[0062] A preset model is generated based on the association between the merged network nodes, the type groups and the merged network nodes.

[0063] For example, reference Figure 1c , the gray network node parameters corresponding to the image type group a are merged. For example, the four gray network node parameters corresponding to the image type group a are added together to obtain a merged parameter, and the merged parameter is used as a new parameter of the four gray network nodes, thereby obtaining four gray network nodes with the same parameter value.

[0064] In some embodiments, the association relationship between the type grouping and the network node after the addition process may also be generated based on the association relationship between the image type and the image type grouping.

[0065] For example, reference Figure 1c , add the parameters of the four gray network nodes corresponding to the image type group a, obtain the added value, and use it as the parameter of the four gray network nodes, thereby obtaining four gray network nodes with the same parameter values.

[0066] In some embodiments, the training data may be preprocessed in advance to improve the accuracy of image recognition and further improve the efficiency of image recognition. Therefore, step a may specifically include the following steps:

[0067] Obtain a training image set and an initial model with labeled image types;

[0068] The initial model is used to calculate the classification vectors of the training images in the training image set;

[0069] Calculating distances between training images based on classification vectors of training images in the training image set;

[0070] Clustering the training images in the training image set based on the distances between the training images to obtain a training image cluster group;

[0071] The training images in the training image clustering group are grouped and labeled by image type to obtain a training image set labeled with image types and image type groups.

[0072] The distance between training images is the vector distance between the classification vectors of the training images. The closer the distance is, the more similar the image contents are.

[0073] Specifically, the vector distance may be a confidence level, and the closer the confidence level is, the more similar the image contents are.

[0074] The method of clustering the training images in the training image set based on the distance between the training images is to classify the first N training images with the closest vector distance to a certain training image into one cluster, and then calculate the vector distance between different clusters, and finally obtain a training image clustering group of M clusters.

[0075] Wherein, N and M are both positive integers and can be set by technical personnel.

[0076] 102. Perform feature extraction on the image to be identified to obtain image features of the image to be identified.

[0077] Among them, the image recognition model can extract features of the image to be recognized to obtain multiple image features of the image to be recognized.

[0078] The image recognition model can include various neural networks for extracting image features. For example, the convolutional neural network in the image recognition model can extract image features. For example, the feature extraction part of convolutional neural networks such as ResNet-50, LeNet-5, AlexNet and VGGNet can be used as the convolutional neural network in the image recognition model, that is, the network part of these CNN networks except the last fully connected layer can be used as the convolutional neural network in the image recognition model to extract image features.

[0079] 103. Use the fully connected first sub-layer to perform group analysis on image features to obtain a group vector of the image to be identified.

[0080] refer to Figure 1c In this embodiment, the fully connected first sublayer may include multiple grouped network nodes. When image features with different image type groups are input into the fully connected first sublayer, the fully connected first sublayer can perform group analysis on the image features to obtain a grouping vector of the image to be identified.

[0081] Specifically, in some embodiments, the fully connected first sublayer may include multiple network node sets, and step 103 may include the following steps:

[0082] Multiple network node sets are used to perform weighted sum processing on image features to obtain multiple grouping vectors.

[0083] The grouping vector may indicate which image classification groups the image feature belongs to.

[0084] It should be noted that the representation of grouped vectors can include matrices, vectors, numbers, etc.

[0085] The obtained multiple grouping vectors may be used in step 104 to determine the type grouping of the image to be identified.

[0086] 104. Determine a type grouping of the image to be identified based on the grouping vector.

[0087] In some embodiments, step 104 may include the following steps:

[0088] (1) determining at least one target grouping vector among a plurality of grouping vectors;

[0089] (2) determining the network node set corresponding to the target grouping vector as the target network node set;

[0090] (3) Determine the type grouping associated with the target network node set as the type grouping of the image to be identified.

[0091] In some embodiments, in order to improve the image recognition accuracy, step (1) "determining at least one target grouping vector from multiple grouping vectors" may specifically include the following steps:

[0092] Get the preset value;

[0093] A vector size comparison is performed on multiple grouping vectors, and a preset number of grouping vectors is determined as a target grouping vector.

[0094] The preset value may be any positive integer and may be set by a technician.

[0095] For example, when the preset value is 3, vector sizes of multiple grouping vectors are compared, and the three largest grouping vectors are determined as target grouping vectors.

[0096] In some embodiments, the type grouping is a type grouping associated with a network node, and step (2) "using the type grouping associated with the target network node set as the type grouping of the image to be identified" may specifically include the following steps:

[0097] Query the type grouping associated with the target network node;

[0098] When there is a type grouping associated with the target network node, the type grouping is used as the type grouping of the image to be identified.

[0099] 105. Use the fully connected second sublayer to perform weighted summation processing on the grouping vectors based on type grouping to obtain the classification vector of the image to be identified.

[0100] Among them, reference Figure 1c, the fully connected second sublayer can also be similar to the fully connected first sublayer, that is, the fully connected second sublayer can include multiple grouped network nodes. When the grouped fully connected second sublayer is used to perform weighted summation processing on the grouped vectors, it is only necessary to group them according to type and select the network nodes of the group in the fully connected second sublayer for weighted summation processing to obtain the classification vector of the image to be identified.

[0101] Therefore, there is no need to use the weighted summation processing of the network nodes in the fully connected second sublayer that are irrelevant to the image type group to be identified. Only the weighted summation processing of a part of the network nodes in the fully connected second sublayer that are associated with the image type group to be identified can accurately obtain the classification vector of the image to be identified. Therefore, this scheme can reduce the amount of calculation while ensuring the accuracy of image recognition.

[0102] It should be noted that classification vectors can be expressed in multiple forms, for example, classification vectors can be expressed in multiple forms such as vectors, matrices, numbers, linked lists, etc.

[0103] 106. Determine the image type of the image to be identified according to the classification vector.

[0104] Specifically, the classification vector output by the fully connected second sub-layer can be input into the normalization layer for normalization processing to obtain the probability that the image to be identified belongs to different image types.

[0105] Among them, the normalization layer may include a sigmoid function, a softmax function, and the like.

[0106] As can be seen from the above, the embodiment of the present invention can obtain an image to be identified and an image recognition model, the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups; feature extraction is performed on the image to be identified to obtain image features of the image to be identified; the image features are grouped and analyzed using the fully connected first sublayer to obtain a grouping vector of the image to be identified; the type grouping of the image to be identified is determined based on the grouping vector; the grouping vector is weighted and summed based on the type grouping using the fully connected second sublayer to obtain a classification vector of the image to be identified; the image type of the image to be identified is determined based on the classification vector. The embodiment of the present invention can perform image recognition through an image recognition model based on an artificial neural network, wherein when the image recognition model is used to calculate the classification vector of the image, the grouping of the image can be first determined in the fully connected first sublayer of the image recognition model, and then the type of the image can be calculated only for the grouping of the image in the fully connected second sublayer of the image recognition model, thereby reducing the amount of calculation consumed by calculating the image type. Therefore, this solution can improve the efficiency of the image recognition method.

[0107] The method described in the above embodiments will be further described in detail below.

[0108] In this embodiment, taking the application of this image recognition method to the recognition of Chinese character images based on optical character recognition as an example, the method of the embodiment of the present invention will be described in detail.

[0109] As Figure 2a shown, the specific process of an image recognition method is as follows:

[0110] 201. Obtain training data and a preset model, and use the preset model to preprocess the training data to obtain a training image set labeled with image types and image type groups.

[0111] First, use a preset model that can already be used for image recognition to perform image recognition on the training data.

[0112] Among them, the training data can be an image containing a Chinese character. The Chinese characters in the Chinese character image can have different styles. For example, there can be multiple images containing the Chinese character "粉", one image of the Chinese character "粉" in each of different styles such as boldface, regular script, and Song typeface. When the preset model performs image recognition on these training data (i.e., the images containing the Chinese character "粉"), it can label these images with the label [粉].

[0113] In some embodiments, during the process of the preset model preprocessing the training data, the confidence of the image belonging to different Chinese character types can be calculated. For example, when an image containing the Chinese character "粉" is recognized by the preset model, the preset model can calculate that the confidence of the image containing the Chinese character "粉" belonging to the Chinese character [粉] type is 1447.9, the confidence of belonging to the Chinese character [纷] type is 1457.9, the confidence of belonging to the Chinese character [籽] type is 1459.1, the confidence of belonging to the Chinese character [扮] type is 1460.2, and so on.

[0114] In this embodiment, the top K image types with the highest confidence can be selected as candidate image types, and clustering operations can be performed based on the confidence comparison between these candidate image types. Finally, J image type groups are generated, where K and J can be any positive integers and can be set by those skilled in the art.

[0115] After that, these training data can be labeled with image type groups.

[0116] For example, the image of the Chinese character "粉" is labeled as [粉, 粉+纷] before clustering, where the label [粉] is the image type of the Chinese character "粉" in this image, and the label [粉+纷] is the image type group of the Chinese character "粉" in this image.

[0117] 202. Use the training image set labeled with image types and image type groups to train a preset model to obtain an image recognition model.

[0118] In this embodiment, the training image set labeled with image types and image type groups includes the association relationship between the image types of the training images and the image type groups. The specific steps of using the training image set labeled with image types and image type groups to train a preset model are as follows:

[0119] Obtain a training image set labeled with image types and image type groups and an initial model. The initial model includes multiple network nodes;

[0120] According to the association relationship between the image types and the image type groups, group the multiple network nodes in the fully connected layer of the initial model to obtain a set of grouped network nodes;

[0121] Perform a merging process on the set of grouped network nodes to obtain a set of network nodes after the merging process;

[0122] According to the association relationship between the image types and the image type groups, generate the association relationship between the type groups and the network nodes after the merging process;

[0123] Generate a preset model based on the network nodes after the merging process and the association relationship between the type groups and the network nodes after the merging process.

[0124] For example, referring to Figure 2b , multiple network nodes (shown as gray nodes in the figure) associated with the labels [powder] and [numerous] in the fully connected layer of the initial model can be grouped together, merged, and this group can be associated with the label [powder + numerous].

[0125] Specifically, in some embodiments, the fully connected layer of the initial model is a matrix W, where the parameter of each network node is w ij , as follows:

[0126]

[0127] Assume that the input vector of the fully connected layer of the initial model is vector X, where the vector parameter of vector X is x j , as follows:

[0128] X = [x1, x2,....x j

[0129] When using the fully connected layer of the initial model to process this vector X, that is, when the matrix W and the vector X are weighted and summed, the fully connected layer can output a vector Y, where the vector parameter of vector Y is y i , and each vector parameter y i ​All can establish a mapping relationship with image classification as follows:

[0130]

[0131] Assume the association relationship between the image type and the image type grouping is:

[0132] There is an association relationship between the image type [powder] and the image type [numerous] and the image type grouping [powder + numerous]. Then, group [w 11 、w 12 ...w 1j and [w 21 、w 22 ...w 2j into one group, and perform a merging process on [w 11 、w 12 ...w 1j and [w 21 、w 22 ...w 2j to obtain a new group [w 11+ w 21 、w 12+ w 22 、...w 1j+ w 2j .

[0133] And establish a mapping relationship between this new group [w 11+ w 21 、w 12+ w 22 、...w 1j+ w 2j and the image type grouping [powder + numerous].

[0134] 203. Obtain the image to be recognized, and use an image recognition model to recognize the image type of the image to be recognized.

[0135] Step 203 can refer to Step 102, Step 103, Step 104, Step 105, Step 106, which will not be elaborated here.

[0136] Assume there are 10,000 kinds of annotated Chinese characters in total. Then, in the traditional image recognition method, the calculation amount of the fully connected layer is 10,000 * the input quantity of the fully connected layer. In this embodiment, the calculation amount of the fully connected layer is (j + (x1 + … + x i )) * the input quantity of the fully connected layer, where x i is the number of different image types in each image type grouping.

[0137] Thus, in this solution, the parameter quantity of the fully connected layer is reduced to 1 / n times that of the traditional method, where 1 / n = the number of image types / the number of image type groupings.

[0138] As can be seen from the above, this embodiment can obtain training data and a preset model, and use the preset model to preprocess the training data to obtain a training image set labeled with image types and image type groups; use the training image set labeled with image types and image type groups to train the preset model to obtain an image recognition model; obtain an image to be recognized, and use the image recognition model to recognize the image type of the image to be recognized. Therefore, this embodiment can greatly reduce the amount of image recognition calculations in complex situations where the number of labels is huge and the parameters of the fully connected layer are numerous, thereby improving the efficiency of the image recognition method.

[0139] In order to better implement the above method, an embodiment of the present invention further provides an image recognition device, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, or a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0140] In some embodiments, the image recognition device may also be integrated into multiple electronic devices. For example, the image recognition device may be integrated into multiple servers, and the image recognition method of the present invention may be implemented by multiple servers.

[0141] For example, in this embodiment, an image recognition device integrated into a single server is taken as an example to describe the method of the embodiment of the present invention in detail.

[0142] For example, Figure 3 As shown, the image recognition device may include an acquisition unit 301, a feature unit 302, an analysis unit 303, a grouping unit 304, a classification unit 305, and a determination unit 306, as follows:

[0143] (I) Acquisition unit 301:

[0144] The acquisition unit 301 can be used to acquire an image to be recognized and an image recognition model. The image recognition model includes a fully connected first sublayer and a fully connected second sublayer. The image recognition model is trained by a training image set labeled with image types and image type groups.

[0145] In some embodiments, the acquisition unit 301 may further include an acquisition subunit and a convergence subunit, which are used before the step of "acquiring the image to be recognized and the image recognition model":

[0146] (1) Get subunits:

[0147] The acquisition subunit can be used to acquire a preset model, a training image set labeled with image types and image type groups, and the preset model includes a fully connected first sublayer.

[0148] (2) Convergence subunit:

[0149] The convergence subunit can be used to train the fully connected first sublayer using a training image set labeled with image types and image type groups until convergence to obtain an image recognition model.

[0150] In some embodiments, the acquisition subunit may include an acquisition submodule and a model submodule, as follows:

[0151] a. Get the submodule:

[0152] The acquisition submodule can be used to acquire a training image set annotated with image types and image type groups, and an initial model, wherein the initial model includes multiple network nodes.

[0153] b. Model submodule:

[0154] The model submodule can be used to perform type grouping processing on multiple network nodes of the initial model based on a training image set labeled with image types and image type groups to obtain a preset model.

[0155] In some embodiments, the training image set labeled with image types and image type groups includes associations between image types and image type groups of the training images, and the model submodule can be specifically used for:

[0156] According to the association relationship between the image types and the image type groups, multiple network nodes of the initial model are grouped to obtain a grouped network node set;

[0157] Merging the grouped network node sets to obtain a merged network node set;

[0158] According to the association relationship between the image type and the image type grouping, an association relationship between the type grouping and the network node after the merging process is generated;

[0159] A preset model is generated based on the association relationship between the merged network nodes, the type groups and the merged network nodes.

[0160] In some embodiments, the acquisition submodule may be specifically used for:

[0161] Obtain a training image set and an initial model with labeled image types;

[0162] The initial model is used to calculate the classification vectors of the training images in the training image set;

[0163] Calculating distances between training images based on classification vectors of training images in the training image set;

[0164] Clustering the training images in the training image set based on the distances between the training images to obtain a training image cluster group;

[0165] The training images in the training image clustering group are grouped and labeled by image type to obtain a training image set labeled with image types and image type groups.

[0166] (ii) Feature unit 302:

[0167] The feature unit 302 may be used to extract features from the image to be identified, and obtain image features of the image to be identified.

[0168] (III) Analysis unit 303:

[0169] The analysis unit 303 may be configured to perform group analysis on image features using the fully connected first sub-layer to obtain a group vector of the image to be identified.

[0170] In some embodiments, the fully connected first sublayer includes a plurality of network node sets, and the analysis unit 303 may be used to:

[0171] Multiple network node sets are used to perform weighted sum processing on image features to obtain multiple grouping vectors.

[0172] (IV) Grouping unit 304:

[0173] The grouping unit 304 may be configured to determine the type grouping of the to-be-recognized image based on the grouping vector.

[0174] In some embodiments, the grouping unit 304 may include a vector sub-unit, a node sub-unit, and a grouping sub-unit, as follows:

[0175] (1) Vector unit:

[0176] The vector subunit may be configured to determine at least one target grouping vector among the plurality of grouping vectors.

[0177] (2) Node subunit:

[0178] The node subunit may be configured to determine the network node set corresponding to the target grouping vector as the target network node set.

[0179] (3) Grouping subunits:

[0180] The grouping subunit may be used to determine the type group associated with the target network node set as the type group of the image to be identified.

[0181] In some embodiments, the type grouping may be a type grouping having an association relationship with a network node, and the grouping subunit may be specifically used for:

[0182] Query the type grouping associated with the target network node;

[0183] When there is a type grouping associated with the target network node, the type grouping is used as the type grouping of the image to be identified.

[0184] In some embodiments, at least one target grouping vector is determined from a plurality of grouping vectors, and the vector subunit may be specifically configured to:

[0185] Get the preset value;

[0186] A vector size comparison is performed on multiple grouping vectors, and a preset number of grouping vectors is determined as a target grouping vector.

[0187] (V) Classification unit 305:

[0188] The classification unit 305 may be configured to perform weighted sum processing on the grouping vectors based on type grouping using the fully connected second sub-layer to obtain a classification vector of the image to be identified.

[0189] (VI) Determining unit 306:

[0190] The determination unit 306 may be configured to determine the image type of the image to be identified according to the classification vector.

[0191] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0192] As can be seen from the above, the image recognition device of this embodiment acquires the image to be recognized and the image recognition model by the acquisition unit, the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups; the feature unit extracts features of the image to be recognized to obtain image features of the image to be recognized; the analysis unit uses the fully connected first sublayer to group and analyze the image features to obtain a grouping vector of the image to be recognized; the grouping unit determines the type grouping of the image to be recognized based on the grouping vector; the classification unit uses the fully connected second sublayer to perform weighted summation processing on the grouping vector based on the type grouping to obtain the classification vector of the image to be recognized; the determination unit determines the image type of the image to be recognized based on the classification vector. Therefore, this solution can improve the efficiency of the image recognition method.

[0193] The embodiment of the present invention further provides an electronic device, which can be a terminal, a server or other device. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, or a personal computer (PC) and other devices; the server can be a single server or a server cluster composed of multiple servers. In some embodiments, the image recognition device can also be integrated into multiple electronic devices. For example, the image recognition device can be integrated into multiple servers, and the image recognition method of the present invention can be implemented by multiple servers. In some embodiments, the xx server and the xx server can also be implemented by one server.

[0194] In this embodiment, the electronic device of this embodiment is XX as an example for detailed description, for example, Figure 4 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present invention, specifically:

[0195] The server may include one or more processing core processors 401, one or more computer-readable storage media memories 402, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will appreciate that Figure 4 The server structure shown in the figure does not constitute a limitation on the server, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0196] The processor 401 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the server and processes data, thereby monitoring the server as a whole. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0197] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0198] The server also includes a power supply 403 for supplying power to various components. In some embodiments, the power supply 403 may be logically connected to the processor 401 through a power management system, so that the power management system can manage charging, discharging, power consumption, and other functions. The power supply 403 may also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0199] The server may further include an input module 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0200] The server may further include a communication module 405. In some embodiments, the communication module 405 may include a wireless module. The server may perform short-range wireless transmission through the wireless module of the communication module 405, thereby providing wireless broadband Internet access for the user. For example, the communication module 405 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0201] Although not shown, the server may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the server will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0202] Obtain an image to be recognized and an image recognition model, the image recognition model comprising a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups;

[0203] Perform feature extraction on the image to be identified to obtain image features of the image to be identified;

[0204] The fully connected first sublayer is used to group and analyze the image features to obtain the group vector of the image to be identified;

[0205] Determine the type grouping of the image to be identified based on the grouping vector;

[0206] The fully connected second sublayer is used to perform weighted summation processing on the grouping vectors based on type grouping to obtain the classification vector of the image to be identified;

[0207] The image type of the image to be identified is determined according to the classification vector.

[0208] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0209] From the above, it can be seen that this solution can improve the efficiency of the image recognition method.

[0210] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0211] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the image recognition methods provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0212] Obtain an image to be recognized and an image recognition model, the image recognition model comprising a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups;

[0213] Perform feature extraction on the image to be identified to obtain image features of the image to be identified;

[0214] The fully connected first sublayer is used to group and analyze the image features to obtain the group vector of the image to be identified;

[0215] Determine the type grouping of the image to be identified based on the grouping vector;

[0216] The fully connected second sublayer is used to perform weighted summation processing on the grouping vectors based on type grouping to obtain the classification vector of the image to be identified;

[0217] The image type of the image to be identified is determined according to the classification vector.

[0218] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0219] Since the instructions stored in the storage medium can execute the steps in any image recognition method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any image recognition method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0220] The above is a detailed introduction to an image recognition method, device and storage medium provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An image recognition method, characterized in that: include: Obtaining an image to be recognized and an image recognition model, wherein the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups; Extracting features of the image to be identified to obtain image features of the image to be identified; Using the fully connected first sublayer to perform group analysis on the image features to obtain a group vector of the image to be identified; Determining the type grouping of the image to be identified based on the grouping vector; Using the fully connected second sublayer to perform weighted sum processing on the grouping vector based on the type grouping, to obtain a classification vector of the image to be identified; Determining the image type of the image to be identified according to the classification vector; The fully connected first sublayer includes a plurality of network node sets, and the fully connected first sublayer is used to perform group analysis on the image features to obtain a group vector of the image to be identified, including: Using multiple network node sets to perform weighted sum processing on the image features to obtain multiple grouping vectors; Determining the type grouping of the to-be-recognized image based on the grouping vector includes: determining at least one target grouping vector among the plurality of grouping vectors; Determine the network node set corresponding to the target grouping vector as the target network node set; The type group associated with the target network node set is determined as the type group of the image to be identified.

2. The image recognition method according to claim 1, characterized in that: The type grouping is a type grouping associated with the network node, and the type grouping associated with the target network node set is used as the type grouping of the image to be identified, including: Querying the type group associated with the target network node; When there is a type group associated with the target network node, the type group is used as the type group of the image to be identified.

3. The image recognition method according to claim 1, characterized in that: Determining at least one target grouping vector among a plurality of grouping vectors comprises: Get the preset value; A vector size comparison is performed on multiple grouping vectors, and a preset number of grouping vectors is determined as a target grouping vector.

4. The image recognition method according to claim 1, characterized in that: Before obtaining the image to be recognized and the image recognition model, the method further includes: Obtaining a preset model and a training image set labeled with image types and image type groups, wherein the preset model includes a preset fully connected first sublayer; The preset fully connected first sublayer is trained using the training image set labeled with image types and image type groups until convergence to obtain an image recognition model.

5. The image recognition method according to claim 4, characterized in that: Get a preset model, a training image set with labeled image types and image type groups, including: Obtaining a training image set labeled with image types and image type groups and an initial model, wherein the initial model includes a plurality of network nodes; Based on the training image set labeled with image types and image type groups, type grouping processing is performed on multiple network nodes of the initial model to obtain a preset model.

6. The image recognition method according to claim 5, characterized in that: The training image set labeled with image types and image type groups includes an association relationship between image types and image type groups of the training images. Based on the training image set labeled with image types and image type groups, multiple network nodes of the initial model are subjected to type grouping processing to obtain a preset model, including: According to the association relationship between the image type and the image type grouping, the multiple network nodes of the initial model are grouped to obtain a grouped network node set; Merging the grouped network node sets to obtain a merged network node set; Generate an association relationship between the type group and the merged network node according to the association relationship between the image type and the image type group; A preset model is generated based on the association relationship between the merged network nodes, the type groups and the merged network nodes.

7. The image recognition method according to claim 5, characterized in that: The step of obtaining a training image set and an initial model with image types and image type groups marked includes: Obtain a training image set and an initial model with labeled image types; Calculating classification vectors of training images in the training image set using the initial model; Calculating distances between the training images based on classification vectors of the training images in the training image set; Clustering the training images in the training image set based on the distances between the training images to obtain a training image cluster group; The training images in the training image cluster group are grouped and labeled by image type to obtain a training image set labeled with image types and image type groups.

8. An image recognition device, characterized in that: include: An acquisition unit, used to acquire an image to be recognized and an image recognition model, wherein the image recognition model includes a fully connected first sublayer and a fully connected second sublayer, and the image recognition model is trained by a training image set labeled with image types and image type groups; A feature unit, used for extracting features of the image to be identified to obtain image features of the image to be identified; An analysis unit, configured to perform group analysis on the image features using the fully connected first sublayer to obtain a group vector of the image to be identified; A grouping unit, configured to determine a type grouping of the image to be identified based on the grouping vector; A classification unit, configured to use the fully connected second sublayer to perform weighted sum processing on the grouping vector based on the type grouping to obtain a classification vector of the image to be identified; a determination unit, configured to determine the image type of the image to be identified according to the classification vector; The fully connected first sublayer includes a plurality of network node sets, and the analysis unit is used for: Using multiple network node sets to perform weighted sum processing on the image features to obtain multiple grouping vectors; The grouping unit is used for: determining at least one target grouping vector among the plurality of grouping vectors; Determine the network node set corresponding to the target grouping vector as the target network node set; The type group associated with the target network node set is determined as the type group of the image to be identified.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the image recognition method according to any one of claims 1 to 7.

Images