Image recognition method and system based on artificial intelligence

Abstract

The invention discloses an image recognition method and system based on artificial intelligence, and belongs to the technical field of artificial intelligence and image recognition. The recognition method comprises the steps that the convolutional neural network is trained; the feature extraction unit extracts deep feature information of an image to be recognized; the trunk pooling layer performs global pooling operation on each layer of a deep feature map; and feature vectors sequentially pass through the nonlinear mapping layer and the classifier. In the feature scale set extraction module, not only is multi-scale convolution adopted for feature extraction, but also input feature maps are fused for multiple times through matrix addition and element corresponding products, and input image information is further utilized from more angles, so that the extraction effect of some hidden feature information is improved, and the extraction efficiency is improved. And a multi-branch dimension reduction structure is adopted, so that the problem of information loss caused by conventional pooling operation is solved, and the whole convolutional neural network has a good classification effect on low-quality images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition and artificial intelligence, and in particular, relates to an image recognition method and system based on artificial intelligence. Background Art

[0002] Image classification and recognition technology is one of the most widely used directions in the field of computer vision. Its subsets include face recognition, OCR recognition, license plate recognition, iris recognition, etc. The general process of image recognition can be simply summarized into the following three steps: image processing, feature extraction, and classification and recognition. Since deep learning has been applied to image recognition tasks, a large number of network models have been developed, among which the leading methods are mostly based on convolutional neural networks. When designing these models, they all assume that the input image quality can meet the expected requirements, and they are not optimized for low-quality image input in actual application scenarios, resulting in unsatisfactory results when applied in actual scenarios. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides an image recognition method and system based on artificial intelligence to better recognize various images in actual scenes.

[0004] In order to achieve the above object, the solution adopted by the present invention is: an image recognition method based on artificial intelligence, applied to a recognition device, the method comprising:

[0005] M100, constructing a convolutional neural network, and training the convolutional neural network using labeled training images;

[0006] The convolutional neural network comprises a feature extraction unit and a classification recognition unit, wherein the feature extraction unit is arranged at the front of the convolutional neural network, and the feature extraction unit comprises a plurality of feature scale set refining modules arranged in series; the classification recognition unit is arranged at the rear of the convolutional neural network, and the classification recognition unit comprises a trunk pooling layer, a nonlinear mapping layer and a classifier arranged in series;

[0007] The feature extraction module performs feature extraction on the feature map as shown in the following mathematical model:

[0008] Fm1=A1f(λ1ω3(FP))

[0009] Fm2=A2f(λ1ω5(FP))

[0010] Fv1=Th1(FP+Fm1+Fm2)

[0011]

[0012] Fv3=λ2ω1(cat(Fm1,Fm2,Fv1,Fv2))

[0013] Fm3=A3f(λ2d3(Fm1))

[0014] Fm4=A4f(λ2d5(Fm2))

[0015] FO=λ3ω1(cat(Fm3,Fm4,Pc(Fv3)))

[0016] Among them, FP feature map represents the input of the feature set extraction module, FO represents the feature map output by the feature set extraction module after feature extraction operation, λ1ω3, λ1ω5, λ2ω1, λ2d3, λ2d5 and λ3ω1 all represent convolution operations on feature maps, the convolution kernel size used in the λ1ω3 convolution operation is 3*3, and the sliding step size is 1, the convolution kernel size used in the λ1ω5 convolution operation is 5*5, and the sliding step size is 1, the convolution kernel size used in the λ2ω1 convolution operation is 1*1, and the sliding step size is is 1, the convolution kernel size used in the λ2d3 convolution operation is 3*3, and the sliding step is 2, the convolution kernel size used in the λ2d5 convolution operation is 5*5, and the sliding step is 2, the convolution kernel size used in the λ3ω1 convolution operation is 1*1, and the sliding step is 1, Pc represents the maximum pooling operation on the feature map, the pooling window size used in the Pc pooling operation is 2*2, and the sliding step is 2, A1f, A2f, A3f and A4f all represent the activation function ReLU, Th1 and Th2 all represent the activation function Tanh, Indicates the element-by-element product operation, and cat indicates the concatenation operation on the feature map;

[0017] M200, obtaining an image to be recognized, and obtaining the convolutional neural network trained in step M100;

[0018] M300, input the image to be identified into the convolutional neural network, and as the image information is transmitted along the trunk of the feature extraction unit, each of the feature set extraction modules performs feature extraction operations on the image information in turn. After the deep feature information of the image to be identified is extracted by the feature extraction unit, the tail output of the feature extraction unit obtains a deep feature map (the deep feature map is also the feature map output by the tail of the last feature set extraction module);

[0019] M400, inputting the deep feature map obtained in step M300 into the backbone pooling layer, using the backbone pooling layer to perform a global pooling operation on each layer of the deep feature map, and generating a feature vector corresponding to the deep feature map;

[0020] M500, the feature vector passes through the nonlinear mapping layer and the classifier in sequence, and then the classifier outputs the classification result to complete the image recognition.

[0021] Furthermore, in step M100, when training the convolutional neural network, a cross entropy function is used as a loss function.

[0022] Furthermore, the nonlinear mapping layer is a fully connected layer.

[0023] Furthermore, a residual connection is provided in the feature cluster extraction module, and the FP feature map is added and fused with the Fv3 feature map through the residual connection.

[0024] Furthermore, a cluster attention mechanism is provided in the feature cluster extraction module, and the cluster attention mechanism is used to modulate the feature map FO output by the feature cluster extraction module.

[0025] Furthermore, the mathematical model of the clustered attention mechanism is:

[0026] GF1=GPα(Fm3)+GPβ(Fm3)

[0027] GF2=GPα(Fm4)+GPβ(Fm4)

[0028] GF3=GPα(Fm5)+GPβ(Fm5)

[0029]

[0030] GU2=GPθ(λAω3(cat(Fm1,Fm2,Fm3)))

[0031] GFU=δ2(GU1+GU2)

[0032] Among them, Fm5 represents the feature map generated after the Pc pooling operation, feature maps Fm3, feature maps Fm4 and feature maps Fm5 are the inputs of the scaled attention mechanism, GPα represents the global maximum pooling operation on the feature map in the channel direction, GPβ represents the global average pooling operation on the feature map in the channel direction, and GPθ represents the global variance pooling operation on the feature map in the channel direction. For a feature map of size C*W*H (channel*width*height), after performing a global pooling operation on it in the channel direction, the output is a matrix of size 1*W*H. δ1 and δ2 both represent the nonlinear activation function sigmoid, Represents the element-by-element multiplication operation, λAω3 represents the convolution operation with a convolution kernel size of 3*3 and a sliding step size of 1, cat represents the splicing operation on the feature map, and GFU represents the spatial modulation map generated and output by the clustered attention mechanism.

[0033] The present invention also provides an artificial intelligence-based image recognition system, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the artificial intelligence-based image recognition method as described above by loading the computer program.

[0034] The beneficial effects of the present invention are:

[0035] (1) Studies have shown that the use of multi-scale convolution kernels in parallel can improve the feature extraction effect. However, for some low-resolution input images, the existing multi-scale feature extraction structure still does not fully utilize the input information. In the feature cluster refining module of the present invention, not only a multi-scale convolution layer (λ1ω3 and λ1ω5) is used to extract features from the feature map FP, but also the feature maps FP, Fm1 and Fm2 are fused multiple times through matrix addition and element-by-element product, so as to further utilize the input image information from more angles, thereby improving the extraction effect of some hidden feature information. Compared with the existing feature extraction module, the feature cluster refining module has a more comprehensive feature extraction capability for low-quality images, and provides more comprehensive and detailed deep feature information for subsequent image recognition operations, which is of great value for improving the recognition capability of low-quality images.

[0036] (2) In the feature cluster extraction module, λ2d3 convolution, λ2d5 convolution and PC pooling operations are used to reduce the dimension (length and width) of different feature maps. This multi-branch pooling structure can not only improve the reuse of feature maps Fm1 and Fm2, but also make up for the information loss problem caused by conventional pooling operations. Even if the network is deep, the feature information of the front end can still be reliably transmitted to the back end. The gradient disappearance phenomenon is not likely to occur during model training, and training is easier. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the convolutional neural network structure of the present invention;

[0038] Figure 2 for Figure 1 The structural diagram of the feature cluster extraction module in the convolutional neural network shown in FIG.

[0039] Figure 3 This is a schematic diagram of the structure of the feature cluster extraction module with cluster attention mechanism;

[0040] Figure 4 Schematic diagram of the structure of the clustered attention mechanism of the present invention;

[0041] In the attached figure:

[0042] 1-image to be recognized, 2-feature extraction unit, 3-feature cluster refinement module, 31-residual connection, 32-residual module, 33-cluster attention mechanism, 34-first balanced convolution layer, 35-first balanced activation function, 36-second balanced convolution layer, 37-second balanced activation function, 4-classification recognition unit, 41-trunk pooling layer, 42-non-linear mapping layer, 43-classifier, 5-classification result. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings:

[0044] Embodiment 1:

[0045] The image recognition method provided by the present invention is a universal image classification and recognition method, and its recognition objects can be faces, OCRs, license plates, etc. By adjusting the images in the training set and training different models, specific image recognition tasks can be completed.

[0046] The following is an exemplary description of the internal data processing flow of the convolutional neural network of the present invention. Figure 1 As shown, the convolutional neural network includes a feature extraction unit 2 and a classification recognition unit 4. The feature extraction unit 2 is arranged at the front of the convolutional neural network. The feature extraction unit 2 includes five feature set refining modules 3 arranged in series. In order to match the complexity of the entire network (non-linear fitting ability) with the difficulty of the actual recognition task, a number of residual modules 32 connected in series can be set in each feature set refining module 3 according to actual needs. The residual module 32 is located at the front end of the feature set refining module 3; if the recognition task is relatively simple, it is also possible to choose not to set the residual module 32 to avoid unnecessary calculations. The internal structure of the feature set refining module 3 of this embodiment is shown in FIG. Figure 2 As shown, a residual module 32 is set at its front end, and the residual module 32 includes a first balanced convolution layer 34, a first balanced activation function 35, a second balanced convolution layer 36 and a second balanced activation function 37 connected in sequence. The first balanced activation function 35 and the second balanced activation function 37 are both ReLU, and the first balanced convolution layer 34 and the second balanced convolution layer 36 are both ordinary convolutions with a convolution kernel of 3*3 and a step size of 1. The feature map at the front end of the residual module 32 is added and fused with the feature map at the back end of the residual module 32 through skip connection.

[0047] For the first feature cluster refining module 3 in the network, its input is a 3-channel image 1 to be identified. If the residual module 32 is not set therein, the λ1ω3 convolution and λ1ω5 convolution play a role in increasing the number of channels of the feature map. The input image channels of the λ1ω3 convolution and λ1ω5 convolution are 3, and the number of channels of the output Fm1 feature map and Fm2 feature map are both 64. In the feature cluster refining module 3 of this embodiment, a residual module 32 is set, and the first balanced convolution layer 34 of the residual module 32 (of the first feature cluster refining module 3) plays a role in increasing the number of channels. Its input is a 3-channel image 1 to be identified, and its output is a feature map of 64 channels. For the following four cluster refining modules 3, the size of the feature map will not be changed before and after the operation of the first balanced convolution layer 34 inside. For all five feature cluster refining modules 3, the size of the feature map will not be changed before and after the operation of the second balanced convolution layer 36, λ1ω3 convolution and λ1ω5 convolution inside.

[0048] Within the same feature cluster extraction module 3, the sizes of the Fm1 feature map, the Fm2 feature map, the Fv1 feature map, the Fv2 feature map, and the Fv3 feature map are exactly the same. By filling the feature map with appropriate padding values, the length and width of the feature map generated after the λ2d3 convolution operation, the λ2d5 convolution operation, and the Pc pooling operation are both half of those before the operation, and the number of feature map channels remains unchanged. In order to further enhance the reuse of feature information, a residual connection 31 is set in the feature cluster extraction module 3. The feature map output by the residual module 32 is added and fused with the Fv3 feature map through the residual connection 31, and the fused feature map is then input into the Pc pooling operation. At the end of the feature set extraction module 3, after the Fm3 feature map, the Fm4 feature map and the Fm5 feature map (output of the Pc pooling operation) are concatenated, the λ3ω1 convolution is used to reduce the number of feature map channels, so that the number of channels of the feature map output by the feature set extraction module 3 is two-thirds of the sum of the number of channels of the Fm3 feature map, the Fm4 feature map and the Fm5 feature map (in the same feature set extraction module 3).

[0049] The classification recognition unit 4 is arranged at the rear of the network, and the classification recognition unit 4 includes a backbone pooling layer 41, a nonlinear mapping layer 42 and a classifier 43 arranged in series. In this embodiment, the backbone pooling layer 41 is a global average pooling layer, and the backbone pooling layer 41 is used to perform a global average pooling operation on each layer of the feature map in the spatial direction, and then generate a feature vector with a length of 2048. The nonlinear mapping layer 42 is implemented by a fully connected layer, and its input nodes are 2048, and the output nodes are set according to the number of categories to be classified. The classifier 43 is implemented by a softmax classifier, and the output of the classifier 43 obtains the classification result 5.

[0050] In order to demonstrate the effect of the image recognition method proposed in the present invention, this embodiment is trained and tested with the help of the public data set SVHN. First, all images in the training set and test set of SVHN are downsampled by 2 times and 4 times respectively to generate corresponding low-resolution images, and then the generated low-resolution images are merged with the original images in SVHN to obtain an extended training set and an extended test set, so that the number of images in the extended training set is three times the number of images in the original training set of SVHN, and the number of images in the extended test set is also three times the number of images in the original test set of SVHN. In order to facilitate the image input network for processing, the size of all images in the extended training set and the extended test set are uniformly adjusted to 224*224 (in pixels).

[0051] The convolutional neural network and Densenet-121 proposed in the present invention were trained using the above-mentioned expanded training set, and then the trained convolutional neural network (of the present invention) and Densenet-121 were tested on the above-mentioned expanded test set. The test results show that the classification and recognition error rate of the convolutional neural network proposed in the present invention is 12.56%, and the recognition error rate of Densenet-121 is 18.44%. It can be seen that the classification and recognition accuracy of the convolutional neural network proposed in the present invention is significantly higher than that of Densenet-121.

[0052] Embodiment 2:

[0053] This embodiment sets up a cluster attention mechanism 33 on the basis of the feature cluster extraction module 3 in embodiment 1. The internal structure of the feature cluster extraction module 3 in embodiment 2 is as follows: Figure 3 As shown in Figure 3, the specific internal structure of the clustered attention mechanism 33 is as follows Figure 4 shown.

[0054] The scaled attention mechanism 33 takes three feature maps as input at the same time. Inside it, on the one hand, conventional global average pooling and global maximum pooling are used to perform pooling operations on the feature maps in the channel direction to generate six two-dimensional matrices, and then the six matrices are fused into one matrix by matrix addition and element-by-element product, and the GU1 attention map is obtained by activation with δ1. On the other hand, the three feature maps input to the scaled attention mechanism 33 are concatenated, and then convolution operation is performed with λAω3. After λAω3 convolution, the number of channels of the output feature map is one-third of the input λAω3 convolution layer. Next, the feature map is subjected to global variance pooling in the channel direction, and the generated two-dimensional matrix is ​​added to the GUI attention map, and the final spatial modulation map GFU is obtained after δ2 activation.

[0055] Low-quality images greatly reduce the distinction between different images due to the lack of a considerable amount of detail information. The present invention sets a cluster attention mechanism 33 in the feature cluster extraction module 3, so that the model can focus on utilizing and mining the feature information of important areas and reduce the interference of unimportant areas. Since the feature maps (Fm3, Fm4 and Fm5) input to the cluster attention mechanism 33 come from different ways of dimensionality reduction processing (λ2d3 convolution, λ2d5 convolution and PC pooling), different dimensionality reduction methods have diversity and certain complementarity, so that the cluster attention mechanism 33 can receive relatively comprehensive feature information and enhance the accuracy of the spatial modulation map to the feature map FO modulation. Moreover, based on actual work experience, the inventor creatively designed the internal structure of the clustered attention mechanism 33, and adopted distributed global average pooling and global maximum pooling to learn the relative relationship of feature information of different spatial positions in feature maps Fm3, Fm4 and Fm5 in more detail, thereby generating the GU1 attention map; on the other hand, the clustered attention mechanism 33 uses λAω3 convolution to integrate the feature information in feature maps Fm3, Fm4 and Fm5, and uses global variance pooling to learn the high-frequency information features of different spatial positions in the integrated feature maps. The vector generated by the global variance pooling is added to GU1 to achieve fine-tuning of GU1, which to a certain extent compensates for the excessive enhancement / suppression caused by repeated learning of some of the same information in feature maps Fm3, Fm4 and Fm5 by distributed global average pooling and global maximum pooling, thereby improving the modulation effect of the clustered attention mechanism 33.

[0056] The convolutional neural network after adding the scaled attention mechanism 33 was trained using the same expanded training set as in Example 1, and was also tested on the above-mentioned expanded test set. The results showed that after adding the scaled attention mechanism 33, the recognition error rate of the convolutional neural network proposed in the present invention was 6.82% on the expanded test set, and the recognition error rate was significantly reduced.

[0057] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. An image recognition method based on artificial intelligence, characterized by: Applied to identification equipment, the method comprises: M100, constructing a convolutional neural network, and training the convolutional neural network using labeled training images; The convolutional neural network comprises a feature extraction unit and a classification recognition unit, wherein the feature extraction unit is arranged at the front of the convolutional neural network, and the feature extraction unit comprises a plurality of feature scale set refining modules arranged in series; the classification recognition unit is arranged at the rear of the convolutional neural network, and the classification recognition unit comprises a trunk pooling layer, a nonlinear mapping layer and a classifier arranged in series; The feature extraction module performs feature extraction on the feature map as shown in the following mathematical model: Fm1=A1f(λ1ω3(FP)) Fm2=A2f(λ1ω5(FP)) Fv1=Th1(FP+Fm1+Fm2) Fv3=λ2ω1(cat(Fm1,Fm2,Fv1,Fv2)) Fm3=A3f(λ2d3(Fm1)) Fm4=A4f(λ2d5(Fm2)) FO=λ3ω1(cat(Fm3,Fm4,Pc(Fv3))) Among them, FP feature map represents the input of the feature set extraction module, FO represents the feature map output by the feature set extraction module after feature extraction operation, λ1ω3, λ1ω5, λ2ω1, λ2d3, λ2d5 and λ3ω1 all represent convolution operations on feature maps, the convolution kernel size used in the λ1ω3 convolution operation is 3*3, and the sliding step size is 1, the convolution kernel size used in the λ1ω5 convolution operation is 5*5, and the sliding step size is 1, the convolution kernel size used in the λ2ω1 convolution operation is 1*1, and the sliding step size is is 1, the convolution kernel size used in the λ2d3 convolution operation is 3*3, and the sliding step is 2, the convolution kernel size used in the λ2d5 convolution operation is 5*5, and the sliding step is 2, the convolution kernel size used in the λ3ω1 convolution operation is 1*1, and the sliding step is 1, Pc represents the maximum pooling operation on the feature map, the pooling window size used in the Pc pooling operation is 2*2, and the sliding step is 2, A1f, A2f, A3f and A4f all represent the activation function ReLU, Th1 and Th2 all represent the activation function Tanh, Indicates the element-by-element product operation, and cat indicates the concatenation operation on the feature map; M200, obtaining an image to be recognized, and obtaining the convolutional neural network trained in step M100; M300, inputting the image to be identified into the convolutional neural network, using the feature extraction unit to extract deep feature information of the image to be identified, and then the tail of the feature extraction unit outputs a deep feature map; M400, inputting the deep feature map obtained in step M300 into the backbone pooling layer, using the backbone pooling layer to perform a global pooling operation on each layer of the deep feature map, and generating a feature vector corresponding to the deep feature map; M500, the feature vector passes through the nonlinear mapping layer and the classifier in sequence, and then the classifier outputs the classification result to complete the image recognition.

2. The image recognition method based on artificial intelligence according to claim 1, characterized in that: In step M100, when training the convolutional neural network, the cross entropy function is used as the loss function.

3. The image recognition method based on artificial intelligence according to claim 1, characterized in that: The nonlinear mapping layer is a fully connected layer.

4. The image recognition method based on artificial intelligence according to claim 1, characterized in that: The feature cluster extraction module is provided with a residual connection, and the FP feature map is added and fused with the Fv3 feature map through the residual connection.

5. The image recognition method based on artificial intelligence according to claim 1, characterized in that: A cluster attention mechanism is provided in the feature cluster extraction module, and the cluster attention mechanism is used to modulate the feature map FO output by the feature cluster extraction module.

6. The method for image recognition based on artificial intelligence according to claim 5, characterized in that: The mathematical model of the clustered attention mechanism is: GF1=GPα(Fm3)+GPβ(Fm3) GF2=GPα(Fm4)+GPβ(Fm4) GF3=GPα(Fm5)+GPβ(Fm5) GU2=GPθ(λAω3(cat(Fm1,Fm2,Fm3))) GFU=δ2(GU1+GU2) Among them, Fm5 represents the feature map generated after the Pc pooling operation, the feature maps Fm3, Fm4 and Fm5 are the inputs of the scaled attention mechanism, GPα represents the global maximum pooling operation on the feature map in the channel direction, GPβ represents the global average pooling operation on the feature map in the channel direction, GPθ represents the global variance pooling operation on the feature map in the channel direction, δ1 and δ2 both represent the nonlinear activation function sigmoid, Represents the element-by-element multiplication operation, λAω3 represents the convolution operation with a convolution kernel size of 3*3 and a sliding step size of 1, cat represents the splicing operation on the feature map, and GFU represents the spatial modulation map generated and output by the clustered attention mechanism.

7. An image recognition system based on artificial intelligence, comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The processor is used to execute the artificial intelligence-based image recognition method as described in any one of claims 1 to 6 by loading the computer program.

Images