Image recognition method and device, storage medium and electronic device

Abstract

Embodiments of the present application provide an image recognition method, the method comprising: constructing a neural network model, and training the neural network model through pre-acquired sample image data to obtain an image recognition model, the neural network model being composed of an input layer, a plurality of image feature capturing modules, a full connection layer, and an output layer in series; wherein the image feature capturing module is composed of a first channel integration module, a first convolution module, a feature map splicing module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer in series; acquiring an image to be recognized, inputting the image to be recognized into the image recognition model; and outputting a classification result of the image to be recognized recognized by the image recognition model. The technical solution of the embodiments of the present application can improve the applicability of image recognition based on a neural network model.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and more specifically, to an image recognition method, apparatus, storage medium, and electronic device. Background Technology

[0002] Currently, neural network models are widely used in image recognition due to their high accuracy and fast processing speed. However, these models often require excessive parameters and computation, necessitating high memory and computing power from the hardware platform. This makes it difficult to implement image recognition on mobile devices with limited hardware resources and low power consumption. Therefore, improving the applicability of neural network models for image recognition is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] The embodiments of this application provide an image recognition method, apparatus, storage medium, and electronic device, which can reduce the number of parameters and computational load of the image recognition model to a certain extent, thereby improving the applicability of image recognition based on the image recognition model.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to a first aspect of the embodiments of this application, an image recognition method is provided, the method comprising:

[0006] A neural network model is constructed and trained using pre-collected sample image data to obtain an image recognition model. The neural network model consists of an input layer, multiple image feature capture modules, a fully connected layer, and an output layer arranged in sequence. The image feature capture module consists of a first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer arranged in sequence.

[0007] The image to be identified is obtained, and the image to be identified is input into the image recognition model so that the image recognition model can classify the image to be identified; the classification result of the image to be identified by the image recognition model is output.

[0008] In some embodiments of this application, based on the foregoing scheme, the first channel integration module is used to reduce the number of original output channels of the original feature map input to the image feature capture module to a preset number of output channels.

[0009] In some embodiments of this application, based on the aforementioned scheme, the feature map stitching module consists of a data processing layer and a stitching layer connected in sequence; the data processing layer is used to process the first feature map input to the feature map stitching module to obtain a second feature map; the stitching layer is used to stitch the second feature map with the first feature map to expand the number of output channels of the first feature map to the original number of output channels.

[0010] In some embodiments of this application, based on the foregoing scheme, the second channel integration module is composed of a channel grouping module, a data exchange module, and a dimension compression module connected in sequence; the channel grouping module is used to divide the feature map of the initial dimension input to the second channel integration module into multiple feature map groups, and expand the initial dimension to a preset dimension; the data exchange module is used to exchange image data between different feature map groups in the multiple feature map groups; the dimension compression module is used to compress the preset dimension to the initial dimension, so as to realize the image data fusion between the different feature map groups.

[0011] In some embodiments of this application, based on the aforementioned scheme, the spatial attention module is composed of a computation module, a splicing module, a convolution module, and an activation module connected in sequence; the computation module is used to perform absolute operations on the feature map input to the spatial attention module to obtain an absolute feature map, and to perform data processing on the absolute feature map to obtain a mean feature map and a maximum feature map with one output channel respectively;

[0012] The concatenation module is used to concatenate the mean feature map and the maximum feature map to obtain an intermediate feature map with two output channels; the convolution module is a 1x1 convolution used to perform dimensionality reduction processing on the intermediate feature map to obtain a weight tensor with one output channel; the activation module is used to activate the weight tensor to obtain a target weight tensor, and output a target feature map based on the target weight tensor and the absolute feature map.

[0013] In some embodiments of this application, based on the foregoing scheme, the first activation layer consists of a first grouped convolutional layer, a first operational layer, a batch normalization layer, a second channel integration module, and a second operational layer, sequentially arranged in sequence. The first operational layer is used to perform absolute operations on the feature map input to the first operational layer to obtain an absolute feature map. The batch normalization layer is used to normalize the absolute feature map to obtain a normalized feature map. The second operational layer is used to determine a target feature value based on the magnitude of a first feature value in the absolute feature map and a second feature value in the normalized feature map, and to construct a target feature map from the target feature value, wherein the first feature value and the second feature value correspond one-to-one.

[0014] In some embodiments of this application, based on the foregoing scheme, the second activation layer includes an activation function, which is a linear rectification function.

[0015] According to a second aspect of the embodiments of this application, an image recognition apparatus is provided. The apparatus includes: a construction unit, configured to construct a neural network model and train the neural network model using pre-collected sample image data to obtain an image recognition model. The neural network model consists of an input layer, multiple image feature capture modules, a fully connected layer, and an output layer arranged sequentially. The image feature capture module consists of a first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer arranged sequentially. An acquisition unit is configured to acquire an image to be recognized and input the image to be recognized into the image recognition model for classification by the image recognition model. An output unit is configured to output the classification result of the image to be recognized identified by the image recognition model.

[0016] According to a third aspect of the embodiments of this application, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores at least one piece of program code, the at least one piece of program code being loaded and executed by a processor to perform the operations performed by the method described in any of the first aspects above.

[0017] According to a fourth aspect of the present application, an electronic device is provided, including one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, the at least one piece of program code being loaded and executed by the one or more processors to perform the operation as described in any of the first aspects above.

[0018] The technical solution proposed in this application constructs a neural network model and trains it using pre-collected sample image data to obtain an image recognition model. The neural network model consists of a sequentially arranged input layer, multiple image feature capture modules, a fully connected layer, and an output layer. The image feature capture module comprises, sequentially, a first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer. The image to be recognized is then input into the image recognition model to obtain the classification result. Because multiple image feature capture modules are designed in the image recognition model, the number of parameters and computational load of the image recognition model can be greatly reduced. This allows the image recognition model provided in this application to run on mobile devices with limited hardware resources and low power consumption to achieve image recognition, thus improving the applicability of image recognition based on neural network models.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0021] Figure 1 A flowchart of an image recognition method according to an embodiment of this application is shown;

[0022] Figure 2 A schematic diagram of the structure of a neural network model according to an embodiment of this application is shown;

[0023] Figure 3 A schematic diagram of the structure of a second channel integration module according to an embodiment of this application is shown;

[0024] Figure 4 A schematic diagram of the structure of a first activation layer according to an embodiment of this application is shown;

[0025] Figure 5 A block diagram of an image recognition apparatus according to an embodiment of this application is shown;

[0026] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0027] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0028] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0029] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0031] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such uses of these terms can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described.

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0034] The following detailed description of some embodiments of this application will be provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0035] It should be noted that the technical solutions of this application are applicable to scenarios including but not limited to image type recognition, image segmentation, face recognition, and other visual applications.

[0036] See Figure 1 The diagram shows a flowchart of an image recognition method according to an embodiment of the present application, which specifically includes steps 110 to 130.

[0037] Step 110: Construct a neural network model and train the neural network model using pre-collected sample image data to obtain an image recognition model. The neural network model consists of an input layer, multiple image feature capture modules, a fully connected layer, and an output layer arranged in sequence. The image feature capture module consists of a first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer arranged in sequence.

[0038] It should be noted that different types of image data can be collected as sample data according to the actual application scenario, or existing image datasets can be used as sample datasets. For example, the CIFAR10 image dataset is used as a sample dataset to train the constructed neural network model. Specifically, the source of the sample image data is not limited in this application.

[0039] It should also be noted that the constructed neural network model can be built based on existing network structures such as AlexNet and VGG11. That is, it can be improved upon the AlexNet and VGG11 network structures, incorporating the image feature capture module designed in this application to recognize images. Alternatively, it can be constructed according to... Figure 2 The neural network model shown is designed to recognize images.

[0040] See Figure 2The diagram illustrates the structure of a neural network model according to an embodiment of this application. The diagram shows four image feature capture modules, each with the same structural design. This application does not limit the specific number of image feature capture modules or the specific parameter settings within each module.

[0041] In some implementations, the first convolutional module can be a 3x3 standard convolutional unit, and the second convolutional module can be a 1x1 standard convolutional unit.

[0042] In some implementations, the first channel integration module in the image feature capture module is used to reduce the number of original output channels of the original feature map input to the image feature capture module to a preset number of output channels.

[0043] Specifically, in one embodiment, the original number of output channels of the original feature map input to the image feature capture module can be multiplied by a preset channel scaling factor to obtain a preset number of output channels, wherein the channel scaling factor is greater than 0 and less than 1. Preferably, the channel scaling factor can be determined to be 0.5.

[0044] It is understandable that the first channel integration module actually reduces the output channels of the feature map input to the first convolution module, thereby reducing the computational load of the first convolution module to a certain extent and making the image recognition model lightweight.

[0045] In some implementations, the feature map stitching module in the image feature capture module consists of a data processing layer and a stitching layer connected in sequence; the data processing layer is used to process the first feature map input to the feature map stitching module to obtain a second feature map; the stitching layer is used to stitch the second feature map with the first feature map to expand the number of output channels of the first feature map to the original number of output channels.

[0046] It should be noted that the data processing layer can be used to invert the feature values ​​in the first feature map to obtain the second feature map (i.e., the negative feature map), or it can be used to process the data to obtain a suitable second feature map according to the specific application scenario. This application does not limit this.

[0047] For example, taking the input feature map as a 2D tensor (matrix), suppose the first feature map is:

[0048]

[0049] After processing by the data processing layer, the second feature map can be obtained as follows:

[0050]

[0051] It should also be noted that the stitching layer can stitch the second feature map according to the depth dimension of the first feature map, or it can stitch the second feature map according to the length and width dimensions of the first feature map. If it is stitched according to the length and width dimensions of the first feature map, the resulting feature map needs to be adjusted in terms of dimensions to adapt the feature map obtained by stitching the length and width dimensions to the feature map that is stitched according to the depth dimension.

[0052] For example, if the above-mentioned first feature map X and second feature map X - If the concatenation is performed along the depth dimension, the resulting feature map will be:

[0053]

[0054] If the first feature map and the second feature map are concatenated according to the length and width dimensions of the first feature map, the resulting feature map is:

[0055]

[0056] At this point, the obtained stitched image Y0 needs to be dimensionally adjusted to adapt the feature map Y0 obtained by stitching the length and width dimensions to a feature map that is adapted to the depth dimension. In one implementation, the feature map Y0 can be adjusted to the following feature map Y2:

[0057]

[0058] It is understood that by readjusting the number of output channels of the feature map in the input feature map stitching module to the original number of output channels, this embodiment can maintain the shape of the neural network model and improve the accuracy of the neural network model in recognizing image data.

[0059] In some implementations, the structure of the second channel integration module in the image feature capture module is as follows: Figure 3 As shown, it consists of a channel grouping module, a data exchange module, and a dimension compression module connected in sequence. The channel grouping module is used to divide the initial dimension feature map input to the second channel integration module into multiple feature map groups and expand the initial dimension to a preset dimension. The data exchange module is used to exchange image data between different feature map groups in the multiple feature map groups. The dimension compression module is used to compress the preset dimension to the initial dimension to achieve image data fusion between different feature map groups.

[0060] See Figure 3This diagram illustrates the structure of a second channel integration module according to an embodiment of this application. In one implementation, if the convolution connected to the second channel integration module is a grouped convolution, the input feature map can be divided into feature map groups based on the number of convolution groups in the grouped convolution. For example, assuming the convolution connected to the second channel integration module is a grouped convolution (i.e., the number of convolution groups g > 1), firstly, the output channels of the input feature map with the initial dimension are randomly divided into g groups, and the initial dimension is expanded to a preset dimension; then, data exchange is performed between different feature groups; finally, the preset dimension is compressed to the initial dimension to complete the image data fusion between different feature map groups, and also to achieve the decoupling of channels and pixel positions in the input feature map.

[0061] In another implementation, the convolution connected to the second channel is a non-grouped convolution (i.e., the number of convolution groups g = 1). Before implementing channel grouping, the first channel integration module can be used to calculate how many feature map groups can be divided into, and how many output channels are contained in each feature map group; then the image data information is fused in the manner of convolution group number g > 1.

[0062] For example, suppose the initial dimension of the feature map X input to the second channel integration module is a four-dimensional tensor (32, 32, 3, 3), where the first dimension represents the number of input channels (32), the second dimension represents the number of output channels (32), and the third and fourth dimensions represent the size of the convolution kernel (3x3). Using the first channel integration module (assuming a channel scaling factor of 0.25), the output channel number is multiplied by the channel scaling factor (32 x 0.25 = 4), resulting in four groups of eight output channels each. This expands the initial dimension to a five-dimensional tensor (32, 8, 4, 3, 3). The image data between different feature map groups is exchanged through the data exchange module, changing the result to (32, 4, 8, 3, 3). Finally, the five-dimensional tensor is compressed back to a four-dimensional tensor (32, 32, 3, 3) through the dimension compression module to obtain a new feature map, achieving image data fusion and decoupling of channels and pixel positions.

[0063] In some implementations, the spatial attention module in the image feature capture module is composed of a computation module, a stitching module, a convolution module, and an activation module connected in sequence. The computation module performs absolute operations on the feature map input to the spatial attention module to obtain an absolute feature map, and performs data processing on the absolute feature map to obtain a mean feature map and a maximum feature map with one output channel. The stitching module stitches the mean feature map and the maximum feature map to obtain an intermediate feature map with two output channels. The convolution module is a 1x1 convolution used to perform dimensionality reduction processing on the intermediate feature map to obtain a weight tensor with one output channel. The activation module activates the weight tensor to obtain a target weight tensor, and outputs a target feature map based on the target weight tensor and the absolute feature map.

[0064] It should be noted that the absolute value of the feature map of the input space attention module is first taken to obtain a non-negative absolute feature map. Then, the mean and maximum values ​​of the absolute feature map are taken in a specified dimension to obtain a mean feature map and a maximum feature map with 1 channel. Then, the mean feature map and the maximum feature map are concatenated by channel to obtain an intermediate feature map with 2 channels. Finally, a 1X1 convolution is used to reduce the dimensionality of the intermediate feature map, realizing the fusion of image data information across channels of the feature map, and outputting a weight tensor with 1 channel.

[0065] Finally, the input is processed by the activation module, which includes an activation layer and a multiplier. The activation layer can use activation functions such as Softmax or Sigmoid to limit the values ​​of the weight tensor to within the range (0,1) as the target weight tensor. The multiplier multiplies the target weight tensor with the absolute feature map to obtain the target feature map, which is then output. In this embodiment, the inclusion of absolute value operations ensures that all values ​​in the output target feature map are non-negative.

[0066] For example, suppose the input feature map X of the input spatial attention module has a dimension of (32,32,3,3). Taking its absolute value yields an absolute feature map Y with the same dimension. Taking the mean and maximum value of Y according to the output channel dimension, we obtain a mean feature map with a dimension of (32,1,3,3) and a maximum feature map with a dimension of (32,1,3,3). After concatenation, we obtain an intermediate feature map with a dimension of (32,2,3,3). Using a 1X1 convolution, we reduce the dimension of the intermediate feature map to obtain a weight tensor Z with a dimension of (32,1,3,3). Finally, the activation layer limits the value of Z to (0,1), and the multiplier multiplies the absolute feature map Y and Z to obtain a target feature map with the same dimension as the feature map X.

[0067] In some implementations, the structure of the first activation layer in the image feature capture module is as follows: Figure 4As shown, it consists of a first grouped convolutional layer, a first operational layer, a batch normalization layer, a second channel integration module, and a second operational layer, which are sequentially connected. The first operational layer is used to perform absolute operations on the feature map input to the first operational layer to obtain an absolute feature map. The batch normalization layer is used to normalize the absolute feature map to obtain a normalized feature map. The second operational layer is used to determine a target feature value based on the magnitude of a first feature value in the absolute feature map and a second feature value in the normalized feature map, and to construct a target feature map from the target feature value, wherein the first feature value and the second feature value correspond one-to-one.

[0068] In some implementations, the first group convolution can be a 1x1 group convolution. The second operational layer can take the maximum value between the first feature value in the absolute feature map and the second feature value in the normalized feature map as the target feature value.

[0069] For example, the first operational layer takes the absolute value of the feature map output by the 1X1 grouped convolution to obtain the absolute feature map X. After passing through the batch normalization layer, a non-negative normalized feature map X1 is obtained. The second operational layer compares the eigenvalues ​​in the absolute feature map X and the normalized feature map Z, and takes the maximum value of the two to form a new target feature map Y. That is, Y(a,b,c,d)=max{X(a,b,c,d),X1(a,b,c,d)}.

[0070] In some implementations, the second activation layer in the image feature capture module includes an activation function, which is a linear rectified function.

[0071] See also Figure 1 Step 120: Obtain the image to be identified, and input the image to be identified into the image recognition model so that the image recognition model can classify the image to be identified.

[0072] See also Figure 1 Step 130: Output the classification result of the image to be recognized by the image recognition model.

[0073] To better illustrate the beneficial effects of the technical solution proposed in this application, Tables 1 to 5 will be used to describe the design of different neural network models incorporating the image feature capture module proposed in this application, the training of the models using the CIFAR10 image dataset, and the comparison of their parameter count, computational cost, and accuracy.

[0074] The explanations for Tables 1, 2, and 3 are as follows:

[0075] Table 1 shows the parameter settings for the AlexNet neural network model and the Alex_Like neural network model, which replaces the Conv3X3+ReLU module in the AlexNet neural network model with the image feature capture module proposed in this application. In the table, parameters 1-5 represent, from left to right, the number of input channels, the number of output channels, the kernel size, the stride size, the padding value, and the number of groups, respectively; parameters 6-8 represent, from left to right, the number of input channels and the number of output channels, respectively; and neither the AlexNet nor the Alex_Like neural network model uses bias (bias = False) in their structural units.

[0076] Table 2 shows the parameter settings for the VGG11 neural network model and the parameter settings for the VGG11_Like neural network model, which replaces Conv in the VGG11 neural network model with the image feature capture module proposed in this application. In the Conv or image feature capture module, the parameters from left to right represent the number of input channels, the number of output channels, the kernel size, the stride size, the padding value, and the number of groups, respectively. In the fully connected layer, the parameters from left to right represent the number of input channels and the number of output channels, respectively. In the pooling layers (Maxpooling and AvgPooling), the parameters from left to right represent the kernel size and the stride size, respectively. Neither the VGG11 neural network model nor the VGG11_Like neural network model uses bias (bias = False).

[0077] Table 3 represents, for example Figure 2 The lightweight neural network model shown consists of an input layer, four image feature capture modules, a fully connected layer, and an output layer arranged sequentially. In the image feature capture modules, the parameters from left to right represent the number of input channels, the number of output channels, the kernel size, the stride size, the padding value, and the number of groups, respectively. In the fully connected layers, the parameters from left to right represent the number of input channels and the number of output channels, respectively. Furthermore, this lightweight neural network model does not use bias (bias = False).

[0078]

[0079] Table 1

[0080]

[0081] Table 2

[0082]

[0083] Table 3

[0084] The explanations for Tables 4 and 5 are as follows:

[0085] Table 4 represents the experimental environment for training the AlexNet neural network model, Alex_Like neural network model, VGG11 neural network model, VGG11_Like neural network model, and lightweight neural network model using the CIFAR10 image dataset.

[0086] Table 5 compares the parameter count, computational cost, and accuracy of the AlexNet, Alex_Like, VGG11, VGG11_Like, and lightweight neural network models after training in image recognition. Experiment 1 compares the Alex_Like and lightweight neural network models with the AlexNet neural network model; Experiment 2 compares the VGG11_Like and VGG11 neural network models.

[0087]

[0088] Table 4

[0089]

[0090] Table 5

[0091] As can be clearly seen from Table 5, the Alex_Like neural network model constructed using the image feature capture module proposed in this application, compared with the AlexNet neural network model, can reduce the number of parameters by 8.27% and the computation by 22.92% under the same parameter settings, and the accuracy is improved by 16.47% compared with the AlexNet neural network model's 75.70%.

[0092] It can also be seen that the VGG11_Like neural network model constructed using the image feature capture module proposed in this application can reduce the number of parameters by 36.11% and the amount of computation by 30.33% under the same parameter settings, compared with the VGG11 neural network model, and the accuracy is improved by 1.50% compared with the 91.80% of the VGG11 neural network model.

[0093] It can also be seen that the lightweight neural network model proposed in this application reduces the number of parameters by 92.18% and the computation by 85.26% compared with the AlexNet neural network model, and improves the accuracy by 15.22% compared with the AlexNet neural network model's 75.70%.

[0094] The experimental data in Tables 1 to 5 above show that by using the neural network model constructed by the image feature capture module proposed in this application, the number of parameters and computation can be greatly reduced when applied to image recognition, and the accuracy of image recognition can be improved to a certain extent. Therefore, the image recognition model provided in this application can run on mobile devices with limited hardware resources and low power consumption, thus improving the applicability of image recognition based on neural network models.

[0095] In some embodiments of this application, an image recognition model is obtained by constructing a neural network model and training it with pre-collected sample image data. The neural network model consists of a sequentially arranged input layer, multiple image feature capture modules, a fully connected layer, and an output layer. The image feature capture module comprises a sequentially arranged first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer. The image to be recognized is then input into the image recognition model to obtain the classification result. Because multiple image feature capture modules are designed in the image recognition model, the number of parameters and computational load of the image recognition model can be greatly reduced. This allows the image recognition model provided in this application to run on mobile devices with limited hardware resources and low power consumption to achieve image recognition, thus improving the applicability of image recognition based on neural network models.

[0096] The following describes an embodiment of the apparatus described in this application, which can be used to execute the image recognition method described above. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the image recognition method described above.

[0097] Figure 5 A block diagram of an image recognition device according to an embodiment of this application is shown.

[0098] Reference Figure 5 As shown, an image recognition device 500 according to an embodiment of this application includes: a construction unit 501, an acquisition unit 502, and an output unit 503.

[0099] The system includes a construction unit 501, which constructs a neural network model and trains it using pre-collected sample image data to obtain an image recognition model. The neural network model consists of a sequentially connected input layer, multiple image feature capture modules, a fully connected layer, and an output layer. The image feature capture module comprises a sequentially connected first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer. An acquisition unit 502 acquires the image to be recognized and inputs it into the image recognition model for classification. An output unit 503 outputs the classification result of the image to be recognized by the image recognition model.

[0100] In some embodiments of this application, based on the foregoing scheme, the construction unit 501 further includes: the first channel integration module is used to reduce the number of original output channels of the original feature map input to the image feature capture module to a preset number of output channels.

[0101] In some embodiments of this application, based on the foregoing scheme, the construction unit 501 further includes: the feature map stitching module is composed of a data processing layer and stitching layers connected in sequence; the data processing layer is used to process the first feature map input to the feature map stitching module to obtain a second feature map; the stitching layer is used to stitch the second feature map and the first feature map together to expand the number of output channels of the first feature map to the original number of output channels.

[0102] In some embodiments of this application, based on the foregoing scheme, the construction unit 501 further includes: the second channel integration module is composed of a channel grouping module, a data exchange module, and a dimension compression module connected in sequence; the channel grouping module is used to divide the feature map of the initial dimension input to the second channel integration module into multiple feature map groups, and expand the initial dimension to a preset dimension; the data exchange module is used to exchange image data between different feature map groups in the multiple feature map groups; the dimension compression module is used to compress the preset dimension to the initial dimension to realize the fusion of image data between the different feature map groups.

[0103] In some embodiments of this application, based on the foregoing scheme, the construction unit 501 further includes: the spatial attention module is composed of a computation module, a splicing module, a convolution module, and an activation module connected in sequence;

[0104] The computation module performs absolute operations on the feature map input to the spatial attention module to obtain an absolute feature map, and performs data processing on the absolute feature map to obtain a mean feature map and a maximum feature map with one output channel respectively; the concatenation module concatenates the mean feature map and the maximum feature map to obtain an intermediate feature map with two output channels; the convolution module is a 1x1 convolution, used to perform dimensionality reduction processing on the intermediate feature map to obtain a weight tensor with one output channel;

[0105] The activation module is used to activate the weight tensor to obtain the target weight tensor, and output the target feature map based on the target weight tensor and the absolute feature map.

[0106] In some embodiments of this application, based on the foregoing scheme, the construction unit 501 further includes: the first activation layer is composed of a first grouped convolutional layer, a first operational layer, a batch normalization layer, a second channel integration module, and a second operational layer in sequence; the first operational layer is used to perform absolute operations on the feature map input to the first operational layer to obtain an absolute feature map; the batch normalization layer is used to normalize the absolute feature map to obtain a normalized feature map; the second operational layer is used to determine a target feature value based on the magnitude of a first feature value in the absolute feature map and a second feature value in the normalized feature map, and to construct a target feature map from the target feature value, wherein the first feature value and the second feature value correspond one-to-one.

[0107] In some embodiments of this application, based on the foregoing scheme, the construction unit 501 further includes: the second activation layer includes an activation function, wherein the activation function is a linear rectifier function.

[0108] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0109] It should be noted that, Figure 6 The computer system 600 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0110] like Figure 6As shown, the computer system 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 602 or programs loaded from storage portion 608 into Random Access Memory (RAM) 603, such as performing the methods described in the above embodiments. The RAM 603 also stores various programs and data required for system operation. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An Input / Output (I / O) interface 605 is also connected to the bus 604.

[0111] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0112] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs various functions defined in the system of this application.

[0113] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0116] In another aspect, this application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image recognition method described in the above embodiments.

[0117] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the image recognition method described in the above embodiments.

[0118] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0119] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0120] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0121] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An image recognition method, characterized in that, The method includes: A neural network model is constructed and trained using pre-collected sample image data to obtain an image recognition model. The neural network model consists of an input layer, multiple image feature capture modules, a fully connected layer, and an output layer arranged in sequence. The image feature capture module consists of a first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer arranged in sequence. The first channel integration module is used to reduce the number of original output channels of the original feature map input to the image feature capture module to a preset number of output channels; the second channel integration module is used to divide the feature map of the initial dimension into multiple feature map groups and realize image data fusion between different feature map groups in the multiple feature map groups; the first activation layer is used to perform absolute operation and normalization processing on the input feature map to determine the target feature value, and construct a second target feature map from the target feature value; the second activation layer includes an activation function, which is a linear rectified function; The image to be identified is obtained and input into the image recognition model so that the image recognition model can classify the image to be identified. Output the classification result of the image to be identified by the image recognition model.

2. The method according to claim 1, characterized in that, The feature map stitching module consists of a data processing layer and stitching layers connected in sequence; The data processing layer is used to process the first feature map input to the feature map splicing module to obtain the second feature map; The splicing layer is used to splice the second feature map and the first feature map to expand the number of output channels of the first feature map to the original number of output channels.

3. The method according to claim 1, characterized in that, The second channel integration module consists of a channel grouping module, a data exchange module, and a dimension compression module connected in sequence; The channel grouping module is used to divide the feature map of the initial dimension input to the second channel integration module into multiple feature map groups, and expand the initial dimension to a preset dimension; The data exchange module is used to exchange image data between different feature map groups in the multiple feature map groups; The dimension compression module is used to compress the preset dimension to the initial dimension in order to achieve image data fusion between the different feature map groups.

4. The method according to claim 1, characterized in that, The spatial attention module is composed of a computation module, a splicing module, a convolution module, and an activation module connected in sequence. The computation module is used to perform absolute operations on the feature map input to the spatial attention module to obtain a first absolute feature map, and to perform data processing on the first absolute feature map to obtain a mean feature map and a maximum feature map with one output channel respectively. The splicing module is used to splice the mean feature map and the maximum feature map to obtain an intermediate feature map with two output channels; The convolution module is a 1x1 convolution, used to perform dimensionality reduction on the intermediate feature map to obtain a weight tensor with one output channel. The activation module is used to activate the weight tensor to obtain the target weight tensor, and output the first target feature map based on the target weight tensor and the first absolute feature map.

5. The method according to claim 1, characterized in that, The first activation layer consists of a first grouped convolutional layer, a first operational layer, a batch normalization layer, a second channel integration module, and a second operational layer, which are sequentially connected in sequence. The first computational layer is used to perform absolute operations on the feature map input to the first computational layer to obtain a second absolute feature map; The batch normalization layer is used to normalize the second absolute feature map to obtain a normalized feature map; The second computational layer is used to determine a target feature value based on the magnitude of the first feature value in the second absolute feature map and the second feature value in the normalized feature map, and to construct a second target feature map from the target feature value, wherein the first feature value and the second feature value correspond one-to-one.

6. An image recognition device, characterized in that, The device includes: A construction unit is used to construct a neural network model and train the neural network model using pre-collected sample image data to obtain an image recognition model. The neural network model consists of an input layer, multiple image feature capture modules, a fully connected layer, and an output layer arranged in sequence. The image feature capture module consists of a first channel integration module, a first convolution module, a feature map stitching module, a second channel integration module, a spatial attention module, a first activation layer, a second convolution module, and a second activation layer arranged in sequence. The first channel integration module is used to reduce the number of original output channels of the original feature map input to the image feature capture module to a preset number of output channels; the second channel integration module is used to divide the feature map of the initial dimension into multiple feature map groups and realize image data fusion between different feature map groups in the multiple feature map groups; the first activation layer is used to perform absolute operation and normalization processing on the input feature map to determine the target feature value, and construct a second target feature map from the target feature value; the second activation layer includes an activation function, which is a linear rectified function; An acquisition unit is used to acquire an image to be identified and input the image to be identified into the image recognition model so that the image recognition model can classify the image to be identified. The output unit is used to output the classification result of the image to be recognized by the image recognition model.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to perform the operations performed by the method as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, It includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, wherein the one or more programs contain instructions for performing the method as described in any one of claims 1 to 5.

Images