Residual capsule network classification model, classification method, device and storage medium for images

The residual capsule network classification model addresses CNN limitations by incorporating a residual extraction module and reconstruction module to enhance feature extraction and image reconstruction, resulting in improved classification accuracy and reduced parameters.

CN114359638BActive Publication Date: 2024-12-03ANHUI UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210022437.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-10
Publication Date
2024-12-03
Estimated Expiration
2042-01-10

AI Technical Summary

Technical Problem

Convolutional neural networks (CNNs) face challenges in maintaining spatial information and relationships between feature positions, leading to accuracy issues in image classification, especially with complex datasets like FASH-MNIST and CIFAR-10, due to limited convolutional extraction layers and high reconstruction errors in fully connected layers.

Method used

A residual capsule network classification model with a residual extraction module that includes multiple residual extraction blocks and a reconstruction module using deconvolution, along with a routing algorithm to convert scalar neurons to vector neurons, enhancing feature extraction and image reconstruction.

Benefits of technology

The model achieves high precision, low reconstruction error, and improved classification accuracy by fully extracting deep features and capturing spatial relationships, reducing model parameters and optimizing network parameters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114359638B_ABST
    Figure CN114359638B_ABST
Patent Text Reader

Abstract

The present invention discloses a residual capsule network classification model, classification method, device and storage medium for an image. The classification model includes a residual extraction module, a capsule module and a reconstruction module. The residual extraction module is used to extract deep information of the image to be classified that is compiled into the capsule layer; the capsule module includes a main capsule layer and a digital capsule. The main capsule layer is used to realize the conversion from scalar neurons to vector neurons, and the iterative update of parameters between the main capsule layer and the digital capsule is realized through a routing algorithm. The digital capsule layer is used to identify the category to which the feature belongs; the reconstruction module reconstructs the image through deconvolution based on the recognition result of the digital capsule layer to obtain the classified image. The reconstructed image of the present invention has strong restoration authenticity and small reconstruction error, and has a high-efficiency, low-manpower and high-precision recognition effect, a small number of model parameters, a fast running speed and high classification accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and relates to a residual capsule network classification model, a classification method, a device and a storage medium for an image. Background Art

[0002] Convolutional neural networks have achieved great success in image classification, target detection and other fields. Their success is due to their powerful ability to extract implicit features and potential information from data. However, convolutional neural networks also have some disadvantages and limitations. For example, they require training samples of various situations to improve generalization, which leads to the need for large samples for training. Spatial information such as position, size and direction will be lost during the pooling process. In addition, the pooling layer can maintain translation invariance and control the number of parameters, but it cannot clearly represent the relationship between feature positions. The same object may be identified as two different objects due to different directions, thus affecting the accuracy of classification.

[0003] Unlike CNN, which uses scalars, CapsNet replaces scalars with vector neurons. The length of the vector represents the probability of the entity's existence, and the direction represents the instantiation parameter. This includes various attributes of the entity in the image, including posture (position, size, direction), deformation, speed, hue, texture, etc. At the same time, CapsNet uses stride convolution and dynamic routing to control the size of parameters, and uses dynamic routing to replace the pooling layer in the convolutional neural network to solve the problem of information loss during the pooling process. CapsNet achieves state-of-the-art performance on the MNIST dataset, and performs better than CNN in the recognition of affine transformed digits.

[0004] CapsNet is an architecture designed specifically for recognizing the MNIST dataset. It is considered to be another revolution in the field of deep learning and has great development prospects in the future. However, CapsNet cannot achieve high-precision classification for datasets such as FASH-MNIST and CIFAR-10. There are two possible reasons: (1) There are only two layers of convolutional extraction modules in front of the main capsule layer. In complex datasets, the images are all multi-channel and high-resolution color images. The two-layer convolution cannot extract deep information from the images, resulting in low recognition rate. In order to extract the significant features and deep abstract features in the images, it is very necessary to increase the depth of the convolutional extraction module. (2) The three-layer fully connected layer has a large reconstruction error for complex datasets, resulting in a decrease in classification accuracy. Finding a useful reconstruction method is very important for the performance of the network. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a residual capsule network classification model for images, which has strong restoration authenticity of reconstructed images and small reconstruction error, and has high efficiency, low manpower and high precision recognition effect, small model parameter amount, fast running speed and high classification accuracy.

[0006] The second objective of the present invention is to provide a classification method for an image residual capsule network classification model.

[0007] A third object of the present invention is to provide an electronic device.

[0008] A fourth object of the present invention is to provide a computer storage medium.

[0009] The technical solution adopted by the present invention is a residual capsule network classification model for images, comprising

[0010] A residual extraction module, the residual extraction module is used to extract deep information of the image to be classified encoded into the capsule layer;

[0011] The capsule module includes a main capsule layer and digital capsules. The main capsule layer is used to realize the conversion from scalar neurons to vector neurons. The iterative update of parameters between the main capsule layer and the digital capsules is realized through a routing algorithm. The digital capsule layer is used to identify the category to which the feature belongs.

[0012] A reconstruction module reconstructs the image through deconvolution based on the recognition result of the digital capsule layer to obtain a classified image.

[0013] Furthermore, the residual extraction module consists of three sequentially connected residual extraction blocks, each residual extraction block includes a filter, four residual units, a 1×1 convolution layer, and a beta-mish activation function layer, and the number of filters contained in the three sequentially connected residual extraction blocks increases exponentially; the residual units in each residual extraction block are densely connected, that is, for each layer of residual units, the feature maps of all previous layers are used as input, and the feature maps output by each layer of residual units are used as input for all subsequent layers.

[0014] Furthermore, the first residual unit decomposes the input feature into n branches, each branch passes through 3×3 convolution, BN layer and beta-mish activation layer in turn, and then fuses the results of n branches to obtain feature map T, performs 1×1 convolution on feature map T and adds it to bypass h1(x), and then activates it through beta-mish function to obtain the output of the first residual unit of the i-th residual extraction block i=1, 2, 3, i is the number of the residual extraction block; As the input of the following three residual units, As the input of the next two residual units, As the input of a subsequent residual unit; where Respectively represent the outputs of the 2nd and 3rd residual units of the i-th residual extraction block, represents the output of the first residual unit of the first residual extraction block;

[0015] The final output of the i-th residual extraction block is: the output of the cascaded 4 residual units is fused with the input of the i-th residual extraction block after passing through a 1×1 convolution and a beta-mish activation layer.

[0016] Furthermore, before inputting into the capsule network, the nonlinearity of the classification model is increased by an activation function beta-mish; the product of the number of capsules and the dimension in the main capsule layer is equal to the number of feature maps input into the main capsule layer; the digital capsule layer receives the output of the main capsule layer, the number of digital capsules depends on the type of image, and each digital capsule corresponds to a classification label.

[0017] Furthermore, the reconstruction module includes

[0018] Fully connected layer, the number is 1;

[0019] There are 4 deconvolution layers. The convolution kernel sizes of the four deconvolution layers are all 3×3, and the strides are 1, 2, 2, and 1 respectively. The number of filters decreases exponentially.

[0020] A classification method of a residual capsule network classification model for an image is performed according to the following steps:

[0021] Step S1, inputting the image to be classified into the residual extraction module of the residual capsule network classification model of the image;

[0022] Step S2, the residual extraction module first extracts the primary features of the image to be classified through a layer of convolution, batch normalization and beta-mish activation layer, and then extracts the deep features of the image to be classified through three consecutively connected residual extraction blocks to obtain the deep information of the image to be classified for extracting the deep information of the image to be classified into the capsule layer;

[0023] Step S3, the deep information extracted by the residual extraction module is incorporated into the main capsule layer of the capsule module to realize the conversion from scalar neurons to vector neurons. Under the constraint of the loss function, the iterative update of the parameters between the main capsule and the digital capsule is realized through the routing algorithm, and the category to which the feature belongs is identified by the digital capsule layer;

[0024] Step S4: Reconstruct the image of the recognition result of the digital capsule layer through a reconstruction module to obtain a classified image.

[0025] Furthermore, the method for extracting deep information in step S2 is specifically as follows:

[0026] The primary feature X0 is input to the first residual extraction block, and the input features are further extracted through a layer of convolution, batch normalization and beta-mish activation layer. The first residual unit in the first residual extraction block first decomposes the input features into n branches. Each branch passes through 3×3 convolution, batch normalization and beta-mish activation layer in turn, and then the results of n branches are fused to obtain the feature map T. After 1×1 convolution and bypass addition, T is activated by the beta-mish function to obtain the output of the first residual unit. Among them, the bypass is obtained by X0 after 1×1 convolution operation and batch normalization;

[0027] X0 and The concatenated result is used as the input feature of the second residual unit of the first residual extraction block, and the output of the second residual unit is obtained through the same operation as the first residual unit. The corresponding bypass formula At this time, i=1,j=2, that is, X0 and Cascade results;

[0028] X0, and The concatenated result is used as the input feature of the third residual unit of the first residual extraction block. The output of the third residual unit is obtained through the same operation as the first residual unit. The corresponding bypass formula At this time, i=1, j=3;

[0029] X0, and The concatenated result is used as the input feature of the fourth residual unit of the first residual extraction block, and the output of the fourth residual unit is obtained through the same operation as the first residual unit. The corresponding bypass formula At this time, i=1, j=4;

[0030] The final output of the first residual extraction block is the output of the cascaded 4 residual units after 1×1 convolution and beta-mish activation layer, and then fused with the input X0 of the first REB, that is, X1;

[0031] If i=2, X1 is input to the first residual unit of the second residual extraction block, and the second residual extraction block finally outputs X2 through the same operation as the first residual extraction block;

[0032] If i=3, X2 is input to the first residual unit of the third residual extraction block, and through the same operation as the first residual extraction block, the third residual extraction block finally outputs X3.

[0033] Furthermore, in step S3, the relationship between the main capsule layer and the digital capsule layer is:

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] Among them, s n is the comprehensive feature vector extracted by the main capsule layer, is the prediction vector of the output of the main capsule layer, v n is the corresponding digital capsule feature vector, indicating the nth digital capsule. One digital capsule corresponds to one category. u k is the output of the mth capsule in the main capsule layer, k represents the number of output features of the mth capsule in the main capsule layer; w kn is u k The weight matrix, c kn yes The coupling coefficient, b kn The initial value of is 0, and squash is a compression function;

[0040] The loss function is:

[0041]

[0042] Among them, n represents the category, K n Indicates whether class n exists, 1 if it exists, and 0 if it does not exist; m + Penalizing false positives, where the n-class exists but is predicted to not exist, will result in a large loss function; m - Penalize false negatives, λ1 is the weight introduced to reduce the error caused by classification errors, and takes a real value between 0 and 1; λ2 is the reconstruction loss of the reconstruction module The proportionality coefficient of X xy , They represent the grayscale value of the pixel at the xth row and yth column of the input image to be classified and the output reconstructed image, respectively. X and Y represent the number of rows and columns of the input image to be classified or the output reconstructed image, respectively.

[0043] An electronic device adopts the above method to achieve classification.

[0044] A computer storage medium stores at least one program instruction, and the at least one program instruction is loaded and executed by a processor to implement the above classification method.

[0045] The beneficial effects of the present invention are:

[0046] The present invention introduces the method of feature map decomposition, extraction and merging to design residual units, and uses dense connections to reconstruct the features extracted by each residual unit to ensure that deep information in the image is fully extracted and incorporated into the capsule module. Small convolution kernels and identity mapping are fused to extract different features, and residual networks are used for feature fusion to ensure that image features used for capsule network coding are fully extracted. This structure can solve the problems of multi-parameters and model degradation caused by simple superposition of network layers, and at the same time help to extract deep features.

[0047] Before encoding the capsule network, the present invention uses the activation function beta-mish to increase the nonlinearity of the classification model, so that it can obtain better gradient flow when it is slightly negative, thereby ensuring that the capsule network can activate more neurons.

[0048] The present invention obtains a reconstruction module through four-layer cascade deconvolution, which captures more spatial relationships in the input image while reducing the number of parameters, thereby ensuring the authenticity of the reconstructed image.

[0049] The present invention uses the error between the original image and the restored image to adjust the parameters of the network, which is beneficial to improving the authenticity of image restoration, optimizing network parameters, and improving image classification accuracy. It has a wide range of applications and flexible and diverse application methods. It solves the problems of low authenticity of image restoration, insufficient optimization of network parameters, and low classification accuracy in existing capsule networks, and is of great significance for achieving accurate image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 1 is a schematic diagram of the structure of a residual capsule network classification model for an image according to an embodiment of the present invention.

[0052] Figure 2 It is a flowchart of a classification method of a residual capsule network classification model for an image according to an embodiment of the present invention.

[0053] Figure 34 is a structural diagram of a residual extraction block module (REB) of a residual capsule network classification model for an image according to an embodiment of the present invention.

[0054] Figure 4 It is a structural diagram of a residual unit (RU) in a residual extraction block module (REB) of a residual capsule network classification model for an image according to an embodiment of the present invention.

[0055] Figure 5 The residual capsule network classification model of the image in the embodiment of the present invention and the classification accuracy and model parameters of different capsule network models on the CIFAR-10 dataset. DETAILED DESCRIPTION

[0056] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] Embodiment 1,

[0058] A residual capsule network classification model for images, such as Figure 1 As shown, it includes a residual extraction module, a capsule module and a reconstruction module connected in sequence; the residual extraction module is used to extract the deep information of the image to be classified that is encoded into the capsule layer; the capsule module is used to realize classification, and it realizes the conversion of scalar neurons to vector neurons through the main capsule layer, and uses the routing algorithm to realize the iterative update of the parameters between the main capsule and the digital capsule, and the digital capsule layer identifies the category to which the feature belongs; the reconstruction module is used to reconstruct the image, and uses deconvolution to reconstruct the image based on the recognition result of the digital capsule layer to obtain the classified image.

[0059] Among them, the residual extraction module is composed of three residual extraction blocks (REBs) connected in sequence; the number of filters contained in the three REBs connected in sequence increases exponentially, and while a small amount of parameters are increased, complementary extraction helps to improve multiple features of classification accuracy. The more filters there are, the more model parameters there are. The compromise value between performance and efficiency is generally determined through experiments. In the embodiment, the number of filters contained in the three REBs connected in sequence are 64, 128, and 256, respectively.

[0060] like Figure 3 As shown in the figure, REB contains four residual units (RU), a 1×1 convolution layer and a beta-mish activation function layer; RU adopts dense connection, that is, for each layer of RU, the feature maps of all previous layers are used as input, and the feature maps output by each layer of RU are used as input of all subsequent layers.

[0061] Before encoding the capsule network, the activation function beta-mish is used to increase the nonlinearity of the classification model, so that it can obtain better gradient flow when it is slightly negative, thereby ensuring that the capsule network can activate more neurons. The structure of the residual extraction module in the embodiment of the present invention more effectively exerts the activation characteristics of beta-mish, and beta-mish also effectively stimulates the nonlinearity between input and output features. Together, it can effectively extract deep information of the image to be classified for inclusion in the capsule layer.

[0062] like Figure 4 As shown in the figure, RU first decomposes the input features into n branches, each branch passes through 3×3 convolution, BN layer and beta-mish activation layer in turn, and then "Concat" fuses the results of n branches to obtain the feature map T. After 1×1 convolution on T, it is added to the bypass h1(x), and then activated by the beta-mish function to obtain the output of the first RU of the i-th residual extraction block (REB) is the number of the residual extraction block (REB); It is the input of the next three RUs. It is the input of the next two RUs. is the input of the next RU. They represent the outputs of the 2nd and 3rd RUs of the i-th residual extraction block (REB), respectively. Can they be understood as Represents the output of the 1st RU of the 1st residual extraction block (REB).

[0063] The final output of the i-th REB is the output of the cascaded 4 RUs, which is fused with the input of the i-th REB after passing through a 1×1 convolution and a beta-mish activation layer.

[0064] The capsule module includes a main capsule layer and a digital capsule layer; the product of the number of capsules and the dimension in the main capsule layer is equal to the number of feature maps input to the main capsule layer, and each output is an 8-dimensional vector, that is, the main capsule layer contains 32 8-dimensional capsules; the filters contained in the three REBs connected in sequence are 64, 128, and 256, respectively, and the input to the main capsule layer is the output of the third REB, a total of 256 feature maps. The digital capsule layer receives the output of the main capsule layer. The number of digital capsules depends on the type of image, and each digital capsule corresponds to a classification label.

[0065] The reconstruction module consists of a fully connected layer and 4 deconvolution layers; the convolution kernel sizes of the 4 deconvolution layers are all 3×3, and the strides are 1, 2, 2, and 1 respectively. The number of filters decreases exponentially to reduce the dimension of the features; in some embodiments, the number of filters is a multiple of 8, and the specific numbers of filters are 64, 32, 16, and 8 respectively.

[0066] Figure 1 In the above, capsule i represents the i-th (i=1,…,32) primary capsule, and Label represents the category to which the image to be classified belongs. A larger number of residual extraction blocks and residual units in each residual extraction block may further improve the classification accuracy while reducing the model classification efficiency; a smaller number may do the opposite. The current number given is the best compromise between classification accuracy and classification efficiency determined through experiments. Experiments show that the residual unit works best when it decomposes the input features into 12 branches.

[0067] Embodiment 2,

[0068] A classification method for a residual capsule network classification model of an image, such as Figure 1-2 As shown, follow these steps:

[0069] Step S1, inputting the image to be classified into the residual extraction module of the residual capsule network classification model of the image;

[0070] Step S2, the residual extraction module first uses a layer of 3×3 convolution, batch normalization and beta-mish activation layer to extract the primary features of the image to be classified, and then further extracts the input features through a layer of 1×1 convolution, batch normalization and beta-mish activation layer, and uses three consecutive residual extraction blocks (REBs) to extract the deep features of the image to be classified, and obtains the deep information of the image to be classified that is used to extract the capsule layer; the primary features and the deep information are both independent and correlated, and each image must contain primary features (edges, contours) and deep features (texture, details).

[0071] The formula for extracting deep information is:

[0072] X0=G 3×3 (X)

[0073]

[0074]

[0075]

[0076]

[0077] T=Con(T1,T2···T n )

[0078] h1(x)=G 1×1 (X i-1 )

[0079]

[0080] Among them, X irepresents the output of the i-th residual extraction block module (REB), represents the output of the jth residual unit (RU) in the i-th REB, i = 1, 2, 3, j = 1, 2, 3, 4, Con(·) is a concatenation operation (i.e., an operation that combines the outputs of four RUs into one result), G 1×1 (·) and G 3×3 (·) are 1×1 and 3×3 convolution operations respectively, T is the feature extraction of n branches of RU T1, T2···T n The fusion result of h1(x) and h2(x) are both bypass. represents the mapping function of the jth RU block in the i-th REB; X represents the image to be classified.

[0081] X0 represents the feature map, i.e., the primary feature, obtained after the image X to be classified passes through a 3×3 convolution, a BN layer, and a beta-mish activation function layer; X0 is the input of three consecutively connected residual extraction blocks (REB).

[0082] X i-1 represents the input of the i-th REB; each REB contains four residual units (RU), a 1×1 convolutional layer and a beta-mish activation function layer, represents the output of the first residual unit (RU) in the i-th REB. represents the output of the j-1th residual unit (RU) in the i-th REB.

[0083] like Figure 4 As shown in the figure, if i=1, X0 is input to the first REB, the first RU (RU1) in the first REB first decomposes the input features into n branches, each branch passes through 3×3 convolution, BN layer and beta-mish activation layer in turn, and then "Concat" fuses the results of n branches to obtain the feature map T, performs 1×1 convolution on T and adds it to the bypass h1(x), and then activates it with the beta-mish function to obtain the output of the first RU Among them, the bypass h1(x) is obtained by X0 after 1×1 convolution operation and batch normalization;

[0084] X0 and The concatenation result is used as the input feature of the second RU (RU2) of the first REB. The output of the second RU is obtained through the same operation as RU1. The corresponding bypass formula At this time, i=1,j=2, that is, X0 and Cascade results.

[0085] X0, and The concatenation result is used as the input feature of RU3 of the first REB. Through the same operation as RU1, the output of the third RU is obtained. The corresponding bypass formula At this time, i=1, j=3.

[0086] X0, and The concatenation result is used as the input feature of RU4 of the first REB. Through the same operation as RU1, the output of the fourth RU is obtained. The corresponding bypass formula At this time, i=1, j=4.

[0087] The final output of the first REB is the output of the cascaded 4 RUs, which is fused with the input X0 of the first REB after 1×1 convolution and beta-mish activation layer, that is, X1.

[0088] If i=2, X1 is input to RU1 of the second REB, and through the same operation as the first REB, the second REB finally outputs X2.

[0089] If i=3, X2 is input to RU1 of the third REB, and through the same operation as the first REB, the third REB finally outputs X3.

[0090] Step S3, the deep information extracted by the residual extraction module is incorporated into the main capsule layer of the capsule module to realize the conversion from scalar neurons to vector neurons. Under the constraint of the loss function, the routing algorithm is used to realize the iterative update of the parameters between the main capsule and the digital capsule, and the digital capsule layer identifies the category to which the feature belongs;

[0091] The relationship between the main capsule layer and the digital capsule layer is,

[0092]

[0093]

[0094]

[0095]

[0096]

[0097] Among them, s n is the comprehensive feature vector extracted by the main capsule layer, is the prediction vector of the output of the main capsule layer, v nis the corresponding digital capsule feature vector, indicating the nth digital capsule. One digital capsule corresponds to one category. u k is the output of the mth capsule in the main capsule layer, k represents the number of output features of the mth capsule in the main capsule layer, k = 1, ..., 1152; w kn is u k The weight matrix, c kn yes The coupling coefficient, b kn The initial value is 0, squash is a compression function, which converts the vector v n The length of is compressed between 0 and 1, compressing small vectors to 0 and large vectors to the unit vector 1. Represents an iterative operation.

[0098] The loss function is:

[0099]

[0100] Among them, n represents the category, K n Indicates whether class n exists, 1 if it exists, and 0 if it does not exist; m + is 0.9, which penalizes false positives. If the n-class exists but is predicted not to exist, the loss function will be very large. - λ is 0.1, which penalizes false negatives. If the n-class does not exist but is predicted to exist, the loss function will be very large. λ1 is the weight introduced to reduce the error caused by misclassification, which is generally a real value between 0 and 1. λ2 is the reconstruction loss of the reconstruction module. The proportionality factor. xy , They represent the grayscale value of the pixel in the xth row (x=1,…,X)th column (y=1,…,Y) of the input image to be classified and the output reconstructed image, respectively. X and Y represent the number of rows and columns of the input image to be classified and the output reconstructed image, respectively.

[0101] The loss function of the present invention includes three parts: (1) the loss caused by the existence or non-existence of a certain category; (2) the error caused by reducing the classification error; (3) the reconstruction loss of the reconstruction module. Under the limitation of the classification method proposed in the embodiment of the present invention, the residual capsule network classification model designed by the present invention is trained under the constraint of the self-defined loss function of the present invention to obtain the optimal parameter model, which effectively improves the classification accuracy of the model.

[0102] Step S4: Reconstruct the image of the recognition result of the digital capsule layer through a reconstruction module of a fully connected layer and four deconvolution layers to obtain a classified image.

[0103] The experiments were conducted in the Keras environment, using the Adam optimizer as the gradient descent algorithm for training, and setting λ1 = 0.5, λ2 = 0.0005 (to avoid the reconstruction loss dominating the training process). The batch size was set to 128, the training was set to 150 cycles, each cycle performed 390 iterations, and the initial learning rate was 0.001. During the training process, the learning rate was reduced in stages to make the network optimal. All experiments were conducted on a GTX2080ti computer with 11GB of memory.

[0104] The residual capsule network classification model of the image proposed by the present invention in the embodiment is abbreviated as Res-CapsNet, the activation function in the model uses beta-mish, and each residual unit (RU) in the residual extraction block (REB) includes 12 branches.

[0105] To verify the performance of the Res-CapsNet model, experiments were conducted on the standard dataset CIFAR-10 and compared with CapsNet [S. Sabour, N. Frosst, and G. E. Hinton, “Dynamic routing between capsules,” in Proc. Int. Conf. Neural Inf. Process. Syst., Nov. 2017, pp. 3856-3866], MS-CapsNet [C. Xiang, L. Zhang, Y. Tang, W. Zou and C. Xu, “MS-CapsNet: A Novel Multi-Scale Capsule Network,” IEEE Signal Processing Letters, vol. 25, no. 12, pp. 1850-1854, Dec. 2018], FSc-CapsNet [T. Han, R. Sun, F. Shao and Y. Sui, “Feature and spatial relationship coding capsule network”. J of Electronic Imaging, vol. 29, no. 2, pp. 023004, 2020], RS-CapsNet [S. Yang et al., "RS-CapsNet: An Advanced Capsule Network," IEEE Access, vol. 8, pp. 85007-85018, 2020], RS-CapsNet(7-ens) [S. Yang et al., "RS-CapsNet: An Advanced Capsule Network," IEEE Access, vol. 8, pp. 85007-85018, 2020], DeepCaps_sau_att [X. Ning et al., "BDARS_CapsNet: Bi-Directional Attention Routing Sausage Capsule Network," in IEEE Access, vol. 8, pp. 59059-59068, 2020], and DCNet++ [S. S. R. Phaye, A. Sikka, A. Dhall, and D. Bathula, “Dense and diverse capsule networks: Making the capsules learn better,” 2018, arXiv:1805.04001] The classification accuracy and model parameter quantity of the seven models were compared, and the results are shown in Table 1 and. Figure 5 , Figure 5 The horizontal and vertical axes represent the number of parameters and classification accuracy, respectively. Obviously, Res-CapsNet(aug) has the highest classification accuracy and the least parameters. Res-CapsNet ranks second in both indicators. RS-CapsNet and RS-CapsNet(7-ens) rank third in terms of parameters. DeepCaps_sau_att ranks third in classification accuracy, but the number of parameters is very high.

[0106] Table 1 Classification accuracy and parameter count of different models for CIFAR-10

[0107] Model CIFAR-10 (%) Parameters(M) CapsNet 68.93 11.7 MS-CapsNet 75.70 11.2 FSc-CapsNet 80.03 34.0 RS-CapsNet 89.81 5.01 RS-CapsNet(7-ens) 91.32 5.01 DeepCaps_sau_att 91.76 13.43 DCNet++ 89.32 13.4 Res-CapsNet 92.13 4.01 Res-CapsNet(aug) 93.33 4.01

[0108] from Figure 5 It can also be seen that the classification accuracy can be further improved by using image data augmentation (IDA) techniques, namely flipping, translation, and rotation, to enhance the image data. Res-CapsNet is the result on CIFAR-10, and Res-CapsNet (aug) is the result on CIFAR-10 after IDA. With the same parameters, the classification accuracy of Res-CapsNet (aug) is 1.30% higher than that of Res-CapsNet.

[0109] In summary, the model structure of Res-CapsNet is reasonable. In terms of comprehensive classification accuracy and model parameters, Res-CapsNet is superior to the current advanced capsule network method. The main reason is that the residual extraction module of Res-CapsNet combines the advantages of residual learning, small convolution kernels and identity mapping. This structure ensures the effective reduction of model parameters on the one hand, and on the other hand, ensures that the feature information of the image is fully extracted, thereby ensuring that there is enough information encoded in the capsule, giving full play to the spatial relationship advantage of the capsule network, and mapping local features to global features.

[0110] The embodiment of the present invention introduces the characteristic of CapsNet that replaces scalars with vector neurons (the length of the vector represents the probability of the entity's existence, and the direction represents the instantiation parameter) based on the designed residual extraction module. As a result, the features extracted by the residual extraction module can include various attributes of the entity in the image, including posture (position, size, direction), deformation, speed, hue, texture, etc., thereby improving the classification accuracy of the model. The model structure of the embodiment of the present invention determines that the number of parameters is small. Compared with CapsNet, Res-CapsNet greatly reduces the number of parameters, with a reduction of 65.73%. The embodiment of the present invention can be applied to coal gangue image classification, standard image classification, target detection and other fields, and has strong generalization, which can solve the problem that the model parameters cannot be promoted in practical applications.

[0111] If the image classification method described in the embodiment of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the image classification method described in the embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.

[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A residual capsule network classification model for an image, characterized in that: include: A residual extraction module, the residual extraction module is used to extract deep information of the image to be classified encoded into the capsule layer; The capsule module includes a main capsule layer and digital capsules. The main capsule layer is used to realize the conversion from scalar neurons to vector neurons. The iterative update of parameters between the main capsule layer and the digital capsules is realized through a routing algorithm. The digital capsule layer is used to identify the category to which the feature belongs. A reconstruction module, wherein the reconstruction module reconstructs the image through deconvolution based on the recognition result of the digital capsule layer to obtain a classified image; The residual extraction module is composed of three consecutively connected residual extraction blocks, each of which contains a filter, four residual units, a 1×1 convolution layer, and a beta-mish activation function layer, and the number of filters contained in the three consecutively connected residual extraction blocks increases exponentially; the residual units in each residual extraction block are densely connected, that is, for each layer of residual units, the feature maps of all previous layers are used as input, and the feature maps output by each layer of residual units are used as input for all subsequent layers; The first residual unit decomposes the input feature into n branches. Each branch passes through 3×3 convolution, BN layer and beta-mish activation layer in turn, and then fuses the results of n branches to obtain feature map T. The feature map T is convolved with 1×1 and added to the bypass h1(x), and then activated by the beta-mish function to obtain the output of the first residual unit of the i-th residual extraction block. i is the number of the residual extraction block; As the input of the following three residual units, As the input of the next two residual units, As the input of a subsequent residual unit; where Respectively represent the outputs of the 2nd and 3rd residual units of the i-th residual extraction block, represents the output of the first residual unit of the first residual extraction block; The final output of the i-th residual extraction block is: the output of the cascaded 4 residual units is fused with the input of the i-th residual extraction block after passing through a 1×1 convolution and a beta-mish activation layer.

2. According to claim 1, the residual capsule network classification model for an image is characterized in that: Before inputting the capsule network, the nonlinearity of the classification model is increased by the activation function beta-mish; the product of the number and dimension of capsules in the main capsule layer is equal to the number of feature maps input to the main capsule layer; The digital capsule layer receives the output of the main capsule layer. The number of digital capsules depends on the type of image, and each digital capsule corresponds to a classification label.

3. According to claim 1, the residual capsule network classification model for an image is characterized in that: The reconstruction module includes: Fully connected layer, the number is 1; There are 4 deconvolution layers. The convolution kernel sizes of the four deconvolution layers are all 3×3, and the strides are 1, 2, 2, and 1 respectively. The number of filters decreases exponentially.

4. A classification method of the residual capsule network classification model of an image according to any one of claims 1 to 3, characterized in that: Follow these steps: Step S1, inputting the image to be classified into the residual extraction module of the residual capsule network classification model of the image; Step S2, the residual extraction module first extracts the primary features of the image to be classified through a layer of convolution, batch normalization and beta-mish activation layer, and then extracts the deep features of the image to be classified through three consecutively connected residual extraction blocks to obtain the deep information of the image to be classified for extracting the deep information of the image to be classified into the capsule layer; Step S3, the deep information extracted by the residual extraction module is incorporated into the main capsule layer of the capsule module to realize the conversion from scalar neurons to vector neurons. Under the constraint of the loss function, the iterative update of the parameters between the main capsule and the digital capsule is realized through the routing algorithm, and the category to which the feature belongs is identified by the digital capsule layer; Step S4: Reconstruct the image of the recognition result of the digital capsule layer through a reconstruction module to obtain a classified image.

5. The classification method of the residual capsule network classification model of an image according to claim 4, characterized in that: The method for extracting deep information in step S2 is specifically as follows: The primary feature X0 is input to the first residual extraction block, and the input features are further extracted through a layer of convolution, batch normalization and beta-mish activation layer. The first residual unit in the first residual extraction block first decomposes the input features into n branches. Each branch passes through 3×3 convolution, batch normalization and beta-mish activation layer in turn, and then the results of n branches are fused to obtain the feature map T. After 1×1 convolution and bypass addition, T is activated by the beta-mish function to obtain the output of the first residual unit. Among them, the bypass is obtained by X0 after 1×1 convolution operation and batch normalization; X0 and The concatenated result is used as the input feature of the second residual unit of the first residual extraction block, and the output of the second residual unit is obtained through the same operation as the first residual unit. The corresponding bypass formula At this time, i=1,j=2, that is, X0 and Cascade results; X0, and The concatenated result is used as the input feature of the third residual unit of the first residual extraction block. The output of the third residual unit is obtained through the same operation as the first residual unit. The corresponding bypass formula At this time, i=1, j=3; X0, and The concatenated result is used as the input feature of the fourth residual unit of the first residual extraction block. The output of the fourth residual unit is obtained through the same operation as the first residual unit. The corresponding bypass formula At this time, i=1, j=4; The final output of the first residual extraction block is the output of the cascaded 4 residual units after 1×1 convolution and beta-mish activation layer, and then fused with the input X0 of the first REB, that is, X1; If i=2, X1 is input to the first residual unit of the second residual extraction block, and the second residual extraction block finally outputs X2 through the same operation as the first residual extraction block; If i=3, X2 is input to the first residual unit of the third residual extraction block, and through the same operation as the first residual extraction block, the third residual extraction block finally outputs X3.

6. The classification method of the residual capsule network classification model of an image according to claim 4, characterized in that: In step S3, the relationship between the main capsule layer and the digital capsule layer is: Among them, s n is the comprehensive feature vector extracted by the main capsule layer, is the prediction vector of the output of the main capsule layer, v n is the corresponding digital capsule feature vector, indicating the nth digital capsule. One digital capsule corresponds to one category. u k is the output of the mth capsule in the main capsule layer, k represents the number of output features of the mth capsule in the main capsule layer; w kn is u k The weight matrix, c kn yes The coupling coefficient, b kn The initial value of is 0, and squash is a compression function; The loss function is: Among them, n represents the category, K n Indicates whether class n exists, 1 if it exists, and 0 if it does not exist; m + Penalizing false positives, where the n-class exists but is predicted to not exist, will result in a large loss function; m - Penalize false negatives, λ1 is the weight introduced to reduce the error caused by classification errors, and takes a real value between 0 and 1; λ2 is the reconstruction loss of the reconstruction module The proportionality coefficient of X xy , They represent the grayscale value of the pixel at the xth row and yth column of the input image to be classified and the output reconstructed image, respectively. X and Y represent the number of rows and columns of the input image to be classified or the output reconstructed image, respectively.

7. An electronic device, characterized in that: The classification is achieved by using the method described in any one of claims 4 to 6.

8. A computer storage medium, characterized in that: The storage medium stores at least one program instruction, and the at least one program instruction is loaded and executed by the processor to implement the classification method according to any one of claims 4 to 6.