Image recognition method, system and storage medium based on cross residual network model

Abstract

The present invention discloses an image recognition method, system and storage medium based on a cross residual network model, which can be applied to the technical field of convolutional neural networks. The method of the present invention comprises the following steps: obtaining an image to be recognized; using a cross residual network model to recognize the image to be recognized, and obtaining an image recognition result; wherein the cross residual network model comprises: at least one cross residual module, the cross residual module comprises a plurality of weight layers and a plurality of jump connection lines, the weight layer comprises a convolution layer, a batch normalization layer and an activation function layer, and the activation function layer is arranged after the addition operator; each of the jump connection lines jumps at least one of the weight layers and all the jump connection lines cross jump at least one of the weight layers. The present invention can improve the accuracy of image recognition results without deepening and expanding the convolution layer to improve the model performance.

Description

Technical Field

[0001] The present invention relates to the technical field of convolutional neural networks, and in particular to an image recognition method, system and storage medium based on a cross residual network model. Background Art

[0002] In the related art, convolutional neural networks have been widely used in the field of computer vision. Expanding and deepening convolutional layers are two important strategies in the design of convolutional neural networks. The Inception architecture of GoogLeNet expanded the width of the convolutional layer to improve accuracy. Subsequent versions of Inception improved the performance of the network through a series of optimizations, such as adding batch normalization (BN), turning the 7×7 convolution kernel into a two-layer series of a 1×7 convolution kernel and a 7×1 convolution kernel, and introducing the idea of ​​residual networks into the model. The residual network (ResNet) uses residual block layers, which can obtain accuracy by increasing the depth of the convolution layer without performance degradation or overfitting. The building blocks of ResNeXt extend the residual blocks in ResNet and enrich the diversity of features. The SENet architecture based on the Squeeze-and-Excitation (SE) block is effectively generalized on different datasets. The SE module increases the depth of channels in ResNet and enhances the channel-wise feature response by distinguishing the interdependencies between channels. SK units expand the channels of ResNeXt and SE blocks, and SKNets can also gain accuracy by deepening the network, that is, stacking more layers of SK units. All the above-mentioned variants of the original ResNet show that performance is improved by deepening and expanding the convolutional layers, so when it is necessary to improve the image recognition effect, it is necessary to deepen and expand the convolutional layers first, which greatly increases the difficulty of designers. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an image recognition method, system and storage medium based on a cross residual network model, which can effectively improve the image recognition effect without deepening and expanding the convolution layer.

[0004] On the one hand, an embodiment of the present invention provides an image recognition method based on a cross residual network model, comprising the following steps:

[0005] Obtain an image to be recognized;

[0006] Using a cross residual network model to identify the image to be identified, to obtain an image recognition result;

[0007] Wherein, the cross residual network model includes:

[0008] At least one cross residual module, the cross residual module includes a plurality of weight layers and a plurality of jump connection lines, the weight layer includes a convolution layer, a batch normalization layer and an activation function layer, and the activation function layer is arranged after the addition operator; each of the jump connection lines jumps at least one of the weight layers and all the jump connection lines cross jump at least one of the weight layers.

[0009] In some embodiments, the cross residual module includes three weight layers and two cross jump connection lines.

[0010] In some embodiments, when the jump connection line includes a dotted line, the convolution layer of the weight layer includes a 1*1 convolution layer or a 3*3 convolution layer; the weight layer including the 1*1 convolution layer is alternately connected with the weight layer including the 3*3 convolution layer.

[0011] In some embodiments, the cross residual module corresponding to the first jump connection line being a dotted line is used as the first cross residual module; and the cross residual module corresponding to the second jump connection line being a dotted line is used as the second cross residual module.

[0012] In some embodiments, the cross residual module includes three weight layers and two cross jump connection lines or includes six weight layers and three cross jump connection lines.

[0013] In some embodiments, the cross residual module including three weight layers and two cross jump connection lines is used as a third cross residual module, and the third cross residual module includes two 1*1 convolutional layers and one 3*3 convolutional layer; the two 1*1 convolutional layers and the one 3*3 convolutional layer are alternately connected.

[0014] In some embodiments, the cross residual module including six weight layers and three cross jump connection lines includes two third cross residual modules; the two third cross residual modules are connected sequentially.

[0015] On the other hand, an embodiment of the present invention provides an image recognition system based on a cross residual network model, comprising:

[0016] An acquisition unit, used for acquiring an image to be recognized;

[0017] A recognition unit, used to recognize the image to be recognized by using a cross residual network model to obtain an image recognition result;

[0018] Wherein, the cross residual network model includes:

[0019] At least one cross residual module, the cross residual module includes a plurality of weight layers and a plurality of jump connection lines, the weight layer includes a convolution layer, a batch normalization layer and an activation function layer, and the activation function layer is arranged after the addition operator; each of the jump connection lines jumps at least one of the weight layers and all the jump connection lines cross jump at least one of the weight layers.

[0020] On the other hand, an embodiment of the present invention provides an image recognition system based on a cross residual network model, comprising:

[0021] at least one memory for storing a program;

[0022] At least one processor is used to load the program to execute the image recognition method based on the cross residual network model.

[0023] On the other hand, an embodiment of the present invention provides a storage medium, which stores a computer-executable program, and when the computer-executable program is executed by a processor, it is used to implement the image recognition method based on the cross residual network model.

[0024] An image recognition method based on a cross residual network model provided by an embodiment of the present invention has the following beneficial effects:

[0025] In this embodiment, at least one cross-residual module is first set in the cross-residual network model, and a plurality of weight layers sequentially connected by convolution layers, batch normalization layers and activation function layers, as well as jump connection lines of cross-jump weight layers are set in the cross-residual module, thereby eliminating the need to deepen and expand the convolution layers to improve the model performance; the cross-residual network model is then used to perform image recognition to improve the accuracy of the image recognition results.

[0026] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments, wherein:

[0028] Figure 1 A flowchart of an image recognition method based on a cross residual network model according to an embodiment of the present invention;

[0029] Figure 2 A structural diagram of a weight layer according to an embodiment of the present invention;

[0030] Figure 3 The WBR representation is used in the embodiment of the present invention. Figure 2 Schematic diagram of

[0031] Figure 4 A schematic diagram of a skip connection line with an addition operator according to an embodiment of the present invention;

[0032] Figure 5 It is a schematic diagram of another skip connection line with an addition operator according to an embodiment of the present invention;

[0033] Figure 6 One embodiment of the present invention Figure 4 Schematic diagram of the corresponding jump connection line without the addition operator;

[0034] Figure 7 Another embodiment of the present invention is Figure 5 Schematic diagram of the corresponding jump connection line without the addition operator;

[0035] Figure 8 Schematic diagram of the structure of Cross-Block according to an embodiment of the present invention;

[0036] Fig. 9 The embodiment of the present invention is represented by WBR Figure 8 The schematic diagram of the Cross-Block structure;

[0037] Fig.10 A schematic diagram of the structure of a Cross-Bottleneck according to an embodiment of the present invention;

[0038] Fig.11 It is a schematic diagram of another Cross-Bottleneck structure of an embodiment of the present invention;

[0039] Fig.12 The embodiment of the present invention is represented by WBR Fig.11 Schematic diagram of the Cross-Bottleneck structure;

[0040] Fig.13 Schematic diagram of the structure of the BasicBlock module in ResNet of an embodiment;

[0041] Fig.14 A schematic diagram of the structure of a Bottleneck module in ResNet according to an embodiment;

[0042] Fig.15 It is a schematic diagram of the structure of C-ResNet18-A in an embodiment of the present invention;

[0043] Fig.16 A schematic diagram of the structure of ResNet* according to an embodiment of the present invention;

[0044] Fig.17This is a comparison chart of C-ResNets and ResNets*Flops in an embodiment of the present invention;

[0045] Fig.18 4 is a comparison chart of the number of parameters of C-ResNets and ResNets* in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0047] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0048] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0049] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0050] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0051] At present, convolutional neural network models generally improve model performance by deepening and expanding convolutional layers, but this approach will increase the amount of data calculation in the model. Based on this, this embodiment provides an image recognition method, system, and storage medium based on a cross residual network model. The difference between the cross residual network module in the cross residual network model in this embodiment and the residual module in the current ResNet and ResNet (v2) is:

[0052] First, the position of the ReLU activation function. The ReLU activation function in the cross residual module of this embodiment appears after the addition, rather than before the addition operation;

[0053] Second, the number of convolutional layers and skip connections. Compared with the Bottleneck module in ResNet, the Cross-Bottleneck module of this embodiment inserts an additional convolutional layer to increase the number of skip connections and cross them together;

[0054] Third, the effect of 1×1 convolution operation on skip connection. When the present embodiment uses skip connection, 1×1 convolution operation can be used to increase the conversion of features without worrying about whether the sizes of both ends of the skip connection match;

[0055] Fourth, the number of channels: Compared with Bottleneck, this embodiment reduces the channel count while maintaining the classification performance.

[0056] The following uses the application of the model of this embodiment in image recognition as an example to illustrate the specific process:

[0057] Reference Figure 1 An embodiment of the present invention provides an image recognition method based on a cross residual network model. This embodiment can be applied to a background processor of an image recognition platform or to a server.

[0058] During the application process, this embodiment includes the following steps:

[0059] S11, obtaining an image to be identified. It is understandable that the image to be identified includes an image acquired in real time at the time, and also includes an image pre-stored by the user.

[0060] S12. Use a cross residual network model to identify the image to be identified and obtain an image recognition result.

[0061] In this embodiment, the cross residual network model includes at least one cross residual module, the cross residual module includes several weight layers and several jump connection lines, specifically, the weight layer includes a convolution layer, a batch normalization layer and an activation function layer, the convolution layer, the batch normalization layer and the activation function layer are sequentially connected, and the activation function layer is set after the addition operator. Each jump connection line jumps at least one weight layer and all jump connections are cross-jumped at least one weight layer.

[0062] It is understandable that the WBR layer can be used to represent the weight layer. Figure 2 The weight layer shown in Figure 1 contains a series of operations in the order of convolution, batch normalization layer BN, and activation function layer ReLU. Figure 2 After the weight layer is converted into the WBR layer representation, we get Figure 3 It can be understood that when the jump connection line contains an addition operator, its connection diagram is as follows: Figure 4 and Figure 5 As shown; when Figure 4 When the jump connection line does not have an addition operator, the connection diagram is as follows Figure 6 As shown; when Figure 5 When the jump connection line does not have an addition operator, the connection diagram is as follows Figure 7 shown.

[0063] Assume that the input of the WBR layer is x and the output is P(x). So WER can be expressed as shown in formula (1), where W, B, σ represent the weight layer, BN and ReLU function respectively:

[0064] P(x)=σ(B(W(x))) Formula (1)

[0065] Similar to ResNet, the cross residual module of this embodiment also has two basic modules: one is Cross-Block, which is applied to networks with fewer layers; the other is Cross-Bottleneck, which is applied to networks with more layers.

[0066] For Cross-Block, Figure 8As shown, its cross residual module includes three weight layers and two cross jump connection lines, and the convolution layer of the weight layer is a 3*3 convolution layer. The purpose of using two cross jump connection lines is to have more jump connection lines in fewer WBR layers, thereby enhancing the transmission of information between layers. When the jump connection line is a dotted line, it means that the weight layer includes a 1×1 convolution operation. There are two variants of Cross-Block in this embodiment. The variant with the first jump connection line as a dotted line is used as the first cross residual module Cross-Block-A1, and the variant with the second jump connection line as a dotted line is used as the second cross residual module Cross-Block-A2. In order to distinguish these variants, Figure 8 It is also called Cross-Block-A. The WBR of Cross-Block-A is as follows Fig. 9 As shown, it can also be expressed by formula (2), where the symbol "+" represents the addition operation. As with the ResNet model, the "+" operation does not generate additional parameters or computational costs.

[0067]

[0068] Assume P m (x) and represents the nesting of m(m≥2)P functions. For example, the two functions shown in formula (3) and formula (4):

[0069] P 2 (x) = P (P (x) + x) Formula (3)

[0070]

[0071] Then the output of Cross-Block-A can be rewritten as P 3 (x)+P(x). Similarly, Cross-Block-A1 and Cross-Block-A2 can be written as shown in formula (5) and formula (6):

[0072]

[0073]

[0074] For Cross-Bottleneck, the cross residual module includes three weight layers and two cross jump connection lines or includes six weight layers and three cross jump connection lines. Specifically, the cross residual module including three weight layers and two cross jump connection lines is used as the third cross residual module, and the third cross residual module includes two 1*1 convolution layers and one 3*3 convolution layer; two 1*1 convolution layers and one 3*3 convolution layer are alternately connected. The cross residual module including six weight layers and three cross jump connection lines includes two third cross residual modules; two third cross residual modules are connected sequentially. For example, Fig.10 The Cross-Bottleneck module with three WER layers and two skip connections is shown in Fig.11 The Cross-Bottleneck module with six WER layers and three skip connections is shown in Figure 1. The six-layer Cross-Bottleneck is composed of two three-layer Cross-Bottlenecks stacked together. The purpose is to keep the number of real-line skip connection lines as much as possible while reducing the number of channels, such as from 256 to 128, so as to reduce the amount of computation. Using a similar strategy, the WBR of the six-layer Cross-Bottleneck is expressed as follows: Fig.12 As shown, it can also be expressed as shown in formula (7):

[0075]

[0076] It is understandable that the basic blocks in ResNet are as follows Fig.13 The BasicBlock module shown has two layers and one skip connection, while the proposed Cross-Block has three layers and two cross skip connections. It can be seen that, except for the number of skip connections and the position of ReLU, such as before and after addition, the superposition of two Cross-Blocks is exactly the same as the superposition of three basic blocks. Therefore, when the network scale is the same, that is, the number of convolutional layers is the same, the computational amount and the number of parameters of the cross residual network (C-ResNet) stacked by Cross-Block are basically the same as the computational amount and the number of parameters of the network composed of the basic blocks in ResNet. However, because the model of this embodiment has more skip connections than ResNet, specifically, the solid line skip connection promotes feature transfer and the dotted line skip connection increases feature conversion, so this embodiment uses a smaller-scale C-ResNet to achieve a better performance network. As Fig.14 The Bottleneck and the Fig.10The difference between the three layers of Cross-Bottleneck shown is the number of channels in the skip connection line and the second convolutional layer. The change in the number of channels in Bottleneck (ResNet) increases the difficulty of designing Cross-Bottleneck, because the design principle of this embodiment is to design as many solid line skip connections as possible instead of dotted line skip connections. Therefore, this embodiment tries to strike a balance between the number of channels and the number of skip connections. Because too few channels are not conducive to feature extraction. So this embodiment will Fig.11 The number of channels in is set to 128 instead of 64.

[0077] When passing Figures 8 to 12 When constructing a cross residual network C-ResNet with cross blocks, C-ResNet can be named according to the residual module type and the number of layers. For example, Fig.15 The C-ResNet18-A shown is stacked by Cross-Block-A, and has 18 layers. In order to enable ResNet to process 32×32 images, that is, it is applied to the ImageNet dataset and can only process images of size 224×224, this embodiment fine-tunes ResNet, that is, the fine-tuned ResNet is called ResNet*, as shown in Fig.16 As shown, the 7×7 convolution and maximum pooling are replaced by two 3×3 convolution layers, and the fully connected layer sets different numbers of neurons to adapt to the number of categories in different data sets. In the subsequent experimental part of this embodiment, it is proved that the classification accuracy of ResNet* is higher than that of ResNet(v2) on the CIFAR-10 and CIFAR-101 data sets. Therefore, this embodiment uses a fine-tuned ResNet (ie, ResNet*) as the baseline for the experiments in this embodiment. In addition, the C-ResNets designed in this embodiment also uses two 3×3 convolution layers in the input stage, so except for the stacked modules, the external environment of ResNet* and C-ResNet is the same.

[0078] C-ResNet-B and C-ResNet-C are stacked by six layers of Cross-Bottlenecks and three layers of Cross-Bottlenecks, respectively, including the number of dotted skip connections and their corresponding FLOPs. The FLOPs of C-ResNets are smaller than those of the corresponding ResNets. Although C-ResNet27-A2 has fewer layers than the fine-tuned ResNet34, it has 5 more dotted skip connections than the fine-tuned ResNet34, which means that 5 more 1×1 convolution operations are completed. Therefore, compared with the fine-tuned ResNet34, C-ResNet27-A2 can be regarded as using skip connections to exchange convolutional layers (equivalent to 32 convolutional layers).

[0079] The above embodiments were tested experimentally:

[0080] Specifically, the ResNet* model and the C-ResNet model were experimented on six data sets: MNIST, FashionMnist, CIFAR-10, CIFAR-100, CALTECH-101, and SVHN. And both models used the same hyperparameters. This embodiment adjusts BN and ReLU after each convolution, uses SGD, the weight decay is 0.0005, the learning rate starts from 0.01, and is divided by 10 every 150 epochs. These models are trained for up to 500 iterations, and the mini-batch size used is 32. This embodiment runs each model 3 times, and then takes the results of the last 20 epochs each time, generating a total of 60 results. Then take the mean and standard deviation of these 60 results as the results of the experiment. In the experiment of this embodiment, except for the residual module itself, the external environment (such as hyperparameter settings, optimization functions, etc.) of C-ResNet and ResNet* is exactly the same. In order to make a fair comparison, this embodiment designs a C-ResNet with similar layers or the same FLOPs to compare the performance with ResNet, and the following results are obtained: Fig.17 and Fig.18 This embodiment uses ResNet* as the baseline of the experiment, not only because its classification accuracy on the CIFAR-10 and CIFAR-100 datasets is much higher than that of ResNet as shown in Table 1, but also because the BasicBlock and Bottleneck modules in ResNet* are exactly the same as those in ResNet.

[0081] Table 1

[0082] CIFAR-10 CIFAR-100 <![CDATA[Resnet100 # ]]> 6.37 - <![CDATA[Resnet164 # ]]> 5.46 24.33 <![CDATA[Resnet1001 # ]]> 4.92 22.71 Fine-tuned Resnet18 4.52 21.42 Fine-tuned Resnet34 4.25 20.23 Fine-tuned Resnet50 3.96 19.45

[0083] For C-ResNet, this embodiment designs and evaluates the models of C-ResNet15, C-ResNet18, C-ResNet27-A2 and C-ResNet27-B2 respectively and the corresponding ResNet*.

[0084] First, experiments were conducted on C-ResNet15 and C-ResNet18: Resnet18 (i.e. ResNet18*) has 18 convolutional layers, each of which has a 3×3 kernel and 8 non-crossing jump connections, including 5 solid lines and 3 dotted lines. C-ResNet15) and C-ResNet18 were constructed to compare the performance of ResNet18*.

[0085] C-ResNet15 has 14 convolutional layers, each of which is 3×3 and 8 cross jumps, including 2 solid lines and 6 dashed lines. Compared with ResNet18*, its computational cost (i.e., FLOPs) and number of parameters are reduced by 22.03% and 24.78%, respectively, and it has lower error rates on 5 datasets: MNIST, FashionMnist, CIFAR-10, CALTECH-101, and SVHN. In particular, the error rate of C-ResNet15 on the CALTECH-101 dataset is 7.96% lower than that of ResNet18*, and the variance is also smaller, indicating that C-ResNet15 is more stable. In addition, C-ResNet15 has the same number of skip connections as ResNet18* and 4 fewer 3×3 convolutional layers, but the ratio of solid lines to dashed lines in the skip connections is different. This means that we can improve the classification accuracy of the residual network by skipping connections and increasing the number of dashed lines while reducing the amount of computation and parameters.

[0086] Similarly, C-ResNet18 has 17 convolutional layers and 10 cross jumps, including 7 solid lines and 3 dotted lines. Compared with ResNet18*, the computational cost and number of parameters of C-ResNet18 are reduced by 5.08% and 23.98% respectively, and the error rate on six datasets is lower. It can be seen that C-ResNet18 greatly reduces the number of parameters, which can significantly reduce the use of video memory. It is further revealed that the number of convolutional layers in C-ResNet18 is less than that in ResNet18*, and the number of dotted jumps is the same. C-ResNet18 only increases the number of jump connection lines (addition operation) by cross jumps, which means that only cross jumps are needed to improve the classification accuracy of the residual network, while reducing the amount of calculation and parameters.

[0087] Second, experiments were conducted on C-ResNet27-A2, which was constructed by stacking more WBR modules (2-2-2-2 structure) to compare performance with the fine-tuned ResNet34 (i.e., ResNet34*). ResNet34* (3-4-6-3 structure) has 34 convolutional layers, each of which is 3×3 and has 16 non-crossed jump connections (or skips), including 13 solid lines and 3 dotted lines. C-ResNet27-A2 has 26 convolutional layers, each of which is 3×3 and has 16 cross-jump connections, including 8 solid lines and 8 dotted lines.

[0088] Compared with ResNet34*, C-ResNet27-A2 reduces the computational cost and the number of parameters by 23.33% and 16.33% respectively. At the same time, the error rate of C-ResNet27-A2 on the datasets MNIST, FashionMnist, CIFAR-10 and SVHN is lower than that of ResNet34*. Since C-ResNet27-A2 has a smaller variance, it can be seen from Table 2 that it also shows stronger stability on the datasets MNIST, FashionMnist, CALTECH-101 and SVHN.

[0089] Third, experiments were conducted on C-ResNet27-B2, which is a variant based on the six-layer Cross-Bottleneck module, in which the second solid line becomes a dashed line for comparison with ResNet50*. C-ResNet27-B2 has 26 convolutional layers, including 3×3 and 1×1 convolutional layers, and 12 cross-jump connections, including 4 solid lines and 8 dashed lines, while ResNet50* has 50 convolutional layers and 16 non-cross-jump connections, including 13 solid lines and 3 dashed lines.

[0090] Compared with ResNet50*, C-ResNet27-B2 has higher accuracy on three datasets and lower accuracy on the other three datasets. Therefore, C-ResNet27-B2 is competitive in classification accuracy. However, due to its significant reduction in computational complexity and parameters, specifically 29.10% and 23.08% respectively, it has a clear advantage over ResNet50*. C-ResNet27-B2 has fewer convolutional layers, fewer total jumps, and more dotted jumps compared to ResNet50*, confirming the conjecture that this embodiment uses cross jumps and dotted jumps instead of solid jumps as much as possible. Through these two methods, this embodiment can increase the density of jumps and maintain feature diversity, thereby reducing the number of convolutional layers and computational complexity without reducing classification performance.

[0091] In summary, compared with ResNet*, Fig.17 , Fig.18 As can be seen from Table 2, C-ResNets provide superior performance in terms of computational cost, number of parameters, and classification accuracy. C-ResNet can be used as a substitute for ResNet in various scenarios*.

[0092] Table 2

[0093]

[0094] In some embodiments, for the setting of jump connections, this embodiment introduces two operations: one is a self-identity mapping, and the corresponding connection is a solid jump line (also called a "solid line"); the other is a 1×1 transformation represented by a dotted jump line (also called a "dotted line"). The 1×1 convolution operation is not only used to match the input-output dimensions, but also helps to enhance feature diversity by increasing the feature transformations involved. In other words, even if the dimensions at both ends are completely matched, it is still beneficial to introduce such a dotted jump line. Compared with ResNets*, C-ResNet has a higher jump line density because it has a cross feature, fewer convolutional layers, and more dotted jump lines, which serves to increase the diversity of features. Therefore, we further discuss here the impact of the number of dotted jump lines on the classification performance of C-ResNets on the datasets CIFAR-10, CIFAR-100, and CALTECH-101.

[0095] For C-ResNet27-A series: Figure 2 The Cross-Block and its variants shown, three (2,2,2,2) networks C-ResNet27-A, C ResNet27-A1 and C-ResNet27-A2, have the same network structure except for the different ratios of solid and dashed lines. The three cross residual networks A, A1 and A2 have three, six and eight dashed line jumps, respectively. Compared with C-ResNet27-A, C-ResNet27-A1 has more dashed line jumps, as shown in Table 3, the classification accuracy on CIFAR-100 and CALTECH-101 is reduced. However, as shown in Table 3, C-ResNet27-A2 has more jumps compared with C-ResNet27-A and C-ResNet27-A1, and the classification accuracy on all three datasets is improved. So arbitrarily incorporating dashed line jumps will not always lead to performance improvement. It can be observed that the number and deployment strategy of dashed lines are important in C-ResNets.

[0096] For the C-ResNet27-B series: This embodiment further verifies the importance of the number of dashed lines through the C-ResNet27-B series. Fig.11The Cross-Bottleneck and its variants shown in the figure, some of the solid line jumps become dotted lines, and the network structures constructed by the four C-ResNet27 (1,1,1,1) networks B, B1, B2 and B3 are the same, but the ratio of solid line and dotted line jumps is different. The four cross residual networks B, B1, B2 and B3 have four, three, eight and twelve dotted line jumps respectively. As can be seen from Table 3, the cross residual network with more dotted line jumps does not have higher classification accuracy on some data sets, which also implies the importance of the dotted line jump structure. The solid line and the dotted line should maintain a certain ratio, and pay attention to adjusting the structure between them.

[0097] Table 3

[0098] params CIFAR-10 CIFAR-100 CALTECH-101 C-ResNet27-A 17.53M 4.31±0.13 21.0±0.29 25.77±0.08 C-ResNet27-A1 17.87M 4.29±0.12 21.35±0.23 26.81±0.92 C-ResNet27-A2 17.87M 4.23±0.1 21±0.12 25.52±0.89 C-ResNet27-B 17.88M 4.14±0.06 20.04±0.18 16.5±0.73 C-ResNet27-B1 16.868M 4.11±0.06 19.96±0.13 16.21±0.45 C-ResNet27-B2 18.26M 4.22±0.15 20.1±0.20 16.34±0.47 C-ResNet27-B3 19.66M 4.26±0.22 20.09±0.24 15.93±0.74 C-ResNet27-C1 16.27M 4.53±0.15 21.58±0.32 16.96±0.49 C-ResNet27-B2 18.26M 4.22±0.15 20.1±0.20 16.34±0.47

[0099] Regarding the trade-off between channels and jumpers, this embodiment uses the Fig.14 The design of the Bottleneck module can easily be Fig.10 A similar design scheme is derived from the three-layer Cross-Bottleneck module in C-ResNet of . However, when building C-ResNet based on this three-layer module, this scheme is prone to cause all jumpers to become dashed jumpers due to channel inconsistency, for example, C-ResNet27-C1. Therefore, this embodiment designs a six-layer Cross-Bottleneck module to maintain a certain ratio between solid lines and dashed lines so that C-ResNet achieves optimal performance. In addition, this embodiment wants to evaluate whether it is feasible to reduce jumpers and increase channels to help enhance feature diversity. Therefore, the following is designed: Fig.11 The six-layer Cross-Bottleneck module shown.

[0100] By comparing and evaluating the performance of C-ResNet27-C1 and C-ResNet27-B2, except for the number of channels and the ratio of solid lines, they have the same network structure, number of convolutional layers, and convolution kernels. C-ResNet27-B2 has fewer total jumps and dashed jumps than C-ResNet27-C1, but it has more channels in the 3×3 convolutional layer. Therefore, this is the trade-off between channels and jumps. It can be seen that by increasing the number of channels and optimizing the configuration strategy of cross jump lines, C-ResNet27-B2 (FlOPs: 0.95G, Parameters: 18.26M) has higher classification accuracy on MNIST, and the parameter complexity and computational cost are higher than C-ResNet27-C1 (FlOPs: 0.88G, Parameters: 16.27M). Although the cost is increased, the classification performance is also improved.

[0101] The embodiment of the present invention provides an image recognition system based on a cross residual network model, comprising:

[0102] An acquisition unit, used for acquiring an image to be recognized;

[0103] A recognition unit, used to recognize the image to be recognized by using a cross residual network model to obtain an image recognition result;

[0104] Wherein, the cross residual network model includes:

[0105] At least one cross residual module, the cross residual module includes a plurality of weight layers and a plurality of jump connection lines, the weight layer includes a convolution layer, a batch normalization layer and an activation function layer, and the activation function layer is arranged after the addition operator; each of the jump connection lines jumps at least one of the weight layers and all the jump connection lines cross jump at least one of the weight layers.

[0106] The contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method.

[0107] The embodiment of the present invention provides an image recognition system based on a cross residual network model, comprising:

[0108] at least one memory for storing a program;

[0109] At least one processor for loading the program to execute Figure 1 The image recognition method based on the cross residual network model shown.

[0110] The contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method.

[0111] An embodiment of the present invention provides a storage medium, wherein a computer executable program is stored, and when the computer executable program is executed by a processor, it is used to implement Figure 1 The image recognition method based on the cross residual network model shown.

[0112] The embodiment of the present invention further provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The image recognition method based on the cross residual network model shown.

[0113] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in the relevant technical field without departing from the purpose of the present invention. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

Claims

1. An image recognition method based on a cross residual network model, characterized in that: The following steps are involved: Obtain an image to be recognized; Using a cross residual network model to identify the image to be identified, to obtain an image recognition result; Wherein, the cross residual network model includes: At least one cross residual module, the cross residual module includes a plurality of weight layers and a plurality of jump connection lines, the weight layer includes a convolution layer, a batch normalization layer and an activation function layer, the activation function layer is arranged after the addition operator; each of the jump connection lines jumps at least one of the weight layers and all the jump connection lines cross jump at least one of the weight layers; Wherein, the cross residual module includes Cross-Block or Cross-Bottleneck; When the cross residual module is the Cross-Block, the cross residual module includes three weight layers and two cross jump connection lines, and the convolution layer of the weight layer is a 3*3 convolution layer; When the cross residual module is the Cross-Bottleneck, the cross residual module includes three weight layers and two cross jump connection lines or includes six weight layers and three cross jump connection lines, and the cross residual module including three weight layers and two cross jump connection lines is used as the third cross residual module, and the third cross residual module includes two 1*1 convolutional layers and one 3*3 convolutional layer; two 1*1 convolutional layers and one 3*3 convolutional layer are alternately connected; the cross residual module including six weight layers and three cross jump connection lines includes two third cross residual modules; two third cross residual modules are connected sequentially.

2. The image recognition method based on the cross residual network model according to claim 1 is characterized in that: When the jump connection line includes a dotted line, the convolution layer of the weight layer includes a 1*1 convolution layer or a 3*3 convolution layer; the weight layer including the 1*1 convolution layer is alternately connected with the weight layer including the 3*3 convolution layer.

3. The image recognition method based on the cross residual network model according to claim 2 is characterized in that: The cross residual module corresponding to the first jump connection line being a dotted line is used as the first cross residual module; the cross residual module corresponding to the second jump connection line being a dotted line is used as the second cross residual module.

4. An image recognition system based on a cross residual network model, characterized in that: include: An acquisition unit, used for acquiring an image to be recognized; A recognition unit, used to recognize the image to be recognized by using a cross residual network model to obtain an image recognition result; Wherein, the cross residual network model includes: At least one cross residual module, the cross residual module includes a plurality of weight layers and a plurality of jump connection lines, the weight layer includes a convolution layer, a batch normalization layer and an activation function layer, and the activation function layer is arranged after the addition operator; each of the jump connection lines jumps at least one of the weight layers and all the jump connection lines cross jump at least one of the weight layers; Wherein, the cross residual module includes Cross-Block or Cross-Bottleneck; When the cross residual module is the Cross-Block, the cross residual module includes three weight layers and two cross jump connection lines, and the convolution layer of the weight layer is a 3*3 convolution layer; When the cross residual module is the Cross-Bottleneck, the cross residual module includes three weight layers and two cross jump connection lines or includes six weight layers and three cross jump connection lines, and the cross residual module including three weight layers and two cross jump connection lines is used as the third cross residual module, and the third cross residual module includes two 1*1 convolutional layers and one 3*3 convolutional layer; two 1*1 convolutional layers and one 3*3 convolutional layer are alternately connected; the cross residual module including six weight layers and three cross jump connection lines includes two third cross residual modules; two third cross residual modules are connected sequentially.

5. An image recognition system based on a cross residual network model, characterized in that: include: at least one memory for storing a program; At least one processor is used to load the program to execute the image recognition method based on the cross residual network model as described in any one of claims 1 to 3.

6. A storage medium, characterized in that: A computer-executable program is stored therein, and when the computer-executable program is executed by a processor, it is used to implement the image recognition method based on the cross residual network model as described in any one of claims 1 to 3.

Images