Image classification method and device

Abstract

The invention provides an image classification method and device. The method comprises the steps that a trained image classification model matched with a target service scene is acquired; wherein the image classification model comprises a convolutional network, an activation function network and a full-connection network from top to bottom, and the convolutional network is obtained by fusing a plurality of parallel feature processing branches and performing re-parameterization; obtaining a to-be-classified image in the target business scene; inputting a to-be-classified image into the image classification model, and processing the to-be-classified image in sequence through the convolutional network, the activation function network and the full-connection network to obtain the probability of at least one prediction category output by the full-connection network; and according to the probability of the at least one prediction category, determining the target category to which the to-be-classified image belongs, so that the image classification model can extract more diversified and deeper feature information from the to-be-classified image, and the accuracy and reliability of image classification are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an image classification method and device. Background Art

[0002] With the rapid development of big data and artificial intelligence technology, image classification technology has been widely used in financial scenarios in various document recognition, risk monitoring, customer service optimization and other aspects. For example, in banking business, a large number of customer information, contracts, invoices and other image files need to be quickly and accurately classified and processed. Image classification technology can automatically identify and classify these image files, greatly improving work efficiency; in customer service, image classification technology is used to identify photos or screenshots uploaded by customers, quickly respond to customer needs, and improve user experience. Therefore, in order to achieve more efficient, safer and more personalized services in the financial field, how to accurately and quickly classify images is very important. Summary of the invention

[0003] The present disclosure provides an image classification method and device to solve one of the technical problems in the related art at least to a certain extent. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image classification method, comprising: obtaining a trained image classification model adapted to a target business scenario; wherein the image classification model comprises, from top to bottom, a convolutional network, an activation function network and a fully connected network, and the convolutional network is obtained by re-parameterizing by fusing a plurality of parallel feature processing branches; obtaining an image to be classified in the target business scenario; inputting the image to be classified into the image classification model, so as to process the image to be classified in turn through the convolutional network, the activation function network and the fully connected network, and obtain the probability of at least one predicted category output by the fully connected network; and determining the target category to which the image to be classified belongs according to the probability of the at least one predicted category.

[0005] According to a second aspect of an embodiment of the present disclosure, an image classification device is provided, comprising: a first acquisition module, for acquiring a trained image classification model adapted to a target business scenario; wherein the image classification model comprises, from top to bottom, a convolutional network, an activation function network and a fully connected network, and the convolutional network is obtained by re-parameterization by fusing multiple parallel feature processing branches; a second acquisition module, for acquiring an image to be classified in the target business scenario; an input module, for inputting the image to be classified into the image classification model, so as to process the image to be classified in turn through the convolutional network, the activation function network and the fully connected network, and obtain the probability of at least one predicted category output by the fully connected network; a determination module, for determining the target category to which the image to be classified belongs according to the probability of the at least one predicted category.

[0006] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image classification method as described in the embodiment of the first aspect of the present disclosure.

[0007] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image classification method as described in the embodiment of the first aspect of the present disclosure.

[0008] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including: a computer program, wherein when the computer program is executed by a processor, the image classification method as described in the embodiment of the first aspect of the present disclosure is implemented.

[0009] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0010] In the technical scheme, the image to be classified is input into the image classification model, and the image to be classified is processed in sequence by the convolutional network, activation function network and fully connected network in the image classification model to obtain the probability of at least one predicted category output by the fully connected network, and the target category to which the image to be classified belongs is determined based on the probability of at least one predicted category. Since the convolutional network in the image classification model is obtained by re-parameterization by fusing multiple parallel feature processing branches, the feature extraction capability of the image classification model is improved, so that the model can extract more diverse and deeper feature information from the image to be classified, thereby improving the accuracy of image classification; the activation function network in the image classification model enables the image classification model to have stronger generalization ability when processing images, so that The image classification model can maintain stable classification performance when facing images of different styles and sources, effectively reducing the risk of misclassification; the fully connected network is used as the output layer of the image classification model to map the feature information extracted by the convolutional network and the activation function network to the category space, and calculate the probability of each category, thereby improving the accuracy and reliability of image classification; among them, the convolution and normalization operations in multiple feature processing branches are integrated into an efficient re-parameterized convolutional layer, and the re-parameterized convolutional layer is further fused to obtain a convolutional network, which reduces the number of layers and parameters of the image classification model, making the image classification model structure more concise, thereby reducing the calculation steps of the image classification model in the reasoning process, making the image classification model more lightweight and efficient in practical applications.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0013] Figure 1 is a flowchart of an image classification method shown in the first embodiment of the present disclosure;

[0014] Figure 2 is a flowchart of an image classification method shown in the second embodiment of the present disclosure;

[0015] Figure 3 is a flowchart of an image classification method shown in the third embodiment of the present disclosure;

[0016] Figure 4 It is a schematic diagram of the principle of feature processing branch re-parameterization shown in an embodiment of the present disclosure;

[0017] Figure 5is a flowchart of an image classification method shown in the fourth embodiment of the present disclosure;

[0018] Figure 6 It is a schematic diagram of the principle of feature processing branch re-parameterization shown in an embodiment of the present disclosure;

[0019] Figure 7 is a flowchart of an image classification method shown in the fifth embodiment of the present disclosure;

[0020] Figure 8 It is a schematic diagram of the principle of feature processing branch re-parameterization shown in an embodiment of the present disclosure;

[0021] Fig. 9 is a structural schematic diagram of an image classification device shown in the sixth embodiment of the present disclosure;

[0022] Fig.10 It is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0025] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0026] It should be understood that in order to improve the efficiency of image classification, lightweight models are used to classify images in related technologies. The lightweight model is mainly implemented in the following two ways: first, low-precision quantization technology of model parameters; second, knowledge distillation technology; however, the low-precision quantization technology of model parameters may cause accuracy loss due to information loss and quantization error; for some specific tasks, knowledge distillation cannot transfer the effective knowledge of the teacher model to the student model, and the distillation efficiency is related to the model selection, and it cannot be plug-and-play.

[0027] Therefore, in order to solve the above problems, the present invention proposes an image classification method and device.

[0028] The image classification method and apparatus according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0029] Figure 1 It is a flowchart of the image classification method shown in the first embodiment of the present disclosure.

[0030] like Figure 1 As shown, the image classification method includes the following steps:

[0031] Step 101: Obtain a trained image classification model adapted to the target business scenario.

[0032] Among them, the image classification model includes a convolutional network, an activation function network and a fully connected network from top to bottom. The convolutional network is obtained by re-parameterizing by fusing multiple parallel feature processing branches.

[0033] In the embodiments of the present disclosure, the target business scenario refers to a specific business demand in the financial field, such as automatic recognition and classification of image files, classification and recognition of image information in financial reports (such as financial charts, trend charts, etc.), classification and recognition of identity documents provided by customers (such as identity cards, passports, etc.), etc. Since each business scenario has its own unique image features and classification requirements, it is necessary to design and train the image classification model in a targeted manner to achieve optimal performance, so that different business scenarios correspond to different image classification models. Therefore, when obtaining the image classification model, it is necessary to consider the specific needs of the target business scenario and select a trained image classification model that is adapted to the target business scenario.

[0034] It should be noted that the trained image classification model adapted to the target business scenario includes, from top to bottom, a convolutional network, an activation function network and a fully connected network. The convolutional network is obtained by re-parameterizing by fusing multiple parallel feature processing branches; among them, the convolutional network is used to extract image features; the activation function network is used to introduce nonlinear transformations to enhance the expressiveness and adaptability of the image classification model; the fully connected network is used to integrate the features extracted by the convolutional network and the activation function network, and output the final classification results.

[0035] In the embodiment of the present disclosure, the image classification model is trained using the following steps:

[0036] (1) Obtain a sample image and obtain the labeled category of the sample image;

[0037] In the disclosed embodiment, image data is collected from various sources, and each image is assigned one or more labels indicating the category to which it belongs.

[0038] (2) Using an image classification model to classify the sample image to obtain the predicted category to which the sample image belongs;

[0039] In the disclosed embodiment, a sample image is input into an image classification model, which outputs a probability value for each category, and based on the output probability values, the category with the highest probability is selected as the predicted category of the sample image;

[0040] (3) The image classification model is trained based on the difference between the predicted category and the labeled category.

[0041] In the disclosed embodiment, a loss value is calculated by comparing the difference between the predicted category and the actual labeled category (a loss function such as cross entropy loss is usually used to quantify this difference). Then, a back propagation algorithm is used to adjust the parameters in the model (such as weights and biases) according to the loss value to minimize the difference between the predicted category and the labeled category. This is repeated until the performance of the model on the training set reaches the expected level or the loss value converges, thereby completing the training of the image classification model.

[0042] Step 102: Obtain an image to be classified in a target business scenario.

[0043] In the disclosed embodiments, when the target business scenario is to classify and identify image information in financial statements, financial charts, trend charts, etc. to be classified are obtained; in the target business scenario, identity documents provided to customers are classified and identified, and identity cards, passports, etc. to be classified are obtained.

[0044] Step 103: input the image to be classified into the image classification model, so as to process the image to be classified in sequence through the convolutional network, the activation function network and the fully connected network, and obtain the probability of at least one predicted category output by the fully connected network.

[0045] In the disclosed embodiment, when the image to be classified is input into the image classification model, the image to be classified is processed by the convolutional network, the activation function network and the fully connected network in sequence; wherein, the convolutional network first performs feature extraction on the image to be classified, and captures local features in the image, such as edges, textures and shapes, through convolution operations, and these features are then nonlinearly transformed through the activation function network to enhance the expressive power and adaptability of the model. In the activation function network, the activation function (such as ReLU, etc.) performs nonlinear processing on the features output by the convolutional network, so that the model can learn and represent more complex functional relationships; finally, the features after feature extraction and nonlinear transformation are sent to the fully connected network. The main task of the fully connected network is to integrate these features and map the integrated features to predefined categories through the operation of weight matrices and bias vectors. The fully connected network predicts the probability of each category, which is used to indicate the possibility that the image to be classified belongs to each category.

[0046] Step 104: Determine the target category to which the image to be classified belongs based on the probability of at least one predicted category.

[0047] As an example, the predicted category with the highest probability is selected as the target category of the image to be classified.

[0048] As another example, a pre-set probability threshold is obtained. When there is a probability exceeding the probability threshold in the probability of at least one predicted category, the predicted category corresponding to the probability exceeding the probability threshold is used as the target category; when the predicted probabilities of all categories are lower than the probability threshold, the image to be classified can be classified into other categories.

[0049] In summary, by inputting the image to be classified into the image classification model, the image to be classified is processed in turn by the convolutional network, activation function network and fully connected network in the image classification model, and the probability of at least one predicted category output by the fully connected network is obtained. Based on the probability of at least one predicted category, the target category to which the image to be classified belongs is determined. Since the convolutional network in the image classification model is obtained by re-parameterization by fusing multiple parallel feature processing branches, the feature extraction ability of the image classification model is improved, so that the model can extract more diverse and deeper feature information from the image to be classified, thereby improving the accuracy of image classification; the activation function network in the image classification model enables the image classification model to have stronger generalization ability when processing images, so that the image classification model can maintain stable classification performance when facing images of different styles and sources, effectively reducing the risk of misclassification; the fully connected network is used as the output layer of the image classification model to map the feature information extracted by the convolutional network and the activation function network to the category space, and calculate the probability of each category, thereby improving the accuracy and reliability of image classification.

[0050] Based on any of the above embodiments, the convolutional network in the image classification model is obtained by re-parameterizing by fusing multiple parallel feature processing branches. Figure 2 The embodiments are described in detail.

[0051] Figure 2 It is a flowchart of the image classification method shown in the second embodiment of the present disclosure.

[0052] like Figure 2 As shown, the convolutional network is generated using the following steps 201 to 202:

[0053] Step 201, for any feature processing branch among the multiple feature processing branches, each network layer in any feature processing branch is reparameterized to obtain a reparameterized convolution layer corresponding to any feature processing branch.

[0054] In order to improve the performance and classification efficiency of the image classification model, for any feature processing branch in the multiple feature processing branches in the image classification model, all network layers in the feature processing branch are identified. These network layers may include convolutional layers, batch normalization layers, activation function layers, etc., and multiple network layers in the feature processing branch are re-parameterized. The goal of parameterization is to merge the functions of multiple network layers into a more efficient convolutional layer, which can replace the original multiple network layers, simplify the model structure and improve computational efficiency.

[0055] Step 202, reparameterize each reparameterized convolutional layer to obtain a convolutional network.

[0056] In order to further simplify the network structure, as a possible implementation method, all the re-parameterized convolutional layers are traversed and their parameters and operations are further integrated to obtain a convolutional network with simple structure and efficient calculation. This convolutional network can quickly give classification results in the inference stage while maintaining performance comparable to the original network.

[0057] In the disclosed embodiment, the convolution weights and biases in each heavily parameterized convolution layer are obtained; the convolution weights in each heavily parameterized convolution layer are added together to obtain the convolution weights of the convolution network; the biases in each heavily parameterized convolution layer are added together to obtain the bias of the convolution network.

[0058] That is to say, for each reparameterized convolutional layer, you first need to extract its convolution weights and bias; next, accumulate the convolution weights of all reparameterized convolutional layers. It should be noted that since the number of output channels and input channels of each convolutional layer may be different, it is necessary to ensure that these convolution weights are dimensionally compatible before accumulation; similarly, accumulate the biases of all reparameterized convolutional layers, and the accumulated result will form a new bias tensor, which will be used as the bias of the entire convolutional network.

[0059] For example, the convolutional network is obtained by fusing three parallel feature processing branches and re-parameterizing them. The convolution weight of the re-parameterized convolutional layer corresponding to the first feature processing branch is W 1 , with a bias of b 1 , the convolution weight of the re-parameterized convolution layer corresponding to the second feature processing branch is W 2 , with a bias of b 2 , the convolution weight of the re-parameterized convolution layer corresponding to the third feature processing branch is W 3 , with a bias of b 3 , Convolutional Network Conv total It is expressed by the following formula:

[0060] Conv total =(W 1 +W 2 +W 3 )*x+(b 1 +b 2 +b 3 ).

[0061] In summary, by reparameterizing each network layer in any feature processing branch of multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to any feature processing branch; each reparameterized convolutional layer is reparameterized to obtain a convolutional network, thereby reparameterizing multiple feature processing branches and constructing the final convolutional network, which can not only simplify the network structure and improve computational efficiency, but also improve the flexibility and scalability of the network.

[0062] On the basis of any of the above embodiments, the multiple feature processing branches include: a first feature processing branch, a second feature processing branch and a third feature processing branch, and each network layer in the first feature processing branch, the second feature processing branch and the third feature processing branch is reparameterized to obtain a corresponding reparameterized convolution layer, and each reparameterized convolution layer is reparameterized to obtain a convolutional network. Figure 3 The embodiments are described in detail.

[0063] Figure 3 It is a flowchart of the image classification method shown in the third embodiment of the present disclosure.

[0064] like Figure 3 As shown, the convolutional network is generated using the following steps:

[0065] Step 301, reparameterize the first convolutional layer and the first batch normalization layer in the first feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the first feature processing branch.

[0066] In the disclosed embodiment, the first feature processing branch includes a first convolutional layer (e.g., a 3*3 convolutional layer) and a first batch normalization (Batch Normalization, BN for short) layer. The first convolutional layer and the first batch normalization layer are fused into one convolutional layer. The convolutional layer contains the recalculated parameters and can directly replace the first convolutional layer and the first batch normalization layer. The convolutional layer obtained by fusing the first convolutional layer and the first batch normalization layer is used as the re-parameterized convolutional layer corresponding to the first feature processing branch.

[0067] Step 302: reparameterize the second convolutional layer and the second batch normalization layer in the second feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the second feature processing branch.

[0068] Among them, the convolution kernel sizes of the first convolution layer and the second convolution layer are different.

[0069] In the embodiment of the present disclosure, the second feature processing branch includes a second convolutional layer (e.g., a 1*1 convolutional layer) and a second batch normalization layer. The second convolutional layer and the second batch normalization layer are fused into a new convolutional layer. The convolutional layer contains the recalculated parameters and can directly replace the second convolutional layer and the second batch normalization layer. The convolutional layer obtained by fusing the second convolutional layer and the second batch normalization layer is used as the re-parameterized convolutional layer corresponding to the second feature processing branch.

[0070] Step 303: reparameterize the third batch normalization layer in the third feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the third feature processing branch.

[0071] In the embodiment of the present disclosure, during forward propagation, the input data of the BN layer is scaled and standardized to have zero mean and unit variance to accelerate the convergence of the network. During backward propagation, the weight matrix of the BN layer is decomposed into two parts, one part is used to calculate the output of the BN layer, and the other part is used to update the weights of other layers, which is specifically expressed as the following formula:

[0072] y = E x;

[0073]

[0074] Among them, y is the output after BN layer processing, E is the scaling factor, and x is the input data after normalization. is the weight of the BN layer, b 3 is the bias term of the BN layer.

[0075] Step 304, reparameterize each reparameterized convolutional layer to obtain a convolutional network.

[0076] For example, Figure 4 As shown in the figure, the convolutional network of the image classification model is obtained by re-parameterizing by fusing three parallel feature processing branches. The first feature processing branch includes a 3*3 convolution layer and a BN layer, the second feature processing branch includes a 1*1 convolution layer and a BN layer, and the third feature processing branch includes a BN layer. Then, the 3*3 convolution layer and the BN layer of the first feature processing branch are fused, which is specifically expressed as the following formula:

[0077]

[0078] Conv 1 =W 1 ·x+b 1 ;

[0079] Among them, x represents the input data, represents the weight of the 3*3 convolutional layer, δ 2 represents the variance of the BN layer, ε represents a small constant, r represents the scaling factor of the BN layer, and β represents the bias of the BN layer. represents the weight of the BN layer, b 1 represents the bias of the BN layer, W 1 represents the weight of the convolutional layer obtained by fusing the 3*3 convolutional layer and the BN layer, b 1 Represents the bias of the convolutional layer obtained by fusing the 3*3 convolutional layer and the BN layer;

[0080] Then, the 1*1 convolution layer and BN layer of the second feature processing branch are fused, which is specifically expressed as the following formula:

[0081] Conv 3*3 =Conv 1*1 ·weight·Pad(1,1,1,1);

[0082]

[0083] Conv 2 =W 2 ·x+b 2 ;

[0084] in, represents the weight matrix of the convolutional layer in the second feature processing branch, represents the weight matrix of the batch normalization layer in the second feature processing branch, Conv 2 represents the re-parameterized convolutional layer corresponding to the second feature processing branch, W 2 represents the weight of the re-parameterized convolutional layer corresponding to the second feature processing branch, b 2 Represents the bias of the re-parameterized convolutional layer corresponding to the second feature processing branch;

[0085] Then, the BN layer in the third feature processing branch is reparameterized, which is specifically expressed as the following formula:

[0086] y = E x;

[0087]

[0088] Among them, y is the output after BN layer processing, E is the scaling factor, and x is the input data after normalization. is the weight of the BN layer, b 3 is the bias term of the BN layer;

[0089] Finally, the weights and biases of the three feature processing branches are added together, and the new weights and biases are reassigned to a new convolution, which is specifically expressed as the following formula:

[0090] Conv total =(W 1 +W 2 +W 3 )*x+(b 1 +b 2 +b 3 );

[0091] Among them, W 1 , b 1 are the convolution weights and biases of the re-parameterized convolutional layer corresponding to the first feature processing branch, W 2 , b 2 are the convolution weights and biases of the re-parameterized convolutional layer corresponding to the second feature processing branch, W 3 , b 3 are the convolution weights and biases of the re-parameterized convolutional layer corresponding to the third feature processing branch, Conv total Represents a convolutional network.

[0092] It should be noted that the embodiment of the present disclosure does not limit the execution sequence of steps 301, 302 and 303. Steps 301, 302 and 303 may be executed sequentially or in parallel. Figure 3 It is merely taken as an example that step 302 is executed after step 301 and step 303 is executed after step 302 .

[0093] In summary, the first convolution layer and the first batch normalization layer in the first feature processing branch among multiple feature processing branches are reparameterized to obtain the reparameterized convolution layer corresponding to the first feature processing branch; the second convolution layer and the second batch normalization layer in the second feature processing branch among multiple feature processing branches are reparameterized to obtain the reparameterized convolution layer corresponding to the second feature processing branch; the third batch normalization layer in the third feature processing branch among multiple feature processing branches is reparameterized to obtain the reparameterized convolution layer corresponding to the third feature processing branch; each reparameterized convolution layer is reparameterized to obtain a convolution network, thereby integrating the convolution and normalization operations in multiple feature processing branches into efficient reparameterized convolution layers, and further fusing the reparameterized convolution layers to obtain the convolution network, thereby reducing the number of layers and parameters of the image classification model, making the structure of the image classification model simpler, and making the image classification model more lightweight and efficient in practical applications.

[0094] On the basis of any of the above embodiments, the multiple feature processing branches include: a first feature processing branch, a second feature processing branch and a third feature processing branch, and each network layer in the first feature processing branch, the second feature processing branch and the third feature processing branch is reparameterized to obtain a corresponding reparameterized convolution layer, and each reparameterized convolution layer is reparameterized to obtain a convolutional network. Figure 5 The embodiments are described in detail.

[0095] Figure 5 It is a flowchart of the image classification method shown in the fourth embodiment of the present disclosure.

[0096] like Figure 5 As shown, the convolutional network is generated using the following steps:

[0097] Step 501, reparameterize the third convolution layer and the fourth batch normalization layer in the first feature processing branch among the multiple feature processing branches to obtain a reparameterized convolution layer corresponding to the first feature processing branch.

[0098] In the disclosed embodiment, the first feature processing branch includes a third convolutional layer (e.g., a 3*3 convolutional layer) and a fourth batch normalization layer. The third convolutional layer and the fourth batch normalization layer are fused into one convolutional layer. The convolutional layer contains the recalculated parameters and can directly replace the third convolutional layer and the fourth batch normalization layer. The convolutional layer obtained by fusing the third convolutional layer and the fourth batch normalization layer is used as the re-parameterized convolutional layer corresponding to the first feature processing branch.

[0099] Step 502: reparameterize the fourth convolutional layer and the fifth batch normalization layer in the second feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the second feature processing branch.

[0100] Among them, the convolution kernel sizes of the third convolution layer and the fourth convolution layer are different.

[0101] In the disclosed embodiment, the second feature processing branch includes a fourth convolutional layer (e.g., a 1*1 convolutional layer) and a fifth batch normalization layer. The fourth convolutional layer and the fifth batch normalization layer are fused into one convolutional layer. The convolutional layer contains the recalculated parameters and can directly replace the fourth convolutional layer and the fifth batch normalization layer. The convolutional layer obtained by fusing the fourth convolutional layer and the fifth batch normalization layer is used as the re-parameterized convolutional layer corresponding to the second feature processing branch.

[0102] Step 503: reparameterize the fifth convolution layer and the sixth batch normalization layer in the third feature processing branch among the multiple feature processing branches to obtain a reparameterized convolution layer corresponding to the third feature processing branch.

[0103] Among them, the convolution kernel sizes of the fourth convolution layer and the fifth convolution layer are the same.

[0104] In the disclosed embodiment, the third feature processing branch includes a fifth convolutional layer (e.g., a 3*1 convolutional layer) and a sixth batch normalization layer. The fifth convolutional layer and the sixth batch normalization layer are fused into one convolutional layer. The convolutional layer contains the recalculated parameters and can directly replace the fifth convolutional layer and the sixth batch normalization layer. The convolutional layer obtained by fusing the fifth convolutional layer and the sixth batch normalization layer is used as the re-parameterized convolutional layer corresponding to the third feature processing branch.

[0105] Step 504, reparameterize each reparameterized convolutional layer to obtain a convolutional network.

[0106] For example, Figure 6 As shown, the convolutional network of the image classification model is obtained by re-parameterizing by fusing three parallel feature processing branches. The first feature processing branch includes a 3*3 convolutional layer and a BN layer, the second feature processing branch includes a 1*3 convolutional layer and a BN layer, and the third feature processing branch includes a 3*1 convolutional layer and a BN layer. Then, the 3*3 convolutional layer and the BN layer in the first feature processing branch are fused to obtain the re-parameterized convolutional layer corresponding to the first feature processing branch, the 1*3 convolutional layer and the BN layer in the second feature processing branch are fused to obtain the re-parameterized convolutional layer corresponding to the second feature processing branch, and the 3*1 convolutional layer and the BN layer in the third feature processing branch are fused to obtain the re-parameterized convolutional layer corresponding to the third feature processing branch. Then, the re-parameterized convolutional layer corresponding to the first feature processing branch, the re-parameterized convolutional layer corresponding to the second feature processing branch, and the re-parameterized convolutional layer corresponding to the third feature processing branch are fused to obtain the convolutional network. Among them, Figure 6 I represents input data, γ 1 , β 1 , δ 1 and u 1 They represent the bias, scaling factor, standard deviation and mean of the BN layer in the first feature processing branch, respectively. 2 , β 2 , δ 2 and u 2 They represent the bias, scaling factor, standard deviation and mean of the BN layer in the second feature processing branch. Similarly, γ 3 , β 3 , δ 3 and u 3 They respectively represent the bias, scaling factor, standard deviation and mean of the BN layer in the third feature processing branch, W represents the weight of the convolutional network, and B represents the bias of the convolutional network.

[0107] It should be noted that the embodiment of the present disclosure does not limit the execution sequence of steps 501, 502 and 503. Steps 501, 502 and 503 may be executed sequentially or in parallel. Figure 5 It is merely taken as an example that step 502 is executed after step 501 and step 503 is executed after step 502 .

[0108] In summary, the third convolutional layer and the fourth batch normalization layer in the first feature processing branch among multiple feature processing branches are reparameterized to obtain the reparameterized convolutional layer corresponding to the first feature processing branch; the fourth convolutional layer and the fifth batch normalization layer in the second feature processing branch among multiple feature processing branches are reparameterized to obtain the reparameterized convolutional layer corresponding to the second feature processing branch; the fifth convolutional layer and the sixth batch normalization layer in the third feature processing branch among multiple feature processing branches are reparameterized to obtain the reparameterized convolutional layer corresponding to the third feature processing branch; each reparameterized convolutional layer is reparameterized to obtain a convolutional network, thereby, by gradually reparameterizing the convolutional layers and batch normalization layers in multiple feature processing branches, and finally integrating these reparameterized convolutional layers into a convolutional network, the reparameterized convolutional network has higher accuracy and stronger generalization ability, improves the accuracy of the image classification model, reduces the calculation steps of the image classification model in the reasoning process, and thus improves the image classification efficiency.

[0109] On the basis of any of the above embodiments, the multiple feature processing branches include: a first feature processing branch, a second feature processing branch, a third feature processing branch and a fourth feature processing branch, and each network layer in the first feature processing branch, the second feature processing branch, the third feature processing branch and the fourth feature processing branch is reparameterized to obtain a corresponding reparameterized convolution layer, and each reparameterized convolution layer is reparameterized to obtain a convolutional network. Figure 7 The embodiments are described in detail.

[0110] Figure 7 It is a flowchart of the image classification method shown in the fifth embodiment of the present disclosure.

[0111] like Figure 7 As shown, the convolutional network is generated using the following steps:

[0112] Step 701, reparameterize the sixth convolution layer and the seventh batch normalization layer in the first feature processing branch among the multiple feature processing branches to obtain a reparameterized convolution layer corresponding to the first feature processing branch.

[0113] In the embodiment of the present disclosure, the first feature processing branch includes a sixth convolutional layer (e.g., a 3*3 convolutional layer) and a seventh batch normalization layer. The sixth convolutional layer and the seventh batch normalization layer are fused into one convolutional layer. The convolutional layer contains the recalculated parameters and can directly replace the sixth convolutional layer and the seventh batch normalization layer. The convolutional layer obtained by fusing the sixth convolutional layer and the seventh batch normalization layer is used as the re-parameterized convolutional layer corresponding to the first feature processing branch.

[0114] Step 702: reparameterize the seventh convolutional layer and the eighth batch normalization layer in the second feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the second feature processing branch.

[0115] Among them, the convolution kernel size of the seventh convolution layer is smaller than the convolution kernel size of the sixth convolution layer.

[0116] In the disclosed embodiment, the second feature processing branch includes a seventh convolutional layer (e.g., a 1*1 convolutional layer) and a seventh batch normalization layer. The seventh convolutional layer and the eighth batch normalization layer are fused into one convolutional layer. The convolutional layer contains the recalculated parameters and can directly replace the seventh convolutional layer and the eighth batch normalization layer. The convolutional layer obtained by fusing the seventh convolutional layer and the eighth batch normalization layer is used as the re-parameterized convolutional layer corresponding to the second feature processing branch.

[0117] Step 703, reparameterize the eighth convolutional layer, the ninth batch normalization layer, the ninth convolutional layer and the tenth batch normalization layer in the third feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the third feature processing branch.

[0118] Among them, the convolution kernel sizes of the ninth convolution layer and the sixth convolution layer are the same.

[0119] In the embodiment of the present disclosure, the third feature processing branch includes an eighth convolutional layer (1*1 convolutional layer), a ninth batch normalization layer, a ninth convolutional layer (3*3 convolutional layer) and a tenth batch normalization layer. By fusing the eighth convolutional layer, the ninth batch normalization layer, the ninth convolutional layer and the tenth batch normalization layer, a re-parameterized convolutional layer corresponding to the third feature processing branch is generated.

[0120] In the present disclosed embodiment, the eighth convolutional layer and the ninth batch normalization layer are reparameterized to obtain a first reparameterized convolutional sublayer; the ninth convolutional layer and the tenth batch normalization layer are reparameterized to obtain a second reparameterized convolutional sublayer; the first reparameterized convolutional sublayer and the second reparameterized convolutional sublayer are reparameterized to obtain a reparameterized convolutional layer corresponding to the third feature processing branch.

[0121] Step 704, reparameterize the tenth convolutional layer, the eleventh batch normalization layer, the first pooling layer, and the twelfth batch normalization layer in the fourth feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the fourth feature processing branch.

[0122] Among them, the convolution kernel sizes of the tenth convolutional layer, the eighth convolutional layer, and the seventh convolutional layer are the same.

[0123] In the embodiment of the present disclosure, the fourth feature processing branch includes the tenth convolution layer (1*1 convolution layer), the eleventh batch normalization layer, the first pooling layer and the twelfth batch normalization layer. By fusing the tenth convolution layer, the eleventh batch normalization layer, the first pooling layer and the twelfth batch normalization layer, the re-parameterized convolution layer corresponding to the fourth feature processing branch is generated.

[0124] In the disclosed embodiment, the tenth convolutional layer and the eleventh batch normalization layer are reparameterized to obtain a third parameterized convolutional sublayer; the first pooling layer and the twelfth batch normalization layer are reparameterized to obtain a fourth parameterized convolutional sublayer; the third parameterized convolutional sublayer and the fourth parameterized convolutional sublayer are reparameterized to obtain a reparameterized convolutional layer corresponding to the fourth feature processing branch.

[0125] Step 705, reparameterize each reparameterized convolutional layer to obtain a convolutional network.

[0126] For example, Figure 8As shown in the figure, the convolutional network of the image classification model is obtained by reparameterizing by fusing four parallel feature processing branches. The first feature processing branch includes a 3*3 convolution layer and a BN layer, the second feature processing branch includes a 1*1 convolution layer and a BN layer, the third feature processing branch includes a 1*1 convolution layer, a BN layer and a 3*3 convolution layer and a BN layer, and the fourth feature processing branch includes a 1*1 convolution layer, a BN layer, an average pooling layer and a BN layer. Then, the 3*3 convolution layer and the BN layer in the first feature processing branch are fused to obtain the reparameterized convolution layer corresponding to the first feature processing branch, the 1x1 convolution in the second feature processing branch is padded (pad) into a 3x3 convolution, and then fused with the BN layer in the second feature processing branch to obtain the reparameterized convolution layer corresponding to the second feature processing branch; the 1*1 convolution layer in the third feature processing branch is fused with the BN layer , obtain the first parameterized convolution sublayer corresponding to the third feature processing branch, fuse the 3*3 convolution layer and the BN layer in the third feature processing branch to obtain the second parameterized convolution sublayer corresponding to the third feature processing branch; then, fuse the first parameterized convolution sublayer and the second parameterized convolution sublayer to obtain the re-parameterized convolution layer corresponding to the third feature processing branch; then, fuse the 1*1 convolution layer and the BN layer in the fourth feature processing branch to obtain the third parameterized convolution sublayer, and fuse the average pooling layer and the BN layer in the fourth feature processing branch to obtain the fourth parameterized convolution sublayer, fuse the third parameterized convolution sublayer and the fourth parameterized convolution sublayer to obtain the re-parameterized convolution layer corresponding to the fourth feature processing branch, finally, finally add the weights of the four feature processing branches and add the biases to obtain the weights and biases of the convolution network.

[0127] It should be noted that the embodiment of the present disclosure does not limit the execution sequence of steps 701, 702 and 703. Steps 701, 702 and 703 may be executed sequentially or in parallel. Figure 7 It is merely taken as an example that step 702 is executed after step 701 and step 703 is executed after step 702 .

[0128] In summary, the sixth convolutional layer and the seventh batch normalization layer in the first feature processing branch among the multiple feature processing branches are reparameterized to obtain the reparameterized convolutional layer corresponding to the first feature processing branch; the seventh convolutional layer and the eighth batch normalization layer in the second feature processing branch among the multiple feature processing branches are reparameterized to obtain the reparameterized convolutional layer corresponding to the second feature processing branch; the eighth convolutional layer, the ninth batch normalization layer, the ninth convolutional layer and the tenth batch normalization layer in the third feature processing branch among the multiple feature processing branches are reparameterized to obtain the reparameterized convolutional layer corresponding to the third feature processing branch; The tenth convolutional layer, the eleventh batch normalization layer, the first pooling layer and the twelfth batch normalization layer in the fourth feature processing branch of the multiple feature processing branches are reparameterized to obtain the reparameterized convolutional layer corresponding to the fourth feature processing branch. Thus, by gradually reparameterizing the convolutional layers and batch normalization layers in the multiple feature processing branches, and finally integrating the reparameterized convolutional layers into a convolutional network, the reparameterized convolutional network has higher accuracy and stronger generalization ability, which improves the accuracy of the image classification model, reduces the calculation steps of the image classification model in the reasoning process, and thus improves the image classification efficiency.

[0129] Corresponding to the image classification method provided in the above embodiment, the present disclosure also provides an image classification device. Since the image classification device provided in the embodiment of the present disclosure corresponds to the image classification method provided in the above embodiment, the implementation method of the image classification method is also applicable to the image classification device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0130] Fig. 9 It is a schematic diagram of the structure of an image classification device shown in the sixth embodiment of the present disclosure.

[0131] like Fig. 9 As shown, the image classification device 900 includes: a first acquisition module 910 , a second acquisition module 920 , an input module 930 and a determination module 940 .

[0132] Among them, the first acquisition module 910 is used to obtain a trained image classification model adapted to the target business scenario; wherein the image classification model includes a convolutional network, an activation function network and a fully connected network from top to bottom, and the convolutional network is obtained by re-parameterization by fusing multiple parallel feature processing branches; the second acquisition module 920 is used to obtain the image to be classified in the target business scenario; the input module 930 is used to input the image to be classified into the image classification model, so as to process the image to be classified in turn through the convolutional network, the activation function network and the fully connected network, and obtain the probability of at least one predicted category output by the fully connected network; the determination module 940 is used to determine the target category to which the image to be classified belongs based on the probability of at least one predicted category.

[0133] As a possible implementation of the embodiment of the present disclosure, the convolutional network is generated using the following modules: a third acquisition module and a processing module.

[0134] Among them, the third acquisition module is used to reparameterize each network layer in any feature processing branch of multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to any feature processing branch; the processing module is used to reparameterize each reparameterized convolutional layer to obtain a convolutional network.

[0135] As a possible implementation method of the embodiment of the present disclosure, multiple feature processing branches include: a first feature processing branch, a second feature processing branch and a third feature processing branch; a processing module, used to reparameterize the first convolution layer and the first batch normalization layer in the first feature processing branch among the multiple feature processing branches to obtain the reparameterized convolution layer corresponding to the first feature processing branch; reparameterize the second convolution layer and the second batch normalization layer in the second feature processing branch among the multiple feature processing branches to obtain the reparameterized convolution layer corresponding to the second feature processing branch; wherein the convolution kernel sizes of the first convolution layer and the second convolution layer are different; reparameterize the third batch normalization layer in the third feature processing branch among the multiple feature processing branches to obtain the reparameterized convolution layer corresponding to the third feature processing branch.

[0136] As a possible implementation method of an embodiment of the present disclosure, a processing module is used to obtain the convolution weights and biases in each re-parameterized convolution layer; add the convolution weights in each re-parameterized convolution layer to obtain the convolution weights of the convolution network; add the biases in each re-parameterized convolution layer to obtain the bias of the convolution network.

[0137] As a possible implementation method of the embodiment of the present disclosure, multiple feature processing branches include: a first feature processing branch, a second feature processing branch and a third feature processing branch, and a processing module is used to reparameterize the third convolution layer and the fourth batch normalization layer in the first feature processing branch of the multiple feature processing branches to obtain the reparameterized convolution layer corresponding to the first feature processing branch; reparameterize the fourth convolution layer and the fifth batch normalization layer in the second feature processing branch of the multiple feature processing branches to obtain the reparameterized convolution layer corresponding to the second feature processing branch; wherein the convolution kernel sizes of the third convolution layer and the fourth convolution layer are different; reparameterize the fifth convolution layer and the sixth batch normalization layer in the third feature processing branch of the multiple feature processing branches to obtain the reparameterized convolution layer corresponding to the third feature processing branch; wherein the convolution kernel sizes of the fourth convolution layer and the fifth convolution layer are the same.

[0138] As a possible implementation of the embodiment of the present disclosure, the multiple feature processing branches include: a first feature processing branch, a second feature processing branch, a third feature processing branch and a fourth feature processing branch; a processing module, which is used to reparameterize the sixth convolutional layer and the seventh batch normalization layer in the first feature processing branch of the multiple feature processing branches to obtain the reparameterized convolutional layer corresponding to the first feature processing branch; reparameterize the seventh convolutional layer and the eighth batch normalization layer in the second feature processing branch of the multiple feature processing branches to obtain the reparameterized convolutional layer corresponding to the second feature processing branch; wherein the convolution kernel size of the seventh convolutional layer is smaller than that of the sixth convolutional layer The convolution kernel size of the convolution layer is as follows: the eighth convolution layer, the ninth batch normalization layer, the ninth convolution layer and the tenth batch normalization layer in the third feature processing branch among the multiple feature processing branches are reparameterized to obtain the reparameterized convolution layer corresponding to the third feature processing branch; wherein the convolution kernel sizes of the ninth convolution layer and the sixth convolution layer are the same; the tenth convolution layer, the eleventh batch normalization layer, the first pooling layer and the twelfth batch normalization layer in the fourth feature processing branch among the multiple feature processing branches are reparameterized to obtain the reparameterized convolution layer corresponding to the fourth feature processing branch; wherein the convolution kernel sizes of the tenth convolution layer, the eighth convolution layer and the seventh convolution layer are the same.

[0139] As a possible implementation method of an embodiment of the present disclosure, a processing module is used to reparameterize the eighth convolutional layer and the ninth batch normalization layer to obtain a first reparameterized convolutional sublayer; reparameterize the ninth convolutional layer and the tenth batch normalization layer to obtain a second reparameterized convolutional sublayer; reparameterize the first reparameterized convolutional sublayer and the second reparameterized convolutional sublayer to obtain a reparameterized convolutional layer corresponding to the third feature processing branch.

[0140] As a possible implementation method of the embodiment of the present disclosure, the processing module is used to reparameterize the tenth convolutional layer and the eleventh batch normalization layer to obtain a third parameterized convolutional sublayer; reparameterize the first pooling layer and the twelfth batch normalization layer to obtain a fourth parameterized convolutional sublayer; reparameterize the third parameterized convolutional sublayer and the fourth parameterized convolutional sublayer to obtain a reparameterized convolutional layer corresponding to the fourth feature processing branch.

[0141] As a possible implementation manner of the embodiment of the present disclosure, the image classification model is trained using the following modules: a fourth acquisition module, a classification module, and a training module.

[0142] Among them, the fourth acquisition module is used to obtain a sample image and obtain the labeled category of the sample image; the classification module is used to classify the sample image using an image classification model to obtain the predicted category to which the sample image belongs; the training module is used to train the image classification model according to the difference between the predicted category and the labeled category.

[0143] The image classification device of the disclosed embodiment inputs the image to be classified into the image classification model, processes the image to be classified in turn through the convolution network, activation function network and fully connected network in the image classification model, obtains the probability of at least one predicted category output by the fully connected network, and determines the target category to which the image to be classified belongs based on the probability of at least one predicted category. Since the convolution network in the image classification model is obtained by re-parameterization by fusing multiple parallel feature processing branches, the feature extraction capability of the image classification model is improved, so that the model can extract more diverse and deeper feature information from the image to be classified, thereby improving the accuracy of image classification; the activation function network in the image classification model enables the image classification model to have stronger generalization ability when processing images, so that the image classification model can maintain stable classification performance when facing images of different styles and sources, effectively reducing the risk of misclassification; the fully connected network is used as the output layer of the image classification model to map the feature information extracted by the convolution network and the activation function network to the category space, and calculate the probability of each category, thereby improving the accuracy and reliability of image classification.

[0144] In an exemplary embodiment, an electronic device is also provided.

[0145] Among them, electronic equipment includes:

[0146] processor;

[0147] a memory for storing processor-executable instructions;

[0148] The processor is configured to execute instructions to implement the text search method proposed in any of the aforementioned embodiments.

[0149] As an example, Fig.10 is a schematic diagram of the structure of an electronic device 1000 shown in an exemplary embodiment of the present disclosure, such as Fig.10 As shown, the electronic device 1000 may further include:

[0150] The memory 1010 and the processor 1020, a bus 1030 connecting different components (including the memory 1010 and the processor 1020), the memory 1010 stores a computer program, and when the processor 1020 executes the program, the image classification method described in the embodiment of the present disclosure is implemented.

[0151] Bus 1030 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0152] The electronic device 1000 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 1000, including volatile and non-volatile media, removable and non-removable media.

[0153] The memory 1010 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1040 and / or cache memory 1050. The server 1000 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 1060 may be used to read and write non-removable, non-volatile magnetic media ( Fig.10 not shown, usually called a "hard drive"). Although Fig.10 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 1030 via one or more data medium interfaces. The memory 1010 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0154] A program / utility 1080 having a set (at least one) of program modules 1070 may be stored, for example, in the memory 1010, such program modules 1070 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 1070 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0155] The electronic device 1000 may also communicate with one or more external devices 1090 (e.g., keyboard, pointing device, display 1091, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or communicate with any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 1092. In addition, the electronic device 1000 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 1093. As shown, the network adapter 1093 communicates with other modules of the electronic device 1000 through a bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0156] The processor 1020 executes various functional applications and data processing by running the programs stored in the memory 1010 .

[0157] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the image classification method of the embodiment of the present disclosure, and will not be repeated here.

[0158] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to complete the image classification method proposed in any of the above embodiments. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0159] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the image classification method proposed in any of the above embodiments.

[0160] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0161] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image classification method, characterized in that: include: Obtain a trained image classification model adapted to the target business scenario; wherein the image classification model includes a convolutional network, an activation function network, and a fully connected network from top to bottom, and the convolutional network is obtained by re-parameterizing by fusing multiple parallel feature processing branches; Acquire the image to be classified in the target business scenario; Inputting the image to be classified into the image classification model, so as to process the image to be classified in sequence through the convolutional network, the activation function network and the fully connected network, and obtaining the probability of at least one predicted category output by the fully connected network; The target category to which the image to be classified belongs is determined according to the probability of the at least one predicted category.

2. The method according to claim 1, characterized in that The convolutional network is generated using the following steps: For any feature processing branch among the multiple feature processing branches, reparameterize each network layer in the any feature processing branch to obtain a reparameterized convolution layer corresponding to the any feature processing branch; Each of the reparameterized convolutional layers is reparameterized to obtain the convolutional network.

3. The method according to claim 2, characterized in that The multiple feature processing branches include: a first feature processing branch, a second feature processing branch and a third feature processing branch; The re-parameterizing each network layer in any feature processing branch to obtain a re-parameterized convolution layer corresponding to any feature processing branch includes: Reparameterizing a first convolutional layer and a first batch normalization layer in a first feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the first feature processing branch; Reparameterizing a second convolutional layer and a second batch normalization layer in a second feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the second feature processing branch; wherein the convolution kernel sizes of the first convolutional layer and the second convolutional layer are different; A third batch normalization layer in a third feature processing branch among the multiple feature processing branches is reparameterized to obtain a reparameterized convolutional layer corresponding to the third feature processing branch.

4. The method according to claim 2, characterized in that: The re-parameterizing each of the re-parameterized convolutional layers to obtain the convolutional network includes: Obtaining convolution weights and biases in each of the re-parameterized convolutional layers; Adding the convolution weights in each of the re-parameterized convolution layers to obtain the convolution weights of the convolution network; The biases in each of the re-parameterized convolutional layers are added together to obtain the bias of the convolutional network.

5. The method according to claim 2, characterized in that: The multiple feature processing branches include: a first feature processing branch, a second feature processing branch and a third feature processing branch; The re-parameterizing each network layer in any feature processing branch to obtain a re-parameterized convolution layer corresponding to any feature processing branch includes: Reparameterizing the third convolution layer and the fourth batch normalization layer in the first feature processing branch among the multiple feature processing branches to obtain a reparameterized convolution layer corresponding to the first feature processing branch; Reparameterizing the fourth convolution layer and the fifth batch normalization layer in the second feature processing branch among the multiple feature processing branches to obtain a reparameterized convolution layer corresponding to the second feature processing branch; wherein the convolution kernel sizes of the third convolution layer and the fourth convolution layer are different; The fifth convolutional layer and the sixth batch normalization layer in the third feature processing branch among the multiple feature processing branches are reparameterized to obtain a reparameterized convolutional layer corresponding to the third feature processing branch; wherein the convolution kernel sizes of the fourth convolutional layer and the fifth convolutional layer are the same.

6. The method according to claim 2, characterized in that The multiple feature processing branches include: a first feature processing branch, a second feature processing branch, a third feature processing branch and a fourth feature processing branch; The re-parameterizing each network layer in any feature processing branch to obtain a re-parameterized convolution layer corresponding to any feature processing branch includes: Reparameterizing the sixth convolutional layer and the seventh batch normalization layer in the first feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the first feature processing branch; Reparameterizing the seventh convolutional layer and the eighth batch normalization layer in the second feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the second feature processing branch; wherein the convolution kernel size of the seventh convolutional layer is smaller than the convolution kernel size of the sixth convolutional layer; Reparameterizing the eighth convolutional layer, the ninth batch normalization layer, the ninth convolutional layer, and the tenth batch normalization layer in the third feature processing branch among the multiple feature processing branches to obtain a reparameterized convolutional layer corresponding to the third feature processing branch; wherein the convolution kernel size of the ninth convolutional layer is the same as that of the sixth convolutional layer; The tenth convolutional layer, the eleventh batch normalization layer, the first pooling layer, and the twelfth batch normalization layer in the fourth feature processing branch among the multiple feature processing branches are reparameterized to obtain a reparameterized convolutional layer corresponding to the fourth feature processing branch; wherein the convolution kernel sizes of the tenth convolutional layer, the eighth convolutional layer, and the seventh convolutional layer are the same.

7. The method according to claim 6, characterized in that The re-parameterizing the eighth convolutional layer, the ninth batch normalization layer, the ninth convolutional layer, and the tenth batch normalization layer in the third feature processing branch among the multiple feature processing branches to obtain the re-parameterized convolutional layer corresponding to the third feature processing branch includes: Reparameterizing the eighth convolutional layer and the ninth batch normalization layer to obtain a first reparameterized convolutional sublayer; Reparameterizing the ninth convolutional layer and the tenth batch normalization layer to obtain a second reparameterized convolutional sublayer; The first re-parameterized convolution sublayer and the second re-parameterized convolution sublayer are re-parameterized to obtain a re-parameterized convolution layer corresponding to the third feature processing branch.

8. The method according to claim 6, characterized in that The re-parameterizing the tenth convolutional layer, the eleventh batch normalization layer, the first pooling layer, and the twelfth batch normalization layer in the fourth feature processing branch in the multiple feature processing branches to obtain the re-parameterized convolutional layer corresponding to the fourth feature processing branch includes: Reparameterizing the tenth convolutional layer and the eleventh batch normalization layer to obtain a third reparameterized convolutional sublayer; Reparameterizing the first pooling layer and the twelfth batch normalization layer to obtain a fourth reparameterized convolutional sublayer; The third parameterized convolution sublayer and the fourth parameterized convolution sublayer are reparameterized to obtain a reparameterized convolution layer corresponding to the fourth feature processing branch.

9. The method according to claim 1, characterized in that: The image classification model is trained using the following steps: Obtaining a sample image and obtaining a labeled category of the sample image; Using the image classification model to classify the sample image to obtain a predicted category to which the sample image belongs; The image classification model is trained according to the difference between the predicted category and the labeled category.

10. An image classification device, characterized in that: include: A first acquisition module acquires a trained image classification model adapted to a target business scenario; wherein the image classification model includes, from top to bottom, a convolutional network, an activation function network, and a fully connected network, and the convolutional network is obtained by re-parameterizing by fusing multiple parallel feature processing branches; A second acquisition module is used to acquire the image to be classified in the target business scenario; An input module, used to input the image to be classified into the image classification model, so as to process the image to be classified through the convolutional network, the activation function network and the fully connected network in sequence, and obtain the probability of at least one predicted category output by the fully connected network; A determination module is used to determine the target category to which the image to be classified belongs according to the probability of the at least one predicted category.

Images