A training method and apparatus for an image classification model

Abstract

This invention discloses a training method and apparatus for an image classification model, relating to the field of artificial intelligence technology. One specific implementation of the method includes: inputting image samples into a convolutional neural network (CNN), using the CNN's classification result as the first classification label for the image samples; pre-training the image classification model using pre-labeled second and first classification labels of the image samples to obtain a pre-trained image classification model; and training the pre-trained image classification model using image samples as input and the second classification label as output. This implementation can learn the inductive bias of the CNN, reduce the amount of data required to train the model, improve training efficiency, and enhance the image classification performance of the trained model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more particularly to the field of computer vision technology, specifically to a training method and apparatus for an image classification model. Background Technology

[0002] With the rapid development of Transformer models (a type of model for seq2seq tasks) in the field of NLP (Natural Language Processing), Transformer models have also begun to be gradually applied to image classification in the visual field. Existing Transformer models directly utilize the basic Transformer architecture, fitting the output by acquiring long-range dependency information at different locations of features. Since Transformer models lack inductive bias compared to convolutional neural networks, they need to learn more of their inherent inductive bias from the data, and consequently require more data for training.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0004] Lacking inductive bias, it requires a large amount of data for training, resulting in low training efficiency and poor image classification performance of the trained model. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a training method and apparatus for an image classification model, which can learn the inductive bias of a convolutional neural network, reduce the amount of data required to train the model, improve training efficiency, and improve the image classification performance of the trained model.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for training an image classification model is provided.

[0007] A method for training an image classification model includes: inputting an image sample into a convolutional neural network, using the classification result of the convolutional neural network on the image sample as a first classification label of the image sample; pre-training the image classification model using a pre-labeled second classification label of the image sample and the first classification label to obtain a pre-trained image classification model; and training the pre-trained image classification model using the image sample as input and the second classification label as output.

[0008] Optionally, the image classification model is a transformer model. The step of pre-training the image classification model using the pre-labeled second classification label and the first classification label of the image samples to obtain a pre-trained image classification model includes: processing the image samples through an embedding layer and inputting them into a transformer model including a multi-head convolutional attention mechanism layer to obtain transformer features of the image samples; generating a first overall loss function based on a first loss function, a second loss function, and a third loss function; wherein the second loss function makes the classification probability output by the transformer model consistent with the second classification label, and the third loss function makes the classification probability output by the transformer model consistent with the first classification label, and the classification probability is the probability that the image sample belongs to each category; and pre-training the transformer model with the first overall loss function approaching 0 as the training objective to obtain a pre-trained transformer model.

[0009] Optionally, the first loss function is determined by: extracting first transformer features and second transformer features from the transformer features; and determining the first loss function based on the first transformer features and the convolutional neural network features, wherein the convolutional neural network features are the features of the image samples obtained through the convolutional neural network.

[0010] Optionally, determining the first loss function based on the first converter features and the convolutional neural network features includes: transforming the first converter features to obtain first converter features corresponding to the dimension of the convolutional neural network features, wherein the transformation includes deformation, interpolation, convolution, layer normalization, and linear rectification; and determining the first loss function based on the first converter features corresponding to the dimension of the convolutional neural network features and the convolutional neural network features.

[0011] Optionally, the second loss function is determined by processing the image samples using a normalized exponential function based on the second transformer features, and then using the second classification label and the processing result of the normalized exponential function to determine the second loss function.

[0012] Optionally, the third loss function is determined by using the processing result of the normalized exponential function and the first classification label as input parameters of the second loss function.

[0013] Optionally, the step of using the image sample as input and the second classification label as output to train the pre-trained image classification model includes: using the processing result of the normalized exponential function and the second classification label as input parameters of the second loss function to determine the second overall loss function; and using the image sample as input and the second classification label as output to train the pre-trained image classification model with the second overall loss function approaching 0 as the training objective.

[0014] According to another aspect of the present invention, a training apparatus for an image classification model is provided.

[0015] A training apparatus for an image classification model includes: a first classification label generation module, used to input image samples into a convolutional neural network and use the classification result of the convolutional neural network on the image samples as the first classification label of the image samples; an image classification model pre-training module, used to pre-train the image classification model using the pre-labeled second classification label and the first classification label of the image samples to obtain a pre-trained image classification model; and an image classification model training module, used to train the pre-trained image classification model by taking the image samples as input and the second classification label as output.

[0016] Optionally, the image classification model is a transformer model, and the image classification model pre-training module is further configured to: process the image sample through an embedding layer and input it into a transformer model including a multi-head convolutional attention mechanism layer to obtain the transformer features of the image sample; generate a first overall loss function based on a first loss function, a second loss function, and a third loss function; wherein the second loss function makes the classification probability output by the transformer model consistent with the second classification label, and the third loss function makes the classification probability output by the transformer model consistent with the first classification label, and the classification probability is the probability that the image sample belongs to each category; and pre-train the transformer model with the first overall loss function approaching 0 as the training objective to obtain a pre-trained transformer model.

[0017] Optionally, the first loss function is determined by: extracting first transformer features and second transformer features from the transformer features; and determining the first loss function based on the first transformer features and the convolutional neural network features, wherein the convolutional neural network features are the features of the image samples obtained through the convolutional neural network.

[0018] Optionally, the image classification model pre-training module is further configured to: transform the first transformer features to obtain first transformer features corresponding to the dimension of the convolutional neural network features, wherein the transformation includes deformation, interpolation, convolution, layer normalization, and linear rectification; and determine the first loss function based on the first transformer features corresponding to the dimension of the convolutional neural network features and the convolutional neural network features.

[0019] Optionally, the second loss function is determined by processing the image samples using a normalized exponential function based on the second transformer features, and then using the second classification label and the processing result of the normalized exponential function to determine the second loss function.

[0020] Optionally, the third loss function is determined by using the processing result of the normalized exponential function and the first classification label as input parameters of the second loss function.

[0021] Optionally, the image classification model training module is further configured to: use the processing result of the normalized exponential function and the second classification label as input parameters of the second loss function to determine the second overall loss function; and use the image sample as input and the second classification label as output, with the second overall loss function approaching 0 as the training objective, to train the pre-trained image classification model.

[0022] According to another aspect of the present invention, an electronic device is provided.

[0023] An electronic device includes: one or more processors; and a memory for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the training method for the image classification model provided in the embodiments of the present invention.

[0024] According to another aspect of the present invention, a computer-readable medium is provided.

[0025] A computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the training method for the image classification model provided in the embodiments of the present invention.

[0026] One embodiment of the above invention has the following advantages or beneficial effects: Image samples are input into a convolutional neural network (CNN), and the classification result of the CNN on the image samples is used as the first classification label of the image samples; the image classification model is pre-trained using the pre-labeled second classification label and the first classification label of the image samples to obtain a pre-trained image classification model; the pre-trained image classification model is trained using image samples as input and the second classification label as output. This allows the model to learn the inductive bias of the CNN, reducing the amount of data required to train the model, improving training efficiency, and enhancing the image classification performance of the trained model.

[0027] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0028] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0029] Figure 1 This is a schematic diagram illustrating the main steps of a training method for an image classification model according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the architecture of a training method for an image classification model according to an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of a process for converting dimensions according to an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of the main modules of a training device for an image classification model according to an embodiment of the present invention;

[0033] Figure 5 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0034] Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0035] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0036] Figure 1This is a schematic diagram illustrating the main steps of a training method for an image classification model according to an embodiment of the present invention.

[0037] like Figure 1 As shown, the training method of the image classification model in one embodiment of the present invention mainly includes the following steps S101 to S103.

[0038] Step S101: Input the image sample into the convolutional neural network and use the classification result of the convolutional neural network on the image sample as the first classification label of the image sample.

[0039] Step S102: Use the pre-labeled second classification label and first classification label of the image samples to pre-train the image classification model to obtain the pre-trained image classification model.

[0040] Image classification models can be Transformer models. A Transformer model is a model used for seq2seq tasks that does not have the recurrent structure of an RNN (Recurrent Neural Network) or the convolutional structure of a CNN (Convolutional Neural Network).

[0041] Using pre-labeled second and first classification labels of image samples, the image classification model is pre-trained to obtain a pre-trained image classification model. This process may include: processing image samples through an embedding layer and inputting them into a transformer model including a multi-head convolutional attention mechanism layer to obtain transformer features of the image samples; generating a first overall loss function based on a first loss function, a second loss function, and a third loss function; wherein the second loss function makes the classification probability output by the transformer model consistent with the second classification label, and the third loss function makes the classification probability output by the transformer model consistent with the first classification label. The classification probability is the probability that the image sample belongs to each category. The classification probability output by the transformer model being consistent with a certain classification label means that the classification probability output by the transformer model indicates that the probability of belonging to a certain category is the highest, and that category is the category corresponding to the classification label; and pre-training the transformer model with the first overall loss function approaching 0 as the training objective to obtain a pre-trained transformer model. Pre-training is the training process performed before step S103, where image samples are used as input and the second classification label is used as output to train the image classification model. Pre-training can be considered as the first stage in the overall training phase, while the training in step S103 is the second stage.

[0042] The first loss function can be determined as follows: extract the first transformer feature and the second transformer feature from the transformer features. Specifically, the last row of the transformer features is used as the second transformer feature, which represents the classification result of the transformer model for the image sample. The features other than the second transformer feature in the transformer features are used as the first transformer feature. Based on the first transformer feature and the convolutional neural network feature, the first loss function is determined. The convolutional neural network feature is the feature of the image sample obtained through the convolutional neural network.

[0043] Determining a first loss function based on the first converter features and the convolutional neural network features may include: transforming the first converter features to obtain first converter features corresponding to the dimensions of the convolutional neural network features, the transformation including deformation, interpolation, convolution, layer normalization, and linear rectification; and determining the first loss function based on the first converter features corresponding to the dimensions of the convolutional neural network features and the convolutional neural network features.

[0044] The second loss function can be determined as follows: Based on the features of the second transformer, image samples are processed using a normalized exponential function, and the second loss function is determined using the second classification label and the processing result of the normalized exponential function. Specifically, the second and third classification labels are used as inputs, and the second loss function is obtained through the cross-entropy loss function. The third classification label is the processing result of the normalized exponential function.

[0045] The third loss function can be determined as follows: the result of the normalized exponential function and the first classification label are used as input parameters for the second loss function to determine the third loss function.

[0046] Step S103: Use the image sample as input and the second classification label as output to train the pre-trained image classification model.

[0047] Training a pre-trained image classification model by using image samples as input and second classification labels as output can include: using the result of the normalized exponential function and the second classification labels as input parameters to a second overall loss function to determine the second overall loss function; and using the second overall loss function approaching 0 as the training objective, training the pre-trained image classification model by using image samples as input and the second classification labels as output. The result of the normalized exponential function is a vector of dimension n obtained by the normalized exponential function.

[0048] Figure 2 This is a schematic diagram of the architecture of a training method for an image classification model according to an embodiment of the present invention.

[0049] like Figure 2As shown, this embodiment of the invention employs a two-stage training method. In the first stage (i.e., the pre-training stage), knowledge distillation is performed on the Transformer to introduce the inductive bias of the CNN (201). In the second stage (i.e., the training stage), the inductive bias inherent in the Transformer is learned based on the learned inductive bias of the CNN (202), thereby further improving the performance of image classification. Knowledge distillation can transfer knowledge from one network to another; the two networks can be isomorphic or heterogeneous.

[0050] In one embodiment, image samples are processed by an embedding layer and then input into a transformer model that includes a multi-head convolutional attention mechanism layer to obtain transformer features of the image samples. Specifically, the image samples are input into the embedding layer to obtain a sequence X, X∈R, corresponding to the image. T×d The sequence X is input into the Transformer layer (i.e., the transformer model) to obtain the Transformer features corresponding to the image sample. The Transformer features can be obtained using the following formula:

[0051] A = Softmax(QK + v) (h) r ij )

[0052] MHCA(X)=AXW V

[0053] Where Q and K are the first and second similarity matrices of sequence X obtained after passing through a fully connected layer, respectively. The product of Q and K represents the similarity (i.e., attention) matrix between points in sequence X. v is a learnable variable defined in the transformer, h is the number of heads in a multi-head attention mechanism layer (generally h > 9), and r ij The positional encoding representing the relative distances of sequence X, i.e., the relative distances between points in sequence X, can be obtained by quadratic encoding of sequence X, where QK and V are the relative distances. (h) r ijThe sum of represents the similarity between each point in sequence X and other points in terms of distance; that is, the closer the relative distance, the higher the similarity. This sum is input into the Softmax function (normalized exponential function) to obtain the intermediate value A for calculating the Transformer features. W is obtained by linearly transforming sequence X through a linear layer, and the dimension of W corresponds to the dimension of QK. MHCA(X) represents the Transformer feature corresponding to this image sample; the Transformer feature is the output of MHCA in the Transformer model. MHCA stands for multi-head convolutional attention. According to the above formula for MHCA(X), this embodiment of the invention adds a learnable positional encoding representing relative distance to the multi-head attention mechanism layer (MHSA) of the Transformer model, enabling the network to focus on local information, thus equivalent to convolution, constituting the multi-head convolutional attention mechanism layer (MHCA) of this embodiment of the invention.

[0054] In one embodiment, a first transformer feature and a second transformer feature are extracted from the transformer features. Specifically, the last row of MHCA(X) is used as the second transformer feature (i.e., category tokens, cls token), representing the classification result of the transformer model for the image sample; MHCA(X) can be a matrix of dimension T×d, and all the remaining tokens in MHCA(X) after removing the category tokens (i.e., the first T-1 rows) are used as the first transformer feature (i.e., content tokens), representing the attribute parameters of the transformer itself.

[0055] In one embodiment, image samples are input into a convolutional neural network, and the classification result of the convolutional neural network on the image samples is used as the first classification label of the image samples; the image samples are pre-labeled to obtain the second classification label of the image samples, which can represent the true classification label of the image samples; the features of the second transformer are processed by a normalized exponential function to obtain a vector of dimension n, where n is the number of categories. The values ​​in this vector of dimension n represent the probability distribution of the image samples belonging to different categories, which can be used as the processing result of the normalized exponential function, i.e., the third classification label.

[0056] Figure 3 This is a schematic diagram of a transformation dimension according to an embodiment of the present invention.

[0057] In one embodiment, such as Figure 3As shown, the first transformer features are transformed to obtain the first transformer features corresponding to the dimensions of the convolutional neural network (CNN) features. Based on the first transformer features corresponding to the dimensions of the CNN features and the CNN features, the first loss function is determined. Specifically, image samples are input into the CNN, and CNN features are obtained based on the output results. Based on the dimensions of the CNN features, the first transformer features are transformed by a matcher. For example, the dimensions of the CNN features can be h×w×d, and the dimensions of the first transformer features can be T-1×d. The first transformer features are processed sequentially through steps such as reshaping, interpolation, convolution, layer normalization, linear rectified activation function, and convolution to convert the dimensions of the first transformer features to h×w×d. Here, h×w can be equal to T-1. Reshaping does not change the value size, but only rearranges the features. Layer normalization is to calculate the mean and variance of each layer of features separately and normalize them. The linear rectified activation function can be the ReLU activation function.

[0058] In this embodiment of the invention, a first loss function is determined based on the first transformer features and the convolutional neural network features corresponding to the dimensions of the convolutional neural network features. Specifically, the calculation formula for the first loss function can be:

[0059] L hidden =MSE(aligner(H) S ),H T )

[0060] Among them, L hidden H represents the first loss function. S H represents the first transformer feature after dimensional transformation. T The 'convolutional neural network' features are represented by the 'aligner' function, which makes the classification probability output by the transformer model consistent with the second classification label. MSE (Mean Square Error) represents the mean square error.

[0061] In one embodiment, a second loss function is determined based on the second and third classification labels, a third loss function is determined based on the first and third classification labels, and a first overall loss function is generated based on the first, second, and third loss functions. Specifically, the second and third classification labels are used as inputs, and the second loss function is obtained through the cross-entropy loss function. The first and third classification labels are used as parameters of the second loss function to obtain the third loss function. The formula for calculating the third loss function can be:

[0062] L logit =LCE (logit,yt)

[0063] Among them, L logit L represents the third loss function. CE Let represent the second loss function, logit represent the result of the normalized exponential function (i.e., the third classification label), and yt represent the first classification label.

[0064] This invention generates a first overall loss function based on a first loss function, a second loss function, and a third loss function. Specifically, the calculation formula for the first overall loss function can be:

[0065] L1=αL CE +(1-α)L logit +βL hidden

[0066] Here, L1 represents the first overall loss function, and α and β are weight coefficients set empirically. The converter model can be pre-trained with the first overall loss function approaching 0 as the training objective, resulting in a pre-trained converter model.

[0067] In one embodiment, the result of the normalized exponential function and the second classification label are used as input parameters to determine the second overall loss function. The training objective is to make the second overall loss function approach 0. Image samples are used as input, and the second classification label is used as output to train the pre-trained image classification model. Specifically, the formula for calculating the second overall loss function can be:

[0068] L2 = L CE (logit,y)

[0069] Where L2 represents the second overall loss function, y represents the second classification label, and logit represents the result of the normalized exponential function. With the training objective of the second overall loss function approaching 0, image samples are used as input and the second classification label as output. A second stage of training is then performed on the pre-trained image classification model to obtain the final image classification model, which can be used to classify images.

[0070] Figure 4 This is a schematic diagram of the main modules of a training device for an image classification model according to an embodiment of the present invention.

[0071] like Figure 4 As shown, the image classification model training device 400 of an embodiment of the present invention mainly includes: a first classification label generation module 401, an image classification model pre-training module 402, and an image classification model training module 403.

[0072] The first classification label generation module 401 is used to input image samples into a convolutional neural network and use the classification result of the convolutional neural network on the image samples as the first classification label of the image samples.

[0073] The image classification model pre-training module 402 is used to pre-train the image classification model using the pre-labeled second classification label and first classification label of the image samples, so as to obtain the pre-trained image classification model.

[0074] The image classification model training module 403 is used to train the pre-trained image classification model by taking image samples as input and the second classification label as output.

[0075] In one embodiment, the image classification model is a transformer model. The image classification model pre-training module 402 is specifically used to: process image samples through an embedding layer and input them into a transformer model including a multi-head convolutional attention mechanism layer to obtain transformer features of the image samples; generate a first overall loss function based on a first loss function, a second loss function, and a third loss function; wherein the second loss function makes the classification probability output by the transformer model consistent with the second classification label, and the third loss function makes the classification probability output by the transformer model consistent with the first classification label, and the classification probability is the probability that the image sample belongs to each category; and pre-train the transformer model with the first overall loss function approaching 0 as the training objective to obtain the pre-trained transformer model.

[0076] In one embodiment, the first loss function can be determined as follows: extracting first transformer features and second transformer features from transformer features; and determining the first loss function based on the first transformer features and convolutional neural network features, where the convolutional neural network features are the features of image samples obtained through a convolutional neural network.

[0077] In one embodiment, the image classification model pre-training module 402 is specifically used to: transform the first transformer features to obtain the first transformer features corresponding to the dimensions of the convolutional neural network features, the transformation including deformation, interpolation, convolution, layer normalization, and linear rectification; and determine the first loss function based on the first transformer features corresponding to the dimensions of the convolutional neural network features and the convolutional neural network features.

[0078] In one embodiment, the second loss function can be determined as follows: based on the features of the second transformer, image samples are processed by a normalized exponential function, and the second loss function is determined using the second classification label and the processing result of the normalized exponential function.

[0079] In one embodiment, the third loss function can be determined by using the processing result of the normalized exponential function and the first classification label as input parameters to the second loss function.

[0080] In one embodiment, the image classification model training module 403 is specifically used to: use the processing result of the normalized exponential function and the second classification label as input parameters of the second loss function to determine the second overall loss function; and use image samples as input and the second classification label as output to train the pre-trained image classification model with the second overall loss function approaching 0 as the training objective.

[0081] Furthermore, the specific implementation details of the training device for the image classification model in this embodiment of the invention have been described in detail in the above-described training method for the image classification model, so they will not be repeated here.

[0082] Figure 5 An exemplary system architecture 500 is shown, which can be used to train an image classification model according to embodiments of the present invention, or to train an image classification model.

[0083] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, and 503, a network 504, and a server 505. Network 504 serves as the medium for providing communication links between terminal devices 501, 502, and 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0084] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0085] Terminal devices 501, 502, and 503 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0086] Server 505 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 501, 502, and 503 (for example only). The backend management server can analyze and process data such as received product information query requests, and feed back the processing results (such as target push information, product information - for example only) to the terminal device.

[0087] It should be noted that the training method of the image classification model provided in this embodiment of the invention is generally executed by the server 505, and correspondingly, the training device of the image classification model is generally set in the server 505.

[0088] It should be understood that Figure 5 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0089] Reference below Figure 6 It shows a schematic diagram of the structure of a computer system 600 suitable for implementing terminal devices or servers of the present invention. Figure 6 The terminal device or server shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0090] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the system 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0091] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0092] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this invention.

[0093] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a first classification label generation module, an image classification model pre-training module, and an image classification model training module. The names of these modules do not necessarily limit the module itself; for example, the first classification label generation module may also be described as "a module for inputting image samples into a convolutional neural network, using the classification result of the convolutional neural network as the first classification label of the image sample."

[0096] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: inputting image samples into a convolutional neural network, using the classification result of the convolutional neural network on the image samples as a first classification label for the image samples; pre-training an image classification model using pre-labeled second classification labels and the first classification labels of the image samples to obtain a pre-trained image classification model; and training the pre-trained image classification model using image samples as input and the second classification labels as output.

[0097] According to the technical solution of this invention, image samples are input into a convolutional neural network (CNN), and the classification result of the CNN on the image samples is used as the first classification label of the image samples. Using the pre-labeled second classification label and the first classification label of the image samples, the image classification model is pre-trained to obtain a pre-trained image classification model. The pre-trained image classification model is then trained using image samples as input and the second classification label as output. This method can learn the inductive bias of the CNN, reduce the amount of data required to train the model, improve training efficiency, and enhance the image classification performance of the trained model.

[0098] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A training method for an image classification model, characterized in that, include: Image samples are input into a convolutional neural network, and the classification result of the convolutional neural network on the image samples is used as the first classification label of the image samples; Using the pre-labeled second classification label and the first classification label of the image sample, the image classification model is pre-trained to obtain the pre-trained image classification model; The image sample is used as input, and the second classification label is used as output to train the pre-trained image classification model. The image classification model is a transformer model. The pre-training of the image classification model using the pre-labeled second classification label and the first classification label of the image samples to obtain the pre-trained image classification model includes: The image samples are processed by the embedding layer and then input into a transformer model that includes a multi-head convolutional attention mechanism layer to obtain the transformer features of the image samples. A first overall loss function is generated based on a first loss function, a second loss function, and a third loss function; wherein, the second loss function makes the classification probability output by the transformer model consistent with the second classification label, and the third loss function makes the classification probability output by the transformer model consistent with the first classification label, and the classification probability is the probability that the image sample belongs to each category; the first loss function is determined by extracting first transformer features and second transformer features from the transformer features, and determining the first loss function based on the first transformer features and convolutional neural network features, wherein the convolutional neural network features are the features of the image sample obtained through a convolutional neural network; With the first overall loss function approaching 0 as the training objective, the converter model is pre-trained to obtain the pre-trained converter model.

2. The method according to claim 1, characterized in that, The step of determining the first loss function based on the features of the first transformer and the features of the convolutional neural network includes: The first transformer features are transformed to obtain the first transformer features corresponding to the dimensions of the convolutional neural network features. The transformation includes deformation, interpolation, convolution, layer normalization, and linear rectification. The first loss function is determined based on the first transformer features corresponding to the dimensions of the convolutional neural network features and the convolutional neural network features.

3. The method according to claim 1, characterized in that, The second loss function is determined as follows: Based on the second transformer features, the image samples are processed by a normalized exponential function, and the second loss function is determined using the second classification label and the processing result of the normalized exponential function.

4. The method according to claim 3, characterized in that The third loss function is determined as follows: The processing result of the normalized exponential function and the first classification label are used as input parameters of the second loss function to determine the third loss function.

5. The method according to claim 4, characterized in that, The step of using the image sample as input and the second classification label as output to train the pre-trained image classification model includes: The processing result of the normalized exponential function and the second classification label are used as input parameters of the second loss function to determine the second overall loss function; With the training objective of the second overall loss function approaching 0, the image samples are used as input and the second classification label is used as output to train the pre-trained image classification model.

6. A training device for an image classification model, characterized in that, include: The first classification label generation module is used to input image samples into a convolutional neural network and use the classification result of the convolutional neural network on the image samples as the first classification label of the image samples. The image classification model pre-training module is used to pre-train the image classification model using the pre-labeled second classification label and the first classification label of the image samples, so as to obtain the pre-trained image classification model. An image classification model training module is used to train the pre-trained image classification model by taking the image sample as input and the second classification label as output. The image classification model is a transformer model, and the image classification model pre-training module is also used for: The image samples are processed by the embedding layer and then input into a transformer model that includes a multi-head convolutional attention mechanism layer to obtain the transformer features of the image samples. A first overall loss function is generated based on a first loss function, a second loss function, and a third loss function; wherein, the second loss function makes the classification probability output by the transformer model consistent with the second classification label, and the third loss function makes the classification probability output by the transformer model consistent with the first classification label, and the classification probability is the probability that the image sample belongs to each category; the first loss function is determined by extracting first transformer features and second transformer features from the transformer features, and determining the first loss function based on the first transformer features and convolutional neural network features, wherein the convolutional neural network features are the features of the image sample obtained through a convolutional neural network; With the first overall loss function approaching 0 as the training objective, the converter model is pre-trained to obtain the pre-trained converter model.

7. The apparatus according to claim 6, characterized in that, The image classification model pre-training module is also used for: The first transformer features are transformed to obtain the first transformer features corresponding to the dimensions of the convolutional neural network features. The transformation includes deformation, interpolation, convolution, layer normalization, and linear rectification. The first loss function is determined based on the first transformer features corresponding to the dimensions of the convolutional neural network features and the convolutional neural network features.

8. The apparatus according to claim 6, characterized in that, The second loss function is determined as follows: Based on the second transformer features, the image samples are processed by a normalized exponential function, and the second loss function is determined using the second classification label and the processing result of the normalized exponential function.

9. The apparatus according to claim 8, characterized in that, The third loss function is determined as follows: The processing result of the normalized exponential function and the first classification label are used as input parameters of the second loss function to determine the third loss function.

10. The apparatus according to claim 9, characterized in that, The image classification model training module is also used for: The processing result of the normalized exponential function and the second classification label are used as input parameters of the second loss function to determine the second overall loss function; With the training objective of the second overall loss function approaching 0, the image samples are used as input and the second classification label is used as output to train the pre-trained image classification model.

11. An electronic device, characterized in that, include: one or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

12. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images