System and method for training image classification model and method for classifying images

Abstract

The present application discloses a computer-implemented method for training an image classification model, a system for training an image classification model, and a computer-implemented method for classifying an image. In one or more embodiments, the compact deep learning model includes fewer layers, which requires fewer floating point operations (FLOPs). An embodiment of a new learning rate function is also provided herein, which can adaptively change the learning rate between two linear functions. In one or more embodiments, during the training process, a combination of half-precision floating point format training and larger batch sizes can also be used to help the training process.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for computer learning that can provide improved computer performance, features, and uses. More specifically, the present disclosure relates to systems and methods for improved deep learning models and improved training of deep learning networks. Background Art

[0002] Deep learning (DL) has found widespread applications in many different fields such as computer vision, natural language processing, medical image processing, etc. Increasingly, deep learning has become the basis for the recent artificial intelligence / machine learning (AI / ML) revolution, which has impacted numerous industries and spawned many more. In most cases, the training process for DL ​​tasks is very long and can be difficult to perform even on modern advanced graphics processing unit (GPU) machines such as those manufactured by Nvidia of Sunnyvale, California. The same is true for the V100 Tensor Core, or Tensor Processing Unit (TPU), manufactured by Google in Mountain View, California. There is a great need to design new models and methods to accelerate the training process of DL models to improve efficiency and save costs.

[0003] As an example, consider deep learning text-to-speech (TTS) systems. Due to their complexity, developing TTS systems can be very labor-intensive and difficult. Recent work on neural TTS systems has demonstrated impressive results, producing pipelines with slightly simpler features, fewer components, and higher quality synthesized speech. There is no consensus on the optimal neural network architecture for TTS. Similar problems exist for other applications of deep learning systems that solve different domains.

[0004] Therefore, there is a need for improved deep learning models and systems and methods for the training of improved deep learning models. Summary of the invention

[0005] One aspect of the present disclosure provides a computer-implemented method for training an image classification model, the method comprising: forming one or more batches, the one or more batches comprising images and labels corresponding to the images, the images and labels corresponding to the images being selected from one or more training data sets, wherein each image has a corresponding label; repeating a set of steps for each training cycle until a stopping condition is reached. The set of steps comprises: inputting the batch into the image classification model; determining the loss of the image classification model given the predicted output of the batch; and updating one or more parameters of the image classification model using the loss, wherein the image classification model comprises: a convolution module comprising a convolution with a set of filters, a batch normalization operation, and an activation operation; a first residual module comprising at least two convolution modules separated by a maximum pooling layer, wherein each convolution module has its own filter bank; a second residual module comprising at least two convolution modules separated by a maximum pooling layer, wherein each convolution module has its own filter bank; and a fully connected layer receiving an input obtained from the output of the second residual module.

[0006] Another aspect of the present disclosure provides a system for training an image classification model, the system comprising: one or more processors; and a non-transitory computer-readable medium comprising one or more sets of instructions, wherein the one or more sets of instructions, when executed by at least one of the one or more processors, causes the following steps to be performed: forming one or more batches, the one or more batches comprising images and labels corresponding to the images, the images and labels corresponding to the images being selected from one or more training data sets, wherein each image has a corresponding label; repeating a set of steps for each training cycle until a stop condition is reached, the set of steps comprising: inputting the batch to the image In a classification model; given the predicted output of the batch, determining the loss of the image classification model; and using the loss to update one or more parameters of the image classification model, wherein the image classification model includes: a convolution module, including a convolution with a set of filters, a batch normalization operation, and an activation operation; a first residual module, including at least two convolution modules separated by a maximum pooling layer, wherein each convolution module has its own filter group; a second residual module, including at least two convolution modules separated by a maximum pooling layer, wherein each convolution module has its own filter group; and a fully connected layer, receiving an input obtained from the output of the second residual module.

[0007] Yet another aspect of the present disclosure provides a computer-implemented method for classifying an image, the method comprising: inputting an input image into a classification model; and outputting a classification label of the input image, wherein the classification model comprises: a convolution module, comprising a convolution with a set of filters, a batch normalization operation, and an activation operation; a first residual module, comprising at least two convolution modules separated by a maximum pooling layer, wherein each convolution module has its own filter group; a second residual module, comprising at least two convolution modules separated by a maximum pooling layer; and a fully connected layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Reference will be made to embodiments of the present disclosure, examples of which may be shown in the accompanying drawings. These drawings are illustrative and not restrictive. Although the present disclosure is generally described in the context of these embodiments, it should be understood that it is not intended to limit the scope of the present disclosure to these specific embodiments. The items in the drawings may not be drawn to scale.

[0009] Figure 1 Depicted are tiny or compact neural architectures designed with one or more residual modules according to embodiments of the present disclosure.

[0010] Figure 2 A residual model architecture according to an embodiment of the present disclosure is illustrated.

[0011] Figure 3 An alternative residual model architecture according to an embodiment of the present invention is illustrated.

[0012] Figure 4 An example learning rate according to an embodiment of the present disclosure is illustrated.

[0013] Figure 5 A method for training an image classification model according to an embodiment of the present disclosure is illustrated.

[0014] Figure 6 Described is a method of using a trained model for image classification according to an embodiment of the present disclosure.

[0015] Figure 7 Depicted is a simplified block diagram of a computing device / information processing system in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In the following description, for the purpose of explanation, specific details are set forth to provide an understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these details. In addition, those skilled in the art will recognize that the embodiments of the present disclosure described below may be implemented in a variety of ways, such as a process, apparatus, system, device, or method on a tangible computer-readable medium.

[0017] The components or modules shown in the accompanying drawings are examples of exemplary embodiments of the present disclosure and are intended to avoid obfuscating the present disclosure. It should also be understood that throughout the discussion, components may be described as separate functional units, which may include subunits, but those skilled in the art will recognize that various components or parts thereof may be divided into separate components, or may be integrated together, including within a single system or component. It should be noted that the functions or operations discussed herein may be implemented as components. Components may be implemented in software, hardware, or a combination thereof.

[0018] In addition, the connections between the components or systems in the drawings are not limited to direct connections. Instead, the data between these components can be modified, reformatted or otherwise changed by intermediate components. In addition, additional or fewer connections can be used. It should also be noted that the terms "connection", "connection" or "communication connection" should be understood to include direct connections, indirect connections through one or more intermediate devices, and wireless connections.

[0019] References in the specification to "one embodiment," "preferred embodiment," "embodiment," or "embodiments" mean that a particular feature, structure, characteristic, or function described in conjunction with the embodiment is included in at least one embodiment of the present disclosure, and may be in more than one embodiment. In addition, the above phrases appearing in various places in the specification do not necessarily all refer to the same embodiment or embodiments.

[0020] Certain terms are used in various places in this specification for the purpose of explanation and should not be construed as limiting. Services, functions, or resources are not limited to a single service, function, or resource; the use of these terms may refer to a combination of related services, functions, or resources, which may be distributed or aggregated.

[0021] The terms "comprising," "including," "include," and "comprising" are to be understood as open ended terms, and any list below is an example and is not meant to be limited to the listed items. A "layer" may include one or more operations.

[0022] Any headings used herein are for organizational purposes only and should not be used to limit the scope of the specification or claims. Each reference / document mentioned in this patent document is incorporated herein by reference in its entirety.

[0023] In addition, those skilled in the art will recognize that: (1) certain steps may be performed optionally; (2) the steps may not be limited to the specific order described herein; (3) certain steps may be performed in a different order; and (4) certain steps may be performed simultaneously.

[0024] It should be noted that any experiments and results provided herein are provided by way of example and were performed under specific conditions using one or more specific embodiments; therefore, these experiments and their results should not be used to limit the scope of the disclosure of this patent document.

[0025] A. General Introduction

[0026] Training a neural network usually means finding the best weights for a specific problem using a set of labeled data. With a given loss function (which is used to measure the performance of the weights between the large number of interconnections in each layer of the neural network), the loss function is usually reduced through iteration using the back propagation (BP) method.

[0027] In most cases, training deep neural networks is very challenging. It usually requires expert knowledge and experience to properly train and obtain the optimal model. It also usually requires a lot of experiments to find the best hyperparameters, such as learning rate, batch size, momentum, etc.

[0028] For typical image classification problems, such as ImageNet and CIFAR10, there have been many works to improve the classification performance. However, the models used for these classification problems usually have a very long training process.

[0029] B. Compact / Tiny Model Architecture Embodiment

[0030] Embodiments of compact image classification models have been developed to reduce training time. The goal is to design models that can be trained efficiently on image data but still have acceptable performance. One way to help reduce training time is to make a tiny / compact model; that is, a model that has fewer parameters and therefore requires fewer operations to train. However, reducing the size of a model tends to reduce its prediction accuracy. Therefore, part of the difficulty lies in determining a way to build a model and still obtain good results.

[0031] exist Figure 1An exemplary novel deep learning compact or tiny structure is described in , wherein the model has fewer layers and narrower width (e.g., fewer filters) than a typical image classification model. To address the gradient saturation problem that may occur in the back-propagation process of training of various deep network structures, a shortcut connection with identity output is applied to the neural network module. In one or more embodiments, the shortcut connection means that the output is directly equal to the input without any additional operations, such as convolution, pooling, etc. For example, Figure 2 and Figure 3 Each contains shortcut connections that bypass one or more convolutional blocks (i.e., Convolution+BN+ReLu) modules.

[0032] Figure 1 A tiny neural structure designed using one or more residual modules according to an embodiment of the present disclosure is shown. In one or more embodiments, the model 135 receives an input 105, which is input into the first module 110. In one or more embodiments, the first module 110 includes a convolutional layer, a batch normalization operation, and a rectified linear unit (ReLu) operation. As depicted, the model 135 also includes one or more residual modules (e.g., residual modules 115 and 120) and a fully connected layer 125, which may include a softmax layer (not shown), from which an output 130 is obtained.

[0033] Figure 2 An example residual module according to an embodiment of the present disclosure is depicted. In the depicted embodiment, the residual module may include a first convolutional block 210, which may include one or more convolutional layers, batch normalization, and a rectified linear unit (ReLu). As shown, the residual module also includes a maximum pooling layer 215 and an additional convolutional block 220. In one or more embodiments, the output of the maximum pooling layer 215 can be combined with the output of the last convolutional module 220 of the residual module 200 to obtain a final output 225.

[0034] Figure 3 An alternative residual module according to an embodiment of the present disclosure is depicted. In the depicted embodiment, the residual module 300 is similar to Figure 2 , but it includes two convolution modules 320 and 325 after the maximum pooling layer 315.

[0035] In one or more embodiments, an image classification model (e.g., Figure 1The model shown in FIG. 1 may have a narrow width by having fewer convolutional filters. Given the reduced model size, it is important that the model is constructed to help increase the feature representation capability of the model. One way to increase the feature representation capability of the model is by using increased filter sizes. For example, in one or more embodiments, the first convolution module 110 may include 32 filters, the first residual module (which may be as shown in FIG. 1 ) ... Figure 3 The embodiment in FIG. 3 may include 64 convolution filters (for the first convolution module 310) and 128 filters (for each of the two other convolution modules 320 and 325), and the second residual module (which may also be Figure 3 ) may include 128 convolution filters (for the first convolution module 310) and 256 filters (for each of the two other convolution modules 320 and 325). It should be noted that other combinations and sizes of filters may be used.

[0036] C. Effective Training Examples

[0037] To speed up the learning process, an embodiment of the improved training process can be used. In one or more embodiments, to help speed up learning, an adaptive learning rate function can be used. For example, in one or more embodiments, a piecewise linear learning rate function can be used, where the function first increases linearly to a peak point and then decreases linearly to a small value close to zero. By using this method, the convergence time can be significantly reduced to about 20% of its original convergence time.

[0038] An exemplary adaptive learning rate function is given below (although different learning rate functions can also be used):

[0039]

[0040] Figure 4 An example learning rate graph according to an embodiment of the present disclosure is illustrated. In one or more embodiments, the learning rate function includes a first linear portion 405 and a second linear portion 410, in which the learning rate increases linearly from zero or near zero to a peak point as the number of training cycles (epochs) increases, and in the second linear portion 410, the learning rate decreases linearly from a peak point (which may be a different peak point) to near zero as the number of training cycles increases. In one or more embodiments, the magnitude of the slope of the first linear portion 405 is greater than the magnitude of the slope of the second linear portion 410. It should be noted that in one or more embodiments, the cyclic learning rate may be implemented as a lookup table.

[0041] In one or more embodiments, to further speed up the training process, reduced bit precision training may also be utilized, as it has been shown to efficiently utilize GPU hardware space. For example, in one or more embodiments, half-precision floating point format (FP 16-bit) may be used to represent all floating point numbers in all operations throughout the training process.

[0042] In one or more embodiments, in addition to the above method, a larger batch size can be used during training, such as 512 images. When determining the batch size, the batch is preferably determined so that the storage requirement of the batch is less than the storage limit of the processor used to train the image classification model, which can be a GPU or multiple GPUs.

[0043] Figure 5 A method for training an image classification model according to an embodiment of the present disclosure is illustrated. In one or more embodiments, the training process includes determining (505) a learning rate for a cycle. As described above, a piecewise linear function that relates the number of training cycles to the learning rate can be used to determine the learning rate for the training cycle. A set of images and their corresponding reference answer (ground truth) labels can be selected (510) from one or more training data sets to form a batch. The batch is input (515) into an image classification model embodiment, which performs label predictions on the images in the batch. Given the output of the model, the loss (520) of the image classification model can be determined. In one or more embodiments, a cross entropy loss function that can include one or more regularization terms is used. The loss can then be used to update (525) the parameters of the image classification model. For example, in one or more embodiments, gradient descent and back propagation can be used to update the model parameters.

[0044] In one or more embodiments, steps 510-525 may be repeated (530) until the loop is complete. Typically, a loop will go through all the training data.

[0045] In one or more embodiments, steps 505-530 (535) may be repeated until a stop condition is reached. Examples of stop conditions include, but are not limited to, number of iterations, number of loops, convergence of the model (e.g., error or loss between consecutive iterations is less than a threshold), divergence (e.g., error or loss between consecutive iterations increases, or spans a set of consecutive iterations exceeding a threshold), processing time, etc. When the training process 500 ends, the trained model may be output (540) for image classification.

[0046] It should be noted that Figure 5The method in is an exemplary embodiment. Those skilled in the art will recognize that the steps can be changed. For example, all learning rate values ​​for different cycles can be calculated and converted into a lookup table for training. Also as an example, instead of forming batches in each iteration, the training data can be randomly distributed to all batches at once at the beginning of the cycle. As previously described, the batch size can be selected to correspond to the memory of the processing system. Other such substitutions can be made and fall within the scope of the present disclosure.

[0047] D. Use the trained model to classify images

[0048] After the model is trained, the trained image classification model can be used to classify one or more images. Figure 6 A method of image classification using a trained model according to an embodiment of the present disclosure is depicted. In one or more embodiments, an unlabeled image is input (605) into a trained image classification model. The image classification model can be a tiny model embodiment as discussed herein, comprising: a convolution module, comprising a convolution with a set of filters, a batch normalization operation, and an activation operation; a first residual module, comprising at least two convolution modules separated by a max pooling layer; a second residual module, comprising at least two convolution modules separated by a max pooling layer; and a fully connected layer, outputting a label. After the image has been input into the trained model, the model outputs (610) a predicted label for the image.

[0049] E. Computing System Embodiments

[0050] In one or more embodiments, aspects of this patent document may be directed to one or more information processing systems / computing systems, may include one or more information processing systems / computing systems, or may be implemented on one or more information processing systems / computing systems. A computing system may include any tool or set of tools operable to calculate, compute, determine, classify, process, send, receive, retrieve, originate, route, convert, store, display, communicate, manifest, detect, record, reproduce, cope with or utilize any form of information, knowledge or data. For example, a computing system may be or may include a personal computer (e.g., a laptop computer), a tablet computer, a tablet phone, a personal digital assistant (PDA), a smart phone, a smart watch, a smart bag, a server (e.g., a blade server or a rack server), a network storage device, a camera or any other suitable device, and may vary in size, shape, performance, function and price. A computing system may include a random access memory (RAM), one or more processing resources (such as a central processing unit (CPU) or hardware or software control logic), a ROM, and / or other types of memory. Additional components of a computing system may include one or more disk drives, one or more network ports for communicating with external devices, and various input and output (I / O) devices (e.g., a keyboard, a mouse, a touch screen, and / or a video display). A computing system may also include one or more buses for transmitting communications between various hardware components.

[0051] Figure 7 700 is a simplified block diagram of a computing device / information processing system (or computing system) according to an embodiment of the present disclosure. It should be understood that the functionality shown in system 700 can be used to support various embodiments of the computing system, although it should be understood that the computing system can be configured differently and include different components, including having Figure 7 Fewer or more components than shown.

[0052] like Figure 7 As shown, computing system 700 includes one or more central processing units (CPU) 701, which provide computing resources and control the computer. CPU 701 can be implemented with a microprocessor, etc., and can also include one or more graphics processing units (GPU) 719 and / or floating point coprocessors for mathematical calculations. System 700 can also include system memory 702, which can be in the form of random access memory (RAM), read-only memory (ROM), or both.

[0053] Multiple controllers and peripherals can also be provided, such as Figure 7As shown. Input controller 703 represents an interface to various input devices 704 such as a keyboard, mouse, touch screen and / or stylus. The computing system 700 may also include a storage controller 707 for interfacing with one or more storage devices 708, each of which includes a storage medium such as a tape or disk or an optical medium that can be used to record a program of instructions for an operating system, utility program and application program, which may include an embodiment of a program that implements various aspects of the present disclosure. The storage device 708 may also be used to store data processed or to be processed according to the present disclosure. The system 700 may also include a display controller 709 for providing an interface to a display device 711, which may be a cathode ray tube (CRT), a thin film transistor (TFT) display, an organic light emitting diode, an electroluminescent panel, a plasma panel or other type of display. The computing system 700 may also include one or more peripheral device controllers or interfaces 705 for one or more peripheral devices 706. Examples of peripheral devices may include one or more printers, scanners, input devices, output devices, sensors, etc. The communication controller 714 can interface with one or more communication devices 715, which enables the system 700 to connect to remote devices via any of a variety of networks including the Internet, cloud resources (e.g., Ethernet cloud, Fibre Channel over Ethernet (FCoE) / Data Center Bridging (DCB) cloud, etc., local area network (LAN), wide area network (WAN), storage area network (SAN)), or via any suitable electromagnetic carrier signal including infrared signals.

[0054] In the illustrated system, all major system components may be connected to bus 716, which may represent more than one physical bus. However, various system components may or may not be physically close to each other. For example, input data and / or output data may be remotely transmitted from one physical location to another physical location. In addition, programs implementing various aspects of the present disclosure may be accessed from a remote location (e.g., a server) over a network. Such data and / or programs may be transmitted via any of a variety of machine-readable media, including, but not limited to: magnetic media such as hard disks, floppy disks, and tapes; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices specifically configured to store or execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices.

[0055] Aspects of the present disclosure may be encoded on one or more non-transitory computer-readable media having instructions for one or more processors or processing units to perform steps. It should be noted that one or more non-transitory computer-readable media may include volatile and / or non-volatile memory. It should be noted that alternative embodiments are possible, including hardware embodiments or software / hardware embodiments. Hardware-implemented functions can be implemented using ASICs, programmable arrays, digital signal processing circuits, etc. Therefore, the term "device" in any claim is intended to cover both software and hardware implementations. Similarly, the term "computer-readable medium" used herein includes software and / or hardware having an instruction program included thereon, or a combination thereof. In view of these implementation substitutions, it should be understood that the drawings and the accompanying descriptions provide those skilled in the art with the functional information required to write program code (i.e., software) and / or manufacture circuits (i.e., hardware) to perform the desired processing.

[0056] It should be noted that embodiments of the present disclosure may also relate to computer products with non-transitory, tangible computer-readable media having computer code thereon for performing various computer-implemented operations. The media and computer code may be those specifically designed and constructed for the purposes of the present disclosure, or they may be of a type known or available to those skilled in the relevant art. Examples of tangible computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and tapes; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices specifically configured to store or execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM devices and RAM devices. Examples of computer code include machine code such as produced by a compiler, and files containing higher-level code executed by a computer using an interpreter. Embodiments of the present disclosure may be implemented in whole or in part as machine executable instructions, which may be in program modules executed by a processing device. Examples of program modules include libraries, programs, routines, objects, components, and data structures. In a distributed computing environment, program modules may be physically located in a local, remote, or both setting.

[0057] Those skilled in the art will recognize that no computing system or programming language is critical to the practice of the present disclosure. Those skilled in the art will also recognize that the above-mentioned multiple elements can be physically and / or functionally separated into sub-modules or combined together.

[0058] It will be appreciated by those skilled in the art that the foregoing embodiments and implementations are exemplary, rather than limiting the scope of the present disclosure. All permutations, enhancements, equivalences, combinations, and improvements thereof that are apparent to those skilled in the art upon reading this specification and studying the accompanying drawings are included within the true spirit and scope of the present disclosure. It should also be noted that the elements of any claim may be arranged differently, including having multiple dependencies, configurations, and combinations.

Claims

1. A computer-implemented method for training an image classification model, the method comprising: forming one or more batches, the one or more batches comprising images and labels corresponding to the images, the images and labels corresponding to the images being selected from one or more training datasets, wherein each image has a corresponding label; A set of steps is repeated for each training cycle until a stopping condition is reached, the set of steps comprising: Inputting the batch into the image classification model, the image classification model comprising a convolutional module, a first residual module, a second residual module and a fully connected layer connected in sequence; the convolutional module comprises a convolution with a set of filters, a batch normalization operation and an activation operation; the first residual module comprises at least two convolutional modules separated by a maximum pooling layer, wherein each convolutional module has its own filter bank; the second residual module comprises at least two convolutional modules separated by a maximum pooling layer, wherein each convolutional module has its own filter bank; the fully connected layer receives an input obtained from the output of the second residual module; determining a loss for the image classification model given the predicted outputs for the batch; and updating one or more parameters of the image classification model using the loss; The method also includes determining a learning rate for a training cycle using a piecewise linear function that relates a number of training cycles to a learning rate.

2. The computer-implemented method of claim 1, wherein: The piecewise linear function includes: A first linear portion, in which the learning rate increases linearly from zero or near zero to a peak point as the number of training cycles increases; and A second linear portion, wherein the learning rate decreases linearly from a peak point to near zero as the number of training cycles increases, wherein the slope of the first linear portion is greater than the slope of the second linear portion.

3. The computer-implemented method of claim 1, wherein: At least one of the first residual module and the second residual module includes an increased number of filters to increase feature representation of the image classification model.

4. The computer-implemented method of claim 1, wherein: At least one of the first residual module and the second residual module further includes two convolution modules located after the maximum pooling layer.

5. The computer-implemented method of claim 1, wherein: The number of filters used for convolution is matched to the parallel capability of the processor units of the system used to train the image classification model.

6. The computer-implemented method of claim 1, wherein: The number of images selected for a batch is determined such that a storage requirement for the batch is less than a memory limit of a processor unit used to train the image classification model.

7. A system for training an image classification model, the system comprising: one or more processors; as well as A non-transitory computer readable medium comprising one or more sets of instructions which, when executed by at least one of the one or more processors, cause the following steps to be performed: forming one or more batches, the one or more batches comprising images and labels corresponding to the images, the images and labels corresponding to the images being selected from one or more training datasets, wherein each image has a corresponding label; A set of steps is repeated for each training cycle until a stopping condition is reached, the set of steps comprising: Inputting the batch into the image classification model, the image classification model comprising a convolutional module, a first residual module, a second residual module and a fully connected layer connected in sequence; the convolutional module comprises a convolution with a set of filters, a batch normalization operation and an activation operation; the first residual module comprises at least two convolutional modules separated by a maximum pooling layer, wherein each convolutional module has its own filter bank; the second residual module comprises at least two convolutional modules separated by a maximum pooling layer, wherein each convolutional module has its own filter bank; the fully connected layer receives an input obtained from the output of the second residual module; determining a loss for the image classification model given the predicted outputs for the batch; and updating one or more parameters of the image classification model using the loss; The one or more sets of instructions, when executed by at least one of the one or more processors, further perform steps including determining a learning rate for a training cycle using a piecewise linear function that relates a number of training cycles to a learning rate.

8. The system of claim 7, wherein: The piecewise linear function includes: A first linear portion, in which the learning rate increases linearly from zero or near zero to a peak point as the number of training cycles increases; and A second linear portion, wherein the learning rate decreases linearly from a peak point to near zero as the number of training cycles increases, wherein the slope of the first linear portion is greater than the slope of the second linear portion.

9. The system of claim 7, wherein: At least one of the first residual module and the second residual module includes an increased number of filters to increase feature representation of the image classification model.

10. The system of claim 7, wherein: At least one of the first residual module and the second residual module further includes two convolution modules located after the maximum pooling layer.

11. The system of claim 7, wherein: The number of images selected for a batch is determined such that a storage requirement for the batch is less than a memory limit of a processor unit used to train the image classification model.

12. A computer-implemented method for classifying an image, the method comprising: Inputting an input image into a classification model, and classifying the input image using the classification model, wherein the classification model is trained using the computer-implemented method according to any one of claims 1 to 6; the classification model comprises a convolution module, a first residual module, a second residual module, and a fully connected layer connected in sequence; the convolution module comprises a convolution with a set of filters, a batch normalization operation, and an activation operation; the first residual module comprises at least two convolution modules separated by a maximum pooling layer, wherein each convolution module has its own filter group; the second residual module comprises at least two convolution modules separated by a maximum pooling layer; and Output the classification label of the input image.

13. The computer-implemented method of claim 12, wherein: At least one of the first residual module and the second residual module further includes two convolution modules located after the maximum pooling layer.

14. The computer-implemented method of claim 12, wherein: At least one of the first residual module and the second residual module further includes: The output of the maximum pooling layer is combined with the output of the last convolution module of the at least one residual module.

15. The computer-implemented method of claim 12, wherein: At least one of the first residual module and the second residual module includes an increased number of filters to increase feature representation of the classification model.

16. The computer-implemented method of claim 12, wherein: The number of filters used for convolution is matched to the parallel capabilities of the processor units used to train the classification model.

Images