AI algorithm optimization method and system based on deep learning technology

Abstract

The invention discloses an AI algorithm optimization method and system based on a deep learning technology, and relates to the technical field of deep learning model optimization, and the method comprises the steps: carrying out the preprocessing of a data set, carrying out the normalization and standardization processing, and dividing the data set into a training set, a verification set and a test set; a self-attention mechanism is introduced into a convolutional layer of the ResNet-18 deep learning model, and the weight of each channel is dynamically adjusted by calculating a self-similarity matrix of a feature map; training the model by using the training set data, and dynamically adjusting the model learning rate in the training process; an optimal hyper-parameter combination is automatically searched through a hyper-parameter tuning method; the number of model parameters and the calculation amount are reduced through a static quantification method; exporting and deploying the model, importing the test set, and testing the performance of the model; according to the method, a self-attention mechanism is introduced into the convolutional layer of the ResNet-18 model, the weight of each channel is dynamically adjusted, the attention of the model to important features is enhanced, and the feature extraction precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning model optimization technology, and in particular to an AI algorithm optimization method and system based on deep learning technology. Background Art

[0002] In recent years, deep learning technology has made significant progress in many fields such as computer vision, natural language processing, speech recognition, etc. Deep neural networks (such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs)) can automatically learn complex feature representations from large amounts of data through multi-level nonlinear transformations, thereby demonstrating excellent performance in various tasks.

[0003] However, existing deep learning technologies still have some shortcomings in these aspects. The existing way of introducing the self-attention mechanism is relatively simple and difficult to adapt to the needs of different tasks. Especially when processing complex feature maps, the model's feature extraction ability is limited. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an AI algorithm optimization method based on deep learning technology to solve the problem of insufficient flexibility of the self-attention mechanism.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an AI algorithm optimization method based on deep learning technology, which comprises:

[0008] Preprocess the data set, normalize and standardize it, and divide the data set into training set, validation set and test set;

[0009] The self-attention mechanism is introduced in the convolutional layer of the ResNet-18 deep learning model to dynamically adjust the weight of each channel by calculating the self-similarity matrix of the feature map;

[0010] Use the training set data to train the model and dynamically adjust the model learning rate during the training process;

[0011] Automatically search for the optimal hyperparameter combination through hyperparameter tuning methods;

[0012] Reduce the number of model parameters and the amount of calculation through static quantization method;

[0013] Export and deploy the model, import the test set, and test the model performance.

[0014] As a preferred solution of the AI ​​algorithm optimization method based on deep learning technology described in the present invention, wherein: the data set is preprocessed, normalized and standardized, and the data set is divided into a training set, a validation set and a test set. The specific steps are as follows:

[0015] Clean the image classification dataset at the initial input layer of the deep learning model to remove noise and outliers;

[0016] Normalize all data in the dataset to the interval [0,1];

[0017] Standardize the normalized data set;

[0018] The processed dataset is divided into training set, validation set and test set.

[0019] As a preferred solution of the AI ​​algorithm optimization method based on deep learning technology described in the present invention, wherein: the self-attention mechanism is introduced into the convolutional layer of the ResNet-18 deep learning model, and the weight of each channel is dynamically adjusted by calculating the self-similarity matrix of the feature map. The specific steps are as follows:

[0020] Introducing the self-attention mechanism in the convolutional layer of the ResNet-18 deep learning model;

[0021] The cleaned, normalized and standardized image data is used as the input of the feature map F in the self-attention mechanism and reshaped;

[0022] The reshaped feature maps are converted into query matrix Q, key matrix K, and value matrix respectively through convolutional layers;

[0023] Calculate the product of the query matrix and the key matrix to obtain the self-similarity matrix S;

[0024] Normalize the self-similar matrix;

[0025] Use the normalized self-similarity matrix to perform weighted fusion on the value matrix to obtain a new feature map F';

[0026] Add the new feature map to the input feature map to form a residual connection

[0027] Adjust the residual blocks of the ResNet-18 model. In each residual block, adjust the structure after the convolution layer and add self-attention mechanism and residual connection.

[0028] The weights of the convolutional layer are initialized using the Kaiming initialization method, and the bias of the batch normalization layer is initialized using a constant 0.

[0029] As a preferred solution of the AI ​​algorithm optimization method based on deep learning technology described in the present invention, wherein: the model is trained using training set data, and the model learning rate is dynamically adjusted during the training process. The specific steps are as follows:

[0030] Import the training set into the ResNet-18 model with the self-attention mechanism for training;

[0031] During the training process, the training set data is enhanced by rotation, flipping, and cropping;

[0032] The weighted cross entropy loss function is used as the standard loss function of the ResNet-18 model, and the expression is as follows:

[0033]

[0034] Among them, L is the loss value of the ResNet-18 model, N is the number of samples, ω i is the i-th sample weight, y i is the i-th true value, p i is the i-th predicted probability, i is the index of the number of samples;

[0035] Use the Adam optimizer to perform preliminary learning rate adjustments and initialize the learning rate, batch size, momentum parameter, and weight decay parameter;

[0036] Monitor and record the gradient changes and loss values ​​during training, and calculate the L2 norm of the gradient after each training batch;

[0037] According to the change of gradient and loss value, the learning rate is adjusted dynamically. The expression is as follows:

[0038]

[0039] Among them, η t+1 and η t are the learning rates of the t+1th and tth training batches, α is the adjustment coefficient of the gradient change, G t and G t-1 are the L2 norms of the gradients of the t-th and t-1-th training batches, β is the adjustment coefficient for the change in loss value, L t and L t-1 are the loss values ​​of the t-th and t-1-th training batches respectively;

[0040] when and Among them, β1 is the maximum threshold of gradient change, γ1 is the minimum change of loss value, and the learning rate is increased;

[0041] when and Among them, β2 is the minimum threshold of gradient change, γ2 is the maximum change of loss value, and the learning rate is reduced.

[0042] As a preferred solution of the AI ​​algorithm optimization method based on deep learning technology described in the present invention, wherein: the optimal hyperparameter combination is automatically searched through the hyperparameter tuning method, and the specific steps are as follows:

[0043] Determine the search space for hyperparameters based on the training process data of the ResNet-18 model;

[0044] By randomly sampling a set of hyperparameters from the hyperparameter search space, a ResNet-18 model with a self-attention mechanism is constructed, training and performance verification are performed, and the negative accuracy of the model is returned;

[0045] The set of hyperparameters with the smallest negative accuracy returned by verification is regarded as the optimal hyperparameter combination;

[0046] The ResNet-18 model with the self-attention mechanism was constructed using the optimal hyperparameters, and the training set was imported for training. After the training, the validation set was used to verify the ResNet-18 model with the self-attention mechanism.

[0047] As a preferred solution of the AI ​​algorithm optimization method based on deep learning technology described in the present invention, wherein: the static quantization method is used to reduce the number of model parameters and the amount of calculation, and the specific steps are as follows:

[0048] Load the ResNet-18 model weights, convert the model weights from floating point numbers to integers, and convert the model activation values ​​from floating point numbers to integers;

[0049] Calibrate the model by running the training set data;

[0050] Use the quantization tools provided by PyTorch to convert the model into a static quantization model;

[0051] Use the validation set to verify the performance of the model after static quantization.

[0052] As a preferred solution of the AI ​​algorithm optimization method based on deep learning technology described in the present invention, the specific steps of exporting and deploying the model, importing the test set, and testing the model performance are as follows:

[0053] Export the verified ResNet-18 model and deploy it;

[0054] Install the model's dependencies and load the model using package management tools and Python package managers;

[0055] Import the test set data into the model and run it;

[0056] Record the running time and memory usage of each sample in the test set;

[0057] Calculate the average inference time and maximum memory usage.

[0058] In a second aspect, the present invention provides an AI algorithm optimization system based on deep learning technology, including a data preparation module, a model design module, a training optimization module, a hyperparameter tuning module, a model compression module and a model testing module;

[0059] The data preparation module is used to preprocess the data set, perform normalization and standardization, and divide the data set into a training set, a validation set, and a test set;

[0060] The model design module is used to introduce a self-attention mechanism into the convolutional layer of the ResNet-18 deep learning model, and dynamically adjust the weight of each channel by calculating the self-similarity matrix of the feature map;

[0061] The training optimization module is used to train the model using the training set data and dynamically adjust the model learning rate during the training process;

[0062] The hyperparameter tuning module is used to automatically search for the optimal hyperparameter combination through a hyperparameter tuning method;

[0063] The model compression module is used to reduce the number of model parameters and the amount of calculation through a static quantization method;

[0064] The model testing module is used to export and deploy models, import test sets, and test model performance.

[0065] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the AI ​​algorithm optimization method based on deep learning technology as described in the first aspect of the present invention is implemented.

[0066] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the AI ​​algorithm optimization method based on deep learning technology as described in the first aspect of the present invention.

[0067] The beneficial effects of the present invention are as follows: the present invention cleans, normalizes and standardizes the data set through image processing tools, and reasonably divides the data set, thereby ensuring the quality and consistency of the data, improving the training efficiency and generalization ability of the model, introducing a self-attention mechanism in the convolutional layer of the ResNet-18 model, dynamically adjusting the weight of each channel, enhancing the model's attention to important features, and improving the accuracy and robustness of feature extraction; by dynamically adjusting the learning rate and automatically searching for the optimal hyperparameter combination, the model can converge faster during the training process, avoiding overfitting and parameter oscillation problems; by exporting and deploying the model, efficient deployment and reasoning of the model are achieved, and the performance and stability of the model in practical applications are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0069] Figure 1 This is a flowchart of the AI ​​algorithm optimization method based on deep learning technology in Example 1.

[0070] Figure 2 This is a module diagram of the AI ​​algorithm optimization system based on deep learning technology in Example 1. DETAILED DESCRIPTION

[0071] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0072] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0073] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0074] Example 1, reference Figure 1 and Figure 2, which is the first embodiment of the present invention, and provides an AI algorithm optimization method based on deep learning technology, comprising the following steps:

[0075] Preprocess the data set, normalize and standardize it, and divide the data set into training set, validation set and test set;

[0076] The self-attention mechanism is introduced in the convolutional layer of the ResNet-18 deep learning model to dynamically adjust the weight of each channel by calculating the self-similarity matrix of the feature map;

[0077] Use the training set data to train the model and dynamically adjust the model learning rate during the training process;

[0078] Automatically search for the optimal hyperparameter combination through hyperparameter tuning methods;

[0079] Reduce the number of model parameters and the amount of calculation through static quantization method;

[0080] Export and deploy the model, import the test set, and test the model performance.

[0081] S1. Preprocess the data set, normalize and standardize it, and divide the data set into training set, validation set and test set.

[0082] Use OpenCV image processing tools to clean the image classification dataset at the initial input layer of the deep learning model to remove noise and outliers;

[0083] Normalize all data in the dataset to the interval [0,1];

[0084] The normalized data set is standardized so that its mean is 0 and standard deviation is 1;

[0085] The processed dataset is divided into training set, validation set and test set.

[0086] S2. The self-attention mechanism is introduced in the convolutional layer of the ResNet-18 deep learning model to dynamically adjust the weight of each channel by calculating the self-similarity matrix of the feature map.

[0087] Introducing the self-attention mechanism in the convolutional layer of the ResNet-18 deep learning model;

[0088] The cleaned, normalized and standardized image data is used as the input of the feature map F in the self-attention mechanism with a shape of (N, C, H, W) and reshaped to (N, C, HW);

[0089] The reshaped feature maps are converted into query matrix Q with shape (N, C', HW), key matrix K with shape (N, C', HW), and value matrix V with shape (N, C, HW) through convolutional layers.

[0090] Calculate the product of the query matrix and the key matrix to obtain the self-similarity matrix S, which is expressed as follows:

[0091] S=Q×K T ;

[0092] Among them, K T refers to the transpose of the key matrix K;

[0093] By applying the Softmax function, the self-similarity matrix is ​​normalized to ensure that it is within a reasonable range;

[0094] The normalized self-similarity matrix is ​​used to perform weighted fusion on the value matrix to obtain a new feature map F′, which is expressed as follows:

[0095] F′=S×V;

[0096] Add the new feature map to the input feature map to form a residual connection. The expression is as follows:

[0097] F out =γ·F′+F;

[0098] Among them, F out It is the output feature map after the self-attention mechanism is processed and residual connections are added. γ is a scalar parameter with an initial value of 0, which is used to control the influence of the self-attention module.

[0099] Adjust the residual blocks of the ResNet-18 model. In each residual block, adjust the structure after the convolution layer and add self-attention mechanism and residual connection.

[0100] Specifically, input feature map F, pass through the first convolution layer, perform convolution operation on the input feature map, output feature map F1, perform batch normalization operation on F1, output F2, apply ReLU activation function to F2, output F3, perform convolution operation on F3 in the second convolution layer, output F4, apply self-attention mechanism to F4, output new feature map F5, perform batch normalization operation on F5, output feature map F6, add feature map F6 to input feature map F to form residual connection, and obtain output feature map F out , for F out Apply the ReLU activation function to obtain the final feature map;

[0101] The weights of the convolutional layer are initialized using the Kaiming initialization method, and the bias of the batch normalization layer is initialized using a constant 0.

[0102] S3. Use the training set data to train the model and dynamically adjust the model learning rate during the training process.

[0103] Import the training set into the ResNet-18 model with the self-attention mechanism for training;

[0104] During the training process, the training set data is enhanced by rotation, flipping, and cropping to increase data diversity;

[0105] The weighted cross entropy loss function is used as the standard loss function of the ResNet-18 model to measure the difference between the predicted value of the ResNet-18 model and the true value. The expression is as follows:

[0106]

[0107] Among them, L is the loss value of the ResNet-18 model, N is the number of samples, ω i is the weight of the i-th sample, which is inversely proportional to the frequency of the sample in the training set, y i is the i-th true value, p i is the i-th predicted probability, i is the index of the number of samples;

[0108] Use the Adam optimizer to perform preliminary learning rate adjustments and initialize the learning rate, batch size, momentum parameter, and weight decay parameter;

[0109] Monitor and record the gradient changes and loss values ​​during training, and calculate the L2 norm of the gradient after each training batch;

[0110] According to the change of gradient and loss value, the learning rate is adjusted dynamically. The expression is as follows:

[0111]

[0112] Among them, η t+1 and η t are the learning rates of the t+1th and tth training batches, α is the adjustment coefficient of the gradient change, G t and G t-1 are the L2 norms of the gradients of the t-th and t-1-th training batches, β is the adjustment coefficient for the change in loss value, L t and L t-1 are the loss values ​​of the t-th and t-1-th training batches respectively;

[0113] when and Among them, β1 is the maximum threshold of gradient change, which is set to 0.1, and γ1 is the minimum value of loss value change, which is set to 0.01, indicating that the gradient changes greatly and the loss value decreases slowly. Increasing the learning rate can accelerate convergence and jump out of the local minimum.

[0114] when and Among them, β2 is the minimum threshold of gradient change, which is set to 0.01, and γ2 is the maximum value of loss value change, which is set to 0.1. Both are set to 0.01, which means that the gradient change is small and the loss value decreases quickly. Reduce the learning rate and fine-tune the model parameters to prevent excessive parameter updates from causing model oscillation.

[0115] S4. Automatically search for the optimal hyperparameter combination through hyperparameter tuning methods.

[0116] Determine the search space for hyperparameters based on the training process data of the ResNet-18 model;

[0117] Hyperparameters include learning rate, batch size, momentum parameter, and weight decay parameter;

[0118] Use the Optuna tuning library to automatically search for hyperparameter combinations;

[0119] By randomly sampling a set of hyperparameters from the hyperparameter search space, a ResNet-18 model with a self-attention mechanism is constructed, training and performance verification are performed, and the negative accuracy of the model is returned;

[0120] The set of hyperparameters with the smallest negative accuracy returned by verification is regarded as the optimal hyperparameter combination;

[0121] The ResNet-18 model with the self-attention mechanism was constructed using the optimal hyperparameters, and the training set was imported for training. After the training, the validation set was used to verify the ResNet-18 model with the self-attention mechanism.

[0122] S5. Reduce the number of model parameters and the amount of calculation through static quantization method.

[0123] Load the ResNet-18 model weights, convert the model weights from floating point numbers to 8-bit integers, and convert the model activation values ​​from floating point numbers to 8-bit integers;

[0124] Calibrate the model by running the training set data;

[0125] Use the quantization tools provided by PyTorch to convert the model into a static quantization model;

[0126] Use the validation set to verify the performance of the statically quantized model to ensure that the quantized model still has good accuracy;

[0127] It should be noted that static quantization not only reduces the number of model parameters and computational complexity, but also maintains the accuracy of the model, making it suitable for resource-constrained environments such as embedded devices and mobile devices.

[0128] S6. Export and deploy the model, import the test set, and test the model performance.

[0129] The verified ResNet-18 model is exported to ONNX format and deployed on the GPU server;

[0130] Install the model's dependencies and load the model using package management tools and Python package managers;

[0131] Import the test set data into the model and run it;

[0132] Record the running time and memory usage of each sample in the test set;

[0133] Calculate the average inference time and maximum memory usage.

[0134] The present embodiment also provides an AI algorithm optimization system based on deep learning technology, including: a data preparation module, a model design module, a training optimization module, a hyperparameter tuning module, a model compression module and a model testing module; the data preparation module is used to preprocess the data set, and perform normalization and standardization processing, and divide the data set into a training set, a validation set and a test set; the model design module is used to introduce a self-attention mechanism in the convolutional layer of the ResNet-18 deep learning model, and dynamically adjust the weight of each channel by calculating the self-similarity matrix of the feature map; the training optimization module is used to train the model using the training set data, and dynamically adjust the model learning rate during the training process; the hyperparameter tuning module is used to automatically search for the optimal hyperparameter combination through the hyperparameter tuning method; the model compression module is used to reduce the number of model parameters and the amount of calculation through the static quantization method; the model testing module is used to export and deploy the model, import the test set, and test the model performance.

[0135] This embodiment also provides a computer device, which is suitable for the AI ​​algorithm optimization method based on deep learning technology, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the AI ​​algorithm optimization method based on deep learning technology proposed in the above embodiment.

[0136] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0137] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the AI ​​algorithm optimization method based on deep learning technology proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0138] In summary, the present invention uses image processing tools to clean, normalize and standardize the data set, and reasonably divides the data set to ensure the quality and consistency of the data, improve the training efficiency and generalization ability of the model, introduces a self-attention mechanism in the convolutional layer of the ResNet-18 model, dynamically adjusts the weight of each channel, enhances the model's attention to important features, and improves the accuracy and robustness of feature extraction. By dynamically adjusting the learning rate and automatically searching for the optimal hyperparameter combination, the model can converge faster during training and avoid overfitting and parameter oscillation problems. By exporting and deploying the model, efficient deployment and reasoning of the model are achieved, and the performance and stability of the model in practical applications are improved.

[0139] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An AI algorithm optimization method based on deep learning technology, characterized in that: include, Preprocess the data set, normalize and standardize it, and divide the data set into training set, validation set and test set; The self-attention mechanism is introduced in the convolutional layer of the ResNet-18 deep learning model to dynamically adjust the weight of each channel by calculating the self-similarity matrix of the feature map; Use the training set data to train the model and dynamically adjust the model learning rate during the training process; Automatically search for the optimal hyperparameter combination through hyperparameter tuning methods; Reduce the number of model parameters and the amount of calculation through static quantization method; Export and deploy the model, import the test set, and test the model performance.

2. The AI ​​algorithm optimization method based on deep learning technology according to claim 1, characterized in that: The data set is preprocessed, normalized and standardized, and the data set is divided into a training set, a validation set and a test set. The specific steps are as follows: Clean the image classification dataset at the initial input layer of the deep learning model to remove noise and outliers; Normalize all data in the dataset to the interval [0,1]; Standardize the normalized data set; The processed dataset is divided into training set, validation set and test set.

3. The AI ​​algorithm optimization method based on deep learning technology according to claim 2, characterized in that: The self-attention mechanism is introduced into the convolutional layer of the ResNet-18 deep learning model. The weight of each channel is dynamically adjusted by calculating the self-similarity matrix of the feature map. The specific steps are as follows: Introducing the self-attention mechanism in the convolutional layer of the ResNet-18 deep learning model; The cleaned, normalized and standardized image data is used as the input of the feature map F in the self-attention mechanism and reshaped; The reshaped feature maps are converted into query matrix Q, key matrix K, and value matrix respectively through convolutional layers; Calculate the product of the query matrix and the key matrix to obtain the self-similarity matrix S; Normalize the self-similar matrix; The normalized self-similarity matrix is ​​used to perform weighted fusion on the value matrix to obtain a new feature map F′; Add the new feature map to the input feature map to form a residual connection Adjust the residual blocks of the ResNet-18 model. In each residual block, adjust the structure after the convolution layer and add self-attention mechanism and residual connection. The weights of the convolutional layer are initialized using the Kaiming initialization method, and the bias of the batch normalization layer is initialized using a constant 0.

4. The AI ​​algorithm optimization method based on deep learning technology according to claim 3, characterized in that: The model is trained using the training set data, and the model learning rate is dynamically adjusted during the training process. The specific steps are as follows: Import the training set into the ResNet-18 model with the self-attention mechanism for training; During the training process, the training set data is enhanced by rotation, flipping, and cropping; The weighted cross entropy loss function is used as the standard loss function of the ResNet-18 model, and the expression is as follows: Among them, L is the loss value of the ResNet-18 model, N is the number of samples, ω i is the i-th sample weight, y i is the i-th true value, p i is the i-th predicted probability, i is the index of the number of samples; Use the Adam optimizer to perform preliminary learning rate adjustments and initialize the learning rate, batch size, momentum parameter, and weight decay parameter; Monitor and record the gradient changes and loss values ​​during training, and calculate the L2 norm of the gradient after each training batch; According to the change of gradient and loss value, the learning rate is adjusted dynamically. The expression is as follows: Among them, η t+1 and η t are the learning rates of the t+1th and tth training batches, α is the adjustment coefficient of the gradient change, G t and G t-1 are the L2 norms of the gradients of the t-th and t-1-th training batches, β is the adjustment coefficient for the change in loss value, L t and L t-1 are the loss values ​​of the t-th and t-1-th training batches respectively; when and Among them, β1 is the maximum threshold of gradient change, γ1 is the minimum change of loss value, and the learning rate is increased; when and Among them, β2 is the minimum threshold of gradient change, γ2 is the maximum change of loss value, and the learning rate is reduced.

5. The AI ​​algorithm optimization method based on deep learning technology according to claim 4, characterized in that: The hyperparameter tuning method is used to automatically search for the optimal hyperparameter combination. The specific steps are as follows: Determine the search space for hyperparameters based on the training process data of the ResNet-18 model; By randomly sampling a set of hyperparameters from the hyperparameter search space, a ResNet-18 model with a self-attention mechanism is constructed, training and performance verification are performed, and the negative accuracy of the model is returned; The set of hyperparameters with the smallest negative accuracy returned by verification is regarded as the optimal hyperparameter combination; The ResNet-18 model with the self-attention mechanism was constructed using the optimal hyperparameters, and the training set was imported for training. After the training, the validation set was used to verify the ResNet-18 model with the self-attention mechanism.

6. The AI ​​algorithm optimization method based on deep learning technology according to claim 5, characterized in that: The static quantization method is used to reduce the number of model parameters and the amount of calculation. The specific steps are as follows: Load the ResNet-18 model weights, convert the model weights from floating point numbers to integers, and convert the model activation values ​​from floating point numbers to integers; Calibrate the model by running the training set data; Use the quantization tools provided by PyTorch to convert the model into a static quantization model; Use the validation set to verify the performance of the model after static quantization.

7. The AI ​​algorithm optimization method based on deep learning technology according to claim 6, characterized in that: The specific steps of exporting and deploying the model, importing the test set, and testing the model performance are as follows: Export the verified ResNet-18 model and deploy it; Install the model's dependencies and load the model using package management tools and Python package managers; Import the test set data into the model and run it; Record the running time and memory usage of each sample in the test set; Calculate the average inference time and maximum memory usage.

8. An AI algorithm optimization system based on deep learning technology, based on the AI ​​algorithm optimization method based on deep learning technology according to any one of claims 1 to 7, characterized in that: Including data preparation module, model design module, training optimization module, hyperparameter tuning module, model compression module and model testing module; The data preparation module is used to preprocess the data set, perform normalization and standardization, and divide the data set into a training set, a validation set, and a test set; The model design module is used to introduce a self-attention mechanism into the convolutional layer of the ResNet-18 deep learning model, and dynamically adjust the weight of each channel by calculating the self-similarity matrix of the feature map; The training optimization module is used to train the model using the training set data and dynamically adjust the model learning rate during the training process; The hyperparameter tuning module is used to automatically search for the optimal hyperparameter combination through a hyperparameter tuning method; The model compression module is used to reduce the number of model parameters and the amount of calculation through a static quantization method; The model testing module is used to export and deploy models, import test sets, and test model performance.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the AI ​​algorithm optimization method based on deep learning technology described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the AI ​​algorithm optimization method based on deep learning technology described in any one of claims 1 to 7 are implemented.

Images