Method for improving calculation speed of model based on mercuric chloride AI processor

By deploying Ascend AI processor in the field of deep learning, optimizing memory access and data flow, and decomposing the computing process into multiple stages, the problems of slow model computing speed and low resource utilization in the existing technology are solved, and efficient training and accurate prediction are achieved.

CN120256106APending Publication Date: 2025-07-04四川华鲲振宇智能科技有限责任公司
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202510328317.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-19
Publication Date
2025-07-04

AI Technical Summary

Technical Problem

How to improve the computing speed of models, improve resource utilization in the field of deep learning, and solve the disadvantages of manual tuning.

Method used

Deploy Ascend AI processor and deep learning model in the server, optimize the memory access mode, adopt a double buffering mechanism and real-time monitoring module, and use preload and precise control of the timing of data flow, decompose the computing process into multiple stages, and optimize shared memory access.

Benefits of technology

It improves the calculation speed and resource utilization of the model, realizes efficient training of the model and the accuracy of prediction results, avoids data conflicts and delays, and maximizes resource utilization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120256106A_ABST
    Figure CN120256106A_ABST
Patent Text Reader

Abstract

The invention relates to a method for improving the calculation speed of a model based on a mercuric chloride AI processor. The method comprises the following steps: deploying the mercuric chloride AI processor and a deep learning model in a server, and carrying out adaptation and optimization on the mercuric chloride AI processor; before the data in the first buffer area is read, predicting and preloading the data to be processed, and loading the data from the global memory to the second buffer area in advance; a double-buffer mechanism is arranged in the mercuration AI processor, interrupt and event trigger points are set, the operation of loading data to a next buffer area is immediately started when a specified calculation stage is finished, and the time sequence of data circulation is accurately controlled; and a control parameter is automatically adjusted based on monitoring data of the real-time monitoring module, the calculation process is decomposed into a plurality of stages, different buffer areas are allocated for each stage, and access to the shared memory is optimized by setting a specified cache replacement strategy and the size of a cache line. According to the process, the calculation speed of the model in the deep learning field is improved, the resource utilization rate is improved, and the defect of manual adjustment and optimization is overcome.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning in artificial intelligence technology, and particularly relates to a method for improving the calculation speed of a model based on Ascend AI processors. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning models are increasingly widely used in various fields. However, deep learning models have high computational complexity and require a large amount of computing resources. As a high-performance artificial intelligence processor, Ascend AI processors provide powerful computing capabilities for the operation of deep learning models. However, how to more effectively improve the calculation speed of the model on Ascend AI processors and further improve the performance of the model is an urgent problem to be solved at present. The present invention is proposed under such a technical background.

[0003] In the field of deep learning, in order to improve the calculation speed of the model, researchers have proposed various implementation schemes. The implementation schemes in the prior art mainly include the following several types:

[0004] Optimization of traditional deep learning models: This scheme mainly focuses on the optimization at the model algorithm level, such as improving the network structure, using more efficient activation functions, optimizing the loss function, etc. However, this optimization scheme mainly relies on software-level improvements and has limited improvement in the performance of hardware accelerators.

[0005] Acceleration using general-purpose GPUs: As a general computing unit, GPUs have been widely used in the training of deep learning models. However, GPUs are not specifically designed for deep learning, and their computing efficiency and resource utilization rate are relatively low. In addition, GPUs consume a large amount of power and are not suitable for use on mobile devices or edge devices.

[0006] Optimization for specific hardware accelerators: In recent years, with the continuous development of deep learning technology, some hardware accelerators specifically designed for deep learning have been introduced, such as Tensor Processing Unit (TPU), NPU, etc. These hardware accelerators have high performance in the calculation of deep learning models. However, most of the existing optimization schemes for these hardware accelerators stay at the simple adaptation level and fail to fully utilize the characteristics of the hardware accelerators to improve the calculation speed of the model.

[0007] Therefore, how to improve the calculation speed of the model in the field of deep learning, improve resource utilization rate, and solve the disadvantages of manual tuning is an urgent technical problem to be solved at present. Summary of the Invention

[0008] The object of the present invention is to provide a method for improving the calculation speed of a model based on the Ascend AI processor, so as to improve the calculation speed of the model in the field of deep learning, improve the resource utilization rate, and solve the shortcomings of manual optimization.

[0009] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0010] A method for improving the calculation speed of a model based on the Ascend AI processor, comprising the following steps:

[0011] S1: Deploy the Ascend AI processor and the deep learning model in the server. Based on the high-speed memory and cache characteristics of the Ascend AI processor, optimize the memory access mode of the model, and perform adaptation and optimization on the Ascend AI processor based on the key operators in the deep learning model;

[0012] S2: Before the data in the first buffer is read, predict the data to be processed through the deep learning model, and based on the prediction result, initiate preloading of the data block to be processed in the global memory, and preload the data from the global memory to the second buffer in advance to achieve parallel processing of data preloading and the current data processing process;

[0013] S3: Arrange a double-buffer mechanism in the Ascend AI processor, set at least two buffers in the Ascend AI processor, including a first buffer and a second buffer, attach a timestamp to the loading and calculation operations of each data packet, so that when the data in the first buffer is read, the data in the second buffer has been loaded and completed according to a predetermined time;

[0014] S4: Set interrupt and event trigger points, and immediately initiate the operation of loading data to the next buffer at the end of the specified calculation stage to achieve precise control of the timing of data flow;

[0015] S5: Monitor the data loading and calculation in real time through a real-time monitoring module, and automatically adjust the control parameters based on the monitoring data of the real-time monitoring module. The control parameters include data loading rate and computing resource allocation;

[0016] S6: Decompose the calculation process into multiple stages, allocate different buffers for each stage, and optimize the access to the shared memory by specifying the cache replacement policy and cache line size setting.

[0017] Preferably, simultaneously in step S1, the key operators in the deep learning model are also optimized, and the specific process is as follows:

[0018] S11: Customize a handwritten operator through the underlying interface, and the underlying interface is the CUDA layer;

[0019] S12: Optimize the operator by fusing the convolutional layer and the pooling layer into one operator in the deep learning model, or fusing multiple fully connected layers into one large fully connected layer, or fusing multiple element-wise operations into one large element-wise operation, or fusing the convolutional layer and the batch normalization layer into one operator.

[0020] Preferably, the deep learning model is a neural network model. The specific process of using the deep learning model to predict the data to be processed in step S2 is as follows:

[0021] S21: Create a sample of the data to be processed within a specified period with labels and input it into the initialized deep learning model. Conduct model training through forward propagation, from the input layer through the hidden layer to the output layer, and calculate and store the results of each layer in the neural network.

[0022] S22: Based on the chain rule in calculus, traverse the network in the reverse order from the output layer through the hidden layer to the input layer through backpropagation, and alternately use forward propagation and backpropagation to update the model parameters using the gradients given by backpropagation.

[0023] S23: Input the current processed data and time information into the deep learning model, and the deep learning model predicts the data to be processed at a specified time based on the current processed data and time information.

[0024] Preferably, during the backpropagation process, repeatedly use the intermediate values stored in the forward propagation to guide the completion of this backpropagation.

[0025] It also includes the process of optimizing the hyperparameters of the deep learning model, specifically as follows:

[0026] S24: Generate a probability distribution model of the parameters in the high-dimensional function space and represent it using a non-linear model.

[0027] S25: Based on the generated probability distribution model, initialize a combination of hyperparameters and evaluate the initialized combination of hyperparameters through a test set.

[0028] S26: Construct a surrogate model of the objective function based on the sample of the data to be processed within a specified period with labels, and adopt a specified strategy to select the next sampling point according to the uncertainty of the surrogate model and the expectation of the objective function.

[0029] S27: After selecting the sampling point, sample the objective function and update the surrogate model, and iteratively execute the process of updating the surrogate model until the preset number of samplings is reached or the specified model performance requirement is met.

[0030] Preferably, the specific process of attaching a timestamp to the loading and calculation operations for each data packet in step S3 is as follows:

[0031] S31: The client sends a request data packet;

[0032] S32: After receiving the request data packet, the Ascend AI processor generates a hash value of the data packet or the calculation operation;

[0033] S33: The Ascend AI processor combines the current time with the hash value of the data packet or the computing operation to generate timestamp information;

[0034] S34: The Ascend AI processor adds the timestamp information to the data packet or computing operation.

[0035] Preferably, the double buffering mechanism deployed in the Ascend AI processor in step S2 is specifically as follows:

[0036] The dual control mechanism of timestamp and event-driven drive is adopted. By attaching a timestamp to the loading and calculation operations of each data packet, when the data in the first buffer is read, the data in the second buffer has been loaded at the scheduled time, and the data loading and calculation are monitored in real time by the real-time monitoring module, and the control parameters are automatically adjusted based on the monitoring data of the real-time monitoring module. Before the data in the first buffer is read, the data to be processed is predicted by the deep learning model, and the preloading of the data blocks to be processed in the global memory is started based on the prediction results, and the data is pre-loaded from the global memory to the second buffer, and the Ascend AI processor is used to execute the data preloading operation in parallel with the current data processing task.

[0037] Preferably, the specific implementation process in step S6 is as follows:

[0038] S61: The computational process of the deep learning model is subdivided into multiple independent and parallel computational stages. Each stage is assigned to a dedicated buffer for data processing. By finely dividing the computational tasks, it is ensured that there is no dependency or the dependency is minimized between the stages, thus realizing a pipelined processing process.

[0039] S62: dynamically adjusting processor resource allocation according to actual processing time and resource requirements of each computing stage to achieve load balancing of each stage of the pipeline and maximize resource utilization, wherein the processor resources include CPU cores and memory bandwidth;

[0040] S63: Reduce the occurrence of cache conflicts and cache misses by specifying a cache replacement strategy and a cache line size setting, and optimize the access to the shared memory.

[0041] Preferably, the specific process in step S63 is as follows:

[0042] S631: For the cache of shared memory, a hybrid cache replacement policy based on the usage frequency and the LRU cache eviction policy is adopted, and a machine learning algorithm is introduced to predict the data access pattern and dynamically adjust the cache replacement logic.

[0043] S632: According to the data access characteristics of the deep learning model, customize the cache line size and layout, allocate memory spaces of specified sizes for the first buffer and the second buffer, write specified data loading and computing logics, and use synchronization primitives to coordinate the data access and computing between different threads or tasks. The synchronization primitives include locks or semaphores.

[0044] The beneficial effects of the present invention include:

[0045] The method for improving the model calculation speed based on the Ascend AI processor provided by the present invention includes deploying the Ascend AI processor and the deep learning model in the server and performing adaptation and optimization on the Ascend AI processor; predicting and preloading the data to be processed before the data in the first buffer is read, and preloading the data from the global memory to the second buffer; arranging a double-buffer mechanism on the Ascend AI processor, setting the interrupt and event trigger points, and immediately starting the operation of loading the data to the next buffer at the end of the specified calculation stage to achieve precise control of the timing of data flow; automatically adjusting the control parameters based on the monitoring data of the real-time monitoring module, decomposing the calculation process into multiple stages, allocating different buffers for each stage, and optimizing the access to the shared memory by specifying the cache replacement policy and the cache line size setting. The above process improves the calculation speed of the model in the field of deep learning, improves the resource utilization rate, and solves the disadvantages of manual tuning.

[0046] First, customize the handwritten operator through the underlying interface. The underlying interface is a process of optimizing the operator in the way that the convolutional layer and the pooling layer in the deep learning model are fused into one operator, or multiple fully connected layers are fused into one large fully connected layer, or multiple element-wise operations are fused into one large element-wise operation, or the convolutional layer and the batch normalization layer are fused into one operator, which can more flexibly implement personalized operators or optimize the performance of existing operators, and can better utilize the parallelization advantage of the computing units of the GPU.

[0047] Secondly, by creating and inputting the to-be-processed data samples within the specified period with tags, model training is carried out through the forward propagation method; based on the chain rule in calculus, through the backward propagation method, the network is traversed in the reverse order from the output layer through the hidden layer to the input layer, and by alternately using forward propagation and backward propagation, the gradient given by the backward propagation is used to update the model parameters; the process of inputting the current processed data and time information into the deep learning model realizes the efficient training of the model, and can realize the real-time optimization and training of the model, improving the accuracy of the prediction results of the to-be-processed data of the model.

[0048] Thirdly, by generating a probability distribution model of the parameters in the high-dimensional function space and representing it with a non-linear model, on the basis of the generated probability distribution model, a combination of hyperparameters is initialized, and the initialized combination of hyperparameters is evaluated through the test set; a surrogate model of the objective function is constructed according to the to-be-processed data samples within the specified period with tags, and according to the uncertainty of the surrogate model and the expectation of the objective function, a specified strategy is adopted to select the next sampling point; after selecting the sampling point, the objective function is sampled and the surrogate model is updated, and the process of updating the surrogate model is iteratively executed until the preset number of samplings is reached or the specified model performance requirement is met, further realizing the process of hyperparameter optimization of the model and further improving the accuracy of the prediction results of the to-be-processed data of the model.

[0049] Thirdly, by the client sending a request data packet, the Ascend AI processor generates the hash value of the data packet or the computing operation; the Ascend AI processor combines the current time with the hash value of the data packet or the computing operation to generate timestamp information; the process of the Ascend AI processor adding the timestamp information to the data packet or the computing operation realizes the seamless switching and data synchronization between the two buffers, precisely controlling the timing of data flow and effectively avoiding data conflicts and delays.

[0050] Finally, by subdividing the computing process of the deep learning model into multiple independent and parallel computing stages, each stage is assigned to a dedicated buffer for data processing to realize the pipelined processing process; according to the actual processing time and resource requirements of each computing stage; by specifying the cache replacement policy and cache line size setting to reduce the occurrence of cache conflicts and cache misses, and realizing the optimization process of accessing the shared memory for the access to the shared memory, realizing the dynamic adjustment of the processor resource allocation, realizing the load balancing of each stage of the pipeline, and maximizing the resource utilization rate. Description of the Drawings

[0051] Figure 1 It is a schematic flowchart of the method for improving the model computing speed based on the Ascend AI processor of the present invention.

[0052] Figure 2Schematic diagram of the process of predicting data to be processed through a deep learning model according to the present invention.

[0053] Figure 3 Schematic diagram of the process of optimizing the access to shared memory according to the present invention. Detailed implementation manners

[0054] The following further describes the present invention in detail with reference to the attached Figures 1 to 3 :

[0055] Embodiment 1

[0056] Referring to the attached Figure 1 shown, a method for improving the model calculation speed based on the Ascend AI processor includes the following steps:

[0057] S1: Deploy the Ascend AI processor and the deep learning model in the server, optimize the memory access mode of the model based on the high-speed memory and cache characteristics of the Ascend AI processor, and perform adaptation and optimization on the Ascend AI processor based on the key operators in the deep learning model;

[0058] S2: Before the data in the first buffer is read, predict the data to be processed through the deep learning model, and preload the data block to be processed in the global memory based on the prediction result, and preload the data from the global memory to the second buffer in advance to realize parallel processing of data preloading and the current data processing process;

[0059] S3: Arrange a double-buffer mechanism in the Ascend AI processor, set at least two buffers in the Ascend AI processor, including a first buffer and a second buffer, attach a timestamp to the loading and calculation operations of each data packet, so that when the data in the first buffer is read, the data in the second buffer has been loaded and completed according to a predetermined time;

[0060] S4: Set interrupt and event trigger points, and immediately start the operation of loading data to the next buffer at the end of the specified calculation stage to realize precise control of the timing of data flow;

[0061] S5: Monitor the data loading and calculation in real time through a real-time monitoring module, and automatically adjust the control parameters based on the monitoring data of the real-time monitoring module. The control parameters include data loading rate and computing resource allocation;

[0062] S6: Decompose the calculation process into multiple stages, allocate different buffers for each stage, and optimize the access to shared memory by specifying the cache replacement policy and cache line size setting.

[0063] In this embodiment, by deploying the Ascend AI processor and the deep learning model in the server and performing adaptation and optimization on the Ascend AI processor; predicting and preloading the data to be processed before the data in the first buffer is read, and preloading the data from the global memory to the second buffer in advance; arranging a double-buffer mechanism on the Ascend AI processor, setting the interrupt and event trigger points, and immediately starting the operation of loading the data to the next buffer at the end of the specified calculation stage to achieve precise control of the timing of data flow; automatically adjusting the control parameters based on the monitoring data of the real-time monitoring module, decomposing the calculation process into multiple stages, allocating different buffers for each stage, and optimizing the access to the shared memory by specifying the cache replacement policy and cache line size setting. The above process improves the calculation speed of the model in the field of deep learning, improves the resource utilization rate, and solves the disadvantages of manual optimization.

[0064] Embodiment 2

[0065] On the basis of Embodiment 1, while performing step S1, the key operators in the deep learning model are further optimized, and the specific process is as follows:

[0066] S11: Customize the handwritten operator through the underlying interface, and the underlying interface is the CUDA layer;

[0067] S12: Optimize the operator by fusing the convolutional layer and the pooling layer into one operator, or fusing multiple fully connected layers into one large fully connected layer, or fusing multiple element-wise operations into one large element-wise operation, or fusing the convolutional layer and the batch normalization layer into one operator in the deep learning model.

[0068] In some cases, the native operators provided by the deep learning framework may not meet the requirements or have insufficient performance. In this case, the handwritten operator can be customized through the underlying interface (such as the CUDA layer) to more flexibly implement personalized operators or optimize the performance of existing operators. The biggest difference between the GPU and the CPU is that the computing unit occupies most of the volume, while the control unit is less. Customizing the handwritten operator can better utilize the parallelization advantage of the computing unit of the GPU, and through the reasonable arrangement of Grid, Block, and Thread, the parallelization of the computing logic is realized. Operator optimization can significantly improve the performance and efficiency of the deep learning model. By reducing the overhead of memory access and calculation, the model can run more efficiently and with higher performance on different hardware platforms. This can not only speed up the training speed of the model, but also improve the inference speed and accuracy of the model.

[0069] In this embodiment, the deep learning model is a neural network model. Refer to Figure 2 , the specific process of predicting the data to be processed by the deep learning model in step S2 is as follows:

[0070] S21: Create a to-be-processed data sample within a specified period that has been marked, and input it into the initialized deep learning model. Conduct model training through forward propagation, from the input layer through the hidden layer to the output layer, and calculate and store the results of each layer in the neural network;

[0071] S22: Based on the chain rule in calculus, traverse the network in the reverse order from the output layer through the hidden layer to the input layer through backward propagation, and by alternately using forward propagation and backward propagation, use the gradient given by backward propagation to update the model parameters;

[0072] S23: Input the current processed data and time information into the deep learning model, and the deep learning model predicts the to-be-processed data at a specified time based on the current processed data and time information.

[0073] During the process of backward propagation, repeatedly utilize the intermediate values stored in forward propagation to guide the completion of this backward propagation;

[0074] It also includes a process of optimizing the hyperparameters of the deep learning model, specifically as follows:

[0075] S24: Generate a probability distribution model of the parameters in the high-dimensional function space and represent it using a non-linear model;

[0076] S25: Based on the generated probability distribution model, initialize a combination of hyperparameters, and evaluate the initialized combination of hyperparameters through a test set;

[0077] S26: Construct a surrogate model of the objective function based on the to-be-processed data samples within a specified period that have been marked, and adopt a specified strategy to select the next sampling point according to the uncertainty of the surrogate model and the expectation of the objective function;

[0078] S27: After selecting the sampling point, sample the objective function and update the surrogate model, and iteratively execute the process of updating the surrogate model until the preset number of samplings is reached or the specified model performance requirement is met.

[0079] By creating and inputting the to-be-processed data samples within the specified period with tags as described above, model training is carried out in the way of forward propagation; the backward propagation traverses the network from the output layer through the hidden layer to the input layer in the reverse order, and by alternately using forward propagation and backward propagation, the gradient given by the backward propagation is used to update the model parameters, realizing the efficient training of the model, and being able to achieve the real-time optimization and training of the model, improving the accuracy of the prediction results of the to-be-processed data of the model. By generating a probability distribution model of the parameters in the high-dimensional function space, on the basis of the generated probability distribution model, a combination of hyperparameters is initialized, and the initialized combination of hyperparameters is evaluated through a test set. A surrogate model of the objective function is constructed according to the to-be-processed data samples within the specified period with tags, and according to the uncertainty of the surrogate model and the expectation of the objective function, a specified strategy is adopted to select the next sampling point; after selecting the sampling point, the objective function is sampled and the surrogate model is updated, and the process of updating the surrogate model is iteratively executed until the preset number of samplings is reached or the specified model performance requirement is met, further realizing the process of hyperparameter optimization of the model and further improving the accuracy of the prediction results of the to-be-processed data of the model.

[0080] Embodiment 3

[0081] Based on Embodiment 1 or Embodiment 2, the specific process of attaching a timestamp to the loading and calculation operations of each data packet in step S3 is as follows:

[0082] S31: The client sends a request data packet;

[0083] S32: After the Ascend AI processor receives the request data packet, it generates a hash value of the data packet or calculation operation;

[0084] S33: The Ascend AI processor combines the current time with the hash value of the data packet or calculation operation to generate timestamp information;

[0085] S34: The Ascend AI processor adds the timestamp information to the data packet or calculation operation.

[0086] By the client sending a request data packet, the Ascend AI processor generates a hash value of the data packet or calculation operation; combining the current time with the hash value of the data packet or calculation operation to generate timestamp information, and the Ascend AI processor adding the timestamp information to the data packet or calculation operation, the seamless switching and data synchronization between the two buffers are realized, the timing of data flow is precisely controlled, and data conflicts and delays are effectively avoided.

[0087] The specific double-buffer mechanism arranged in the Ascend AI processor in step S2 is as follows:

[0088] Adopt the dual control mechanism driven by the time stamp and event-driven. By attaching a time stamp to the loading and calculation operations of each data packet, when the data in the first buffer is read, the data in the second buffer has been loaded completely according to a predetermined time. The real-time monitoring module monitors the data loading and calculation in real time, and automatically adjusts the control parameters based on the monitoring data of the real-time monitoring module. Before the data in the first buffer is read, the data to be processed is predicted through a deep learning model, and the preloading of the data block to be processed in the global memory is started based on the prediction result. The data is preloaded from the global memory to the second buffer in advance, and the Ascend AI processor executes the data preloading operation in parallel with the current data processing task.

[0089] Through reasonable task allocation and scheduling strategies, ensure that each computing node can cooperate efficiently, reduce the waste of computing resources, and improve the overall computing efficiency. Utilize the double-buffer mechanism of the Ascend AI processor to disperse the model calculation tasks to multiple computing nodes to achieve parallel computing, which is an important means to improve the model calculation speed. The core of the double-buffer mechanism lies in the alternating use of two buffers to reduce the frequent read and write operations directly to the disk, thereby improving the performance and stability of the system. The two buffers are usually an area in the memory and a file on the disk. Through the above alternating use strategy and the mechanism to ensure the correct and efficient transmission and synchronization of data between the two buffers, the Ascend AI processor is the Ascend 910 processor.

[0090] See Figure 3 , the specific implementation process in step S6 is as follows:

[0091] S61: Subdivide the calculation process of the deep learning model into multiple independent and parallel calculation stages, and each stage is assigned to a dedicated buffer for data processing. By finely dividing the calculation tasks, ensure that there is no dependence or minimize the dependence between each stage, and achieve a pipelined processing process;

[0092] S62: Dynamically adjust the processor resource allocation according to the actual processing time and resource requirements of each calculation stage to achieve load balancing of each stage of the pipeline and maximize the resource utilization rate. The processor resources include CPU cores and memory bandwidth;

[0093] S63: Reduce the occurrence of cache conflicts and cache misses by specifying the cache replacement policy and cache line size settings, and optimize the access to the shared memory for the access to the shared memory.

[0094] In this embodiment, the specific process in step S63 is as follows:

[0095] S631: For the cache of shared memory, a hybrid cache replacement policy based on usage frequency and the LRU cache eviction policy is adopted, and a machine learning algorithm is introduced to predict the data access pattern and dynamically adjust the cache replacement logic.

[0096] S632: According to the data access characteristics of the deep learning model, customize the cache line size and layout, allocate memory spaces of specified sizes for the first buffer and the second buffer, write the specified data loading and calculation logic, and use synchronization primitives, including locks or semaphores, to coordinate data access and calculation between different threads or tasks.

[0097] In summary, the method for improving the model calculation speed based on the Ascend AI processor provided by the present invention customizes handwritten operators through the underlying interface, fuses the convolutional layer and the pooling layer in the deep learning model into one operator for operator optimization, more flexibly realizes personalized operators or optimizes the performance of existing operators, and can better utilize the parallelization advantage of the computing units of the GPU. Train the model in the forward propagation manner; traverse the network from the output layer through the hidden layer to the input layer in the reverse order in the backward propagation manner, and alternately use forward propagation and backward propagation to update the model parameters using the gradients given by the backward propagation; input the current processed data and time information into the deep learning model, realizing efficient training of the model, and being able to achieve real-time optimization and training of the model, improving the accuracy of the prediction results of the data to be processed by the model.

[0098] By generating a probability distribution model of the parameters in the high-dimensional function space and representing it using a non-linear model, based on the generated probability distribution model, initialize a combination of hyperparameters, and evaluate the initialized combination of hyperparameters through the test set; construct a surrogate model of the objective function according to the labeled data samples to be processed within a specified period, and adopt a specified strategy to select the next sampling point; after selecting the sampling point, sample the objective function and update the surrogate model, further realizing the process of hyperparameter optimization of the model, and further improving the accuracy of the prediction results of the data to be processed by the model.

[0099] The client sends a request data packet, and the Ascend AI processor generates the hash value of the data packet or the computing operation; combines the current time with the hash value of the data packet or the computing operation to generate timestamp information. The process of the Ascend AI processor adding the timestamp information to the data packet or the computing operation realizes seamless switching and data synchronization between two buffers, precisely controls the timing of data flow, and effectively avoids data conflicts and delays. By subdividing the computing process of the deep learning model into multiple independent and parallel computing stages, a pipelined processing process is realized; according to the actual processing time and resource requirements of each computing stage; by specifying the cache replacement policy and cache line size settings, cache conflicts and cache misses are reduced, and for the access to shared memory, an optimization process for the access to shared memory is realized, realizing dynamic adjustment of processor resource allocation, achieving load balancing in each stage of the pipeline, and maximizing resource utilization.

Claims

1. A method for improving the calculation speed of a model based on Ascend AI processors, characterized in that, It includes the following steps: S1: Deploy Ascend AI processors and deep learning models in the server. Based on the high-speed memory and cache characteristics of the Ascend AI processors, optimize the memory access mode of the models, and perform adaptation and optimization on the Ascend AI processors based on the key operators in the deep learning models; S2: Before the data in the first buffer is read, use the deep learning model to predict the data to be processed, and based on the prediction results, initiate preloading of the data blocks to be processed in the global memory, and preload data from the global memory to the second buffer in advance to achieve parallel processing of data preloading and the current data processing process; S3: Implement a double-buffer mechanism in the Ascend AI processors. Set at least two buffers in the Ascend AI processors, including a first buffer and a second buffer. Attach timestamps to the loading and computing operations of each data packet, so that when the data in the first buffer is read, the data in the second buffer has been loaded completely according to a predetermined time; S4: Set interruption and event trigger points, and immediately initiate the operation of loading data to the next buffer at the end of the specified computing stage to achieve precise control of the timing of data flow; S5: Use a real-time monitoring module to monitor data loading and computing in real time, and automatically adjust control parameters based on the monitoring data of the real-time monitoring module. The control parameters include data loading rate and computing resource allocation; S6: Decompose the computing process into multiple stages, allocate different buffers for each stage, and optimize the access to shared memory by specifying cache replacement policies and cache line sizes.

2. The method for improving the calculation speed of a model based on the Ascend AI processor according to claim 1, wherein Meanwhile in step S1, the key operators in the deep learning model are also optimized. The specific process is as follows: S11: Customize handwritten operators through the underlying interface, and the underlying interface is the CUDA layer; S12: Optimize the operators by fusing the convolutional layer and the pooling layer into one operator in the deep learning model, or fusing multiple fully connected layers into one large fully connected layer, or fusing multiple element-wise operations into one large element-wise operation, or fusing the convolutional layer and the batch normalization layer into one operator.

3. The method for improving the model calculation speed based on the Ascend AI processor according to claim 1, wherein The deep learning model is a neural network model. The specific process of using the deep learning model to predict the data to be processed in step S2 is as follows: S21: Create data samples to be processed within a specified period with labels and input them into the initialized deep learning model. Perform model training through forward propagation, from the input layer through the hidden layer to the output layer, and calculate and store the results of each layer in the neural network; S22: Based on the chain rule in calculus, traverse the network in the reverse order from the output layer through the hidden layer to the input layer through backward propagation, and alternately use forward propagation and backward propagation to update the model parameters using the gradients given by backward propagation; S23: Input the current processed data and time information into the deep learning model, and the deep learning model predicts the data to be processed at a specified time based on the current processed data and time information.

4. The method for improving the model calculation speed based on the Ascend AI processor according to claim 3, wherein, During the process of backward propagation, repeatedly use the intermediate values stored in forward propagation to guide the completion of this backward propagation; It also includes a process of optimizing the hyperparameters of the deep learning model, which is as follows: S24: Generate a probability distribution model of the parameters in the high-dimensional function space and represent it using a non-linear model; S25: Based on the generated probability distribution model, initialize a combination of hyperparameters and evaluate the initialized combination of hyperparameters through a test set; S26: Construct a surrogate model of the objective function according to the to-be-processed data samples within a specified period with labels, and adopt a specified strategy to select the next sampling point according to the uncertainty of the surrogate model and the expectation of the objective function; S27: After selecting the sampling point, sample the objective function and update the surrogate model, and iteratively execute the process of updating the surrogate model until the preset number of samplings is reached or the specified model performance requirement is met.

5. A method for improving the calculation speed of a model based on the Ascend AI processor according to claim 1, characterized in that, The specific process of attaching a timestamp to the loading and calculation operations of each data packet in step S3 is as follows: S31: The client sends a request data packet; S32: After receiving the request data packet, the Ascend AI processor generates a hash value of the data packet or the calculation operation; S33: The Ascend AI processor combines the current time with the hash value of the data packet or the calculation operation to generate timestamp information; S34: The Ascend AI processor adds the timestamp information to the data packet or the calculation operation.

6. A method for improving the calculation speed of a model based on the Ascend AI processor according to claim 1, wherein, The specific double-buffer mechanism deployed in the Ascend AI processor in step S2 is as follows: Adopt a dual control mechanism driven by the timestamp and event-driven. By attaching a timestamp to the loading and calculation operations of each data packet, when the data in the first buffer is read, the data in the second buffer has been loaded in advance according to a predetermined time. The real-time monitoring module monitors the data loading and calculation in real time, and automatically adjusts the control parameters based on the monitoring data of the real-time monitoring module. Before the data in the first buffer is read, the deep learning model is used to predict the to-be-processed data, and based on the prediction result, the to-be-processed data block in the global memory is pre-loaded. The data is pre-loaded from the global memory to the second buffer in advance, and the data pre-loading operation is executed in parallel with the current data processing task using the Ascend AI processor.

7. A method for improving the calculation speed of a model based on the Ascend AI processor according to claim 1, characterized in that, The specific implementation process in step S6 is as follows: S61: Subdivide the calculation process of the deep learning model into multiple independent and parallel calculation stages, and each stage is assigned to a dedicated buffer for data processing. By finely dividing the calculation tasks, ensure that there is no dependence or minimize the dependence between stages, and achieve a pipelined processing process; S62: Dynamically adjust the processor resource allocation according to the actual processing time and resource requirements of each calculation stage to achieve load balancing of each stage of the pipeline and maximize the resource utilization rate. The processor resources include CPU cores and memory bandwidth; S63: Reduce the occurrence of cache conflicts and cache misses by specifying a cache replacement policy and cache line size settings, and optimize the access to the shared memory for the access to the shared memory.

8. A method for improving the calculation speed of a model based on the Ascend AI processor according to claim 7, characterized in that The specific process in step S63 is as follows: S631: For the cache of shared memory, a hybrid cache replacement policy based on usage frequency and the LRU cache eviction policy is adopted, and a machine learning algorithm is introduced to predict the data access pattern and dynamically adjust the cache replacement logic; S632: According to the data access characteristics of the deep learning model, customize the cache line size and layout, allocate memory spaces of specified sizes for the first buffer and the second buffer, write the specified data loading and calculation logic, and use synchronization primitives, including locks or semaphores, to coordinate data access and calculation between different threads or tasks.

Citation Information

Cited By

  • Data storage optimization method and system based on real-time operating system

    CN120508408A

  • Realization method of AI visual analysis engine based on mercuric chloride architecture

    CN121349658A

  • Method, device and equipment for instantly judging in-orbit cloud phase state of meteorological satellite

    CN121861500A

  • Data processing method and device, electronic equipment and storage medium

    CN121957913A