Batch processing method and device based on request data, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a batch processing method and device based on request data, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the steps of obtaining a request data prediction instruction, wherein the request data prediction instruction comprises a to-be-predicted time period; determining historical periodic sequence data based on the to-be-predicted time period, wherein the historical periodic sequence data comprises multiple pieces of historical request data sorted based on time granularity; and calling a pre-trained request data prediction model based on the plurality of historical request data to perform request data prediction to obtain target request sequence data, the target request sequence data being used for adjusting batch processing parameters of a preset request processing model, and the batch processing parameters being used for determining the amount of input data corresponding to the request processing model. According to the embodiment of the invention, the execution efficiency of the batch processing parameter-based model can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a request data-based batch processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the wide deployment of deep learning models in various online applications, especially in high-concurrency and low-latency scenarios such as recommendation systems, natural language processing and speech recognition, the performance bottleneck of model inference services gradually emerges, and the response speed and throughput of the system have been difficult to meet the actual demand.

[0003] At present, in order to improve the throughput and reduce the response delay under limited computing resources, the existing technology usually adopts a batching method as an optimization means, that is, by collecting multiple request data in a short time, the request data is combined into a batch to input the model for inference processing. However, there are often differences between different request data, and the simple splicing method adopted by the related technology is easy to introduce a large number of padding operations, which leads to the quality of the predicted batch processing parameters being not enough, thereby reducing the execution efficiency of the model based on the batch processing parameters. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a request data-based batch processing method and device, electronic equipment and storage medium, which can improve the execution efficiency of the model based on the batch processing parameters.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application provides a request data-based batch processing method, which comprises:

[0006] Obtaining a request data prediction instruction, the request data prediction instruction comprising a to-be-predicted time period;

[0007] Determining historical period sequence data based on the to-be-predicted time period, the historical period sequence data comprising a plurality of historical request data sorted based on time granularity;

[0008] Calling a pre-trained request data prediction model based on the plurality of historical request data to perform request data prediction, obtaining target request sequence data, the target request sequence data being used to adjust batch processing parameters of a preset request processing model, the batch processing parameters being used to determine the number of input data corresponding to the request processing model.

[0009] In some embodiments, the training process of the request data prediction model specifically comprises:

[0010] Obtaining training sample data, the training sample data comprising sample historical period sequence data corresponding to a sample time period and sample label sequence data, the sample historical period sequence data comprising a plurality of historical sample request data sorted based on a time granularity;

[0011] Calling the request data prediction model based on the plurality of historical sample request data to perform request data prediction, to obtain predicted request sequence data;

[0012] Adjusting model parameters of the request data prediction model according to the predicted request sequence data and the sample label sequence data.

[0013] In some embodiments, the adjusting model parameters of the request data prediction model according to the predicted request sequence data and the sample label sequence data comprises:

[0014] Performing loss calculation according to the predicted request sequence data and the sample label sequence data to obtain a model loss value;

[0015] Adjusting model parameters of the request data prediction model according to the model loss value.

[0016] In some embodiments, the pre-trained request data prediction model comprises a feature extraction unit and a perception calculation unit, and the calling the pre-trained request data prediction model based on the plurality of historical request data to perform request data prediction to obtain the target request sequence data comprises:

[0017] Calling the feature extraction unit based on the plurality of historical request data to perform feature extraction to obtain historical request features;

[0018] Calling the perception calculation unit based on the historical request features to perform request data prediction to obtain the target request sequence data.

[0019] In some embodiments, the feature extraction unit comprises a feature extraction subunit and an attention subunit, and the calling the feature extraction unit based on the plurality of historical request data to perform feature extraction to obtain historical request features comprises:

[0020] Inputting the plurality of historical request data to the feature extraction subunit to perform feature extraction to obtain initial historical request features, the initial historical request features comprising a historical request feature vector corresponding to each of the historical request data;

[0021] Inputting the historical request feature vector corresponding to each of the historical request data to the attention subunit to perform feature fusion to obtain the historical request features.

[0022] In some embodiments, the historical period sequence data before the target time information is determined based on the target period, the historical period sequence data includes a plurality of historical request data sorted based on time granularity, including:

[0023] Obtaining period attribute data corresponding to the to-be-predicted time period, the period attribute data being used to indicate a period type corresponding to the to-be-predicted time period;

[0024] According to the period attribute data, target historical request data is extracted from the plurality of historical request data, the period type corresponding to the target historical request data being the same as the period type corresponding to the to-be-predicted time period.

[0025] In some embodiments, after the pre-trained request data prediction model is called based on the plurality of historical request data to perform request data prediction, and the target request sequence data is obtained, the method further includes:

[0026] Obtaining a preset batch processing parameter of a request processing model;

[0027] According to the target request sequence data, the batch processing parameter is adjusted to obtain a target batch processing parameter, the target batch processing parameter being used to determine the number of input data corresponding to the request processing model in the to-be-predicted time period.

[0028] To achieve the above object, a second aspect of the embodiment of the present application proposes a batch processing device based on request data, the device includes:

[0029] An instruction obtaining module is configured to obtain a request data prediction instruction, the request data prediction instruction including a to-be-predicted time period;

[0030] A data obtaining module is configured to determine historical period sequence data based on the to-be-predicted time period, the historical period sequence data including a plurality of historical request data sorted based on time granularity, and the historical request data corresponding to the same time length as the to-be-predicted time period;

[0031] A data prediction module is configured to call a pre-trained request data prediction model based on the plurality of historical request data to perform request data prediction, and obtain target request sequence data, the target request sequence data being used to adjust a preset batch processing parameter of a request processing model, the batch processing parameter being used to determine the number of input data corresponding to the request processing model.

[0032] To achieve the above object, a third aspect of the embodiment of the present application proposes an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor implementing the method of the first aspect when executing the computer program.

[0033] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0034] The batch processing method and device based on request data, the electronic device and the storage medium provided by the present application can improve the execution efficiency of the model based on the batch processing parameter.

[0035] The present application determines the plurality of historical request data based on time granularity sorting through the to-be-predicted time period, and determines the target request sequence data based on the plurality of historical request data by calling the pre-trained request data prediction model. In this way, the batch processing parameter of the request processing model can be dynamically adjusted according to the target request sequence data, thereby avoiding the problem of reduced model execution efficiency caused by the mismatch between the pre-set batch processing parameter and the actual request quantity of the request data. Therefore, the present application can improve the execution efficiency of the model based on the batch processing parameter. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is the flowchart of the batch processing method based on request data provided by the embodiments of the present application;

[0037] Figure 2 is the flowchart of step S102 in Figure 1

[0038] Figure 3 is the flowchart of the request data prediction model training process provided by the embodiments of the present application;

[0039] Figure 4 is the flowchart of step S303 in Figure 3

[0040] Figure 5 is the flowchart of step S103 in Figure 1

[0041] Figure 6 is the flowchart of step S501 in Figure 5

[0042] Figure 7 is​​​​Figure 1 Flow chart of steps after step S103;

[0043] Figure 8 is a flow chart of a specific application of the batch processing method based on request data provided by the embodiments of the present application;

[0044] Figure 9 is a structural schematic diagram of the batch processing device based on request data provided by the embodiments of the present application;

[0045] Figure 10 is a hardware structural schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0047] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flow chart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flow chart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0049] First, the meanings of several terms involved in the present application are analyzed:

[0050] Request data: refers to the service call requests received by the model inference service system within a specific time period, including the number of requests and request types in the specific time period, etc. For example, in an image recognition service, the request data can refer to the number of times of uploading images for recognition by the operation object within a certain hour period; in a text processing service, the request data can refer to the frequency of submitting text analysis by the operation object within a certain minute period; in a voice recognition service, the request data can refer to the number of times of requesting voice-to-text conversion by the operation object within a certain day.

[0051] Convolutional Neural Network (CNN): A deep neural network model used for processing data with grid structure. In the request data prediction task, CNN automatically extracts spatial features from the request data through convolutional layers, and then predicts future request data based on the extracted features.

[0052] Attention Mechanism: A mechanism used to enhance the processing capability of neural network models, which can automatically learn and focus on key parts of input data, helping the model more effectively capture important features and patterns in request data.

[0053] The widespread use of batch processing methods provides optimization means for model inference services, thereby improving the throughput and resource utilization of the system. However, existing batch processing methods still have obvious deficiencies in terms of strategy intelligence and system adaptability. For example, static batch processing methods (such as the timing batch processing method of TensorFlow Serving) uniformly process request data by setting a fixed batch processing time window. Although this method is simple to implement and easy to deploy, it cannot adaptively adjust the batch processing parameters in the case of dense or sparse request data, which can easily lead to high response delay or resource waste of the system. Alternatively, dynamic batch processing methods (such as the dynamic batch processing method based on queue length and maximum waiting time supported by NVIDIA Triton Inference Server) process request data by setting a minimum batch size and a maximum request waiting time. Although this method provides more flexible triggering strategies, it is based on simple threshold judgment and lacks the ability to predict future request data trends, making it difficult to cope with large fluctuations or periodic changes in request data.

[0054] In addition, the model inference service system used by the related technology usually assumes that the request data distribution is relatively stable, and sets batch processing parameters in advance based on this assumption to batch process request data. However, such a system is difficult to adapt to high dynamic, sudden or periodic fluctuation scenarios, often showing problems such as decreased throughput, increased response delay, and even frequent system idling. Moreover, some systems fail to group request data reasonably according to the size of the request data when processing request data of different sizes, resulting in redundant padding, memory fragmentation, and resource waste in model execution. Based on this, the embodiments of the present application propose a batch processing method based on request data, in order to improve the execution efficiency of the model based on batch processing parameters.

[0055] The request data-based batch processing method provided by the embodiments of the present application can be applied to a terminal, a server, or software running in the terminal or the server. In some embodiments, the terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, etc.; the server can be configured as a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms; and the software can be an application that implements the request data-based batch processing method, but is not limited to the above forms.

[0056] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0057] It should be noted that in each specific embodiment of the present application, when it is necessary to perform relevant processing according to the information of the operation object and the historical request data of the operation object, etc., which are related to the identity or characteristics of the operation object, the permission or consent of the operation object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the operation object, the separate permission or separate consent of the operation object will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the separate permission or separate consent of the operation object, the necessary operation object-related data for enabling the embodiments of the present application to operate normally will be obtained.

[0058] Figure 1 is an optional flowchart of the request data-based batch processing method provided by the embodiments of the present application, Figure 1 The method in can include but is not limited to steps S101 to S103.

[0059] In step S101, a request data prediction instruction is acquired, the request data prediction instruction including a to-be-predicted time period;

[0060] In step S102, historical period sequence data is determined based on the to-be-predicted time period, the historical period sequence data including a plurality of historical request data sorted based on a time granularity;

[0061] In step S103, a pre-trained request data prediction model is called based on the plurality of historical request data to perform request data prediction, to obtain target request sequence data.

[0062] In step S101 of some embodiments, the request data prediction instruction can refer to an instruction issued in a scenario in which request data prediction is needed. For example, the request data prediction instruction can refer to an instruction issued in a voice recognition service to predict the number of requests for voice-to-text conversion during a holiday; or the request data prediction instruction can refer to an instruction issued in an image recognition service to predict the number of requests for image recognition during the weekend. In the embodiments of the present application, the request data prediction instruction can be acquired by manually generating an instruction on a terminal, for example, the to-be-predicted time period can be manually input on the terminal to generate the request data prediction instruction; the request data prediction instruction can also be acquired by automatically generating an instruction on a server, for example, the server can automatically trigger the generation of the request data prediction instruction at a specific time point before a new function of the system is put into operation according to the time point of putting the new function into operation. It can be understood that the specific acquisition method of the request data prediction instruction is not limited in the embodiments of the present application. The request data prediction instruction includes a to-be-predicted time period. The to-be-predicted time period can refer to a time range in which request data prediction is needed. The to-be-predicted time period includes a specific time point and a specific period in which request data prediction is needed. The specific time point can refer to a time point at which request data prediction is started. The specific period can refer to a length of time in which request data prediction is needed. For example, if a popular article is planned to be published at 10:00 on X month X day of XXX, in a text processing service, the request data for text analysis after the publication of the popular article needs to be predicted, the specific time point can be 10:00, the specific period can be 30 minutes, and the to-be-predicted time period is from 10:00 to 10:30. Or, if an online voice conference is planned to start at 14:00 and end at 16:00 on X month X day of XXX, in a voice recognition service, if the number of requests for voice-to-text conversion during the online voice conference is to be predicted, the specific time point can be 14:30, the specific period can be 1 hour, and the to-be-predicted time period is from 14:30 to 15:30. It can be understood that the specific time point and the specific period can be freely adjusted according to actual needs, so that different to-be-predicted time periods can be determined.

[0063] In step S102 of some embodiments, the historical period sequence data can refer to sequence data including a plurality of historical request data sorted based on time granularity. The historical request data can refer to request data before the to-be-predicted time period. The time granularity sorting can refer to arranging the plurality of historical request data according to a preset time interval (such as every minute or every hour, etc.). For example, in a text processing service, if the to-be-predicted time period is 30 minutes (i.e. 10:00 to 10:30) after a popular article is published at 10:00, the historical period sequence data can include three historical request data sorted based on time granularity, which are the request number of text analysis from 9:30 to 10:00, the request number of text analysis from 9:00 to 9:30, and the request number of text analysis from 8:30 to 9:00. Or, in an image recognition service, if the to-be-predicted time period is 1 hour (i.e. 14:15 to 15:15) after a line online art exhibition is opened at 14:15, the historical period sequence data can include two historical request data sorted based on time granularity, which are the request number of image recognition from 14:00 to 14:15 and the request number of image recognition from 13:45 to 14:00. It can be understood that the number of historical request data included in the historical period sequence data and the time length corresponding to the historical request data can be freely adjusted according to actual needs.

[0064] Referring to Figure 2 In some embodiments, step S102 includes but is not limited to steps S201 to S202:

[0065] Step S201, obtaining period attribute data corresponding to the to-be-predicted time period;

[0066] Step S202, extracting target historical request data from the plurality of historical request data according to the period attribute data.

[0067] In steps S201-S202 of some embodiments, the period attribute data can refer to data used to indicate a period type corresponding to the to-be-predicted time period. The period type can include weekdays, weekends, holidays, and system version update days, etc. For example, if the to-be-predicted time period is from 14:00 to 15:00 on October 1, XXX, the period attribute data corresponding to the to-be-predicted time period is a holiday; or if the system version update day is May 1, XXX, and the to-be-predicted time period is from 08:00 to 08:10 on the day, the period attribute data corresponding to the to-be-predicted time period is the system version update day. The target historical request data can refer to historical request data in the plurality of historical request data that has the same period type as the to-be-predicted time period. For example, if the period types of the plurality of historical request data are all the same as the period type of the to-be-predicted time period, the target historical request data is the plurality of historical request data; or if only one historical request data in the plurality of historical request data has the same period type as the to-be-predicted time period, the target historical request data is the one historical request data. In the embodiments of the present application, after the target historical request data is extracted, the pre-trained request data prediction model can be called based on the target historical request data to perform request data prediction, and target request sequence data is obtained.

[0068] It can be understood that, in the embodiments of the present application, the period attribute data corresponding to the to-be-predicted time period is used to extract target historical request data with the same period attribute from the plurality of historical request data, and the pre-trained request data prediction model is called based on the target historical request data to perform request data prediction. In this way, the model can refer to historical request data similar to the to-be-predicted time period when predicting, and can better learn the request data features in a specific period type, thereby improving the accuracy of the request data prediction model.

[0069] In step S103 of some embodiments, the request data prediction model can refer to a model having a request data prediction capability. For example, the request data prediction model can refer to a Long Short-Term Memory (LSTM) model, which is called to perform sequence processing on the plurality of historical request data to capture long-term dependencies in time series, thereby realizing request data prediction; or the request data prediction model can also refer to a Gated Recurrent Unit (GRU) model, which is called to perform sequence processing on the plurality of historical request data to efficiently learn time dependencies in sequence data, thereby realizing request data prediction; or the request data prediction model can also refer to a Convolutional Neural Network (CNN) model, which is called to perform spatial feature extraction and processing on the plurality of historical request data to capture local features and patterns in the data, thereby realizing request data prediction. It can be understood that the specific structure of the request data prediction model is not limited in the embodiments of the present application. Request data prediction can refer to predicting a predicted value of request data under the condition that a plurality of historical request data is determined. The target request sequence data can refer to a predicted value of request data corresponding to a to-be-predicted time period. For example, in a text analysis service, if a popular article is scheduled to be published at 10:00, the to-be-predicted time period is 30 minutes after the popular article is published (i.e., 10:00 to 10:30), and the target request sequence data can refer to a predicted value of the number of text analysis requests from 10:00 to 10:30.

[0070] In the embodiments of the present application, the target request sequence data is used to adjust the batch processing parameter of the preset request processing model. The preset request processing model can refer to a neural network model used to process a specific type of request. For example, if the type of request data is a text analysis request, the preset request processing model can refer to a text analysis model. The batch processing parameter can refer to the number of request data that can be processed by the request processing model at a time. For example, if the request processing model is a text analysis model and the batch processing parameter is set to 50, it means that the text analysis model can process 50 text analysis requests at a time. In the embodiments of the present application, the predicted value of request data corresponding to the to-be-predicted time period can be used to dynamically adjust the batch processing parameter of the request processing model, so as to optimize the request processing efficiency and resource consumption of the model. For example, if the predicted value of request data corresponding to the to-be-predicted time period increases, the batch processing parameter can be increased, so that the model processes more requests at a time, thereby improving the execution efficiency of the model; or if the predicted value of request data corresponding to the to-be-predicted time period decreases, the batch processing parameter can be reduced, so that the model processes fewer requests at a time, thereby reducing resource consumption.

[0071] It can be understood that the request data prediction model can be pre-trained. As follows, the training process of the request data prediction model is described. Referring to Figure 3 The training process of the request data prediction model can include, but is not limited to, steps S301 to S303:

[0072] Step S301, obtaining training sample data;

[0073] Step S302, calling the request data prediction model based on the plurality of historical sample request data to perform request data prediction to obtain predicted request sequence data;

[0074] Step S303, adjusting the model parameters of the request data prediction model according to the predicted request sequence data and the sample label sequence data.

[0075] In step S301 of some embodiments, the training sample data can refer to data used to train the request data prediction model. The training sample data includes sample historical period sequence data corresponding to a sample time period and sample label sequence data. Wherein, the sample time period can refer to a time range used to construct the training sample data. For example, in an image processing service, the sample time period can be XXX year X month X day 8:00 to 8.05; or, in a speech recognition service, the sample time period can be XXX year X month X day 14:10 to 15:10. The sample historical period sequence data can refer to sequence data including a plurality of historical sample request data sorted based on time granularity. Wherein, the historical sample request data can refer to request data before the sample time period. For example, in a text processing service, if the sample time period is XXX year X month X day 8:00 to 9.00, the sample historical period sequence data can include three historical sample request data, which are the request number of text analysis from 6.50 to 7.50, the request number of text analysis from 5.40 to 6.40 and the request number of text analysis from 4.30 to 5.30 of the same day. Or, in an image processing service, if the sample time period is XXX year X month X day 14:00 to 15.00, the sample historical period sequence data can include two historical sample request data, which are the request number of image recognition from 13:50 to 14.00 and the request number of image recognition from 13:40 to 13:50 of the same day. It can be understood that the number of historical sample request data included in the sample historical period sequence data and the time length corresponding to the historical sample request data can be freely adjusted according to actual needs. The sample label sequence data can refer to the true value of the request data corresponding to the sample time period.

[0076] In step S302 of some embodiments, the predicted request sequence data can refer to a predicted value of the request data corresponding to the sample time period obtained after calling the request data prediction model to predict the request data according to the plurality of historical sample request data. For example, in an image processing service, if the sample time period is XXX year X month X day 14:00-15:00, and the two historical sample request data are the request times of image recognition at 13:50-14:00 and the request times of image recognition at 13:40-13:50 on the same day, the predicted request sequence data can refer to a predicted value of the request times of image recognition at XXX year X month X day 14:00-15:00 obtained after calling the request data prediction model to predict the request data according to the two historical sample request data.

[0077] It should be noted that the plurality of historical sample request data used when training the request data prediction model is consistent in quantity with the plurality of historical request data input when calling the model to predict the request data. For example, if the predicted request sequence data of each training sample data includes two historical sample request data, two historical request data need to be input when calling the preset neural network model to predict the request data.

[0078] In step S303 of some embodiments, updating the model parameters of the request data prediction model can be achieved by calculating the error between the predicted request sequence data of the model and the sample label sequence data, and then adjusting the model parameters according to the error. It can be understood that the embodiments of the present application can use a plurality of training sample data to train the model by continuously iterating the parameters, so as to gradually improve the prediction performance of the model.

[0079] In step S303 of some embodiments, please refer to Figure 4 In some embodiments, step S303 can include but is not limited to steps S401-S402:

[0080] In step S401, loss calculation is performed according to the predicted request sequence data and the sample label sequence data to obtain a model loss value;

[0081] In step S402, the model parameters of the request data prediction model are adjusted according to the model loss value.

[0082] In step S401 of some embodiments, the model loss value can refer to the difference between the predicted request sequence data predicted by the request data prediction model and the sample label sequence data, i.e., the model loss value can be calculated according to the predicted request sequence data and the sample label sequence data. For example, the model loss value can be calculated according to the following formula 1.

[0083]

[0084] wherein, L1 represents the model loss value, t represents the number of training sample data, Z k represents the sample label sequence data corresponding to the kth training sample data, represents the predicted request sequence data of the kth training sample data, and k can be [1, t].

[0085] In step S402 of some embodiments, the parameter adjustment of the model parameters of the request data prediction model can be achieved by calculating the gradient value of the model based on the model loss value through the back propagation algorithm, and then adjusting the model parameters of the request data prediction model according to the gradient value. For example, the stochastic gradient descent (SGD) method can be used to calculate the gradient value of the request data prediction model based on the model loss value, and the adjustment direction and amplitude of the model parameters can be determined according to the gradient value, and then the model parameters are adjusted accordingly to realize the parameter update of the request data prediction model. Alternatively, the adaptive moment estimation (Adam) method can also be used to calculate the gradient value of the request data prediction model based on the model loss value, and the adjustment direction and amplitude of the model parameters can be determined according to the gradient value, and then the model parameters are adjusted accordingly to realize the parameter update of the request data prediction model. It can be understood that the parameter adjustment method of the request data prediction model is not limited in the present application, and a suitable optimization algorithm can be selected according to actual needs.

[0086] Please refer to Figure 5 In some embodiments, the pre-trained request data prediction model includes a feature extraction unit and a perception calculation unit, and step S103 can include but is not limited to steps S501 to S502:

[0087] Step S501, calling the feature extraction unit based on a plurality of historical request data to perform feature extraction, and obtaining historical request features;

[0088] Step S502, calling the perception calculation unit based on the historical request features to perform request data prediction, and obtaining target request sequence data.

[0089] In step S501 of some embodiments, the feature extraction unit can refer to a unit for performing feature extraction on the plurality of historical request data. For example, the feature extraction unit can be a hidden layer in an LSTM model, for performing sequence processing on the plurality of historical request data to capture long-term dependencies in the time series; or the feature extraction unit can also be a convolutional layer in a CNN model, for performing spatial feature extraction and processing on the plurality of historical request data to capture local features and patterns in the data. It can be understood that the specific structure of the feature extraction unit can be freely adjusted according to actual needs. The historical request feature can refer to a feature obtained by calling the feature extraction unit to perform feature extraction on the plurality of historical request data. For example, if the feature extraction unit is a hidden layer in an LSTM model, the hidden layer is used to analyze the relationship between each historical request data and the historical request data before and after it, extract long-term dependencies in the time series, and generate the historical request feature according to the extracted long-term dependencies.

[0090] In step S502 of some embodiments, the perception computing unit can refer to a unit for performing request data prediction on the historical request feature. For example, the perception computing unit can refer to an output layer in an LSTM model or a fully connected layer in a CNN model. It can be understood that the specific structure of the perception computing unit can be freely adjusted according to actual needs. The target request sequence data can refer to sequence data obtained by calling the perception computing unit to perform request data prediction on the historical request feature. For example, if the perception computing unit is a fully connected layer in a CNN model, the fully connected layer is used to perform weighted calculation and nonlinear transformation on the historical request feature to obtain the target request sequence data.

[0091] It can be understood that the embodiments of the present application first perform feature extraction on the plurality of historical request data through the feature extraction unit of the request data prediction model, and then predict the target request sequence data based on the extracted features through the perception computing unit. In this way, the feature extraction and prediction of the request data can be realized through two modules respectively, and the overfitting risk caused by the functional coupling of a single module can be avoided, that is, the accuracy of the prediction of the request data prediction model can be improved.

[0092] Please refer to Figure 6 In some embodiments, the feature extraction unit includes a feature extraction subunit and an attention subunit, and step S501 can further include but is not limited to steps S601 to S602:

[0093] In step S601, the plurality of historical request data is input to the feature extraction subunit for feature extraction, to obtain initial historical request features, and the initial historical request features include a historical request feature vector corresponding to each historical request data;

[0094] In step S602, the historical request feature vector corresponding to each historical request data is input into the attention subunit for feature fusion to obtain a historical request feature.

[0095] In step S601 of some embodiments, the feature extraction subunit can refer to a unit in the feature extraction unit for extracting features of each historical request data. For example, if the feature extraction unit is a convolution layer, the feature extraction subunit can be a convolution unit in the convolution layer that performs convolution operation to extract features of the request data by sliding and scanning the convolution kernel; or if the feature extraction unit is a hidden layer, the feature extraction subunit can also be a hidden layer unit (such as an input gate, a forget gate, and an output gate) in the hidden layer that calculates the dependency between multiple historical request data through a gating mechanism and extracts features of the request data according to the dependency. It can be understood that the specific structure of the feature extraction subunit can be adjusted according to actual needs, and the number of feature extraction subunits is the same as the number of input historical request data. The initial historical request feature can refer to a vector set including the historical request feature vector corresponding to each historical request data. The historical request feature vector can refer to a vector obtained by calling the feature extraction subunit to extract features based on multiple historical request data. For example, if the feature extraction subunit is a convolution unit in the convolution layer that performs convolution operation, and there are three historical request data, then three convolution units can be used to extract features of the three historical request data respectively to obtain the historical request feature vector corresponding to each historical request data, and the initial historical request feature is a vector set composed of the historical request feature vectors corresponding to the three historical request data.

[0096] In step S602 of some embodiments, the attention subunit can refer to a unit in the feature extraction subunit for feature fusion of the historical request feature vector corresponding to each historical request data. For example, the attention subunit can be a self-attention mechanism unit (Self-Attention) or a multi-head attention mechanism unit (Multi-Head Attention). It can be understood that the specific structure of the attention subunit can be adjusted according to actual needs. Feature fusion can refer to a process of fusing multiple historical request feature vectors into a feature that can represent historical period sequence data. The historical request feature can refer to a feature obtained by calling the attention subunit to perform feature fusion according to the historical request feature vector corresponding to each historical request data. For example, if the attention subunit is a self-attention mechanism unit, the self-attention mechanism unit assigns a corresponding weight coefficient to each historical request feature vector by calculating the correlation of the historical request feature vector corresponding to each historical request data, and performs weighted calculation on the historical request feature vector according to the corresponding weight coefficient assigned to each historical request feature vector, and sums the weighted historical request feature vectors to obtain the historical request feature.

[0097] It can be understood that, by the feature extraction subunit, the plurality of historical request data are subjected to deep feature extraction to obtain a historical request feature vector of each historical request data, and by the attention subunit, the historical request feature vector of each historical request data is subjected to feature fusion to generate a historical request feature which can better represent the sample historical period sequence data. In this way, the request data prediction model proposed in the embodiments of the present application can focus on the historical request data most relevant to the to-be-predicted time period in the plurality of historical request data, avoid complex calculation and processing of redundant information, thereby improving the execution efficiency of the request data prediction model and reducing the consumption of computing resources.

[0098] Please refer to Figure 7 In some embodiments, after step S103, steps S701 to S702 can be further included, but are not limited to:

[0099] In step S701, a batch parameter of a preset request processing model is obtained.

[0100] In step S702, the batch parameter is adjusted according to the target request sequence data to obtain a target batch parameter.

[0101] In steps S701 to S702 of some embodiments, the preset request processing model can refer to a deep learning model used for processing request data. For example, the request processing model can be a Transformer model used for processing request data of a text translation type; or the request processing model can be a speech representation self-supervised learning model (Wav2Vec: Unsupervised Pre-training for Speech Representation Learning) used for processing request data of a speech recognition type. It can be understood that the specific structure of the request processing model can be freely adjusted according to the request type of the request data. The batch parameter can refer to the batch size of the preset request processing model. For example, the batch parameter can be 8 or 12. It can be understood that the batch parameter corresponding to the to-be-predicted time period can be preset according to the historical request data. The batch parameter is used to determine the number of request data that the request processing model can receive and process at a time.

[0102] The target batch processing parameter refers to the number of input data points for the request processing model during the prediction period. For example, based on historical request data, the batch processing parameter for the prediction period can be pre-set to 8. By comparing the target request sequence data with the historical request data, if the number of predicted request data points is greater than the number of historical request data points, the batch processing parameter can be increased to obtain the target batch processing parameter, such as adjusting it from 8 to 12. This increases the number of request data points that the request processing model can simultaneously receive and process during the prediction period, avoiding resource shortages and processing delays caused by excessive request volume, thereby improving system throughput. Alternatively, if the number of predicted request data points is less than the number of historical request data points, the batch processing parameter can be decreased to obtain the target batch processing parameter, such as adjusting it from 8 to 6. This reduces the number of request data that the request processing model can receive and process simultaneously during the prediction period, avoiding resource waste and excessively long response times due to insufficient request volume, thereby reducing system resource consumption; or, if the number of predicted request data is equal to the number of historical request data, the batch processing parameters remain unchanged to maintain the existing throughput and response time levels, avoiding system fluctuations caused by unnecessary parameter adjustments and ensuring the stability of processing efficiency.

[0103] It is understood that the embodiments of this application dynamically adjust the batch processing parameters of the request processing model through the target request sequence data, so that the model can better adapt to the changes in the amount of request data, avoid uneven distribution of computing resources due to fluctuations in the number of requests, and thus maintain a balance between the system's response speed and throughput.

[0104] Please see Figure 8 This application also provides a flowchart of a specific application of a batch processing method based on request data, including streaming request sequence acquisition, periodic feature encoding, dynamic periodicity awareness, online learning and parameter updating, and model platform decision-making. Streaming request sequence acquisition refers to receiving request logs from a model service system and organizing them into a time series at a fixed time granularity (e.g., per second or per minute). The time series acquired by the streaming request sequence module can be denoted as R = [r1, r2, ..., r...]. i ], where i represents the total number of time points recorded in the time series, r i r represents the time point of the i-th record in this time series. i For a pair <ts i ,rn i >, ts i rn represents the timestamp corresponding to the time point of the i-th record. i This represents the number of requests corresponding to the time point of the i-th record.

[0105] Furthermore, periodic feature encoding refers to transforming a time series R into structured periodic and contextual features to enhance the model's ability to perceive and learn changes in the time dimension, thereby obtaining a specific periodic feature sequence (i.e., historical periodic sequence data). Periodic feature encoding transforms the time series R into a periodic feature sequence X = [x1, x2, ..., x] according to a set period size (i.e., the length of the time period to be predicted) and data aggregation rules (sorting data before the time period to be predicted based on time granularity). j Where j represents the total number of time periods recorded in the periodic feature sequence, and x j This represents the request data (i.e., historical request data) corresponding to the time period of the j-th record in the periodic feature sequence, where each x... j For a triple <p j ,rn j ,l j >,p j Indicates the sequence number of the corresponding time period, rn j This indicates the number of requests within that time period. j This represents the attribute information for that time period. For example, if the request logs of the model service system are organized into a time series by second, then the time series R can be [(00:00:01,3),(00:00:02,3),…,(00:10:00,5)]. Aggregating the time series R in 1-minute intervals (i.e., the time period to be predicted is 1 minute) yields the periodic feature sequence X as [(1,10,N),(2,9,N)…,(10,8,N)], where N represents the attribute information for that time period (such as holidays).

[0106] Furthermore, dynamic periodic sensing can refer to the processing of a received periodic feature sequence X by a feature extraction unit (which can be labeled "f"). θ Extracting key change patterns from the request data in X, and combining this with temporal context information to enhance perceptual accuracy, yields higher-order fusion features (i.e., historical request features). These higher-order fusion features can be denoted as Y = f θ (X) = [y1, y2, ..., y m Where m represents the number of historical request features, y m x represents j The corresponding m-th historical request feature. Then, the obtained high-order fusion feature Y is further processed by a perceptual computing unit (which can be labeled "g"). θ Predict the number of requests for the time period to be predicted (i.e., the target request sequence data). The number of requests for the time period to be predicted can be tagged as...

[0107] It can be understood that the embodiments of the present application can simultaneously predict the corresponding request times of multiple to-be-predicted time periods through the feature extraction unit and the perception calculation unit. And integrate the corresponding request times of multiple to-be-predicted time periods into a prediction period feature sequence Make the model service system can directly from the prediction period feature sequence The corresponding request data of the to-be-predicted time period need to be obtained. Wherein, n represents the total number of to-be-predicted time periods recorded in the period feature sequence, Indicates the predicted value of the request data corresponding to the nth to-be-predicted time period in the prediction period feature sequence (i.e. the target request sequence data).

[0108] After determining the true value of the request data corresponding to the to-be-predicted time period, online learning and parameter updating can refer to adjusting the predicted value of the request data corresponding to the to-be-predicted time period And the true value of the request data corresponding to the to-be-predicted time period, the prediction error of the two is calculated by the mean squared error function (Mean Squared Error, MSE), and the model parameters of the request prediction model (i.e. adjust the parameters of the feature extraction unit and the perception calculation unit) are adjusted according to the prediction error by using the gradient descent method (such as the SGD method), and the periodic cognitive expression in the model is optimized. It can be understood that the embodiments of the present application can continuously compare the difference between the true value and the predicted value of the request data to guide the adjustment of the model parameters, continuously improve the understanding and reasoning accuracy of the model to the request change trend, so as to build a closed loop mechanism of "perception-reasoning-adjustment", and gradually improve the accuracy of model prediction.

[0109] Finally, the model platform decision can refer to comparing the predicted value of the request data corresponding to the to-be-predicted time period with the historical request data, and adjusting the service batch processing parameters in real time according to the comparison result and the load capacity of the model service system, so as to improve the resource utilization and service stability of the system.

[0110] It can be understood that the embodiments of the present application combine the feature extraction unit and the perception calculation unit of multi-scale period extraction and time context modeling, which can simultaneously capture the request behavior features of the period feature sequence, thereby realizing the ability to effectively cope with the reasoning service request fluctuation. And by introducing the online learning and parameter updating function, the system can receive the prediction result and the true result in real time, build a prediction error feedback mechanism, and automatically update the model parameters according to the loss function, so as to continuously improve the reasoning accuracy and system response ability of the model service.

[0111] Please refer to Figure 9 The embodiments of the present application also provide a batch processing device based on request data, which can realize the batch processing method based on request data as described above, and the device comprises:

[0112] The instruction obtaining module 901 is configured to obtain a request data prediction instruction, the request data prediction instruction comprising a to-be-predicted time period;

[0113] The data obtaining module 902 is configured to determine historical period sequence data based on the to-be-predicted time period, the historical period sequence data comprising a plurality of historical request data sorted based on time granularity;

[0114] The data prediction module 903 is configured to call a pre-trained request data prediction model to perform request data prediction based on the plurality of historical request data, to obtain target request sequence data, the target request sequence data being used to adjust a batch processing parameter of a preset request processing model, the batch processing parameter being used to determine a quantity of input data corresponding to the request processing model.

[0115] The specific implementation of the request data-based batch processing apparatus is basically the same as that of the above-described specific embodiments of the request data-based batch processing method, and thus will not be described herein again.

[0116] The embodiments of the present application further provide an electronic device, which comprises a memory and a processor, the memory storing a computer program, and the processor implementing the above-described request data-based batch processing method when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0117] Please refer to Figure 10 , Figure 10 The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0118] The processor 1001 can be implemented in a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0119] The memory 1002 can be implemented in a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1002 and are called and executed by the processor 1001 to implement the request data-based batch processing method of the embodiments of the present application.

[0120] The input / output interface 1003 is configured to realize information input and output.

[0121] The communication interface 1004 is configured to realize communication interaction between the device and other devices, and the communication can be realized through a wired manner (for example, a USB, a network cable, or the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, or the like).

[0122] The bus 1005 is configured to transmit information between various components (for example, the processor 1001, the memory 1002, the input / output interface 1003, and the communication interface 1004) of the device.

[0123] The processor 1001, the memory 1002, the input / output interface 1003, and the communication interface 1004 are connected to each other through the bus 1005 to realize communication connection between the device.

[0124] The computer readable storage medium provided in the embodiment of the present application stores a computer program, and the computer program is executed by the processor to realize the above-mentioned batch processing method based on request data.

[0125] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0126] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0127] Those skilled in the art can understand that the technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figure, or combine certain steps, or different steps.

[0128] The apparatus embodiments described above are merely exemplary, and the units described as separate units can or can not be physically separate, i.e., can be located in one place, or can be distributed over multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0129] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0130] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims of the foregoing drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so

[0131] It should be understood that in this application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0132] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0133] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0134] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0135] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, and various program storage media.

[0136] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for batch processing of request data, characterized in that, The method comprises: obtaining a request data prediction instruction, the request data prediction instruction comprising a to-be-predicted time period; determining historical period sequence data based on the to-be-predicted time period, the historical period sequence data comprising a plurality of historical request data sorted based on time granularity; calling a pre-trained request data prediction model based on the plurality of historical request data to perform request data prediction, to obtain target request sequence data, the target request sequence data being used to adjust a batch processing parameter of a preset request processing model, the batch processing parameter being used to determine the number of input data corresponding to the request processing model.

2. The method of claim 1, wherein, The training process of the request data prediction model specifically comprises: obtaining training sample data, the training sample data comprising sample historical period sequence data corresponding to a sample time period and sample label sequence data, the sample historical period sequence data comprising a plurality of historical sample request data sorted based on time granularity; calling the request data prediction model based on the plurality of historical sample request data to perform request data prediction, to obtain predicted request sequence data; performing parameter adjustment on model parameters of the request data prediction model according to the predicted request sequence data and the sample label sequence data.

3. The method of claim 2, wherein, The parameter adjustment on the model parameters of the request data prediction model according to the predicted request sequence data and the sample label sequence data comprises: performing loss calculation according to the predicted request sequence data and the sample label sequence data, to obtain a model loss value; performing parameter adjustment on the model parameters of the request data prediction model according to the model loss value.

4. The method of claim 1, wherein, The pre-trained request data prediction model comprises a feature extraction unit and a perception calculation unit, and the calling of the pre-trained request data prediction model based on the plurality of historical request data to perform request data prediction, to obtain the target request sequence data, comprises: calling the feature extraction unit based on the plurality of historical request data to perform feature extraction, to obtain historical request features; calling the perception calculation unit based on the historical request features to perform request data prediction, to obtain the target request sequence data.

5. The method of claim 4, wherein, The feature extraction unit comprises a feature extraction subunit and an attention subunit, and the calling of the feature extraction unit based on the plurality of historical request data to perform feature extraction, to obtain historical request features, comprises: inputting the plurality of historical request data into the feature extraction subunit to perform feature extraction, to obtain initial historical request features, the initial historical request features comprising a historical request feature vector corresponding to each historical request data; inputting the historical request feature vector corresponding to each historical request data into the attention subunit to perform feature fusion, to obtain the historical request features.

6. The method of claim 1, wherein, The determination of historical period sequence data before the target time information based on the target period, the historical period sequence data comprising a plurality of historical request data sorted based on time granularity, comprises: obtaining period attribute data corresponding to the to-be-predicted time period, the period attribute data being used to indicate a period type corresponding to the to-be-predicted time period; According to the period attribute data, target historical request data is extracted from the plurality of historical request data, and the period type corresponding to the target historical request data is the same as the period type corresponding to the to-be-predicted time period.

7. The method of claim 1, wherein, After the pre-trained request data prediction model is called based on the plurality of historical request data to perform request data prediction and the target request sequence data is obtained, the method further includes: obtaining a batch processing parameter of a preset request processing model; adjusting the batch processing parameter according to the target request sequence data to obtain a target batch processing parameter, and the target batch processing parameter is used to determine the number of input data corresponding to the request processing model in the to-be-predicted time period.

8. A data request-based batch processing apparatus, characterized by comprising: The apparatus includes: an instruction obtaining module configured to obtain a request data prediction instruction, the request data prediction instruction including a to-be-predicted time period; a data obtaining module configured to determine historical period sequence data based on the to-be-predicted time period, the historical period sequence data including a plurality of historical request data sorted based on a time granularity; a data prediction module configured to call a pre-trained request data prediction model based on the plurality of historical request data to perform request data prediction and obtain target request sequence data, the target request sequence data being used to adjust a batch processing parameter of a preset request processing model, and the batch processing parameter being used to determine the number of input data corresponding to the request processing model.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the request data-based batch processing method in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the request data-based batch processing method in any one of claims 1 to 7.

Images