Resource scheduling method and device for twin training tasks, medium and product

Abstract

The invention discloses a twinborn training task resource scheduling method and device, a medium and a product, and the method comprises the steps: obtaining a digital twinborn of a to-be-trained algorithm, and generating a parameterized feature vector according to the attribute feature of the digital twinborn; according to the parameterized feature vector, generating a digital twinborn training distribution strategy based on an information concentration evaluation reinforcement learning method; and according to the digital twinborn training allocation strategy, allocating corresponding computing resources to the training task of the digital twinborn of the current algorithm to be trained. The method is suitable for carrying out cluster training on the digital twinborn algorithm, computing resources can be reasonably distributed, and the training efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a resource scheduling method, device, medium and product for twin training tasks. Background Art

[0002] There are a large number of heterogeneous twin objects in the digital twin scenario. Using deep learning technology to perform specialized modeling of the multi-level data generated by these different digital twins can produce more valuable industrial analysis results.

[0003] In the scheduling of computing resources for algorithm training on digital twin platforms, existing resource scheduling methods can be roughly divided into static scheduling, dynamic scheduling, and priority scheduling. In static scheduling strategies, resource allocation is based on predefined rules and policies, and resource allocation is fixed and does not adjust according to changes in task requirements. Dynamic scheduling strategies monitor the resource usage of servers and computing nodes in real time and use dynamic scheduling algorithms (such as resource utilization optimization algorithms and load balancing algorithms) to dynamically adjust resources. Priority scheduling strategies assign priorities to different tasks based on their importance and urgency. High-priority tasks receive priority computing resources when resources are scarce.

[0004] However, existing resource scheduling methods suffer from issues such as low resource utilization and inefficient scheduling. Static scheduling strategies are simple to implement, but are prone to resource waste or shortages when resource demands change dynamically. Dynamic scheduling strategies can improve resource utilization, but are complex to implement, place high demands on scheduling algorithms, and can be unstable when resource demands fluctuate drastically. Priority scheduling strategies can ensure resource requirements for critical tasks, but require appropriate priority settings to avoid long-term resource occupation and resource starvation for low-priority tasks. In digital twin platforms, the heterogeneity of neural network input data is also a concern. For the same network parameters, the resulting training resource costs vary due to differences in the information content of the data itself. Data with more prominent information converges more easily, allowing training to complete quickly even with high training costs. Data with less prominent information requires longer time to adjust to appropriate weights. Current resource scheduling solutions lack algorithm training resource scheduling solutions tailored to the heterogeneous nature of digital twin data patterns. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a resource scheduling method, device, medium and product for twin training tasks, which are suitable for cluster training of digital twin algorithms, can reasonably allocate computing resources and improve training efficiency.

[0006] To achieve the above objectives, an embodiment of the present invention provides a resource scheduling method for a twin training task, comprising:

[0007] Obtaining a digital twin of the algorithm to be trained, and generating a parameterized feature vector based on the attribute characteristics of the digital twin;

[0008] According to the parameterized feature vector, a digital twin training allocation strategy is generated based on a reinforcement learning method of information density evaluation;

[0009] According to the digital twin training allocation strategy, corresponding computing resources are allocated to the training task of the digital twin of the current algorithm to be trained.

[0010] As an improvement to the above solution, the step of obtaining a digital twin of the algorithm to be trained and generating a parameterized feature vector based on the attribute characteristics of the digital twin includes:

[0011] Load the digital twin object of the algorithm to be trained from the digital twin database;

[0012] Load the twin environment, twin device, twin data interface, computational graph and data file format of the algorithm required for the digital twin, and training hyperparameters of the digital twin algorithm to generate parameterized feature vectors.

[0013] As an improvement to the above solution, the digital twin training allocation strategy is generated based on the parameterized feature vector and the reinforcement learning method based on information density evaluation, including:

[0014] According to the parameterized feature vector, a single-step training method is used to perform trial sampling during the training process to obtain a single-step training cost;

[0015] Estimate the algorithm computation time based on the single-step training cost;

[0016] Establishing an information complexity index based on Lempel-Ziv complexity, multiscale entropy, sample entropy, and approximate entropy; the information complexity index is used to measure information density;

[0017] Calculating the rate of decline of the loss function according to the information complexity index to calculate the number of training rounds;

[0018] Establishing a state space for reinforcement learning based on the algorithm calculation time and the training rounds;

[0019] Generate action space based on server resource type;

[0020] The final training time of the cluster task is used as a reward, and the mapping relationship between the state space and the action space is iterated through reinforcement learning;

[0021] Build and optimize a DQN-based adaptive resource allocation strategy as a training allocation strategy for digital twins.

[0022] As an improvement to the above solution, the method of calculating the rate of decrease of the loss function according to the information complexity index to obtain the number of training rounds includes:

[0023] Fitting the relationship between the information complexity index and the loss function by the least squares method or the exponential smoothing method to obtain the rate of decrease of the loss function;

[0024] The number of training rounds is calculated based on the decreasing speed of the loss function and the preset loss rate.

[0025] As an improvement to the above solution, the construction and optimization of the DQN-based adaptive resource allocation strategy as the digital twin training allocation strategy includes:

[0026] Select the neural network template to initialize the DQN network;

[0027] Based on the initial network weights, several spatial states are selected and action spaces are generated accordingly. Based on the action spaces, corresponding rewards are obtained to construct a "state-action-reward" sample.

[0028] Randomly obtain samples of the "state-action-reward" and calculate the target value:

[0029] Using the target value and gradient descent algorithm, the network weights are updated until the resource allocation strategy meets the preset requirements, thereby obtaining the digital twin training allocation strategy.

[0030] As an improvement to the above solution, the allocation of corresponding computing resources for the training task of the digital twin of the current algorithm to be trained according to the digital twin training allocation strategy includes:

[0031] Generate a state space based on the training task of the digital twin of the algorithm to be trained;

[0032] Assigning a strategy to the state space and the digital twin training to generate a probability vector for the next action;

[0033] According to the probability distribution of the next action, the training order and corresponding computing resource conditions of each algorithm to be trained are obtained.

[0034] An embodiment of the present invention further provides a resource scheduling device for a twin training task, comprising:

[0035] A feature vector generation module is used to obtain the digital twin of the algorithm to be trained and generate a parameterized feature vector based on the attribute characteristics of the digital twin;

[0036] An allocation strategy generation module is used to generate a digital twin training allocation strategy based on the parameterized feature vector and a reinforcement learning method based on information concentration evaluation;

[0037] The computing resource allocation module is used to allocate corresponding computing resources to the training task of the digital twin of the current algorithm to be trained according to the digital twin training allocation strategy.

[0038] An embodiment of the present invention also provides a resource scheduling device for twin training tasks, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the resource scheduling method for twin training tasks as described in any one of the above items.

[0039] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to perform a resource scheduling method for a twin training task as described in any one of the above.

[0040] An embodiment of the present invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or the computer instructions are executed by a processor, it implements the resource scheduling method for the twin training task as described in any one of the above.

[0041] Compared with the existing technology, the resource scheduling method, device, medium and product for twin training tasks disclosed in the present invention are suitable for cluster training of digital twin algorithms, can reasonably allocate computing resources and improve training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of a resource scheduling method for a twin training task provided by an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of a process for generating a digital twin training allocation strategy in an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of the structure of a neural network in an embodiment of the present invention;

[0045] Figure 4 It is a structural diagram of a resource scheduling device for a twin training task provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0047] In the description of this application, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application.

[0048] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. Throughout this application, unless otherwise specified, "plurality" means two or more.

[0049] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0050] See also Figure 1 , is a flow chart of a resource scheduling method for a twin training task provided by an embodiment of the present invention. The embodiment of the present invention provides a resource scheduling method for a twin training task, including steps S11 to S13:

[0051] S1. Obtain a digital twin of the algorithm to be trained, and generate a parameterized feature vector based on the attribute characteristics of the digital twin;

[0052] S2. Generate a digital twin training allocation strategy based on the parameterized feature vector and a reinforcement learning method based on information concentration evaluation;

[0053] S3. According to the digital twin training allocation strategy, corresponding computing resources are allocated to the training task of the digital twin of the current algorithm to be trained.

[0054] It's important to note that within digital twin platforms, this issue also involves the heterogeneity of neural network input data. For the same network parameters, varying degrees of information in the data itself can lead to varying training resource costs. Data with more prominent information converges more quickly, allowing for faster training even at higher costs. Data with less prominent information requires longer to adjust to appropriate weights. Current resource scheduling solutions haven't yet considered information density and implemented corresponding measures. There are no methods for evaluating information density in digital twin data, no design solutions for reinforcement learning reward functions based on information density, and no algorithm training resource scheduling solutions tailored to the heterogeneous nature of digital twin data patterns.

[0055] In an embodiment of the present invention, in order to solve the problem of how to generate a training resource allocation strategy based on the data characteristics of the twin when generating supporting algorithms for a large number of twin clusters with different three-dimensional structures, motion modes, sampling frequencies, and signal modes in a digital twin cluster platform, compared with similar general computing resource scheduling solutions, the embodiment of the present invention aims to solve the dynamic scheduling of computing tasks mainly based on physical simulator clusters and deep learning clusters, and generate parameterized feature vectors by obtaining the digital twins of the algorithm to be trained and their attribute characteristics. According to the parameterized feature vectors, a digital twin training allocation strategy is generated based on the reinforcement learning method of information concentration evaluation to realize resource scheduling during algorithm training that takes into account the information concentration of sample data, thereby allocating corresponding computing resources to the training tasks of the digital twins of the current algorithm to be trained according to the digital twin training allocation strategy.

[0056] Compared to general computing resource scheduling methods, the technical means of the embodiments of the present invention take into account the information density of the reinforcement learning method, realizing universal cost evaluation of heterogeneous data deep learning cluster training tasks. This achieves more accurate cost estimation for a specific algorithm training task, thereby achieving more efficient training task scheduling. The present invention is applicable to cluster training of digital twin algorithms, can reasonably allocate computing resources, and improve training efficiency.

[0057] As a preferred embodiment, the present invention is further implemented on the basis of the above embodiment. Figure 2 , is a flow chart of generating a digital twin training allocation strategy in an embodiment of the present invention. Step S1, i.e., obtaining a digital twin of the algorithm to be trained and generating a parameterized feature vector based on the attribute characteristics of the digital twin, includes:

[0058] Load the digital twin object of the algorithm to be trained from the digital twin database;

[0059] Load the twin environment, twin device, twin data interface, computational graph and data file format of the algorithm required for the digital twin, and training hyperparameters of the digital twin algorithm to generate parameterized feature vectors.

[0060] Specifically, step S1 is used to load all twin objects that need to configure the algorithm from the digital twin database, and generate a parameterized feature set that can be optimized by reinforcement learning based on the attribute characteristics of the twins.

[0061] Wherein, step S1 specifically includes the following steps:

[0062] Step S1.1: Load the twin device, including the 3D model file of the twin device, the device physical property and behavior property configuration file, the device material and rendering related files, the configuration file of the interaction rules between devices, etc.

[0063] Step S1.2: Load the twin data interface. The data interface is a program call format for requesting physical simulation results and real-time sensor results, including the dimension, length, data structure, and file reading and writing formats of various time series data.

[0064] Step S1.3: Load the computational graph and data file format of the algorithm required by the twin, including: algorithm parameter scale, number of special structures, activation function form, output layer form. In addition, the optimization strategy also needs to read relevant attributes of the dataset, including but not limited to: data file format, data dimensions, data quantity, number of dataset categories, data label file format, etc.

[0065] Step S1.4: Load the training hyperparameters of the twin algorithm. Generate the corresponding feature vector based on the hyperparameters set during the algorithm training, such as learning rate, batch size, dropout, regularization, and optimizer.

[0066] As a preferred embodiment, step S12, i.e., generating a digital twin training allocation strategy based on the parameterized feature vector and the reinforcement learning method of information density evaluation, includes:

[0067] According to the parameterized feature vector, a single-step training method is used to perform trial sampling during the training process to obtain a single-step training cost;

[0068] Estimate the algorithm computation time based on the single-step training cost;

[0069] Establishing an information complexity index based on Lempel-Ziv complexity, multiscale entropy, sample entropy, and approximate entropy; the information complexity index is used to measure information density;

[0070] Calculating the rate of decline of the loss function according to the information complexity index to calculate the number of training rounds;

[0071] Establishing a state space for reinforcement learning based on the algorithm calculation time and the training rounds;

[0072] Generate action space based on server resource type;

[0073] The final training time of the cluster task is used as a reward, and the mapping relationship between the state space and the action space is iterated through reinforcement learning;

[0074] Build and optimize a DQN-based adaptive resource allocation strategy as a training allocation strategy for digital twins.

[0075] In an embodiment of the present invention, step S2 is used to obtain four types of file attributes according to steps 1.1 to 1.4, including a three-dimensional model configuration file, a twin data interface, a deep learning algorithm structure file, and training hyperparameters. The reading time and memory overhead required for different attributes are determined by advance testing, and used as a reference in the subsequent scheduling link. By establishing a complexity index based on Lempel-Ziv complexity, multi-scale entropy, sample entropy, and approximate entropy, a general computational overhead evaluation function for digital twins is obtained, and the state space of reinforcement learning is established based on this. An action space can be generated based on server hardware resources, and the final training time of the cluster task is used as a reward. The mapping relationship between the iterative state space and the optimal action space is iterated through reinforcement learning, and finally an adaptive resource allocation strategy based on DQN (Deep Q-Network) is realized.

[0076] like Figure 3 The figure shows a schematic diagram of the structure of the neural network used in the adaptive resource allocation strategy in the embodiment of the present invention. It consists of a convolutional layer and a multi-layer perceptron. The convolutional layer reduces the multimodal digital twin data into high-dimensional features through various convolution functions and inputs them into the multi-layer perceptron. The subsequent perceptron passes through multiple layers of fully connected layers and finally outputs the probability distribution of resource allocation. For example, it is necessary to generate the following resource-related recommended values: the number of threads, CPU usage, memory usage, disk storage usage, server number, graphics card number, whether memory compression is allowed, whether parallelism is allowed, the maximum number of parallel operations, etc., which should be set for each algorithm image in the cluster.

[0077] Specifically, the implementation of step S2 is divided into the following steps:

[0078] Step S2.1: Sampling the training process. Based on the quantized state space and action space, a sampling process is performed on a cluster of small-parameter algorithms, mirroring parameters are set according to the default initial strategy, and training is performed. The sampling process aims to obtain the computational overhead of the algorithm, which is used to estimate the current state of the cluster before actual training. Specifically, the process performs single-step inference and back gradient propagation on all training tasks in the cluster to obtain the time and resources required for each algorithm training step. In addition to randomly selecting a batch of algorithms for single-step training, the sampling process also randomly selects different samples for multiple experiments.

[0079] Step S2.2: Define the state space and action space. The state S is the state of the cluster task completion and resource consumption. The state space consists of four dimensions: the number of trained algorithms, the number of algorithms being trained, the number of algorithms to be trained, and the remaining computing resources. The action space A is the action space that determines the next action to be taken. The action space contains several optional actions, and the single-step training speed is the reward R. This generates a decision sequence like the following:

[0080] (S t ,a t ,r t ,s t+1 );

[0081] Among them, S t is the current state of the cluster at time t, a t represents the action taken at time t, r t Represents taking action a t The reward obtained after t+1 Represents taking action a t The state of the cluster at the next time t+1.

[0082] The decision sequence records the state, action, and reward information of the agent at different moments. By analyzing and learning these sequences, the agent can continuously optimize its strategy so that it can obtain the maximum cumulative reward during long-term training.

[0083] Specifically, for algorithm training based on digital twin data, the present invention designs a state space based on information density evaluation and process sampling. The current state of the cluster consists of all task vectors, as shown below:

[0084] [Features of Task 1, Features of Task 2, Features of Task 3, …]

[0085] The characteristics of each task include various attributes obtained in steps S1.1 to S1.4, such as how many columns the time series to be processed has, whether the storage file is csv or txt, whether the real-time database is mysql or influxdb, etc. These attributes can be used to estimate the memory overhead and time required to obtain data from the digital twin cluster. The present invention designs a method for estimating the algorithm calculation cost based on steps 1 and 2.1. During the sampling process of step 2.1, each algorithm is trained in a single step to obtain the training cost corresponding to all the attributes of each task. Since the dimension of the attribute is fixed, the single-step training cost Delay is regarded as the function value, and each attribute is regarded as a function variable, denoted as F, and its mapping relationship is obtained by function approximation:

[0086] Delay=Test(F1,F2,F3,F4,F5,F6,…);

[0087] In this way, for the tasks that are not sampled later, the algorithm calculation time can be estimated without experimentation by directly using the properties obtained in step 1:

[0088]

[0089] The characteristics of each task also include information density, which refers to the amount of information contained in the data. After the server reads and writes specific variables from the file and stores them in the floating-point space, the time cost during training also depends on how much useful information the data itself contains. Data with more information is more efficient to train, while data with less information is less efficient to train. new In addition to estimating the training overhead of new algorithms through digital twin file properties, it is also necessary to evaluate how different information densities affect the training overhead of an algorithm.

[0090] Therefore, the embodiment of the present invention designs a complexity index composed of multiple information complexity indicators to measure the concentration of information, including: Lempel-Ziv complexity - En lZ , multi-scale entropy - MSE, sample entropy - SampEn, approximate entropy - ApEn.

[0091] En lZ =LempelZiv(n(t));

[0092] SampEn(m,r)=-log(B(m+1,r));

[0093]

[0094] MSE(m,r,r)={SampEn(m,r,U τ )|τ=1,2,…,τ max};

[0095] Among them, B, C, and Lempel-Ziv are all related dedicated functions, m is the length of the sliding window subsequence used for intermediate calculations, r is the similarity tolerance, which is used to define the extent to which the distance between two vectors is considered "similar", and N represents the length of the sequence n(t).

[0096] Lempel-Ziv complexity is used to measure the complexity of a sequence. The higher the complexity, the richer the information contained in the sequence. Sample entropy reflects the randomness and complexity of the sequence. Approximate entropy is used to measure the similarity of the sequence. Multiscale entropy takes into account the changes in entropy values ​​at different time scales.

[0097] In various software, functions can be used directly to calculate the Lempel-Ziv complexity, multiscale entropy, sample entropy, and approximate entropy functions of a sequence. These four indicators are integrated into the information complexity index:

[0098]

[0099] Preferably, the calculating the rate of decrease of the loss function according to the information complexity index to calculate the number of training rounds includes:

[0100] Fitting the relationship between the information complexity index and the loss function by the least squares method or the exponential smoothing method to obtain the rate of decrease of the loss function;

[0101] The number of training rounds is calculated based on the decreasing speed of the loss function and the preset loss rate.

[0102] Specifically, in step S2.1, different samples are randomly sampled, so for each algorithm, we can obtain how much its loss function decreases in samples of different concentrations. By fitting the relationship between the two using the least squares method or exponential smoothing method, we can obtain the following mapping:

[0103] ΔLoss=En(Index);

[0104] Therefore, for data that has not been used by the algorithm, the rate of decrease of the loss function can be estimated directly by the information density of the sample, and the number of training rounds can be inferred from this:

[0105]

[0106] Based on the method introduced in the embodiment of the present invention, the algorithm features and data features of the untested digital twin algorithm training task can be mapped to the current state of the cluster:

[0107] S = (algorithm calculation time, training rounds);

[0108] Different actions can be designed according to different scenario requirements. For example, the action space can include three strategies:

[0109] A = {start the smallest algorithm, start the largest algorithm, start the medium algorithm};

[0110] Different optional actions can be designed according to the specific situation and used as the action space.

[0111] Step S2.3: Select a neural network template to initialize the DQN network.

[0112] In the embodiment of the present invention, a convolutional autoencoder is selected as the basic network and the parameters are initialized, and the network is composed of a combination of convolutional layers and multi-layer perceptrons with a reasonable number of structural layers.

[0113] Step S2.4: Optimize the policy neural network.

[0114] Preferably, the constructing and optimizing of a DQN-based adaptive resource allocation strategy as a digital twin training allocation strategy includes:

[0115] Select the neural network template to initialize the DQN network;

[0116] Based on the initial network weights, several spatial states are selected and action spaces are generated accordingly. Based on the action spaces, corresponding rewards are obtained to construct a "state-action-reward" sample.

[0117] Randomly obtain samples of the "state-action-reward" and calculate the target value:

[0118] Using the target value and gradient descent algorithm, the network weights are updated until the resource allocation strategy meets the preset requirements, thereby obtaining the digital twin training allocation strategy.

[0119] Specifically, regardless of the structure adopted, the process of optimizing the strategy neural network can be performed according to the following steps:

[0120] The network weight is W t In the case of , several states s are selected and actions a are generated accordingly. A series of rewards r are obtained according to the actions, and the records of these attempts are stored in a storage structure D.

[0121] Randomly take some state-action-reward samples from D and use them to calculate the target value:

[0122] r+γmax a′ Q(s′,a′,Wt);

[0123] Then use the target value and gradient descent algorithm to update the network weights:

[0124] W t+1 =W t +Δw;

[0125] Repeat the above process until the policy meets the requirements, and then save the generated policy.

[0126] As a preferred embodiment, step S3, i.e., allocating corresponding computing resources for the training task of the digital twin of the current algorithm to be trained according to the digital twin training allocation strategy, includes:

[0127] Generate a state space based on the training task of the digital twin of the algorithm to be trained;

[0128] Assigning a strategy to the state space and the digital twin training to generate a probability vector for the next action;

[0129] According to the probability distribution of the next action, the training order and corresponding computing resource conditions of each algorithm to be trained are obtained.

[0130] In this embodiment of the present invention, step S3 is used to allocate corresponding resources for the current twin training task using the generated twin training allocation strategy. Specifically, step S3 includes the following three steps:

[0131] Step S3.1: Generate the state space based on the current remaining resources of the twin algorithm training cluster and the server. First, generate the state space based on the server environment and the algorithm type pre-set on the twin platform.

[0132] Step S3.2: Input the state space into the neural network structure to generate the probability of each current action.

[0133] Step S3.3: According to the action probability distribution, the order of algorithm cluster training and the corresponding resource conditions are selected in sequence.

[0134] The implementation details of the present invention are described in detail below through specific embodiment scenarios.

[0135] For example, a factory has established a digital twin dashboard and wants to batch generate neural network prediction algorithms for key digital twins. These key equipment include three six-degree-of-freedom industrial robotic arms, two six-degree-of-freedom CNC milling machines, and three single-degree-of-freedom belt conveyors. This may require building a batch of deep learning time series prediction algorithms with varying structures for 3×6+2×6+3×1=33 electromechanical systems. These algorithms need to be trained in a cluster. Because servers have limited GPUs, memory, bandwidth, and throughput threads, different task sequencing and resource allocation strategies during training will affect the final cluster training time. This technology can be used to generate the optimal resource allocation and sequencing strategy for this cluster training task. Its implementation specifically involves the following steps:

[0136] Step 1: Read the properties of the digital twin and generate a parameterized state vector for the algorithm. Taking the temperature prediction algorithm of a machine tool screw twin as an example, this step specifically includes the following steps.

[0137] Step 1.1: Load the twin environment information, determine the required data type and storage format based on the monitoring tasks assigned to each object in the scene, and learn that the "temperature prediction" algorithm and "vibration amplitude prediction" algorithm need to be trained for a certain screw.

[0138] Step 1.2: Load the twin device. From the leadscrew device information, we know that the temperature signal sampling rate is 10 Hz. According to the processing task instructions, we know that a complete signal segment lasts for 45 seconds. From the sensor nameplate, we know that the conversion coefficient between the reading and the actual temperature is 0.166.

[0139] Step 1.3: Load the data interface. According to the data-related properties of the screw twin, the temperature reading protocol is http, the reading method is get, the return value is json, and the query key is temp.

[0140] Step 1.4: Load the dataset and computation graph for the algorithm required by the twin. From the database loading information, we know that the screw temperature data has 6 columns, the sequence length is 60,000 points, and the expected prediction step length is 40 seconds. From the algorithm computation graph, we know that the prediction algorithm is a standard LSTM, with a prediction dimension of 2, 2 hidden layers, and a hidden layer size of 100.

[0141] Step 1.5: Load the training hyperparameters for the twin algorithm. The LSTM algorithm is trained with a learning rate of 0.01, a dropout ratio of 0.15, and an expected prediction accuracy of 5%. Following steps 1.1 through 1.5, generate the following dimensionless feature vector for the i-th algorithm to be started, the screw temperature prediction algorithm:

[0142] F i=[10,45,0.166,http,get,json,6,60000,40,LSTM,2,2,100,0.01,0.15,0.05]

[0143] Repeating steps 1.1 to 1.5 for 33 twin objects will yield the same number of features as the number of algorithms to be trained. The key feature of this step is the use of twin information to supplement the parameterized information vector of each algorithm, which includes the structural properties of the neural network and the properties of the twin data format. The key points of developing algorithms for industrial digital twin platforms are different from those of ordinary deep learning algorithm development. Its cluster training strategy also needs to consider the computational time and space costs of reading and writing real data from the twin system. This information records the storage format, storage quantity, and corresponding read and write costs of real-time IoT data waiting to be predicted from different sources in the system. In addition, there is also the information density of the specific sample data, which indicates the richness of the information content contained in the data.

[0144] Step 2: Based on the algorithm training tasks to be optimized and the total server resource pool, DQN is used to determine the optimal cluster training task execution order and resource allocation strategy. Assuming 100 different time series prediction algorithms need to be generated, then, without considering resource constraints, the possible algorithm cluster training orders are virtually infinite based on permutations and combinations. Given limited resources, finding the optimal cluster training order is a vast and daunting task, as resource recycling and reuse are involved. Therefore, assume that the server has 1024GB of RAM, 320GB of total graphics memory, a graphics cluster of four H100 graphics cards, 100TB of hard disk storage space, a maximum thread count of 70,000, and a core count limit of 1024.

[0145] Step 2 specifically includes the following steps:

[0146] Step 2.1: Trial sampling during the training process. In order of order, have 100 algorithms perform single-step reasoning in a common environment. Record the fluctuations in time (4 seconds), CPU usage (1600Mb), memory usage (600Mb), bandwidth (3Mb / s), and core count (4). This gives the cost of each algorithm iteration. For example, the properties of the screw temperature prediction algorithm used in Step 1 are recorded as follows:

[0147]

[0148] According to the state definition method described in step 2.1 above, the above attributes are converted into corresponding calculation time, and the concentration is converted into the number of training times, which can be recorded as follows:

[0149] Cost = (algorithm calculation time, training rounds);

[0150] All costs are consolidated into:

[0151] Cost={cost1,cost2,cost3,…cost 100};

[0152] Step 2.2: Define the state space and action space. The state space consists of four dimensions: {number of trained algorithms, number of algorithms being trained, number of algorithms to be trained, and remaining computing resources}. Based on the example given, a current state can be represented by a seven-tuple:

[0153] s=[4,40,56,40,15,1200,48];

[0154] This vector indicates the current status: 4 algorithms have completed training, 40 algorithms are being trained, 56 algorithms are waiting for training, 40 GB of running memory remains, 15 GB of video memory remains, 1200 idle threads remain, and 48 idle cores remain.

[0155] Action is the next action that can be selected. For example, the action space can include {start an algorithm with the smallest number of parameters, start an algorithm with the largest number of parameters, start an algorithm with the smallest sample size, start an algorithm with the largest sample size, start an algorithm with the largest sample size × parameter size, start an algorithm with the smallest sample size × parameter size}. Action is the possibility of a series of behaviors, recorded as a vector. For six behaviors, the action vector may be:

[0156] a=[0.9,0.8,0.15,0.11,0.01,0.005];

[0157] This vector indicates that for the current state, taking action 1 or action 2 is a better decision. This defines the state space and action space.

[0158] Step 2.3: Initialize the DQN network using a standard multilayer perceptron (MLP). The network's input is a seven-tuple state vector, and its output is a sextet action vector. The network has 10 hidden layers, each containing 40 neurons. This neural network is denoted by function N.

[0159] The neurons are obtained by summing the weighted bias and activation of all neurons in the previous layer:

[0160]

[0161] The activation function is:

[0162]

[0163] Step 2.4: Optimize the neural network using Q-learning and stochastic gradient descent. The goal of Q-learning is to obtain and update the corresponding a for different s. Reinforcement learning and backpropagation are performed alternately to update the network.

[0164] The network weight is W t In the case of , according to the cost obtained in the previous step 2.1, a small virtual image with limited resources is randomly created, which is insufficient to support the simultaneous training of all algorithms. Several random single-step inferences are performed in it to obtain the remaining tasks and remaining resources of different initial cluster training systems. The state at each inference is s t :

[0165] S(t)=[s1(t),s2(t),s3(t),s4(t),s5(t),s6(t),s7(t)];

[0166] Input one of the vectors S(t) into the network N and get the corresponding A(t):

[0167] A(t)=N(S(t));

[0168] A(t)=[a1(t),a2(t),a3(t),a4(t),a5(t),a6(t)];

[0169] Select different actions based on A(t), and perform the next single-step reasoning based on the actions, repeating several times until the remaining resources are insufficient to start any new algorithm. At this point, the inverse of the number of remaining algorithms is recorded as the reward r for this attempt:

[0170]

[0171] By selecting different initial S(t) inputs and feeding them into N, we obtain a series of [S(t), A(t), r]. It's important to note that only the initial state, action, and reward at the end of the trial are saved during each storage. Subsequent S(t+1) and A(t+1) do not need to be saved. Furthermore, for each S(t), in addition to selecting the action with the highest probability recommended by the network, other actions are also tried in sequence.

[0172] These attempts are stored in a storage structure D. Randomly sample state-action-reward pairs from D to calculate the theoretically optimal target action. Specifically, if a set of 100 samples with the same state but different actions are found in D, calculate the target value:

[0173] γ = 0.1;

[0174]

[0175] value(a(t))=R+γmax a Q(s,a,W);

[0176] max a Q(s,a,W) represents the maximum benefit generated by selecting behavior a when the initial state is S. R is the average benefit of the reward obtained by state S in this batch of samples. Value represents the actual benefit that can be obtained by taking different actions a in the remaining resource state S.

[0177] This will result in six different practical benefits:

[0178] value(a1(t)),value(a2(t)),value(a3(t)),value(a4(t)),value(a5(t)),value(a5(t));

[0179] By calculating the proportion of each behavior's benefit to the total benefit, we can get the priority of the actual benefits of taking 6 different behaviors in state S(t). For example, the actual benefit vectors of the six behaviors are:

[0180] value=[0.45,0.19,1.41,0.34,0.16,0.2];

[0181] The proposed action vector assumed in step 2.2 is:

[0182] a=[0.9,0.8,0.15,0.11,0.01,0.005];

[0183] Then the loss function is:

[0184]

[0185] The loss obtained using the input S(t) By backpropagating, the weights and biases of the neurons can be updated. Repeat the above process until the strategy meets the requirements, and then save the generated network weights W max .

[0186] Step 3: Utilize W max , making task allocation decisions for the current 100 tasks. Steps 1 and 2 are both single-step iterations in the virtual image to train the agent's scheduling strategy. After sufficient training time, the agent can be started in the actual server environment. The agent's work is divided into the following steps:

[0187] Step 3.1: Randomly start the training tasks of several algorithms to obtain the current state and state vector.

[0188] Step 3.2: Input the state vector to the network N(W max ), and get the next action probability vector.

[0189] Step 3.3: Take the action with the highest probability.

[0190] Repeat 3.1 to 3.3 until the algorithm is fully trained.

[0191] Utilizing the technical means of the embodiments of this invention, the proposed digital twin algorithm training function strategy generation mechanism addresses the new task scenario of digital twins. This mechanism addresses the issue of how large-scale digital twins can automatically train their own corresponding function algorithms. This proposed process for generating resource allocation strategies eliminates the time required for manual allocation and readjustment, improving the efficiency of the digital twin development platform. Specifically, attribute evaluation methods and information density evaluation methods were designed to address the heterogeneous nature of digital twins. The reinforcement learning implemented in this way improves optimization efficiency and scheduling accuracy in digital twin scenarios.

[0192] This paper innovatively proposes a resource scheduling method for deep learning cluster algorithms. This method employs a reinforcement learning approach based on information density evaluation, establishes a training time evaluation model using a single-step sampling process, and establishes an information density evaluation model based on an information complexity index. Based on these two models, the state information of the algorithm and its untrained dataset in an unstarted training task is generated, thereby forming a reinforcement learning state space. By performing single-step training trial sampling on small batch algorithms, the relationship between multidimensional attributes and the time spent on a single round of training is established. The possible training rounds for a particular algorithm are then established by performing trial sampling on multiple experimental data sets and using an information complexity index composed of multiple information entropies. This solves the problem of difficulty in estimating the training cost of a particular algorithm based on the input data structure and neural network architecture in digital twin cluster algorithm training scenarios. Finally, the untrained algorithm is evaluated, its state is generated, and dynamic decisions are made. A dynamic startup order is set for the cluster tasks to be trained, ultimately reducing the training time of the algorithm cluster by over 20%.

[0193] The existing resource scheduling methods for algorithm training on digital twin platforms have problems such as low resource utilization and low scheduling efficiency. The present invention optimizes the computing efficiency and scheduling effect of the existing scheduling methods, improves the resource utilization and task execution efficiency of the system, is suitable for solving the resource scheduling problem of algorithm training in a complex digital twin environment, and can automatically generate a system resource allocation strategy during training, which can produce industrial analysis results with reference value. The present invention first establishes an evaluation model for the multi-level data generated by large-scale and diverse digital twins, generates a parameterized feature set that can be optimized by reinforcement learning, and then generates a state space based on the parameterized feature set, generates an action space based on the hardware resources of the server, and uses operating efficiency and resource surplus as rewards. Through reinforcement learning, the mapping relationship between the iterative state space and the optimal action space is iterated, and the generated twin training allocation strategy is used to allocate corresponding resources for the current twin training task.

[0194] See also Figure 4 , is a structural diagram of a resource scheduling device for a twin training task provided by an embodiment of the present invention. The embodiment of the present invention provides a resource scheduling device 10 for a twin training task, including:

[0195] A feature vector generation module 11 is used to obtain the digital twin of the algorithm to be trained and generate a parameterized feature vector based on the attribute characteristics of the digital twin;

[0196] An allocation strategy generation module 12 is used to generate a digital twin training allocation strategy based on the parameterized feature vector and a reinforcement learning method based on information density evaluation;

[0197] The computing resource allocation module 13 is used to allocate corresponding computing resources to the training task of the digital twin of the current algorithm to be trained according to the digital twin training allocation strategy.

[0198] It should be noted that the resource scheduling device for a twin training task provided in an embodiment of the present invention is used to execute all the process steps of the resource scheduling method for a twin training task in the above embodiment. The working principles and beneficial effects of the two correspond one to one, so they will not be repeated here.

[0199] An embodiment of the present invention also provides another resource scheduling device for twin training tasks, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the resource scheduling method for the twin training task as described in any one of the above items.

[0200] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to perform a resource scheduling method for a twin training task as described in any one of the above.

[0201] An embodiment of the present invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or the computer instructions are executed by a processor, it implements the resource scheduling method for the twin training task as described in any one of the above.

[0202] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0203] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A resource scheduling method for twin training tasks, characterized in that: include: Obtaining a digital twin of the algorithm to be trained, and generating a parameterized feature vector based on the attribute characteristics of the digital twin; According to the parameterized feature vector, a digital twin training allocation strategy is generated based on a reinforcement learning method of information density evaluation; According to the digital twin training allocation strategy, corresponding computing resources are allocated to the training task of the digital twin of the current algorithm to be trained.

2. The resource scheduling method for twin training tasks according to claim 1, characterized in that: The step of obtaining a digital twin of the algorithm to be trained and generating a parameterized feature vector based on the attribute characteristics of the digital twin includes: Load the digital twin object of the algorithm to be trained from the digital twin database; Load the twin environment, twin device, twin data interface, computational graph and data file format of the algorithm required for the digital twin, and training hyperparameters of the digital twin algorithm to generate parameterized feature vectors.

3. The resource scheduling method for twin training tasks according to claim 1, characterized in that: The method of generating a digital twin training allocation strategy based on the parameterized feature vector and the information density evaluation reinforcement learning method includes: According to the parameterized feature vector, a single-step training method is used to perform trial sampling during the training process to obtain a single-step training cost; Estimate the algorithm computation time based on the single-step training cost; Establishing an information complexity index based on Lempel-Ziv complexity, multiscale entropy, sample entropy, and approximate entropy; the information complexity index is used to measure information density; Calculating the rate of decline of the loss function according to the information complexity index to calculate the number of training rounds; Establishing a state space for reinforcement learning based on the algorithm calculation time and the training rounds; Generate action space based on server resource type; The final training time of the cluster task is used as a reward, and the mapping relationship between the state space and the action space is iterated through reinforcement learning; Build and optimize a DQN-based adaptive resource allocation strategy as a training allocation strategy for digital twins.

4. The resource scheduling method for twin training tasks according to claim 3, characterized in that: Calculating the rate of decrease of the loss function according to the information complexity index to obtain the number of training rounds includes: Fitting the relationship between the information complexity index and the loss function by the least squares method or the exponential smoothing method to obtain the rate of decrease of the loss function; The number of training rounds is calculated based on the decreasing speed of the loss function and the preset loss rate.

5. The resource scheduling method for twin training tasks according to claim 3, characterized in that: The construction and optimization of the DQN-based adaptive resource allocation strategy as a digital twin training allocation strategy includes: Select the neural network template to initialize the DQN network; Based on the initial network weights, several spatial states are selected and action spaces are generated accordingly. Based on the action spaces, corresponding rewards are obtained to construct a "state-action-reward" sample. Randomly obtain samples of the "state-action-reward" and calculate the target value: Using the target value and gradient descent algorithm, the network weights are updated until the resource allocation strategy meets the preset requirements, thereby obtaining the digital twin training allocation strategy.

6. The resource scheduling method for twin training tasks according to claim 1, characterized in that: Allocating corresponding computing resources for the training task of the digital twin of the current algorithm to be trained according to the digital twin training allocation strategy includes: Generate a state space based on the training task of the digital twin of the algorithm to be trained; Assigning a strategy to the state space and the digital twin training to generate a probability vector for the next action; According to the probability distribution of the next action, the training order and corresponding computing resource conditions of each algorithm to be trained are obtained.

7. A resource scheduling device for twin training tasks, characterized in that: include: A feature vector generation module is used to obtain the digital twin of the algorithm to be trained and generate a parameterized feature vector based on the attribute characteristics of the digital twin; An allocation strategy generation module is used to generate a digital twin training allocation strategy based on the parameterized feature vector and a reinforcement learning method based on information concentration evaluation; The computing resource allocation module is used to allocate corresponding computing resources to the training task of the digital twin of the current algorithm to be trained according to the digital twin training allocation strategy.

8. A resource scheduling device for twin training tasks, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the resource scheduling method for the twin training task as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the resource scheduling method for the twin training task as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes a computer program or computer instructions, and when the computer program or the computer instructions are executed by a processor, the resource scheduling method for the twin training task as described in any one of claims 1 to 6 is implemented.

Images