Distributed power model updating method, device and electronic device based on model incremental training

Abstract

The present invention provides a distributed power model update method, device and electronic device based on model incremental training. The implementation scheme is as follows: when the local power incremental data of the distributed power node is insufficient, the local model parameter matrix of the local power model in the distributed power node is subjected to singular value decomposition to obtain a left singular matrix, a diagonal matrix and a right singular matrix; based on the main singular values ​​in the diagonal matrix, the corresponding column vectors are extracted from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix; based on the difference matrix between the local model parameter matrix and the important parameter matrix, the non-important parameter matrix is ​​determined, and the non-important parameter matrix is ​​subjected to low-rank decomposition to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix, so as to perform a global model update and then update the local power model of each power node. The present invention can improve the speed and accuracy of model training and avoid model drift.

Description

Technical Field

[0001] The present invention relates to the fields of computer and power technology, and in particular to a distributed power model updating method, device and electronic equipment based on model incremental training. Background Art

[0002] In some power technology fields, deploying multiple distributed power nodes and global nodes can improve the accuracy of predictions of power-related indicators or functions, thereby improving the accuracy of distributed power dispatching.

[0003] Among them, in each distributed power node, the local power model is trained using local power data, and then the trained local power model parameters are sent to the global node. The global node aggregates the local power model parameters of each distributed power node to obtain the global power model parameters, and returns the global power model parameters to each distributed power node to update the local power model of each distributed power node.

[0004] However, with the continuous increase in the amount of power data and model scale, traditional model training methods face many challenges in terms of real-time performance, computing resources, energy consumption, and communication delay. Summary of the Invention

[0005] The present invention provides a distributed power model updating method, device and electronic device based on model incremental training, which can solve at least one of the above technical problems.

[0006] According to one aspect of the present invention, a distributed power model updating method based on model incremental training is provided, comprising:

[0007] When the amount of local power incremental data of the distributed power node is less than a preset threshold, performing singular value decomposition on a local model parameter matrix of the local power model in the distributed power node to obtain a left singular matrix, a diagonal matrix, and a right singular matrix;

[0008] Based on the main singular values ​​in the diagonal matrix, extract corresponding column vectors from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix;

[0009] Determine a non-important parameter matrix based on a difference matrix between the local model parameter matrix and the important parameter matrix, and perform low-rank decomposition on the non-important parameter matrix to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix;

[0010] Based on the local power increment data, the local first low-rank parameter matrix and the local second low-rank parameter matrix are updated, and the update result is sent to the global scheduling server, so that the global scheduling server updates the global first low-rank parameter matrix and the global second low-rank parameter matrix;

[0011] The local power model of the distributed power node is updated based on the product of the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server, and the important parameter matrix.

[0012] According to another aspect of the present invention, a distributed power model updating device based on model incremental training is provided, comprising:

[0013] a singular value decomposition module, configured to perform singular value decomposition on a local model parameter matrix of a local power model in a distributed power node when the amount of local power incremental data of the distributed power node is less than a preset threshold, to obtain a left singular matrix, a diagonal matrix, and a right singular matrix;

[0014] An important parameter determination module is used to extract corresponding column vectors from the left singular matrix and the right singular matrix based on the main singular values ​​in the diagonal matrix to obtain an important parameter matrix;

[0015] A low-rank parameter determination module is used to determine a non-important parameter matrix based on a difference matrix between the local model parameter matrix and the important parameter matrix, and decompose the non-important parameter matrix to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix;

[0016] a global model updating module, configured to update the local first low-rank parameter matrix and the local second low-rank parameter matrix in the local model parameter matrix based on the local power increment data of the distributed power node, and send the updated result to the global scheduling server, so that the global scheduling server updates the global first low-rank parameter matrix and the global second low-rank parameter matrix;

[0017] A local model updating module is used to update the local power model of the distributed power node based on the product between the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server, and the important parameter matrix.

[0018] According to the technical solution of the present invention, when the local power incremental data of the distributed power node is insufficient, the local model parameter matrix of the local power model in the distributed power node is subjected to singular value decomposition to obtain a left singular matrix, a diagonal matrix and a right singular matrix; based on the main singular values ​​in the diagonal matrix, the corresponding column vectors are extracted from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix; based on the difference matrix between the local model parameter matrix and the important parameter matrix, the non-important parameter matrix is ​​determined, and the non-important parameter matrix is ​​subjected to low-rank decomposition to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix; based on the local power incremental data of the distributed power node, the local first low-rank parameter matrix and the local second low-rank parameter matrix in the local model parameter matrix are updated. In this way, when the incremental data is insufficient, the important parameters in the local model parameters can be retained and not updated, and only the non-important parameters are updated, which can effectively maintain the adaptability of the original task of the model and avoid drift of the local power model. In addition, the non-important parameter matrix is ​​subjected to low-rank decomposition and then updated to reduce the amount of data and improve the efficiency of model update. Furthermore, the updated results are sent to the global scheduling server, causing the global scheduling server to update the global first low-rank parameter matrix and the global second low-rank parameter matrix. Subsequently, the local power models of the distributed power nodes are updated based on the product of the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server, as well as the important parameter matrix. In this way, by utilizing the global model parameters that can aggregate the local power model parameters of each distributed node and thereby updating the local power models of the distributed nodes, the model accuracy of the local power models of each distributed node can be improved, and drift of the local power models can be avoided.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present invention.

[0021] Figure 1 is a flow chart of a distributed power model updating method based on model incremental training according to an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a model updating process according to an embodiment of the present invention;

[0023] Figure 3 This is a structural block diagram of a distributed power model updating device based on model incremental training according to an embodiment of the present invention;

[0024] Figure 4 is a block diagram of an electronic device for implementing the method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, and various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] Figure 1 This is a flowchart of a distributed power model updating method based on model incremental training according to an embodiment of the present invention.

[0027] like Figure 1 As shown, the distributed power model updating method based on model incremental training includes:

[0028] S110, when the amount of local power incremental data of the distributed power node is less than a preset threshold, performing singular value decomposition on a local model parameter matrix of a local power model in the distributed power node to obtain a left singular matrix, a diagonal matrix, and a right singular matrix;

[0029] S120, based on the main singular values ​​in the diagonal matrix, extract corresponding column vectors from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix;

[0030] S130, determining a non-important parameter matrix based on a difference matrix between the local model parameter matrix and the important parameter matrix, and performing low-rank decomposition on the non-important parameter matrix to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix;

[0031] S140, based on the local power increment data of the distributed power node, updating the local first low-rank parameter matrix and the local second low-rank parameter matrix in the local model parameter matrix, and sending the update result to the global scheduling server, so that the global scheduling server updates the global first low-rank parameter matrix and the global second low-rank parameter matrix;

[0032] S150 , updating the local power model of the distributed power node based on the product of the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server, and the important parameter matrix.

[0033] In practical applications, the power system may include multiple distributed power nodes and a global dispatch server. Figure 2 As shown, the global scheduling server is Figure 2 In the dispatch server, the distributed power nodes include nodes 1 to 4. Of course, other nodes can also be included, but they are not shown in the figure. Each node uploads status data, such as load, temperature, power consumption and other parameters to the dispatch server. At the same time, each node also uploads the local differential updated model parameters to the dispatch server. The dispatch server uses the status data of each node to calculate the weight of each node. In addition, for nodes with ARM architecture, the weight of the node is updated using the dispatch coefficient. Thus, the model parameters provided by each node are aggregated using the weight of each node to obtain the global model parameters, which are distributed to each node to update the local model parameters of each node.

[0034] It is understandable that the methods provided by the embodiments of the present invention can be applied to any distributed power node. Each distributed power node can employ edge devices equipped with ARM chips, allowing them to primarily handle data preprocessing, feature extraction, and some lightweight inference tasks, while also leveraging their low power consumption for long-term online caching. Because the important parameter matrix remains fixed, the ARM side only needs to store it during initial loading, which reduces the pressure on communication and energy consumption.

[0035] It can be understood that the data volume of the local power incremental data is less than a preset data volume threshold, indicating that the local power incremental data of the distributed power node is insufficient.

[0036] It can be understood that the initial local power models of each distributed power node are generally the same, and after the process of local update, global update and then local update, the local power models obtained are still the same on each node.

[0037] Exemplarily, singular value decomposition is performed on the local model parameter matrix of the local power model in the distributed power node to obtain a left singular matrix, a diagonal matrix, and a right singular matrix, as follows:

[0038] For the local model parameter matrix , the singular value decomposition result is:

[0039] ;

[0040] in, is a left singular matrix, is a diagonal matrix, is a right singular matrix. The left singular matrix and the right singular matrix are orthogonal matrices, and their column vectors are singular vectors.

[0041] In one example, the singular values ​​with larger values ​​in the diagonal matrix are taken as the primary singular values. The column vectors corresponding to the primary singular values ​​in the left and right singular matrices are merged to obtain the key parameter matrix. When local incremental data is insufficient, training and updating the key parameter matrix is ​​omitted, and only the non-key parameter matrices are trained and updated. This can prevent drift or overfitting of the local power model.

[0042] In one example, statistics are taken based on the weights of the local power model, and parameters whose weights change greatly during the model training process are selected. Based on these parameters, their positions in the diagonal matrix are determined, and the singular values ​​at this position are used as the main singular values. These values ​​can be combined with the singular values ​​with larger values ​​in the above diagonal matrix as the main singular values. The column vectors corresponding to the main singular values ​​in the left singular matrix and the right singular matrix are merged to obtain the important parameter matrix. When the local incremental data is insufficient, the important parameter matrix is ​​not trained and updated, and only the non-important parameter matrix is ​​trained and updated, which can avoid drift or overfitting of the local power model.

[0043] In one example, based on the local power model during the last training process, the model parameters whose gradient sensitivity is greater than a preset threshold are determined, the position of the model parameters in the diagonal matrix is ​​determined, and the singular value at this position is used as the main singular value. The main singular value here can be combined with the singular value with a larger value in the above-mentioned diagonal matrix as all the main singular values. In this way, the important parameter matrix can include parameters with small singular values ​​but actually high gradient sensitivity. Then, when updating non-important parameters, these parameters with small singular values ​​but actually high gradient sensitivity can be avoided from participating in the model update, thereby further avoiding model drift or overfitting.

[0044] For example, the difference matrix between the local model parameter matrix and the important parameter matrix is ​​used as the non-important parameter matrix. The low-rank decomposition of the non-important parameter matrix can be: ;

[0045] in, is the matrix of non-important parameters, is the local first low-rank parameter matrix, is the local second low-rank parameter matrix, , , .

[0046] Exemplarily, the local first low-rank parameter matrix and the local second low-rank parameter matrix in the local model parameter matrix are continuously iteratively updated, and the local first low-rank parameter matrix of the last two updates is subtracted to obtain the local first low-rank parameter difference matrix, and the local second low-rank parameter matrix of the last two updates is subtracted to obtain the local second low-rank parameter difference matrix. In this way, since the amount of data of the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix is ​​small, the transmission speed can be increased when transmitted to the global scheduling server. In addition, due to the small amount of data, the calculation speed of the global model parameter aggregation can also be increased.

[0047] Exemplarily, the global scheduling server globally aggregates the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix of each distributed power node, and uses the global aggregation result to update the global first low-rank parameter matrix and the global second low-rank parameter matrix stored in the server, and sends the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix to each distributed power node.

[0048] Exemplarily, a matrix obtained by adding the product of the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix to the important parameter matrix is ​​used as the local model parameter matrix of the local power model of the distributed power node, specifically as follows:

[0049] ;

[0050] in, is the local model parameter matrix, is the important parameter matrix, and represents the global first low-rank parameter matrix and the global second low-rank parameter matrix, represents the updated non-important parameter matrix.

[0051] Exemplarily, when the amount of local power incremental data of a distributed power node is greater than a preset threshold, the local model parameter matrix of the local power model in the distributed power node is subjected to low-rank decomposition to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix. The model training process of steps S140 and S150 is performed based on these two matrices. In this way, when the amount of data is sufficient, the local model parameter matrix is ​​directly subjected to low-rank decomposition, and the local model parameters are updated based on the results of the low-rank decomposition. In this way, not only can the model accuracy and the model training speed be improved, but also, since the local power incremental data is sufficient, model drift and overfitting will not occur.

[0052] According to the above embodiment, when the local power incremental data of the distributed power node is insufficient, the local model parameter matrix of the local power model in the distributed power node is subjected to singular value decomposition to obtain a left singular matrix, a diagonal matrix and a right singular matrix; based on the main singular values ​​in the diagonal matrix, the corresponding column vectors are extracted from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix; based on the difference matrix between the local model parameter matrix and the important parameter matrix, the non-important parameter matrix is ​​determined, and the non-important parameter matrix is ​​subjected to low-rank decomposition to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix; based on the local power incremental data of the distributed power node, the local first low-rank parameter matrix and the local second low-rank parameter matrix in the local model parameter matrix are updated. In this way, when the incremental data is insufficient, the important parameters in the local model parameters can be retained and not involved in the update, and only the non-important parameters are updated, which can effectively maintain the adaptability of the original task of the model and avoid drift of the local power model. In addition, the non-important parameters are subjected to low-rank decomposition and then updated to reduce the amount of data and improve the efficiency of model update. Furthermore, the updated results are sent to the global scheduling server, causing the global scheduling server to update the global first low-rank parameter matrix and the global second low-rank parameter matrix. Subsequently, the local power models of the distributed power nodes are updated based on the product of the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server, as well as the important parameter matrix. In this way, by utilizing the global model parameters that can aggregate the local power model parameters of each distributed node and thereby updating the local power models of the distributed nodes, the model accuracy of the local power models of each distributed node can be improved, and drift of the local power models can be avoided.

[0053] In one embodiment, based on the main singular values ​​in the diagonal matrix, corresponding column vectors are extracted from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix, including: sorting the diagonal singular values ​​in the diagonal matrix from large to small, and intercepting the first N diagonal singular values ​​as the main singular values, where N is a positive integer greater than 1; based on the singular vectors of each main singular value in the left singular matrix and the singular vectors in the right singular matrix, the important parameter matrix is ​​determined.

[0054] For example, determine the column position of the main singular value, extract the left singular vector of the column position in the left singular matrix, and extract the right singular vector of the column position in the right singular matrix, form matrices of the extracted left singular vectors and right singular vectors respectively, and perform inverse singular value decomposition on the matrix composed of the main singular values ​​to obtain the important parameter matrix, where the number of rows and columns of the important parameter matrix is ​​the same as the number of rows and columns of the local model parameter matrix.

[0055] In this way, the important parameter matrix is ​​subsequently subtracted from the local model parameter matrix to obtain the unimportant parameter matrix, which is the difference matrix mentioned above.

[0056] According to the above embodiment, the singular value with a larger numerical value is used as the main singular value, and the singular vectors in the left singular matrix and the singular vectors in the right singular matrix corresponding to the main singular value are combined to obtain the important parameter matrix. In this way, the local model parameter matrix is ​​subtracted from the important parameter matrix to obtain the unimportant parameter matrix. Only the local model is updated for the unimportant parameter matrix, which can avoid model drift or overfitting of the local power model when the training data is insufficient.

[0057] In one embodiment, based on the main singular values ​​in the diagonal matrix, corresponding column vectors are extracted from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix, including: determining the first position in the diagonal matrix corresponding to the first power model parameter whose gradient sensitivity in the local power model is greater than a preset threshold; determining the main singular values ​​based on the singular values ​​at the first position; and determining the important parameter matrix based on the singular vectors of each main singular value in the left singular matrix and the singular vectors in the right singular matrix.

[0058] It can be understood that the first power model parameter may include one or more, such that the first position may include one or more, and such that the main singular value may include one or more.

[0059] It is understandable that when there is insufficient training data, updating the model parameters with excessive gradient sensitivity will cause the model to drift, thereby affecting the model accuracy. Therefore, in this example, these model parameters can be corresponding to the first position in the diagonal matrix, and the singular value at this position can be used as the main singular value, which can be combined with the above-mentioned first N diagonal singular values ​​as the final main singular value. In this way, even if the singular value of a model parameter is too small but its gradient sensitivity is too large, it is still used as an important parameter. In this way, when there is insufficient training data, avoiding important parameters and only training unimportant parameters can improve model accuracy while avoiding model drift.

[0060] In one embodiment, the above-mentioned determining the main singular value based on the singular value at the first position includes: sorting the diagonal singular values ​​in the diagonal matrix from large to small, and determining the first N diagonal singular values, where N is a positive integer greater than 1; and obtaining the main singular value based on the first N diagonal singular values ​​and the singular value at the first position.

[0061] Exemplarily, the first N diagonal singular values ​​are combined with the singular value at the first position to obtain the main singular value.

[0062] It is understandable that the singular value at the first position may have a smaller value and may not be arranged in the first N diagonal singular values, but because it corresponds to a model parameter with excessive gradient sensitivity. Therefore, it is necessary to treat the singular value at the first position and the first N diagonal singular values ​​as the main singular values. In this way, in addition to including parameters with large singular values, the important parameter matrix can also include parameters with small singular values ​​but actually high gradient sensitivity. Thus, when updating non-important parameters, it is possible to avoid these parameters with small singular values ​​but actually high gradient sensitivity, as well as parameters with large singular values, from participating in the model update, further avoiding model drift or overfitting.

[0063] In one embodiment, a distributed power node runs in a computing power platform with an accelerator card, and based on local power increment data of the distributed power node, a local first low-rank parameter matrix and a local second low-rank parameter matrix are updated based on the local power increment data, including:

[0064] Divide the local power incremental data into multiple batches of data, and perform the following local training on each batch: train the local power model based on the batch data to obtain a local loss; and adjust the efficiency factor based on the gradient information of the local loss and the hardware characteristics corresponding to the computing power platform to update the local first low-rank parameter matrix and the local second low-rank parameter matrix to obtain an updated local first low-rank parameter matrix and an updated local second low-rank parameter matrix;

[0065] If the number of local training times does not meet the preset requirement, updating the local power model based on the product of the updated local first low-rank parameter matrix and the updated local second low-rank parameter matrix, and the important parameter matrix, for the next batch of local training;

[0066] When the number of local training times reaches the preset requirement, the local first low-rank parameter differential matrix is ​​determined based on the difference between the updated local first low-rank parameter matrix and the local first low-rank parameter matrix before updating, and the local second low-rank parameter differential matrix is ​​determined based on the difference between the updated local second low-rank parameter matrix and the local second low-rank parameter matrix before updating.

[0067] Understandably, the computing power platform can use hardware such as Ascend 910B, Kunpeng 920 CPU or ARM chips for calculations, which can accelerate the model training process and improve training accuracy.

[0068] Exemplarily, based on the gradient information of the local loss and the efficiency factor adjusted according to the hardware characteristics of the computing power platform, the local first low-rank parameter matrix and the local second low-rank parameter matrix are updated to obtain the updated local first low-rank parameter matrix and the updated local second low-rank parameter matrix as follows:

[0069] ;

[0070] ;

[0071] in, is the local first low-rank parameter matrix, is the gradient information corresponding to the local first low-rank parameter matrix, is the local second low-rank parameter matrix, is the gradient information corresponding to the local second low-rank parameter matrix, Indicates the hardware feature adjustment efficiency factor, Represents the learning rate.

[0072] For example, the hardware feature adjustment efficiency factor can be dynamically changed based on conditions such as the accelerator card's temperature, load, and bandwidth. For example, if the feature adjustment efficiency factor is greater than or equal to 1 and the hardware is detected to be in a high temperature or overloaded state, the value of the hardware feature adjustment efficiency factor can be converged toward 1 to ensure numerical stability. Otherwise, the value of the hardware feature adjustment efficiency factor can be appropriately increased based on 1 to accelerate training.

[0073] Exemplarily, the product of the updated local first low-rank parameter matrix and the updated local second low-rank parameter matrix is ​​added to the important parameter matrix, and the resulting matrix is ​​the model parameter of the updated local power model of this batch, which can be used for the next batch of local training.

[0074] For example, the preset requirement may be that the number of local training times reaches the total number of batches of the batch data, or that the number of training times reaches a preset threshold.

[0075] According to the above implementation, by continuously updating the model, using the difference between the last two updated local first low-rank parameter matrices as the local first low-rank parameter differential matrix, and using the difference between the last two updated local second low-rank parameter matrices as the local second low-rank parameter differential matrix, a differential matrix with extremely small data volume can be obtained and transmitted to the global scheduling server, which can reduce the amount of data as much as possible and also minimize the amount of calculation of model parameters during global aggregation.

[0076] In one embodiment, the above-mentioned sending of the update results to the global scheduling server includes: determining the global synchronization period based on the network bandwidth and network delay between each distributed power node and the global scheduling server; when the time difference between the last time the parameter matrix was sent to the global scheduling server and the current time is the same as the global synchronization period, sending the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix to the global scheduling server.

[0077] Exemplarily, the sum of the inverse of the network bandwidth and the network delay is taken as the global synchronization period.

[0078] For example, the global synchronization period may be limited by using a lower limit value and an upper limit value of the synchronization period to prevent the model synchronization from being too frequent or being excessively delayed.

[0079] For example, when the network bandwidth is high and the latency is low, the sum of the reciprocal of the network bandwidth and the network latency is low, and the global synchronization period is closer to the lower limit of the synchronization period. Conversely, when the network bandwidth is low and the latency is high, i.e., when the network is poor, the global synchronization period becomes longer, but does not exceed the upper limit of the synchronization period. If it exceeds the upper limit, the global synchronization period is set as the upper limit of the synchronization period.

[0080] Understandably, the global synchronization period determines how often global model parameter aggregation is performed. If the current time is the global synchronization period from the last global model parameter aggregation, each node transmits its local first low-rank parameter difference matrix and local second low-rank parameter difference matrix to the global scheduling server. In this way, the global scheduling server can perform global aggregation on each node's local first low-rank parameter difference matrix and local second low-rank parameter difference matrix.

[0081] In one embodiment, a global synchronization period is determined based on the current network bandwidth and the current network delay between each distributed power node and the global dispatch server, including: based on the network bandwidth hyperparameter and the network delay hyperparameter, respectively taking the weighted sum of the inverse of the current network bandwidth and the current network delay to obtain an expected synchronization period; taking the maximum value between the lower limit value of the synchronization period and the expected synchronization period, and taking the minimum value between the maximum value and the upper limit value of the synchronization period to obtain the global synchronization period.

[0082] Exemplarily, the global synchronization period may be:

[0083] ;

[0084] in, Indicates the global synchronization period, Indicates the upper limit of the synchronization period. Indicates the lower limit of the synchronization period. represents the network bandwidth hyperparameter, Indicates the current network bandwidth. represents the network delay hyperparameter, Indicates the current network delay.

[0085] According to the above embodiment, the global synchronization period is determined according to the network bandwidth and network delay, which can avoid the model synchronization of each distributed power node being too frequent or too delayed. It can also avoid the incremental data collected by each node being too insufficient due to too frequent synchronization, resulting in insufficient accuracy of the model update in one period, or avoid the incremental data collected by each node being accumulated due to too delayed synchronization, causing the model update in the next period to be too slow.

[0086] In one embodiment, the process of updating the global first low-rank parameter matrix and the global second low-rank parameter matrix by the global scheduling server includes: the global scheduling server determines the aggregation weight of each distributed power node based on the load, temperature and power consumption of each distributed power node; the global scheduling server performs weighted aggregation on the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix provided by each distributed power node based on the aggregation weight of each distributed power node, and uses the weighted aggregation results to update the global first low-rank parameter matrix and the global second low-rank parameter matrix of the global scheduling server.

[0087] Exemplarily, the load, temperature, and power consumption of each distributed power node are respectively normalized and scored to obtain a load score, a temperature score, and a power consumption score of each distributed power node.

[0088] Exemplarily, a weighted sum is performed on the load score, temperature score, power consumption score, and error of the distributed power node, and the inverse of the weighted sum result is used as the aggregation weight of the distributed power node.

[0089] For example, The aggregate weight of distributed power nodes is:

[0090]

[0091] in, Indicates the The aggregate weight of distributed power nodes, 、 and Respectively represent The load score, temperature score and power consumption score of each distributed power node, 、 and Represent load weight, temperature weight and power consumption weight respectively, Indicates error.

[0092] Exemplarily, the weighted aggregation result is used to update the global first low-rank parameter matrix and the global second low-rank parameter matrix of the global scheduling server respectively, as follows:

[0093] ;

[0094] ;

[0095] in, and represents the global first low-rank parameter matrix and the global second low-rank parameter matrix, represents the total number of distributed power nodes, and Indicates the A first local low-rank parameter difference matrix and a local second low-rank parameter difference matrix of each distributed power node.

[0096] Figure 3 This is a structural block diagram of a distributed power model updating device based on model incremental training according to an embodiment of the present invention.

[0097] like Figure 3 As shown, a distributed power model updating device based on model incremental training includes:

[0098] a singular value decomposition module 310 configured to perform singular value decomposition on a local model parameter matrix of a local power model in the distributed power node when the amount of local power incremental data of the distributed power node is less than a preset threshold, to obtain a left singular matrix, a diagonal matrix, and a right singular matrix;

[0099] An important parameter determination module 320 is configured to extract corresponding column vectors from the left singular matrix and the right singular matrix based on the main singular values ​​in the diagonal matrix to obtain an important parameter matrix;

[0100] A low-rank parameter determination module 330 is configured to determine a non-important parameter matrix based on a difference matrix between the local model parameter matrix and the important parameter matrix, and decompose the non-important parameter matrix to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix;

[0101] A global model updating module 340 is configured to update the local first low-rank parameter matrix and the local second low-rank parameter matrix in the local model parameter matrix based on the local power increment data of the distributed power node, and send the updated result to the global scheduling server so that the global scheduling server updates the global first low-rank parameter matrix and the global second low-rank parameter matrix;

[0102] The local model updating module 350 is used to update the local power model of the distributed power node based on the product between the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server, and the important parameter matrix.

[0103] In one embodiment, the important parameter determination module includes:

[0104] a first singular value determining unit, configured to sort the diagonal singular values ​​in the diagonal matrix from largest to smallest, and intercept the first N diagonal singular values ​​as the main singular values, where N is a positive integer greater than 1;

[0105] The first matrix determination unit is configured to determine the important parameter matrix based on the singular vectors of the main singular values ​​in the left singular matrix and the singular vectors in the right singular matrix.

[0106] In one embodiment, the important parameter determination module array includes:

[0107] a position determining unit, configured to determine, based on a first power model parameter in the local power model having a gradient sensitivity greater than a preset threshold, a first position corresponding to the first power model parameter in the diagonal matrix;

[0108] a second singular value determining unit, configured to determine the main singular value based on the singular value at the first position;

[0109] The second matrix determination unit is configured to determine the important parameter matrix based on the singular vectors of the main singular values ​​in the left singular matrix and the singular vectors in the right singular matrix.

[0110] In one embodiment, the second singular value determining unit is specifically configured to:

[0111] Sort the diagonal singular values ​​in the diagonal matrix from largest to smallest, and determine the first N diagonal singular values, where N is a positive integer greater than 1;

[0112] The main singular value is obtained based on the first N diagonal singular values ​​and the singular value at the first position.

[0113] In one embodiment, the global model updating module includes:

[0114] a local training unit, configured to divide the local power incremental data into a plurality of batches of data, and perform the following local training on a batch-by-batch basis: training the local power model based on the batch data to obtain a local loss; and adjusting an efficiency factor based on gradient information of the local loss and hardware characteristics corresponding to the computing power platform, updating the local first low-rank parameter matrix and the local second low-rank parameter matrix to obtain updated local first low-rank parameter matrix and updated local second low-rank parameter matrix;

[0115] A local model updating unit, configured to update the local power model for a next batch of local training based on the product of the updated local first low-rank parameter matrix and the updated local second low-rank parameter matrix, and the important parameter matrix, if the number of local training times does not meet a preset requirement;

[0116] A differential calculation unit is used to determine the local first low-rank parameter differential matrix based on the difference between the updated local first low-rank parameter matrix and the local first low-rank parameter matrix before updating, and to determine the local second low-rank parameter differential matrix based on the difference between the updated local second low-rank parameter matrix and the local second low-rank parameter matrix before updating, when the number of local training times reaches the preset requirement.

[0117] In one embodiment, the global model updating module includes:

[0118] a synchronization period determining unit, configured to determine a global synchronization period based on a network bandwidth and a network delay between each of the distributed power nodes and the global dispatch server;

[0119] A differential result sending unit is used to send the local first low-rank parameter differential matrix and the local second low-rank parameter differential matrix to the global scheduling server when the time difference between the last time the parameter matrix was sent to the global scheduling server and the current time is the same as the global synchronization period.

[0120] In one embodiment, the synchronization period determination unit is specifically configured to:

[0121] Based on a network bandwidth hyperparameter and a network delay hyperparameter, performing weighted summation on the inverse of the current network bandwidth and the current network delay, respectively, to obtain an estimated synchronization period;

[0122] The global synchronization period is obtained by taking the maximum value between the lower limit value of the synchronization period and the estimated synchronization period, and taking the minimum value between the maximum value and the upper limit value of the synchronization period.

[0123] In one embodiment, the process of the global scheduling server updating the global first low-rank parameter matrix and the global second low-rank parameter matrix includes:

[0124] The global scheduling server determines the aggregation weight of each of the distributed power nodes based on the load, temperature and power consumption of each of the distributed power nodes;

[0125] The global scheduling server performs weighted aggregation on the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix provided by each distributed power node based on the aggregation weight of each distributed power node, and uses the weighted aggregation results to update the global first low-rank parameter matrix and the global second low-rank parameter matrix of the global scheduling server respectively.

[0126] For the description of specific functions and examples of each module and submodule of the system in the embodiment of the present invention, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0127] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0128] According to an embodiment of the present invention, the present invention further provides a system and a readable storage medium.

[0129] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0130] like Figure 4As shown, electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of electronic device 800. Computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to bus 804.

[0131] Multiple components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0132] The computing unit 801 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the distributed power model update method based on incremental model training. For example, in some embodiments, the distributed power model update method based on incremental model training can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the distributed power model update method based on incremental model training described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other appropriate manner (eg, by means of firmware) to execute the distributed power model updating method based on model incremental training.

[0133] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0138] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0139] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved. This is not limited herein.

[0140] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A distributed power model updating method based on model incremental training, characterized in that: include: When the amount of local power incremental data of the distributed power node is less than a preset threshold, performing singular value decomposition on a local model parameter matrix of the local power model in the distributed power node to obtain a left singular matrix, a diagonal matrix, and a right singular matrix; Based on the main singular values ​​in the diagonal matrix, corresponding column vectors are extracted from the left singular matrix and the right singular matrix respectively to obtain an important parameter matrix, wherein the main singular values ​​include: the singular values ​​in the diagonal matrix ranked from large to small, where N is a positive integer greater than 1, and / or the singular values ​​at the first position in the diagonal matrix corresponding to the model parameters in the local power model whose gradient sensitivity is greater than a preset threshold; Determine a non-important parameter matrix based on a difference matrix between the local model parameter matrix and the important parameter matrix, and perform low-rank decomposition on the non-important parameter matrix to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix; Based on the local power increment data, the local first low-rank parameter matrix and the local second low-rank parameter matrix are updated, and the update results are sent to the global scheduling server; wherein the update results include a local first low-rank parameter difference matrix determined based on the difference between the local first low-rank parameter matrix before and after the update, and a local second low-rank parameter difference matrix determined based on the difference between the local second low-rank parameter matrix before and after the update; the global scheduling server is used to globally aggregate the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix from each of the distributed power nodes, and update the global first low-rank parameter matrix and the global second low-rank parameter difference matrix respectively based on the global aggregation; The product of the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server is summed with the important parameter matrix to obtain an updated local model parameter matrix of the local power model of the distributed power node.

2. The method according to claim 1, characterized in that The method extracts corresponding column vectors from the left singular matrix and the right singular matrix based on the main singular values ​​in the diagonal matrix to obtain an important parameter matrix, including: The important parameter matrix is ​​determined based on the singular vectors of the respective main singular values ​​in the left singular matrix and the singular vectors in the right singular matrix.

3. The method according to claim 1, characterized in that The distributed power node runs in a computing power platform having an accelerator card, and the updating of the local first low-rank parameter matrix and the local second low-rank parameter matrix based on the local power increment data includes: Dividing the local power incremental data into a plurality of batches of data, and performing the following local training on each batch: training the local power model based on the batch data to obtain a local loss; and adjusting the efficiency factor based on gradient information of the local loss and hardware characteristics corresponding to the computing power platform, updating the local first low-rank parameter matrix and the local second low-rank parameter matrix to obtain updated local first low-rank parameter matrix and updated local second low-rank parameter matrix; If the number of local training times does not meet the preset requirement, updating the local power model based on the product of the updated local first low-rank parameter matrix and the updated local second low-rank parameter matrix, and the important parameter matrix, for the next batch of local training; When the number of local training times reaches the preset requirement, the local first low-rank parameter difference matrix is ​​determined based on the difference between the updated local first low-rank parameter matrix and the local first low-rank parameter matrix before updating, and the local second low-rank parameter difference matrix is ​​determined based on the difference between the updated local second low-rank parameter matrix and the local second low-rank parameter matrix before updating.

4. The method according to claim 3, characterized in that The step of sending the update result to the global scheduling server includes: determining a global synchronization period based on the network bandwidth and network delay between each of the distributed power nodes and the global dispatch server; When the time difference between the last time the parameter matrix was sent to the global scheduling server and the current time is the same as the global synchronization period, the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix are sent to the global scheduling server.

5. The method according to claim 4, characterized in that The determining of the global synchronization period based on the current network bandwidth and the current network delay between each of the distributed power nodes and the global dispatch server includes: Based on a network bandwidth hyperparameter and a network delay hyperparameter, performing weighted summation on the inverse of the current network bandwidth and the current network delay, respectively, to obtain an estimated synchronization period; The global synchronization period is obtained by taking the maximum value between the synchronization period lower limit value and the estimated synchronization period and taking the minimum value between the maximum value and the synchronization period upper limit value; The process of the global scheduling server updating the global first low-rank parameter matrix and the global second low-rank parameter matrix includes: The global scheduling server determines the aggregation weight of each of the distributed power nodes based on the load, temperature and power consumption of each of the distributed power nodes; The global scheduling server performs weighted aggregation on the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix provided by each distributed power node based on the aggregation weight of each distributed power node, and uses the weighted aggregation results to update the global first low-rank parameter matrix and the global second low-rank parameter matrix of the global scheduling server respectively.

6. A distributed power model updating device based on model incremental training, characterized in that: include: a singular value decomposition module, configured to perform singular value decomposition on a local model parameter matrix of a local power model in a distributed power node when the amount of local power incremental data of the distributed power node is less than a preset threshold, to obtain a left singular matrix, a diagonal matrix, and a right singular matrix; an important parameter determination module, configured to extract corresponding column vectors from the left singular matrix and the right singular matrix based on the main singular values ​​in the diagonal matrix, respectively, to obtain an important parameter matrix, wherein the main singular values ​​include: the singular values ​​in the diagonal matrix ranked from largest to smallest, where N is a positive integer greater than 1, and / or the singular values ​​at the first position in the diagonal matrix corresponding to model parameters in the local power model having a gradient sensitivity greater than a preset threshold; A low-rank parameter determination module is used to determine a non-important parameter matrix based on a difference matrix between the local model parameter matrix and the important parameter matrix, and decompose the non-important parameter matrix to obtain a local first low-rank parameter matrix and a local second low-rank parameter matrix; A global model update module is used to update the local first low-rank parameter matrix and the local second low-rank parameter matrix in the local model parameter matrix based on the local power increment data of the distributed power node, and send the update result to the global scheduling server; wherein the update result includes a local first low-rank parameter difference matrix determined based on the difference between the local first low-rank parameter matrix before and after the update, and a local second low-rank parameter difference matrix determined based on the difference between the local second low-rank parameter matrix before and after the update; the global scheduling server is used to globally aggregate the local first low-rank parameter difference matrix and the local second low-rank parameter difference matrix from each of the distributed power nodes, and update the global first low-rank parameter matrix and the global second low-rank parameter difference matrix respectively based on the global first low-rank parameter difference matrix and the global second low-rank parameter difference matrix obtained by global aggregation; A local model updating module is used to sum the product of the updated global first low-rank parameter matrix and the updated global second low-rank parameter matrix from the global scheduling server with the important parameter matrix to obtain an updated local model parameter matrix of the local power model of the distributed power node.

7. An electronic device, characterized in that: include: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the processor, and the processor is used to obtain the instructions from the memory and execute the instructions, so that the processor can execute the distributed power model updating method based on model incremental training according to any one of claims 1-5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to be provided to a computer to instruct the computer to execute the distributed power model updating method based on model incremental training according to any one of claims 1 to 5.

Images