Model training method and device, computer equipment and readable storage medium

Abstract

The invention relates to a model training method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of receiving multiple groups of first model parameters fed back by multiple clients, determining the first model parameters as sample points to obtain a sample data set, determining distance inverse number density of the sample points for each sample point based on the distance between the sample point and adjacent sample points of the sample point, and determining the distance inverse number density of the sample points based on the distance inverse number density. The method comprises the following steps: determining a local density correction term of a sample point, clustering a sample data set based on the distance inverse number density and the local density correction term, determining the local density of the sample point, and based on the local density of each sample point, dividing clients with similar data distribution into the same client set to obtain a plurality of client sets, and scheduling the plurality of clients. Clients in the same client set share model parameters, and the model is jointly trained in a federal learning mode. By adopting the method, the model training precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of artificial intelligence technology, data security and privacy are becoming more and more important in the model training process. Researchers have integrated data from different sources to build a distributed model training framework, but the diverse data sources will generate huge communication costs and affect the training efficiency of the model. For example, in the process of dispatching the distribution network, data processing models are usually used to process related power grid business data. In the process of training the data processing model, a large amount of power grid business data is required. However, the power grid business data is usually distributed among multiple nodes of the distribution network. The types and sources of power grid business data between different nodes are very different. Even if a distributed model training framework is built, there are still huge communication costs between multiple nodes of the distribution network and the central server, which affects the training efficiency of the classification model.

[0003] In the traditional solution, through the federated learning framework, multiple nodes of the distribution network perform preliminary training on the data processing model locally, and then the central server integrates the training results of each node and trains again to obtain the global model. In this way, without exchanging data samples, it is possible to train a global model based on distributed data across multiple nodes, which improves the training speed of the data processing model.

[0004] However, in traditional solutions, since there are many clients or nodes providing training data in the federated learning framework, the training data between different clients or nodes is very different, which leads to large differences in model parameters when clients or nodes participate in federated learning to train local models, and there is still a technical problem of low model training accuracy. Summary of the invention

[0005] Based on this, it is necessary to provide a model training method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of model training in response to the above technical problems.

[0006] In a first aspect, the present application provides a model training method, which is applied to a central server of a federated learning framework, wherein the federated learning framework includes the central server and a plurality of clients connected to the central server, and the method includes:

[0007] Receiving multiple groups of first model parameters fed back by multiple clients, determining the first model parameters as sample points, and obtaining a sample data set, wherein the first model parameters are obtained by the clients training a to-be-trained model based on a local business training data set;

[0008] For each of the sample points, based on the distance between the sample point and its adjacent sample points, determine the distance inverse density of the sample point, based on the distance inverse density, determine the local density correction term of the sample point, based on the distance inverse density and the local density correction term, determine the local density of the sample point, the adjacent sample points are sample points within a preset distance range of the sample point;

[0009] Based on the local density of each of the sample points, clustering the sample data set, dividing clients with similar data distribution into the same client set, and obtaining multiple client sets;

[0010] Schedule multiple clients so that clients in the same client set share model parameters and jointly train the model through federated learning.

[0011] In one embodiment, scheduling multiple clients includes:

[0012] Send model training messages to multiple clients so that clients in the same client set can share model parameters and jointly train the model through federated learning;

[0013] receiving a second model parameter fed back by each client, and determining a global model parameter based on a plurality of the second model parameters;

[0014] Send a model training message carrying the global model parameters to clients in different client sets, so that clients in the same client set jointly train the model based on the global model parameters through federated learning, and return to the step of receiving the second model parameters fed back by clients in different client sets until a preset training end condition is reached and the model training is stopped.

[0015] In one embodiment, determining the local density correction term of the sample point based on the inverse distance density includes:

[0016] The local density correction term is determined based on the magnitude relationship between the inverse distance density of the sample point and the inverse distance density of the adjacent sample points.

[0017] In one embodiment, the sample data set is clustered based on the local density of each sample point, and clients with similar data distribution are divided into the same client set to obtain multiple client sets, including:

[0018] Determining the high local density point distance of each of the sample points according to the local density of each of the sample points;

[0019] Determining a decision value of each of the sample points based on the local density and the distance between the high local density points;

[0020] Based on the second-order difference of the decision value of each of the sample points, a cluster center is selected from the sample data set, and sample points that are not cluster centers are assigned to the cluster centers to obtain a plurality of sample clusters;

[0021] Based on the multiple sample clusters, clients with similar data distribution are divided into the same client set to obtain multiple client sets.

[0022] In one embodiment, the selecting a cluster center from the sample data set based on the second-order difference of the decision value of each sample point includes:

[0023] Determining, according to the second-order differences of the decision values ​​of the sample points, target sample points where the second-order differences have a sudden change;

[0024] Based on the target sample point, a cluster center is screened out from the sample data set, and the second-order difference of the decision value of the cluster center is greater than the second-order difference of the decision value of the target sample point.

[0025] In a second aspect, the present application provides a method for processing power grid service data, the method comprising:

[0026] Obtain power grid business data;

[0027] Calling the trained data processing model to process the power grid service data to obtain a data processing result;

[0028] Wherein, the data processing model is based on historical power grid business data and is trained by the model training method of any one of claims 1 to 5.

[0029] In a third aspect, the present application also provides a model training device, comprising:

[0030] A data receiving module, used to receive multiple groups of first model parameters fed back by multiple clients, determine the first model parameters as sample points, and obtain a sample data set, wherein the first model parameters are obtained by the clients training a to-be-trained model based on a local business training data set;

[0031] A local density determination module is used to determine, for each of the sample points, the distance inverse density of the sample point based on the distance between the sample point and its adjacent sample points, determine the local density correction term of the sample point based on the distance inverse density, and determine the local density of the sample point based on the distance inverse density and the local density correction term, wherein the adjacent sample points are sample points within a preset distance range of the sample point;

[0032] A clustering module, used to cluster the sample data set based on the local density of each of the sample points, and divide clients with similar data distribution into the same client set to obtain multiple client sets;

[0033] The model training module is used to schedule multiple clients so that clients in the same client set share model parameters and jointly train the model through federated learning.

[0034] In a fourth aspect, the present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned model training method embodiment when executing the computer program.

[0035] In a fifth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned model training method embodiment are implemented.

[0036] In a sixth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps in the above-mentioned model training method embodiment.

[0037] The above-mentioned model training method, apparatus, computer equipment, computer-readable storage medium and computer program product are different from traditional schemes. The above-mentioned scheme clusters the sample data set through the first model parameter obtained by the client based on local training data training, and then divides the clients with similar data distribution into the same client set. Under the premise of protecting the data privacy of each client, the clients with similar local training data distribution are divided into one category, thereby allowing clients with similar data distribution to share model training parameters, jointly train and optimize models, so that the federated learning process can better cope with and process training data with diverse data characteristics, and improve the training accuracy of the model; and in the process of clustering the sample data set based on the local density of the sample points, the influence of adjacent sample points on the local density of the sample points is also processed through the local density correction term, thereby increasing the possibility of sample points in sparse areas being identified as cluster centers, obtaining a more accurate clustering process, and further improving the model training accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 An application environment diagram of a model training method in an embodiment;

[0040] Figure 2 A schematic diagram of a flow chart of a model training method in one embodiment;

[0041] Figure 3 A flowchart of the steps of scheduling a client in one embodiment;

[0042] Figure 4 A schematic diagram of a process flow of client clustering steps in one embodiment;

[0043] Figure 5 A schematic diagram of a flow chart of a method for processing power grid business data in one embodiment;

[0044] Figure 6 is a structural block diagram of a model training device in one embodiment;

[0045] Figure 7 is a structural block diagram of a power grid service data processing device in one embodiment;

[0046] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The model training method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the client 102 communicates with the central server 104 through the network. The data storage system can store data that the central server 104 needs to process. The data storage system can be integrated on the central server 104, or placed on the cloud or other network servers.

[0049] Specifically, the central server 104 can first send an initialized model to be trained to each client 102, and schedule each client 102 to train the model to be trained. Each client 102 trains the model to be trained based on the local business training data set, obtains and feeds back multiple sets of first model parameters to the central server 104, and then, the central server 104 determines the first model parameter as a sample point to obtain a sample data set, and determines the inverse distance density of the sample point based on the distance between the sample point and the adjacent sample point of the sample point, determines the local density correction term of the sample point based on the inverse distance density, and determines the local density of the sample point based on the inverse distance density and the local density correction term. Finally, the central server 104 clusters the sample data set based on the local density of each sample point, divides the clients 102 with similar data distribution into the same client set, obtains multiple client sets, and schedules multiple clients 102 so that the clients 102 in the same client set share model parameters and jointly train the model through federated learning.

[0050] The client 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc. The central server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0051] In an exemplary embodiment, Figure 2 As shown, a model training method is provided, which is applied to Figure 1 Taking the central server 104 in the example as an example, the following steps are included:

[0052] S100, receiving multiple groups of first model parameters fed back by multiple clients, determining the first model parameters as sample points, and obtaining a sample data set.

[0053] Among them, each client has pre-stored local business training data. For the sake of data security and privacy, the local business training data between the clients are not shared. Before receiving the first model parameters, the central server first sends the model to be trained to each client, and instructs each client to train the model to be trained based on the local business training data set to obtain the first model parameters, and then each client feeds back the first model parameters obtained to the central server.

[0054] Furthermore, the central server uses the first model parameter as a sample point to obtain a sample data set to provide a data basis for the subsequent clustering process. Since the local business training data of each client is different, the first model parameters obtained through training may not be the same. However, the first model parameter can reflect the type of local business training data and data distribution information of the client to a certain extent. Therefore, it can be used as a basis for clustering in this solution to divide clients with similar data distribution into the same client set.

[0055] S200, for each sample point, based on the distance between the sample point and its adjacent sample points, determine the inverse distance density of the sample point, based on the inverse distance density, determine the local density correction term of the sample point, and based on the inverse distance density and the local density correction term, determine the local density of the sample point.

[0056] Among them, adjacent sample points refer to sample points within a preset distance range of the sample point, and the preset distance may be a cutoff distance. Local density is an indicator that reflects the density of sample points in its neighborhood. The larger the local density, the more sample points there are near the sample point, and vice versa.

[0057] The inverse distance density refers to the inverse of the mean distance between a sample point and its adjacent sample points. The mean distance can be the Euclidean distance. For example, the calculation formula for the mean distance of sample point i is shown in formula (1):

[0058] (1)

[0059] In formula (1), n ​​represents the number of adjacent sample points of sample point i. represents the Euclidean distance between sample point i and its adjacent sample point j, and NN represents the set of all adjacent sample points of sample point i. Furthermore, the inverse distance density of sample point i is the inverse of its distance mean, which can be expressed as:

[0060] (2)

[0061] Furthermore, considering that the density distribution of sample points in the sample data set may not be uniform, the sample points in the sparse area are relatively large, resulting in a lower local density of sample points in the sparse area, which means that in the subsequent process of finding cluster centers, the sample points in the sparse area may be easily ignored, resulting in the loss of some clusters and affecting the overall clustering effect. Therefore, in order to better adapt to the sample data set with uneven sample point density distribution, this embodiment adds a correction to the local density of the sample points.

[0062] Specifically, if the local density correction term is not considered, the local density of the sample point is equal to its inverse distance density. The larger the distance mean, the smaller the inverse distance density. Therefore, the inverse distance density of the sample points in the sparse area is small. In order to improve the inverse distance density of the sample points in the sparse area and increase the possibility of being selected as the cluster center, the local density correction term can be a positive number, and the sum of the inverse distance density and the local density correction term is taken as the corrected local density.

[0063] S300, clustering the sample data set based on the local density of each sample point, and dividing clients with similar data distribution into the same client set to obtain multiple client sets.

[0064] Following the above steps, the sample data set is clustered based on the local density of each sample point. The clustering methods include but are not limited to K-means clustering, hierarchical clustering and other clustering methods. For example, the central server will first identify the area with higher local density based on the local density of each sample point, and divide the sample points in the area into the same category. In this way, by analyzing the local density of the sample points, the central server can quickly divide the sample points with similar data distribution into the same category to obtain multiple sample clusters. Furthermore, since the sample point is the first model parameter, the first model parameter can reflect the data distribution of the local business training data of each client. Therefore, based on the multiple sample clusters obtained by clustering, clients with similar data distribution can be divided into the same client set to obtain multiple client sets.

[0065] S400, scheduling multiple clients so that clients in the same client set share model parameters and jointly train the model through federated learning.

[0066] Finally, the central server schedules each client to perform iterative training on the model to be trained, where clients in the same client set train the model together and share model parameters during the training process. In this way, clients in the same client set can indirectly share knowledge without sharing local business training data, learn from each other during the model training process, and perform more effective model training based on similar data characteristics. Moreover, due to the similarity of data distribution, the model can converge faster.

[0067] Exemplarily, for each client, the client can first divide the local business training data to obtain multiple groups of business training data. During the iterative training process, each round of training uses a group of business training data to perform stochastic gradient descent method to train the model, rather than using the entire data set at one time, so that the model parameters can be gradually updated.

[0068] In this embodiment, different from the traditional solution, the above-mentioned solution clusters the sample data set through the first model parameter obtained by the client based on the local training data training, and then divides the clients with similar data distribution into the same client set. Under the premise of protecting the data privacy of each client, the clients with similar local training data distribution are divided into one category, so that the clients with similar data distribution can share model training parameters, jointly train and optimize the model, so that the federated learning process can better cope with and process the training data with diverse data characteristics, and improve the training accuracy of the model; and in the process of clustering the sample data set based on the local density of the sample points, the influence of the local density of the sample points on the local density of the sample points is also processed through the local density correction term, so as to improve the possibility of the sample points in the sparse area being identified as the cluster center, and obtain a more accurate clustering process, thereby further improving the model training accuracy. .

[0069] In an exemplary embodiment, Figure 3 As shown, S400 includes:

[0070] S410, sending a model training message to multiple clients so that clients in the same client set share model parameters and jointly train the model through federated learning.

[0071] S420, receiving the second model parameters fed back by each client, and determining a global model parameter based on the multiple second model parameters.

[0072] S430, sending a model training message carrying global model parameters to clients in different client sets, so that clients in the same client set can jointly train the model based on the global model parameters through federated learning.

[0073] Following the above embodiment, the central server coordinates the clients in the same client set to jointly perform model training. Specifically, the central server may send a model training message to the clients in each client set to instruct each client to perform iterative training on the training model based on the local business training data, and the clients in the same client set may share model parameters during the iterative training process. Specifically, the clients in each client set may each perform iterative training based on the local business training data, and then each client set feeds back the second model parameters obtained by the training to the central server. The central server determines the global model parameters based on the second model parameters fed back by the clients in the same client set. In this way, the central server only needs to aggregate the model parameters of the clients in the same client set, rather than the model parameters of all clients, to reduce the negative migration phenomenon caused by the data differences of different client sets, so as to better utilize the similarity of data distribution to improve the performance of the model. In addition, in the next round of iteration, the clients in another client set may be selected to jointly train the model through federated learning until the preset training end condition is reached and the model training is stopped. The training end condition may be reaching the preset number of training iterations or reaching the preset model training accuracy. Among them, after each client receives the global model parameters, it uses the global model parameters as the initial state, and further trains and optimizes the model based on the local business training data. It should be noted that in each round of model training process mentioned above, not all clients in the same client set need to participate in model training, but a preset number of clients can be randomly selected from the same client set, and the selected clients jointly train the model to be trained, and upload the second model parameters obtained through training to the central server.

[0074] In this embodiment, after clients with similar data distribution are divided into the same client set, the clients in the same client set jointly train the model, and the model parameters are shared during the training process to collaboratively optimize the model, thereby better utilizing the data distribution similarity of the same client set to improve model performance, and reducing the negative impact of data distribution differences of different client sets on global model parameters, thereby reducing the negative migration phenomenon.

[0075] In an exemplary embodiment, determining a local density correction term of a sample point based on the inverse distance density includes: determining the local density correction term based on a magnitude relationship between the inverse distance density of the sample point and the inverse distance density of adjacent sample points.

[0076] Following the above embodiment, the local density of the sample point can be corrected based on the local density correction term. The corrected local density expression is shown in formula (3):

[0077] (3)

[0078] In formula (3), represents the inverse distance density of sample point i, Represents the local density correction term, which characterizes the relationship between the inverse distance density of the sample point and the inverse distance density of the adjacent sample points. For sample point i, when the inverse distance density of its adjacent sample points is less than that of sample point i, the local density correction term is positive, when the inverse distance density of its adjacent sample points is greater than that of sample point i, the local density correction term is negative, and when the inverse distance density of its adjacent sample points is less than that of sample point i, the local density correction term is zero.

[0079] In this embodiment, the local density correction term is determined based on the relationship between the inverse distance density of the sample point and the inverse distance density of the adjacent sample points, so that the local density of the potential cluster center point in the sparse area will be adaptively increased, and the possibility of it being identified as the cluster center will also increase accordingly, thereby improving the accuracy of the clustering results.

[0080] In an exemplary embodiment, Figure 4 As shown, S300 includes:

[0081] S310, determining the high local density point distance of each sample point according to the local density of each sample point.

[0082] S320, determining a decision value of each sample point based on the local density and the distance between the points with high local density.

[0083] S330 , based on the second-order difference of the decision value of each sample point, a cluster center is selected from the sample data set, and sample points that are not cluster centers are assigned to the cluster centers, so as to obtain a plurality of sample clusters.

[0084] S340, based on multiple sample clusters, dividing clients with similar data distribution into the same client set to obtain multiple client sets.

[0085] The high local density point distance refers to the closest distance to the sample point i among all the sample points with higher local density than the sample point i, which can be expressed by formula (4):

[0086] (4)

[0087] When the local density of sample point i is the largest among all sample points, the distance of high local density points is expressed by formula (5):

[0088] (5)

[0089] The decision value can be obtained by multiplying the local density and the distance to the high local density point, as shown in formula (6):

[0090] (6)

[0091] The decision value can be used to characterize the possibility of a sample point being selected as a cluster center. The sample point with a larger decision value is more likely to be in the core position of the data distribution center, and thus is more likely to be selected as a cluster center. Specifically, the decision values ​​of all sample points can be sorted from high to low, and the second-order difference of the sorted decision values ​​is calculated. The cluster center is selected from the sample data set based on the calculation result of the second-order difference. The calculation formula is shown in equations (7) and (8):

[0092] (7)

[0093] (8)

[0094] in, represents the second-order difference of the decision value, Represents the first-order difference of the decision value.

[0095] After the cluster centers are screened out, the sample points that are not cluster centers are assigned to the smallest cluster centers to form multiple sample clusters. Finally, based on multiple sample clusters, clients with similar data distribution are divided into the same client set to obtain multiple client sets.

[0096] In this embodiment, unlike the traditional solution of drawing a decision diagram and manually selecting cluster centers from the decision diagram, when two sample points are too close or almost overlapping, the cluster center may be missed, affecting the clustering accuracy. The solution in this embodiment uses the local density of sample points, the distance between high local density points, and the decision value to screen out the cluster centers, which can effectively reduce the omission of cluster centers and improve the accuracy of the clustering results.

[0097] In an exemplary embodiment, S330 includes: determining a target sample point where the second-order difference has a sudden change according to the second-order difference of the decision value of each sample point, and selecting a cluster center from the sample data set based on the target sample point.

[0098] Following the above steps, since there are significant changes in the decision values ​​of cluster centers and non-cluster centers, this key feature can be used to automatically identify and determine the location of mutation points, and then quickly determine the cluster centers. Specifically, sample points with larger decision values ​​have obvious fluctuations and are more likely to be cluster centers, while sample points with relatively small decision values ​​are relatively stable. In order to present this fluctuation characteristic more clearly, the second-order difference of the decision value is used to quantify the amplitude of the fluctuation, thereby more clearly highlighting the difference between the center point and the non-center point.

[0099] Specifically, define M as the target sample point where the second-order difference of the decision value changes suddenly, and all sample points with decision values ​​greater than M are considered to be cluster centers. The expression of the target sample point M is shown in formula (9):

[0100] (9)

[0101] In this embodiment, the possibility of a sample point becoming a cluster center is quantified by the second-order difference of the decision value, which can effectively reduce the phenomenon of cluster centers being missed due to manual selection of cluster centers from the decision graph, improve the accuracy of clustering, and further improve the training accuracy of the model in the federated learning process.

[0102] In order to make a clearer description of the model training method provided in this application, a specific embodiment is described below, and the specific embodiment includes the following steps:

[0103] S501, receiving multiple groups of first model parameters fed back by multiple clients, determining the first model parameters as sample points, and obtaining a sample data set.

[0104] S502, for each sample point, based on the distance between the sample point and its adjacent sample points, determine the inverse distance density of the sample point, based on the relationship between the inverse distance density of the sample point and the inverse distance density of the adjacent sample points, determine the local density of the sample point based on the inverse distance density and the local density correction term.

[0105] S503, determining the high local density point distance of each sample point according to the local density of each sample point, and determining the decision value of each sample point based on the local density and the high local density point distance.

[0106] S504, determining a target sample point where the second-order difference has a sudden change according to the second-order difference of the decision value of each sample point, and selecting a cluster center from the sample data set based on the target sample point.

[0107] S505 , based on multiple sample clusters, dividing clients with similar data distribution into the same client set to obtain multiple client sets.

[0108] S506, sending a model training message to multiple clients, so that clients in the same client set share model parameters and jointly train the model through federated learning.

[0109] S507, receiving the second model parameters fed back by each client, and determining the global model parameters based on the multiple second model parameters.

[0110] S508, sending a model training message carrying global model parameters to clients in different client sets, so that clients in the same client set can jointly train the model based on the global model parameters through federated learning.

[0111] In one embodiment, Figure 5 As shown, a method for processing power grid business data is provided, comprising the following steps:

[0112] S601, obtaining power grid service data.

[0113] S602, calling the trained data processing model to process the power grid business data to obtain the data processing result.

[0114] The data processing model is based on historical power grid business data and is trained by the model training method in any of the above model training method embodiments. The power grid business data includes but is not limited to power consumption, power generation, equipment operation data, environmental data, etc. The data processing model can be a data classification model, a power generation prediction model, etc.

[0115] For example, in the distribution network, it is necessary to use a trained data processing model to process the power grid business data. However, when training the data processing model, since the training data is distributed in each node of the distribution network, the data processing model can be trained using a model training method such as any of the above-mentioned model training method embodiments, and the specific training process will not be described in detail.

[0116] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0117] Based on the same inventive concept, the embodiment of the present application also provides a model training device for implementing the model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model training device embodiments provided below can refer to the limitations on the model training method above, and will not be repeated here.

[0118] In an exemplary embodiment, Figure 6 As shown, a model training device 700 is provided, comprising: a data receiving module 710, a local density determination module 720, a clustering module 730 and a model training module 740, wherein:

[0119] The data receiving module 710 is used to receive multiple groups of first model parameters fed back by multiple clients, determine the first model parameters as sample points, and obtain a sample data set, wherein the first model parameters are obtained by training a to-be-trained model by the client based on a local business training data set;

[0120] A local density determination module 720 is used to determine, for each sample point, the distance inverse density of the sample point based on the distance between the sample point and its adjacent sample points, determine a local density correction term for the sample point based on the distance inverse density, and determine the local density of the sample point based on the distance inverse density and the local density correction term, wherein the adjacent sample points are sample points within a preset distance range of the sample point;

[0121] A clustering module 730 is used to cluster the sample data set based on the local density of each sample point, and divide the clients with similar data distribution into the same client set to obtain multiple client sets;

[0122] The model training module 740 is used to schedule multiple clients so that clients in the same client set share model parameters and jointly train the model through federated learning.

[0123] In an exemplary embodiment, the model training module 740 is also used to send model training messages to multiple clients so that clients in the same client set share model parameters, jointly train the model through federated learning, receive second model parameters fed back by clients in different client sets, determine global model parameters based on multiple second model parameters, send model training messages carrying global model parameters to clients in different client sets so that clients in the same client set jointly train the model based on the global model parameters through federated learning, receive second model parameters fed back by clients in different client sets again, until a preset training end condition is reached, and stop model training.

[0124] In an exemplary embodiment, the local density determination module 720 is further configured to determine a local density correction term based on the magnitude relationship between the inverse distance density of the sample point and the inverse distance density of adjacent sample points.

[0125] In an exemplary embodiment, the clustering module 730 is also used to determine the distance of each sample point to a high local density point based on the local density of each sample point, determine the decision value of each sample point based on the local density and the distance to the high local density point, screen out cluster centers from the sample data set based on the second-order difference of the decision values ​​of each sample point, and assign non-cluster center sample points to the cluster centers to obtain multiple sample clustering clusters; based on the multiple sample clustering clusters, divide clients with similar data distribution into the same client set to obtain multiple client sets.

[0126] In an exemplary embodiment, the clustering module 730 is also used to determine the target sample point where the second-order difference has a sudden change based on the second-order difference of the decision value of each sample point, and based on the target sample point, screen out the cluster center from the sample data set, and the second-order difference of the decision value of the cluster center is greater than the second-order difference of the decision value of the target sample point.

[0127] Based on the same inventive concept, the embodiment of the present application also provides a model training device for implementing the above-mentioned power grid business data processing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more power grid business data processing device embodiments provided below can refer to the above-mentioned limitations on the power grid business data processing method, and will not be repeated here.

[0128] In an exemplary embodiment, Figure 7 As shown, a power grid service data processing device 800 is provided, including: a data acquisition module 810 and a data processing module 820, wherein:

[0129] The data acquisition module 810 is used to acquire power grid service data.

[0130] The data processing module 820 is used to call the trained data processing model to process the power grid business data and obtain the data processing results.

[0131] Each module in the above-mentioned model training device and power grid business data processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0132] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as the local density of sample points. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model training method is implemented. In addition, the database of the computer device can also be used to store data such as power grid business data, and when the computer program is executed by the processor, a power grid business data processing method is implemented.

[0133] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned model training method embodiment when executing the computer program.

[0135] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned model training method embodiment are implemented.

[0136] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned model training method embodiment when executed by a processor.

[0137] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the above-mentioned embodiment of the method for processing power grid service data are implemented.

[0138] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned power grid service data processing method embodiment are implemented.

[0139] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned power grid service data processing method embodiment when executed by a processor.

[0140] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0142] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0143] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model training method, characterized in that: A central server applied to a federated learning framework, wherein the federated learning framework includes the central server and a plurality of clients connected to the central server, and the method includes: Receiving multiple groups of first model parameters fed back by multiple clients, determining the first model parameters as sample points, and obtaining a sample data set, wherein the first model parameters are obtained by the clients training a to-be-trained model based on a local business training data set; For each of the sample points, based on the distance between the sample point and its adjacent sample points, determine the distance inverse density of the sample point, based on the distance inverse density, determine the local density correction term of the sample point, based on the distance inverse density and the local density correction term, determine the local density of the sample point, the adjacent sample points are sample points within a preset distance range of the sample point; Based on the local density of each of the sample points, clustering the sample data set, dividing clients with similar data distribution into the same client set, and obtaining multiple client sets; Schedule multiple clients so that clients in the same client set share model parameters and jointly train the model through federated learning.

2. The method according to claim 1, characterized in that Scheduling multiple clients, including: Send model training messages to multiple clients so that clients in the same client set can share model parameters and jointly train the model through federated learning; receiving a second model parameter fed back by each client, and determining a global model parameter based on a plurality of the second model parameters; Send a model training message carrying the global model parameters to clients in different client sets, so that clients in the same client set jointly train the model based on the global model parameters through federated learning, and return to the step of receiving the second model parameters fed back by clients in different client sets until a preset training end condition is reached and the model training is stopped.

3. The method according to claim 1, characterized in that The determining of the local density correction term of the sample point based on the inverse distance density includes: The local density correction term is determined based on the magnitude relationship between the inverse distance density of the sample point and the inverse distance density of the adjacent sample points.

4. The method according to claim 1, characterized in that: Based on the local density of each of the sample points, the sample data set is clustered, and clients with similar data distribution are divided into the same client set to obtain multiple client sets, including: Determining the high local density point distance of each of the sample points according to the local density of each of the sample points; Determining a decision value of each of the sample points based on the local density and the distance between the high local density points; Based on the second-order difference of the decision value of each of the sample points, a cluster center is selected from the sample data set, and sample points that are not cluster centers are assigned to the cluster centers to obtain a plurality of sample clusters; Based on the multiple sample clusters, clients with similar data distribution are divided into the same client set to obtain multiple client sets.

5. The method according to claim 4, characterized in that The step of selecting a cluster center from the sample data set based on the second-order difference of the decision value of each sample point includes: Determining, according to the second-order differences of the decision values ​​of the sample points, target sample points where the second-order differences have a sudden change; Based on the target sample point, a cluster center is screened out from the sample data set, and the second-order difference of the decision value of the cluster center is greater than the second-order difference of the decision value of the target sample point.

6. A method for processing power grid business data, characterized in that: The method comprises: Obtain power grid business data; Calling the trained data processing model to process the power grid service data to obtain a data processing result; Wherein, the data processing model is based on historical power grid business data and is trained by the model training method of any one of claims 1 to 5.

7. A model training device, characterized in that: The device comprises: A data receiving module, used to receive multiple groups of first model parameters fed back by multiple clients, determine the first model parameters as sample points, and obtain a sample data set, wherein the first model parameters are obtained by the clients training a to-be-trained model based on a local business training data set; A local density determination module is used to determine, for each of the sample points, the distance inverse density of the sample point based on the distance between the sample point and its adjacent sample points, determine the local density correction term of the sample point based on the distance inverse density, and determine the local density of the sample point based on the distance inverse density and the local density correction term, wherein the adjacent sample points are sample points within a preset distance range of the sample point; A clustering module, used to cluster the sample data set based on the local density of each of the sample points, and divide clients with similar data distribution into the same client set to obtain multiple client sets; The model training module is used to schedule multiple clients so that clients in the same client set share model parameters and jointly train the model through federated learning.

8. A power grid business data processing device, characterized in that: The device comprises: A data acquisition module, used to acquire power grid business data; A data processing module is used to call the trained data processing model to process the power grid business data and obtain a data processing result; Wherein, the data processing model is based on historical power grid business data and is trained by the model training method of any one of claims 1 to 5.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images