Power grid business data processing methods, devices and computer equipment

Abstract

This application relates to a model training method, apparatus, computer device, computer-readable storage medium, and computer program product. The method includes: receiving multiple sets of first model parameters from multiple clients; determining the first model parameters as sample points to obtain a sample dataset; for each sample point, determining the inverse distance density of the sample point based on the distance between the sample point and its neighboring sample points; determining a local density correction term for the sample point based on the inverse distance density; determining the local density of the sample point based on the inverse distance density and the local density correction term; clustering the sample dataset based on the local densities of each sample point; grouping clients with similar data distributions into the same client set to obtain multiple client sets; scheduling multiple clients so that clients in the same client set share model parameters; and jointly training the model through federated learning. This method can improve model training accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer equipment, and computer-readable storage medium for processing power grid business data. Background Technology

[0002] With the development of artificial intelligence technology, data security and privacy are increasingly valued during model training. Researchers are building distributed model training frameworks by integrating data from different sources. However, the diverse data sources generate significant communication costs, impacting model training efficiency. For example, in the process of dispatching distribution networks, data processing models are typically used to handle relevant power grid business data. Training these models requires a large amount of power grid business data. However, this data is usually distributed across multiple nodes in the distribution network, with varying types and sources. Even with a distributed model training framework, significant communication costs still exist between the multiple nodes of the distribution network and the central server, affecting the training efficiency of classification models.

[0003] In traditional solutions, a federated learning framework is used to initially train the data processing model locally at multiple nodes in the distribution network. Then, a central server integrates the training results from each node and trains the model again to obtain a global model. In this way, a global model can be trained across multiple nodes based on distributed data without exchanging data samples, thus improving the training speed of the data processing model.

[0004] However, in traditional solutions, due to the large number of clients or nodes providing training data in the federated learning framework, the training data varies greatly between different clients or nodes. This results in significant differences in model parameters when clients or nodes participate in federated learning to train local models, and the technical problem of low model training accuracy still exists. Summary of the Invention

[0005] Therefore, it is necessary to provide a power grid business data processing method, apparatus, computer equipment, and computer-readable storage medium with higher model training accuracy and higher data processing accuracy to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a power grid business data processing method applied to a central server of a federated learning framework, wherein the federated learning framework includes the central server and multiple clients connected to the central server, and the method includes:

[0007] Acquire power grid business data, which includes power consumption, power generation, equipment operation data, and environmental data;

[0008] The trained data processing model is invoked to process the power grid business data and obtain the data processing results.

[0009] The data processing model is trained in the following manner:

[0010] The system receives multiple sets of first model parameters from multiple clients, determines the first model parameters as sample points, and obtains a sample dataset. The first model parameters are obtained by the client training the data processing model to be trained based on the local power grid business training dataset.

[0011] For each sample point, the inverse distance density of the sample point is determined based on the distance between the sample point and its neighboring sample points. Based on the relationship between the inverse distance density of the sample point and the inverse distance density of its neighboring sample points, a local density correction term is determined. Based on the inverse distance density and the local density correction term, the local density of the sample point is determined. The neighboring sample points are sample points within a preset distance range of the sample point.

[0012] Based on the local density of each sample point, the distance between high local density points of each sample point is determined. Based on the local density and the distance between high local density points, the decision value of each sample point is determined. Based on the second difference of the decision values ​​of each sample point, cluster centers are selected from the sample dataset, and sample points that are not cluster centers are assigned to the cluster centers, resulting in multiple sample clusters. Based on the multiple sample clusters, clients with similar data distributions are divided into the same client set, resulting in multiple client sets.

[0013] Multiple clients are scheduled so that clients in the same client set can share model parameters and jointly train the model through federated learning to obtain a trained data processing model.

[0014] In one embodiment, multiple clients are scheduled, including:

[0015] Send model training messages to multiple clients so that clients in the same client set can share model parameters and train the model together through federated learning.

[0016] Receive the second model parameters fed back by each client, and determine the global model parameters based on multiple second model parameters;

[0017] Send a model training message carrying the global model parameters to clients in different client sets, so that clients in the same client set can jointly train the model based on the global model parameters through federated learning. Then return to the step of receiving the second model parameters fed back by clients in different client sets, until the preset training termination condition is reached and the model training stops.

[0018] In one embodiment, the step of selecting cluster centers from the sample dataset based on the second-order difference of the decision values ​​of each of the sample points includes:

[0019] Based on the second difference of the decision values ​​of each sample point, the target sample point where the second difference changes abruptly is determined;

[0020] Based on the target sample points, cluster centers are selected from the sample dataset, and the second difference of the decision value of the cluster center is greater than the second difference of the decision value of the target sample points.

[0021] In one embodiment, the first model parameter is used to characterize the type and distribution of the client's local power grid business data.

[0022] In one embodiment, the local density correction term is negative when the inverse distance density of the adjacent sample points is greater than the inverse distance density of the sample points.

[0023] Secondly, this application provides a power grid business data processing device, the device comprising:

[0024] The data acquisition module is used to acquire power grid business data, which includes power consumption, power generation, equipment operation data and environmental data.

[0025] The data processing module is used to call the trained data processing model to process the power grid business data and obtain the data processing results.

[0026] The data processing model is trained in the following manner:

[0027] The system receives multiple sets of first model parameters from multiple clients, determines the first model parameters as sample points, and obtains a sample dataset. The first model parameters are obtained by the client training the data processing model to be trained based on the local power grid business training dataset.

[0028] The local density determination module is used to determine the inverse distance density of each sample point based on the distance between the sample point and its neighboring sample points, determine a local density correction term based on the relationship between the inverse distance density of the sample point and the inverse distance density of its neighboring sample points, and determine the local density of the sample point based on the inverse distance density and the local density correction term, wherein the neighboring sample points are sample points within a preset distance range of the sample point;

[0029] The clustering module is used to determine the distance between high local density points of each sample point based on the local density of each sample point, determine the decision value of each sample point based on the local density and the distance between high local density points, filter out cluster centers from the sample dataset based on the second difference of the decision values ​​of each sample point, and assign non-cluster centers to the cluster centers to obtain multiple sample clusters. Based on the multiple sample clusters, clients with similar data distributions are divided into the same client set to obtain multiple client sets.

[0030] The model training module is used to schedule multiple clients, enabling clients in the same client set to share model parameters and jointly train the model through federated learning to obtain a trained data processing model.

[0031] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above-described embodiments of the power grid business data processing method.

[0032] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in any of the above-described embodiments of the power grid business data processing method.

[0033] The aforementioned power grid business data processing method, apparatus, computer equipment, and computer-readable storage medium differ from traditional solutions. This solution uses first model parameters trained on local training data by the client to cluster the sample dataset, thereby grouping clients with similar data distributions into the same client set. While protecting the data privacy of each client, grouping clients with similar local training data distributions into one category allows them to share model training parameters, jointly train and optimize the model, enabling the federated learning process to better handle training data with diverse characteristics and improve model training accuracy. Furthermore, during the clustering of the sample dataset based on the local density of sample points, a local density correction term addresses the influence of adjacent sample points on the local density of sample points, increasing the likelihood that sample points in sparse regions are identified as cluster centers, resulting in a more accurate clustering process and further improving the accuracy of power grid business data processing. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a diagram illustrating the application environment of a model training method in one embodiment;

[0036] Figure 2 This is a flowchart illustrating a model training method in one embodiment;

[0037] Figure 3 This is a flowchart illustrating the steps of scheduling a client in one embodiment;

[0038] Figure 4 This is a flowchart illustrating the client-side clustering steps in one embodiment;

[0039] Figure 5 This is a flowchart illustrating a power grid business data processing method in one embodiment;

[0040] Figure 6 This is a structural block diagram of a model training device in one embodiment;

[0041] Figure 7 This is a structural block diagram of a power grid business data processing device in one embodiment;

[0042] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] The model training method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, client 102 communicates with central server 104 via a network. The data storage system can store the data that central server 104 needs to process. The data storage system can be integrated onto central server 104, or it can be located in the cloud or on other network servers.

[0045] Specifically, the central server 104 can first distribute an initial model to be trained to each client 102, and schedule each client 102 to train the model. Each client 102 trains the model based on its local business training dataset, obtains and feeds back multiple sets of first model parameters to the central server 104. Then, the central server 104 determines the first model parameters as sample points, obtains a sample dataset, and determines the inverse distance density of the sample points based on the distance between the sample points and their neighboring sample points. Based on the inverse distance density, it determines the local density correction term of the sample points. Based on the inverse distance density and the local density correction term, it determines the local density of the sample points. Finally, the central server 104 clusters the sample dataset based on the local density of each sample point, dividing the clients 102 with similar data distributions into the same client set, obtaining multiple client sets, and scheduling multiple clients 102 so that the clients 102 in the same client set can share model parameters and jointly train the model through federated learning.

[0046] The client 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The central server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0047] In one exemplary embodiment, such as Figure 2 As shown, a model training method is provided, which can be applied to... Figure 1 Taking the central server 104 as an example, the following steps are included:

[0048] S100 receives multiple sets of first model parameters from multiple clients, determines the first model parameters as sample points, and obtains the sample dataset.

[0049] Each client pre-stores its local business training data. For data security and privacy reasons, the local business training data is not shared between clients. Before receiving the first model parameters, the central server sends the model to be trained to each client and instructs each client to train the model based on its local business training dataset to obtain the first model parameters. Then, each client feeds back its obtained first model parameters to the central server.

[0050] Furthermore, the central server uses the first model parameters as sample points to obtain a sample data set, providing a data foundation for the subsequent clustering process. Since the local business training data of each client is different, the first model parameters obtained from the training may not be the same. However, the first model parameters can reflect the type and distribution of the client's local business training data to a certain extent. Therefore, in this scheme, they can be used as the basis for clustering to group clients with similar data distributions into the same client set.

[0051] S200: For each sample point, determine the inverse distance density of the sample point based on the distance between the sample point and its neighboring sample points. Based on the inverse distance density, determine the local density correction term of the sample point. Based on the inverse distance density and the local density correction term, determine the local density of the sample point.

[0052] Here, adjacent sample points refer to sample points within a preset distance range of the sample point, which can be a cutoff distance. Local density is an indicator that reflects the density of sample points in their neighborhood; the higher the local density, the more sample points are near that sample point, and vice versa.

[0053] The inverse distance density is the reciprocal of the mean distance between a sample point and its neighboring sample points. The mean distance can be the Euclidean distance. For example, the formula for calculating the mean distance of sample point i is shown in equation (1):

[0054] (1)

[0055] In equation (1), n ​​represents the number of neighboring sample points of sample point i. Let represent the Euclidean distance between sample point i and its neighboring sample point j, and NN represent the set of all neighboring sample points of sample point i. Further, the inverse distance density of sample point i is the reciprocal of its mean distance, expressed by equation (2):

[0056] (2)

[0057] Furthermore, considering that the density distribution of sample points in the sample dataset may not be uniform, with larger intervals between sample points in sparse regions, resulting in lower local density, this means that sample points in sparse regions may be easily ignored during the subsequent cluster center search, leading to the loss of some clusters and affecting the overall clustering effect. Therefore, to better adapt to sample datasets with uneven sample point density distribution, this embodiment adds a feature to correct the local density of sample points.

[0058] Specifically, if the local density correction term is not considered, the local density of a sample point is equal to its inverse distance density. The larger the mean distance, the smaller the inverse distance density. Therefore, the inverse distance density of sample points in sparse regions is small. In order to increase the inverse distance density of sample points in sparse regions and increase the probability of them being selected as cluster centers, the local density correction term can be a positive number. The sum of the inverse distance density and the local density correction term is used as the corrected local density.

[0059] S300 clusters the sample dataset based on the local density of each sample point, dividing clients with similar data distributions into the same client set, resulting in multiple client sets.

[0060] Following the steps above, the sample dataset is clustered based on the local density of each sample point. Clustering methods include, but are not limited to, K-means clustering and hierarchical clustering. For example, the central server first identifies regions with high local density based on the local density of each sample point and groups the sample points in those regions into the same category. Thus, by analyzing the local density of sample points, the central server can quickly group sample points with similar data distributions into the same category, resulting in multiple sample clusters. Furthermore, since the sample points are the first model parameters, and the first model parameters reflect the data distribution of the local business training data of each client, the multiple sample clusters obtained through clustering can group clients with similar data distributions into the same client set, resulting in multiple client sets.

[0061] S400 schedules multiple clients, allowing clients in the same client set to share model parameters and jointly train the model through federated learning.

[0062] Finally, the central server schedules each client to iteratively train the model to be trained. Clients in the same client set train the model together and share model parameters during the training process. In this way, clients in the same client set can indirectly share knowledge without sharing their local business training data, learn from each other during the model training process, and perform more effective model training based on similar data characteristics. Furthermore, due to the similarity of data distribution, the model can converge faster.

[0063] For example, for each client, the client can first divide the local business training data into multiple sets of business training data. During the iterative training process, each round of training uses a set of business training data to train the model using stochastic gradient descent, instead of using the entire dataset at once. This allows the model parameters to be updated gradually.

[0064] In this embodiment, unlike traditional methods, the above method uses the first model parameters trained by the client based on local training data to cluster the sample dataset. This groups clients with similar data distributions into the same client set, protecting the data privacy of each client while allowing them to share model training parameters and jointly train and optimize the model. This enables the federated learning process to better handle training data with diverse characteristics, improving model training accuracy. Furthermore, during the clustering of the sample dataset based on the local density of sample points, a local density correction term addresses the influence of adjacent sample points on the local density of sample points. This increases the likelihood of sample points in sparse regions being identified as cluster centers, resulting in a more accurate clustering process and further improving model training accuracy.

[0065] In one exemplary embodiment, such as Figure 3 As shown, S400 includes:

[0066] S410 sends model training messages to multiple clients so that clients in the same client set can share model parameters and jointly train the model through federated learning.

[0067] S420 receives the second model parameters fed back by each client and determines the global model parameters based on multiple second model parameters.

[0068] S430: Send a model training message carrying global model parameters to clients in different client sets, so that clients in the same client set can jointly train the model based on the global model parameters through federated learning.

[0069] Following the above embodiments, the central server coordinates clients within the same client set to jointly train the model. Specifically, the central server can send model training messages to clients in each client set, instructing each client to iteratively train the model to be trained based on its local business training data. Clients within the same client set can share model parameters during iterative training. Specifically, each client in each client set can iteratively train based on its local business training data. Subsequently, each client set feeds back the trained second model parameters to the central server. The central server determines the global model parameters based on the second model parameters fed back by clients within the same client set. In this way, the central server only needs to aggregate the model parameters of clients within the same client set, rather than aggregating the model parameters of all clients, reducing negative transfer caused by data differences between different client sets, thereby better utilizing the similarity of data distribution to improve model performance. Furthermore, in the next iteration, clients from another client set can be selected to jointly train the model using federated learning until a preset training termination condition is reached, at which point model training stops. The training termination condition can be reaching a preset number of training iterations or reaching a preset model training accuracy. After receiving the global model parameters, each client uses them as the initial state to further train and optimize the model based on its local business training data. It should be noted that in each round of model training, not all clients in the same client set need to participate in the model training. Instead, a preset number of clients can be randomly selected from the same client set to train the model together and upload the trained second model parameters to the central server.

[0070] In this embodiment, after grouping clients with similar data distributions into the same client set, the clients in the same client set jointly train the model, and share model parameters during the training process, collaboratively optimizing the model. This allows for better utilization of the similarity in data distribution within the same client set to improve model performance, and reduces the negative impact of differences in data distribution between different client sets on global model parameters, thereby reducing negative transfer phenomena.

[0071] In an exemplary embodiment, determining a local density correction term for a sample point based on the inverse distance density includes: determining the local density correction term based on the magnitude relationship between the inverse distance density of the sample point and the inverse distance density of neighboring sample points.

[0072] Following the above embodiments, the local density of the sample points can be corrected based on the local density correction term, and the expression for the corrected local density is shown in equation (3):

[0073] (3)

[0074] In equation (3), This represents the inverse distance density of sample point i. The local density correction term represents the relationship between the inverse distance density of a sample point and the inverse distance density of its neighboring sample points. For sample point i, the local density correction term is positive when the inverse distance density of its neighboring sample points is less than that of sample point i, negative when the inverse distance density of its neighboring sample points is greater than that of sample point i, and zero when the inverse distance density of its neighboring sample points is less than that of sample point i.

[0075] In this embodiment, a local density correction term is determined based on the relationship between the inverse distance density of a sample point and the inverse distance density of its neighboring sample points. This causes the local density of potential cluster centers in sparse regions to increase adaptively, thereby increasing the likelihood that they will be identified as cluster centers and improving the accuracy of the clustering results.

[0076] In one exemplary embodiment, such as Figure 4 As shown, S300 includes:

[0077] S310, determine the distance between high local density points of each sample point based on the local density of each sample point.

[0078] S320 determines the decision value for each sample point based on local density and the distance between points with high local density.

[0079] S330, based on the second-order difference of the decision values ​​of each sample point, selects cluster centers from the sample dataset and assigns non-cluster centers to the cluster centers, thus obtaining multiple sample clusters.

[0080] S340, based on multiple sample clusters, divides clients with similar data distributions into the same client set, resulting in multiple client sets.

[0081] Wherein, the distance to high local density points refers to the closest distance to sample point i among all sample points with local density higher than sample point i, which can be expressed by equation (4):

[0082] (4)

[0083] When the local density of sample point i is the largest among all sample points, the distance of high local density points is represented by equation (5):

[0084] (5)

[0085] The decision value can be obtained by multiplying the local density and the distance between points with high local density, as shown in equation (6):

[0086] (6)

[0087] Decision values ​​can be used to characterize the probability that a sample point will be selected as a cluster center. The larger the decision value, the more likely the sample point is to be in a core position in the data distribution center, and thus more likely to be selected as a cluster center. Specifically, the decision values ​​of all sample points can be sorted in descending order, and the second difference of the sorted decision values ​​can be calculated. Based on the calculation results of the second difference, cluster centers can be selected from the sample dataset. The calculation formulas are shown in equations (7) and (8):

[0088] (7)

[0089] (8)

[0090] in, The second difference represents the decision value. This represents the first difference of the decision value.

[0091] After identifying cluster centers, sample points that are not cluster centers are assigned to the nearest cluster centers, forming multiple sample clusters. Finally, based on these multiple sample clusters, clients with similar data distributions are grouped into the same client set, resulting in multiple client sets.

[0092] In this embodiment, unlike the traditional approach of drawing a decision map and manually selecting cluster centers from it, where cluster centers may be missed when two sample points are too close or almost overlap, affecting clustering accuracy, the approach in this embodiment uses the local density of sample points, the distance between high local density points, and decision values ​​to filter cluster centers. This effectively reduces the omission of cluster centers and improves the accuracy of clustering results.

[0093] In an exemplary embodiment, S330 includes: determining the target sample point where the second-order difference abruptly occurs based on the second-order difference of the decision values ​​of each sample point, and selecting cluster centers from the sample dataset based on the target sample point.

[0094] Following the steps above, since there are significant variations in decision values ​​between cluster centers and non-cluster centers, this key feature can be used to automatically identify and determine the location of abrupt change points, thereby quickly identifying cluster centers. Specifically, sample points with larger decision values ​​exhibit obvious fluctuations and are more likely to be cluster centers, while sample points with relatively smaller decision values ​​are more stable. To more clearly present this fluctuation characteristic, the second difference of the decision values ​​is used to quantify the amplitude of the fluctuations, thus more clearly highlighting the differences between centers and non-centers.

[0095] Specifically, M is defined as the target sample point where the second-order difference of the decision value undergoes a sudden change. All sample points with decision values ​​greater than M are considered cluster centers. The expression for the target sample point M is shown in equation (9):

[0096] (9)

[0097] In this embodiment, the probability of a sample point becoming a cluster center is quantified by the second difference of the decision value. This can effectively reduce the phenomenon of cluster centers being missed due to manual selection of cluster centers from the decision graph, improve the accuracy of clustering, and thus improve the training accuracy of the model in the federated learning process.

[0098] To provide a clearer explanation of the model training method provided in this application, a specific embodiment is described below, which includes the following steps:

[0099] S501 receives multiple sets of first model parameters from multiple clients, determines the first model parameters as sample points, and obtains the sample dataset.

[0100] S502, for each sample point, determine the inverse distance density of the sample point based on the distance between the sample point and its neighboring sample points, determine the local density correction term based on the magnitude relationship between the inverse distance density of the sample point and the inverse distance density of its neighboring sample points, and determine the local density of the sample point based on the inverse distance density and the local density correction term.

[0101] S503, based on the local density of each sample point, determine the distance to the high local density points of each sample point, and based on the local density and the distance to the high local density points, determine the decision value of each sample point.

[0102] S504. Based on the second difference of the decision values ​​of each sample point, determine the target sample point where the second difference changes abruptly, and based on the target sample point, select cluster centers from the sample dataset.

[0103] S505, based on multiple sample clusters, divides clients with similar data distributions into the same client set, resulting in multiple client sets.

[0104] S506 sends model training messages to multiple clients so that clients in the same client set can share model parameters and jointly train the model through federated learning.

[0105] S507 receives the second model parameters fed back by each client and determines the global model parameters based on multiple second model parameters.

[0106] S508, send model training messages carrying global model parameters to clients in different client sets, so that clients in the same client set can jointly train the model based on the global model parameters through federated learning.

[0107] In one embodiment, such as Figure 5 As shown, a method for processing power grid business data is provided, including the following steps:

[0108] S601, Obtain power grid business data.

[0109] S602 calls the trained data processing model to process the power grid business data and obtain the data processing results.

[0110] The data processing model is trained based on historical power grid business data using any of the model training methods described in the above examples. Power grid business data includes, but is not limited to, electricity consumption, power generation, equipment operation data, and environmental data. The data processing model can be a data classification model, a power generation prediction model, etc.

[0111] For example, in a distribution network, a trained data processing model is needed to process power grid business data. However, when training the data processing model, since the training data is distributed across various nodes in the distribution network, the model training method described in any of the above-mentioned model training method embodiments can be used to train the data processing model. The specific training process will not be elaborated further.

[0112] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0113] Based on the same inventive concept, this application also provides a model training apparatus for implementing the model training method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more model training apparatus embodiments provided below can be found in the limitations of the model training method described above, and will not be repeated here.

[0114] In one exemplary embodiment, such as Figure 6 As shown, a model training apparatus 700 is provided, including: a data receiving module 710, a local density determination module 720, a clustering module 730, and a model training module 740, wherein:

[0115] The data receiving module 710 is used to receive multiple sets of first model parameters fed back by multiple clients, determine the first model parameters as sample points, and obtain a sample dataset. The first model parameters are obtained by the client training the model to be trained based on the local business training dataset.

[0116] The local density determination module 720 is used to determine the inverse distance density of each sample point based on the distance between the sample point and its neighboring sample points, determine the local density correction term of the sample point based on the inverse distance density, and determine the local density of the sample point based on the inverse distance density and the local density correction term. The neighboring sample points are sample points within a preset distance range of the sample point.

[0117] Clustering module 730 is used to cluster the sample dataset based on the local density of each sample point, and to divide clients with similar data distribution into the same client set to obtain multiple client sets;

[0118] The model training module 740 is used to schedule multiple clients, enabling clients in the same client set to share model parameters and jointly train the model through federated learning.

[0119] In an exemplary embodiment, the model training module 740 is further configured to send model training messages to multiple clients so that clients in the same client set can share model parameters and jointly train the model through federated learning; receive second model parameters fed back by clients in different client sets; determine global model parameters based on multiple second model parameters; send model training messages carrying global model parameters to clients in different client sets so that clients in the same client set can jointly train the model based on global model parameters through federated learning; and receive second model parameters fed back by clients in different client sets again until a preset training termination condition is reached and model training is stopped.

[0120] In an exemplary embodiment, the local density determination module 720 is further configured to determine a local density correction term based on the magnitude relationship between the inverse distance density of the sample point and the inverse distance density of the adjacent sample points.

[0121] In an exemplary embodiment, the clustering module 730 is further configured to determine the distance between high local density points of each sample point based on the local density of each sample point, determine the decision value of each sample point based on the local density and the distance between high local density points, filter out cluster centers from the sample dataset based on the second difference of the decision values ​​of each sample point, and assign non-cluster centers to the cluster centers to obtain multiple sample clusters, and divide clients with similar data distributions into the same client set based on the multiple sample clusters to obtain multiple client sets.

[0122] In an exemplary embodiment, the clustering module 730 is further configured to determine the target sample point whose second-order difference has abruptly changed based on the second-order difference of the decision value of each sample point, and to select cluster centers from the sample dataset based on the target sample point, wherein the second-order difference of the decision value of the cluster center is greater than the second-order difference of the decision value of the target sample point.

[0123] Based on the same inventive concept, this application also provides a model training device for implementing the power grid business data processing method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more power grid business data processing device embodiments provided below can be found in the limitations of the power grid business data processing method described above, and will not be repeated here.

[0124] In one exemplary embodiment, such as Figure 7 As shown, a power grid business data processing device 800 is provided, including: a data acquisition module 810 and a data processing module 820, wherein:

[0125] The data acquisition module 810 is used to acquire power grid business data.

[0126] The data processing module 820 is used to call the trained data processing model to process the power grid business data and obtain the data processing results.

[0127] The modules in the aforementioned model training device and power grid business data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0128] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data such as the local density of sample points. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the processor executes the computer program, it implements a model training method. Furthermore, the database can also be used to store data such as power grid business data, and when the processor executes the computer program, it implements a power grid business data processing method.

[0129] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0130] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described model training method embodiment.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the above-described model training method embodiment.

[0132] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described model training method embodiment.

[0133] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodiment of the power grid business data processing method.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the above-described power grid business data processing method embodiment.

[0135] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described power grid business data processing method embodiment.

[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0139] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing power grid business data, characterized in that, A central server is applied to a federated learning framework, the federated learning framework including the central server and multiple clients connected to the central server, the method comprising: Acquire power grid business data, which includes power consumption, power generation, equipment operation data, and environmental data; The trained data processing model is invoked to process the power grid business data and obtain the data processing results. The data processing model is trained in the following manner: The system receives multiple sets of first model parameters from multiple clients, determines the first model parameters as sample points, and obtains a sample dataset. The first model parameters are obtained by the client training the data processing model to be trained based on the local power grid business training dataset. For each sample point, the inverse distance density of the sample point is determined based on the distance between the sample point and its neighboring sample points. Based on the relationship between the inverse distance density of the sample point and the inverse distance density of its neighboring sample points, a local density correction term is determined. Based on the inverse distance density and the local density correction term, the local density of the sample point is determined. The neighboring sample points are sample points within a preset distance range of the sample point. Based on the local density of each sample point, the distance between high local density points of each sample point is determined. Based on the local density and the distance between high local density points, the decision value of each sample point is determined. Based on the second difference of the decision values ​​of each sample point, cluster centers are selected from the sample dataset, and sample points that are not cluster centers are assigned to the cluster centers, resulting in multiple sample clusters. Based on the multiple sample clusters, clients with similar data distributions are divided into the same client set, resulting in multiple client sets. Multiple clients are scheduled so that clients in the same client set can share model parameters and jointly train the model through federated learning to obtain a trained data processing model.

2. The method according to claim 1, characterized in that, Scheduling multiple clients, including: Send model training messages to multiple clients so that clients in the same client set can share model parameters and train the model together through federated learning. Receive the second model parameters fed back by each client, and determine the global model parameters based on multiple second model parameters; Send a model training message carrying the global model parameters to clients in different client sets, so that clients in the same client set can jointly train the model based on the global model parameters through federated learning. Then return to the step of receiving the second model parameters fed back by clients in different client sets, until the preset training termination condition is reached and the model training stops.

3. The method according to claim 1, characterized in that, The second-order difference based on the decision values ​​of each of the sample points is used to select cluster centers from the sample dataset, including: Based on the second difference of the decision values ​​of each sample point, the target sample point where the second difference changes abruptly is determined; Based on the target sample points, cluster centers are selected from the sample dataset, and the second difference of the decision value of the cluster center is greater than the second difference of the decision value of the target sample points.

4. The method according to claim 1, characterized in that, The first model parameter is used to characterize the type and distribution of the client's local power grid business data.

5. The method according to claim 1, characterized in that, When the inverse distance density of adjacent sample points is greater than the inverse distance density of the sample points, the local density correction term is negative.

6. A power grid business data processing device, characterized in that, The device includes: The data acquisition module is used to acquire power grid business data, which includes power consumption, power generation, equipment operation data and environmental data. The data processing module is used to call the trained data processing model to process the power grid business data and obtain the data processing results. The data processing model is trained in the following manner: The system receives multiple sets of first model parameters from multiple clients, determines the first model parameters as sample points, and obtains a sample dataset. The first model parameters are obtained by the client training the data processing model to be trained based on the local power grid business training dataset. The local density determination module is used to determine the inverse distance density of each sample point based on the distance between the sample point and its neighboring sample points, determine a local density correction term based on the relationship between the inverse distance density of the sample point and the inverse distance density of its neighboring sample points, and determine the local density of the sample point based on the inverse distance density and the local density correction term, wherein the neighboring sample points are sample points within a preset distance range of the sample point; The clustering module is used to determine the distance between high local density points of each sample point based on the local density of each sample point, determine the decision value of each sample point based on the local density and the distance between high local density points, filter out cluster centers from the sample dataset based on the second difference of the decision values ​​of each sample point, and assign non-cluster centers to the cluster centers to obtain multiple sample clusters. Based on the multiple sample clusters, clients with similar data distributions are divided into the same client set to obtain multiple client sets. The model training module is used to schedule multiple clients, enabling clients in the same client set to share model parameters and jointly train the model through federated learning to obtain a trained data processing model.

7. The apparatus according to claim 6, characterized in that, The model training module is also used to send model training messages to multiple clients so that clients in the same client set can share model parameters and jointly train the model through federated learning. It receives second model parameters from each client, determines global model parameters based on multiple second model parameters, and sends model training messages carrying the global model parameters to clients in different client sets so that clients in the same client set can jointly train the model through federated learning based on the global model parameters. It returns to the step of receiving second model parameters from clients in different client sets until a preset training termination condition is reached and model training stops.

8. The apparatus according to claim 6, characterized in that, The clustering module is further configured to determine the target sample point whose second-order difference has abruptly changed based on the second-order difference of the decision values ​​of each sample point, and to select cluster centers from the sample dataset based on the target sample point, wherein the second-order difference of the decision values ​​of the cluster centers is greater than the second-order difference of the decision values ​​of the target sample points.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images