Cluster-based parallel segmentation learning method, device, equipment and storage medium

Abstract

The application belongs to the technical field of computers and discloses a parallel segmentation learning method and device based on clusters, equipment and a storage medium. The method comprises the following steps: obtaining a plurality of to-be-learned user terminals and user communication information of each to-be-learned user terminal; performing cluster division on each to-be-learned user terminal according to each to-be-learned user terminal and the user communication information of each to-be-learned user terminal, and determining a plurality of target user clusters; and performing cluster serial segmentation learning according to an aggregated user terminal model and each target user cluster to obtain a target user terminal model, wherein the aggregated user terminal model is obtained by performing parallel segmentation learning on each target user cluster according to the target spectrum resources of each to-be-learned user terminal. Through the above method, the overall training time delay in the segmentation learning process is effectively reduced, the efficiency of segmentation learning is improved, the negative effects caused by network heterogeneity and dynamics are inhibited, and the convergence and accuracy of existing segmentation learning technology are ensured.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a cluster-based parallel segmentation learning method, apparatus, device, and storage medium. Background Technology

[0002] Segmentation learning is a mainstream distributed learning scheme that enables AI (Artificial Intelligence) models to be trained across user terminals (such as vehicles and mobile terminals) with the assistance of edge servers in the Internet of Vehicles (IoV) without sharing local data on the user terminals. Existing segmentation learning schemes, when dealing with a large number of users, require the edge server to train one user first, then move to the next user sequentially. The entire model training process is controlled by the network controller located on the edge server, resulting in significant model training latency. Because users train the model sequentially, the training latency in existing segmentation learning schemes is cumulative and proportional to the number of users. When the number of users is large, model training leads to substantial latency, significantly increasing the overall training time. Therefore, there is an urgent need for a segmentation learning method that can handle multiple users, reducing training latency and shortening training time. Summary of the Invention

[0003] The main objective of this invention is to provide a cluster-based parallel segmentation learning method, apparatus, device, and storage medium, aiming to solve the technical problem of how to reduce training latency and shorten training time in segmentation learning when multiple user terminals are present in the prior art.

[0004] To achieve the above objectives, the present invention provides a cluster-based parallel segmentation learning method, which includes:

[0005] Acquire multiple client-side learning targets and user communication information for each client-side learning target;

[0006] Based on the learning user terminals and their user communication information, the learning user terminals are divided into clusters to determine multiple target user clusters.

[0007] The target user terminal model is obtained by performing cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster. The aggregated user terminal model is obtained by performing parallel segmentation learning in each target user cluster based on the target spectrum resources of each user terminal to be learned.

[0008] Optionally, the step of dividing each learning user terminal into clusters based on each learning user terminal and its user communication information, and determining multiple target user clusters, includes:

[0009] Based on a random aggregation strategy, each user terminal to be learned is divided into clusters to obtain multiple first user clusters;

[0010] The first training delay is determined based on the user communication information of multiple first user clusters and each user terminal to be learned.

[0011] Randomly select from each first user cluster according to a random sampling strategy to determine the exchange user terminal and the first user cluster where the exchange user terminal is located.

[0012] Multiple second user clusters are obtained by randomly associating the interactive user terminal, the first user cluster to which the exchange user terminal is located, and multiple first user clusters.

[0013] The first training delay and the second training delay are determined based on the user communication information of multiple second user clusters and each user terminal to be learned.

[0014] Multiple target user clusters are determined based on the first training delay and the second training delay.

[0015] Optionally, determining multiple target user clusters based on the first training delay and the second training delay includes:

[0016] The exploration probability is calculated based on the first training delay and the second training delay to determine the target exploration probability;

[0017] The target exploration probability is compared with a preset cluster variable to determine the comparison result;

[0018] When the comparison result shows that the target exploration probability is greater than the preset cluster variable, the number of user terminals in each second user cluster is determined.

[0019] When the number of users in each second user cluster is different, each second user cluster is treated as multiple target user clusters.

[0020] Optionally, after obtaining the number of user terminals in each second user cluster when the comparison result shows that the target exploration probability is greater than the preset cluster variable, the method further includes:

[0021] When the number of user terminals in each second user cluster is the same, the delayed user terminals in each second user cluster are determined based on the user communication information of each user terminal to be learned.

[0022] Based on the delayed user terminals in each second user cluster and multiple second user clusters, random associations are performed to determine multiple first association results and the first association delay of each first association result;

[0023] The first association delays of each first association result are compared, and the first target result is determined from multiple first association results based on the comparison results;

[0024] Based on the results of the first objective, multiple third-user clusters are identified;

[0025] Multiple target user clusters are identified based on multiple third-party user clusters.

[0026] Optionally, determining multiple target user clusters based on multiple third user clusters includes:

[0027] The fast user terminals in each third user cluster are determined based on the user communication information of each user terminal to be learned.

[0028] Based on the fast user terminals in each third user cluster and multiple third user clusters, random association is performed to determine multiple second association results and the second association delay of each second association result;

[0029] The second association delays of each second association result are compared, and the second target result is determined from multiple second association results based on the comparison results;

[0030] Based on the results of the second objective, multiple target user clusters are identified.

[0031] Optionally, the step of performing cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model includes:

[0032] The first and second target clusters are determined according to the preset cluster training order;

[0033] Determine the aggregated user terminal model based on the first target cluster;

[0034] The aggregated client model is sent to the second target cluster so that the second target cluster can feed back the target client model.

[0035] Optionally, the cluster-based parallel segmentation learning method further includes:

[0036] The user-end model is distributed to each learning user terminal in the target user cluster so that each learning user terminal in the target user cluster can perform data sampling in parallel to generate fragmented data, and feed back the fragmented data and sampling data labels according to the target spectrum resources of each learning user terminal in the target user cluster.

[0037] The pulverization data gradient is determined based on the pulverization data and the labels of the sampling data;

[0038] The shredded data gradient is sent to each learning user terminal in each target user cluster, so that each learning user terminal in each target user cluster updates its user terminal model based on the feedback of the shredded data gradient.

[0039] The aggregated client model for each target user cluster is determined based on each updated client model.

[0040] Optionally, determining the aggregated user client model for each target user cluster based on each updated user client model includes:

[0041] Obtain the number of client terminals in each target user cluster and the number of client terminals to be learned in each target user cluster;

[0042] The aggregated user terminal model for each target user cluster is obtained by weighting and aggregating the number of user terminals in each target user cluster, the number of user terminals to be learned in each target user cluster, and the update model of each user.

[0043] Optionally, before obtaining the target user terminal model by performing cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster, the method further includes:

[0044] According to the resource allocation strategy, spectrum resources are allocated to each learning user terminal in each target user cluster to determine the initial resources of each learning user terminal.

[0045] The user training latency of each user terminal to be learned is calculated based on the user communication information and the initial resources of each user terminal to be learned.

[0046] The training latency of each user terminal is sorted, and the target learning user terminal is determined based on the sorting results.

[0047] Spectrum resources are allocated to the target learning user terminal to determine the allocated resources for the target learning user terminal;

[0048] The remaining resource quantity is determined based on the initial resources of each learning user terminal and the allocated resources of the target learning user terminal;

[0049] When the remaining resources are equal to the preset resources, the initial resources of each learning user terminal are updated according to the allocated resources of the target learning user terminal, and the target spectrum resources of each learning user terminal in each target user cluster are determined.

[0050] Furthermore, to achieve the above objectives, the present invention also proposes a cluster-based parallel segmentation learning device, which includes:

[0051] The acquisition module is used to acquire multiple user terminals to be learned and user communication information of each user terminal;

[0052] The segmentation module is used to segment the learning user terminals into clusters based on each learning user terminal and the user communication information of each learning user terminal, and to determine multiple target user clusters.

[0053] The learning module is used to perform cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model. The aggregated user terminal model is obtained by performing parallel segmentation learning in each target user cluster based on the target spectrum resources of each user terminal to be learned.

[0054] Furthermore, to achieve the above objectives, the present invention also proposes a cluster-based parallel segmentation learning device, which includes: a memory, a processor, and a cluster-based parallel segmentation learning program stored in the memory and executable on the processor. The cluster-based parallel segmentation learning program is configured to implement the cluster-based parallel segmentation learning method described above.

[0055] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a cluster-based parallel segmentation learning program, which, when executed by a processor, implements the cluster-based parallel segmentation learning method as described above.

[0056] This invention acquires multiple user terminals to be learned and their user communication information; divides the user terminals to be learned into clusters based on their communication information to determine multiple target user clusters; and performs serial segmentation learning of the clusters based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model. The aggregated user terminal model is obtained through parallel segmentation learning within each target user cluster based on the target spectrum resources of each user terminal to be learned. Through this method, multiple user terminals to be learned are divided into clusters to obtain optimal target user clusters. Serial segmentation learning between clusters is then performed based on the aggregated user terminal model and each target user cluster to obtain the trained target user terminal model. The aggregated user terminal model is obtained through parallel segmentation learning within each target user cluster based on the target spectrum resources of each user terminal to be learned, thereby effectively reducing the overall training latency and shortening the overall training time during the segmentation learning process, improving the efficiency of segmentation learning, suppressing the negative impacts of network heterogeneity and dynamism, and ensuring the convergence and accuracy of existing segmentation learning techniques. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the structure of a cluster-based parallel segmentation learning device in the hardware operating environment involved in the embodiments of the present invention;

[0058] Figure 2 This is a flowchart illustrating the first embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0059] Figure 3 This is a single-user terminal segmentation learning diagram of an embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0060] Figure 4 This is a multi-user serial learning graph of an embodiment of the cluster-based parallel segmentation learning method of the present invention;

[0061] Figure 5 This is a schematic diagram of cluster partitioning in an embodiment of the cluster-based parallel segmentation learning method of the present invention;

[0062] Figure 6 This is a schematic diagram of inter-cluster serial learning in an embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0063] Figure 7 This is a flowchart illustrating the resource allocation process of an embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0064] Figure 8 This is a schematic diagram of model updating and aggregation in an embodiment of the cluster-based parallel segmentation learning method of the present invention;

[0065] Figure 9 This is a flowchart illustrating the second embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0066] Figure 10 This is a schematic diagram of the cluster partitioning process in an embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0067] Figure 11 This is a flowchart illustrating the third embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0068] Figure 12 This is a graph illustrating intra-cluster parallel learning in an embodiment of the cluster-based parallel segmentation learning method of the present invention.

[0069] Figure 13 This is a schematic diagram of the overall process of an embodiment of the cluster-based parallel segmentation learning method of the present invention;

[0070] Figure 14 This is a structural block diagram of the first embodiment of the cluster-based parallel segmentation learning device of the present invention.

[0071] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0072] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0073] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a cluster-based parallel segmentation learning device in the hardware operating environment involved in the embodiments of the present invention.

[0074] like Figure 1 As shown, the cluster-based parallel partitioning learning device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0075] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on cluster-based parallel segmentation learning devices and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0076] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a cluster-based parallel partitioning learning program.

[0077] exist Figure 1In the cluster-based parallel segmentation learning device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the cluster-based parallel segmentation learning device of the present invention can be set in the cluster-based parallel segmentation learning device, and the cluster-based parallel segmentation learning device calls the cluster-based parallel segmentation learning program stored in the memory 1005 through the processor 1001 and executes the cluster-based parallel segmentation learning method provided in the embodiment of the present invention.

[0078] This invention provides a cluster-based parallel segmentation learning method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of a cluster-based parallel segmentation learning method of the present invention.

[0079] Cluster-based parallel segmentation learning methods include the following steps:

[0080] Step S10: Obtain multiple learning client terminals and user communication information of each learning client terminal.

[0081] It should be noted that the execution entity in this embodiment is the network controller in the edge server of the cluster-based parallel segmentation learning system. The cluster-based parallel segmentation learning system consists of an edge server and multiple user terminals to be learned, and all of these user terminals need to perform segmentation learning.

[0082] It is understandable that during the segmentation learning process on a user's end, such as Figure 3 As shown, the AI ​​model is divided into two parts at the segmentation layer: a front-end model running on the user's device and a back-end model running on the edge server. In existing segmentation learning schemes, the process of training an AI model for each user device includes two phases: forward propagation and back propagation. In the forward propagation phase, the user device uses its local data to perform forward propagation of its user-side model and sends the output of the segmentation layer, referred to as smashed data, to the server. The edge server uses the received smashed data to perform back propagation of the server-side model, updating the model during the back propagation phase and sending the gradients of the smashed data to the user device. The user then uses the gradients of the received smashed data to update their user-side model, thus completing one round of AI model training. When extending existing segmentation learning schemes to scenarios with multiple users, the edge server needs to train one user device first, and then move to the next user device sequentially, as shown... Figure 4As shown, before starting model training, the first user downloads the latest client-side model from the edge server. Then, the first client-side trains the AI ​​model by interacting with the edge server and using its local dataset. After the client-side completes model training, the currently updated client-side model is transferred to the next client-side, which then continues the training process using its own local dataset. When all clients-side have completed model training, the client-side uploads the final trained model to the edge server, thus completing one round of training. The entire process requires multiple rounds of training until the trained AI model reaches satisfactory performance metrics. This training method leads to long training latency. To reduce training latency when performing segmentation learning on multiple client-sides, this embodiment proposes a cluster-based parallel segmentation learning method.

[0083] In practical implementation, the user client to be learned refers to the user client that needs to perform segmentation learning together with the edge server to obtain an AI model that meets the requirements. During the training process, each user client to be learned refers to one user. The user communication information of each user client to be learned refers to the real-time channel status and computing power of each user client to be learned.

[0084] It should be noted that when a model training or segmentation learning start command is received, the network controller in the edge server obtains all the user terminals to be learned participating in segmentation learning, as well as the communication information of each user to be learned.

[0085] Step S20: Based on the learning user terminals and their user communication information, divide each learning user terminal into clusters to determine multiple target user clusters.

[0086] It should be noted that the network controller can utilize the user communication information of each learning endpoint to cluster the learning endpoints. Learning endpoints with similar performance can be grouped into the same cluster, thus accommodating the heterogeneity of multiple learning endpoints. This heterogeneity refers to the differences in computing and communication capabilities among the learning endpoints. Grouping learning endpoints with similar performance into the same cluster results in multiple target user clusters. These multiple user clusters represent the optimal clustering strategy for all learning endpoints, further reducing the latency of the segmentation learning model training. Each target user cluster includes at least one learning endpoint. For example, as... Figure 5 As shown, there are N user terminals to be learned. After using the user communication information of user terminals 1 to N to be learned to divide them into clusters, M target user clusters 1 to M are obtained.

[0087] Step S30: Perform cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model. The aggregated user terminal model is obtained by performing parallel segmentation learning in each target user cluster based on the target spectrum resources of each user terminal to be learned.

[0088] It should be noted that when a target cluster user begins segmentation learning, the edge server will distribute the latest user-end model stored in its storage to all user terminals to be learned in that target cluster user. When all user terminals to be learned in the target user cluster receive the user-end model distributed by the edge server, all user terminals to be learned in the target user cluster will begin to extract a small batch of data samples from their local datasets in parallel, perform forward propagation on the received latest user-end model, and generate shredded data at the model segmentation layer. Each user terminal to be learned will transmit its own shredded data at the model segmentation layer and the sampled small batch of data labels to the edge server according to the allocated target spectrum resources. The edge server performs forward propagation of the server-side model based on all the fragmented data sent by all learning clients in the target user cluster. It then calculates the loss function based on the backpropagation results and mini-batch data labels. The server-side model is updated using the gradient of the loss function and backpropagation. The gradient calculation for the fragmented data is performed at the model segmentation layer. The edge server sends the gradient of the fragmented data to the corresponding learning clients in the target cluster. Upon receiving the fragmented data and its gradient, each learning client updates its own client model through backpropagation, thus outputting its updated client model. Each learning client transmits its updated client model to the edge server based on its target spectrum resources. The edge server performs weighted aggregation of all updated client models to obtain the aggregated client model corresponding to the target user cluster.

[0089] Understandably, after determining the aggregated user terminal model for each target user cluster, the network controller determines the next target user cluster to be trained according to the division of multiple target user clusters and the preset cluster training order. The network controller broadcasts the aggregated user terminal model of the previous target user cluster to all the user terminals to be trained in the next target user cluster. After receiving the aggregated user terminal model of the previous target user cluster, all the user terminals to be trained in the next target user cluster begin to segment and learn in parallel, thereby obtaining the aggregated user terminal model of the next target user cluster. This continues until the last target user cluster completes the parallel segmentation and learning within the cluster. The aggregated user terminal model obtained by the last target user cluster is the target user terminal model.

[0090] In the specific implementation, when parallel segmentation learning begins within a target user cluster, the network controller obtains the user communication information of each user terminal to be learned within the current target user cluster, and allocates spectrum resources for each user terminal to be learned based on the user communication information, thereby obtaining the allocation result of the target spectrum resources for each user terminal to be learned within the current target cluster.

[0091] It should be noted that, in order to ensure the accuracy of serial learning, the step of performing serial segmentation learning of clusters based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model includes: determining the first target cluster and the second target cluster according to a preset cluster training order; determining the aggregated user terminal model based on the first target cluster; and sending the aggregated user terminal model to the second target cluster so that the second target cluster can feed back the target user terminal model.

[0092] Understandably, the first target cluster refers to the first target user cluster among multiple target user clusters to begin parallel segmentation learning within the cluster. The second target cluster refers to all target user clusters except the first target cluster. The first target cluster is determined according to a preset cluster order. The network controller broadcasts the latest client-side model stored in the edge server to each client-side node in the first target cluster to begin learning. Each client-side node in the first target cluster uses the client-side model to begin segmentation learning in parallel, ultimately obtaining the aggregated client-side model of the first target cluster. This aggregated client-side model is then distributed to one of the target user clusters in the second target cluster according to a preset cluster training order. Parallel segmentation learning begins in this second target user cluster, determining its aggregated client-side model. Training continues sequentially in the second target cluster according to the preset cluster training order until the last target user cluster in the second target cluster completes its parallel segmentation learning. The aggregated client-side model obtained from the last target user cluster is the target user-side model. For example, as... Figure 5 As shown, there are M target user clusters 1 to M. The first target cluster is determined as target user cluster 1, and the second target clusters are 2 to M, according to a preset cluster training order. The latest user-end model is sent to target user cluster 1, and each user terminal to be trained in target user cluster 1 undergoes parallel segmentation learning to obtain the aggregated user-end model of target user cluster 1. The training order in the second target cluster is determined to be 2 to M, according to the preset cluster training order. Then, the aggregated user-end model of target user cluster 1 is sent to target user cluster 2, and the aggregated user-end model of target user cluster 2 is determined. The aggregated user-end model of target user cluster 2 is then sent to target user cluster 3, and so on, until target user cluster M is trained. The resulting aggregated user-end model of target user cluster M is the target user-end model.

[0093] In the specific implementation, to consider the dynamic nature of the network and further reduce the training latency of each target user cluster, this embodiment proposes a resource allocation strategy based on a greedy algorithm. Furthermore, before obtaining the target user terminal model by performing cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster, the method further includes: allocating spectrum resources for each user terminal to be learned in each target user cluster according to the resource allocation strategy to determine the initial resources of each user terminal to be learned; calculating the user terminal training latency based on the user communication information of each user terminal to be learned and the initial resources of each user terminal to be learned, thus obtaining the user terminal training latency of each user terminal to be learned. Latency; sort the training latency of each user terminal, and determine the target learning user terminal based on the sorting result; allocate spectrum resources to the target learning user terminal to determine the allocated resources of the target learning user terminal; determine the remaining resource quantity based on the initial resources of each user terminal to be learned and the allocated resources of the target learning user terminal; when the remaining resource quantity is a preset resource quantity, update the initial resources of each user terminal to be learned based on the allocated resources of the target learning user terminal to determine the target spectrum resources of each user terminal to be learned in each target user cluster, and the target spectrum resources of each user terminal to be learned are the number of subcarriers owned by each user terminal to be learned.

[0094] It should be noted that when initiating parallel segmentation learning within a target user cluster, the network controller obtains the user communication information of each user terminal to be learned within the current target user cluster and the number K of user terminals to be learned. The network controller allocates spectrum resources using subcarriers as the basic unit. It is assigned to the user If the number of subcarriers is given, then the resource allocation strategy is as follows: .

[0095] Understandably, according to the resource allocation strategy, during the initialization phase, each learning user in the current target user cluster is allocated a subcarrier. This determines the initial resources of each learning endpoint in the current target user cluster. Using the user communication information and initial resources of each learning endpoint in the current target user cluster, the training latency of each learning endpoint can be calculated. The training latency of each learning endpoint is the user-end training latency. The training latency is .

[0096] In the specific implementation, the user terminals to be learned are sorted according to their training latency, and the user terminal with the longest training latency in the current target user cluster is determined, denoted as . The user terminal with the longest training latency in the current target user cluster is the target user learning terminal. An additional subcarrier is allocated to the target learning terminal according to the resource allocation strategy. The number of subcarriers currently owned by the target learning user terminal is the allocated resource for the target learning user terminal. The user terminal training latency of the target learning user terminal is updated. The number of unallocated subcarriers is determined based on the total number of subcarriers in the communication band, the initial resources of each learning user terminal (excluding the target learning user terminal), and the allocated resources of the target learning user terminal. This unallocated number of subcarriers is the remaining resource quantity. When the remaining resource quantity is the preset resource quantity, it indicates that all subcarriers in the communication band have been allocated. In this embodiment, the preset resource quantity is 0. At this time, the allocated resources are updated based on the allocated resources of the target learning user terminal and the initial resources of each learning user terminal to determine the target spectrum resources for each learning user terminal in the current target user cluster.

[0097] It should be noted that when the remaining resources are not the preset number, it means that all subcarriers in the communication band have not yet been allocated. At this time, the user terminal training delay of the target learning user terminal and the user terminal training delay of the other user terminals to be learned need to be used to determine the user terminal to be learned with the slowest training speed, i.e. the longest training delay, in the current target user cluster. An additional subcarrier is allocated to this user terminal to be learned, and the user terminal training delay of this user terminal to be learned is updated until all subcarriers are allocated. The target spectrum resources to be learned in the current target user cluster are determined by using the subcarriers owned by each user terminal to be learned after the subcarrier allocation is completed.

[0098] It is understandable that, such as Figure 7 As shown, when starting parallel segmentation learning within a target user cluster, the user communication information of each learning user terminal in the current target user cluster is obtained, and a subcarrier is allocated to each learning user terminal in the current target user cluster. Then, the user terminal training latency of each learning user terminal is determined. When there are remaining subcarriers, an additional subcarrier is allocated to the learning user terminal with the longest user terminal training latency, and the corresponding user terminal training latency is updated. It is then determined whether there are any remaining subcarriers. If there are no remaining subcarriers, a spectrum resource allocation strategy is output based on the number of subcarriers available to each learning user terminal in the current target user cluster, and the target spectrum resources for each learning user terminal are determined. Subcarriers are allocated incrementally, that is, each learning user terminal is initially allocated one subcarrier, and during the iteration process, more subcarriers are allocated to the learning user terminal with the slowest training speed until all subcarriers are allocated, thereby reducing the training latency within the cluster.

[0099] In specific implementations, such as Figure 8 As shown, Figure 8 This illustrates the timeline of parallel segmentation learning within each target user cluster. Each user client needs to update its own model, and the edge server needs to aggregate the user client models and await model updates from all user clients. This process includes two phases: In the first phase, each user client performs parallel processing of user client model distribution, forward propagation of the user client model, and transmission of shredded data. Once the edge server receives all the shredded data, it performs forward and backward propagation of the server-side model. In the second phase, each user client performs parallel processing of gradient transmission of shredded data, backward propagation of the user client model, and transmission of the user client model. When the edge server receives all user client models, it performs aggregation of the user client models. The resource allocation strategy proposed in this embodiment is applied to both phases, utilizing the target spectrum resources of each user client to further reduce training latency within the cluster.

[0100] This embodiment acquires multiple user terminals to be learned and their user communication information; it then divides the user terminals to be learned into clusters based on their communication information to determine multiple target user clusters; and performs serial segmentation learning on the clusters according to the aggregated user terminal model and each target user cluster to obtain the target user terminal model. The aggregated user terminal model is obtained through parallel segmentation learning within each target user cluster based on the target spectrum resources of each user terminal to be learned. By dividing the multiple user terminals to be learned into clusters using the above method, optimal target user clusters are obtained. Serial segmentation learning between clusters is then performed based on the aggregated user terminal model and each target user cluster to obtain the trained target user terminal model. The aggregated user terminal model is obtained through parallel segmentation learning within each target user cluster based on the target spectrum resources of each user terminal to be learned, thus effectively reducing the overall training latency during the segmentation learning process, shortening the overall training time, improving the efficiency of segmentation learning, suppressing the negative impacts of network heterogeneity and dynamism, and ensuring the convergence and accuracy of existing segmentation learning techniques.

[0101] refer to Figure 9 , Figure 9 This is a flowchart illustrating a second embodiment of a cluster-based parallel segmentation learning method according to the present invention.

[0102] Based on the first embodiment described above, step S20 in the cluster-based parallel segmentation learning method of this embodiment includes:

[0103] Step S21: Divide each learning user terminal into clusters according to the random aggregation strategy to obtain multiple first user clusters.

[0104] It should be noted that the overall training latency during the training process is the number of training epochs required before the model converges. and the delay of each training round The product of, i.e. Where K is the cluster size, and It is a function related to the cluster size. The calculation process is as follows: the required number of training rounds depends on the data distribution of the given AI model. Each client-side learning object extracts a portion of the dataset from the overall dataset as a representative dataset, and the model is trained on this basis. The number of training rounds required is measured according to the cluster size. Experiments show that... The value increases with the cluster size K, indicating that a larger cluster size slows down the model's convergence speed. (Function) The calculation process: This embodiment uses theoretical analysis of the average channel conditions and average computing power of the user terminal to be trained, as well as the computing power of the edge server, to obtain the training latency for each round of the model. Analysis shows that... The latency decreases as the cluster size K increases, indicating that with more client-side devices undergoing parallel training, a larger cluster size results in lower training latency. Given... and Based on this, the overall training latency can be derived. This minimizes latency and yields the optimal cluster size. Therefore, multiple user terminals to be trained need to be divided into multiple target user clusters, and the overall training latency based on the obtained target user clusters needs to be determined.

[0105] It is understandable that each user terminal to be learned is randomly divided according to the random aggregation strategy to obtain multiple first user clusters after division, and each first user cluster includes at least one user terminal to be learned.

[0106] Step S22: Determine the first training delay based on the user communication information of multiple first user clusters and each user terminal to be learned.

[0107] It should be noted that, since each user terminal under each first user cluster adopts a parallel learning method when performing segmentation learning, and a serial learning method is adopted between clusters, the training latency corresponding to each first user cluster under the current partitioning method is calculated based on the user communication information of each user terminal. Based on the training latency corresponding to each first user cluster, the overall training latency under the current partitioning method is determined. Overall training latency This is the first training delay.

[0108] Step S23: Randomly select users from each first user cluster according to a random sampling strategy to determine the exchange user terminal and the first user cluster where the exchange user terminal is located.

[0109] It should be noted that the random selection strategy refers to randomly selecting two user terminals to be learned from two independent first user clusters. These randomly selected user terminals from the two first user clusters are the exchange user terminals. For example, from the first user cluster... Select the user terminal to be learned and from the first user cluster Select the user terminal to be learned User side to be learned and This refers to the user terminal for switching.

[0110] Step S24: Randomly associate the interactive user terminal, the first user cluster where the exchange user terminal is located, and multiple first user clusters to obtain multiple second user clusters.

[0111] It should be noted that randomly associating a particular switching user terminal with the other switching user terminals belonging to the first user cluster results in a new cluster partitioning strategy, leading to multiple randomly associating first user clusters. These multiple randomly associating first user clusters constitute multiple second user clusters. For example, from the first user cluster... Select the user terminal to be learned and from the first user cluster Select the user terminal to be learned The user terminal to be learned Assigned to the first user cluster The user terminal to be learned Assigned to the first user cluster This results in multiple second user clusters after the user terminals to be learned are re-divided.

[0112] Step S25: Determine the first training delay and the second training delay based on the user communication information of multiple second user clusters and each user terminal to be learned.

[0113] It should be noted that the training latency for each second user cluster under the current partitioning method is calculated based on the user communication information of each user terminal to be learned. Based on the training latency of each second user cluster, the overall training latency under the current partitioning method is determined. Overall training latency This is the second training delay.

[0114] Step S26: Determine multiple target user clusters based on the first training delay and the second training delay.

[0115] It should be noted that after determining the first training delay and the second training delay, the optimal cluster partitioning strategy for all learning user terminals can be determined, resulting in multiple target user terminals. Further, determining multiple target user clusters based on the first training delay and the second training delay includes: calculating the exploration probability based on the first training delay and the second training delay to determine the target exploration probability; comparing the target exploration probability with a preset cluster variable to determine the comparison result; when the comparison result indicates that the target exploration probability is greater than the preset cluster variable, determining the number of user terminals in each second user cluster; and when the number of user terminals in each second user cluster is different, treating each second user cluster as multiple target user clusters.

[0116] Understandably, according to the formula The exploration probability during the cluster partitioning process can be calculated using the first and second training delays to determine the target exploration probability. .in, It is a smoothing factor used to control the trend during the exploration process of new cluster partitioning strategies. A higher value indicates a higher probability of exploration. Additionally, preset cluster variables... It is generated randomly within a uniform distribution in the interval [0, 1].

[0117] In the specific implementation, the target exploration probability is... and preset cluster variables In comparison, If the algorithm fails, the second user cluster is updated with the latest cluster partitioning strategy. Then, the steps of random sampling, random association, and calculating the target exploration probability are repeated based on the second user cluster until the algorithm converges, thereby obtaining the globally optimal decision. The multiple second user clusters at the time of algorithm convergence are then used as the target user cluster. For example... Figure 10 As shown, in If the first user cluster is still used as the latest cluster partitioning strategy, the algorithm continues with random sampling, random association, and calculation of target exploration probabilities based on the first user cluster until convergence, thus obtaining the globally optimal decision. The exploration probability depends on the performance gain of the new cluster partitioning strategy relative to the previous cluster partitioning strategy, thereby avoiding the obtained cluster partitioning strategy from getting trapped in local optima. By iteratively exploring multiple cluster partitioning strategies, the optimal cluster partitioning strategy is obtained, and multiple target user clusters are determined.

[0118] It should be noted that in practical applications, the number of clusters N may not be an integer multiple of cluster M. Furthermore, if all clusters use the same cluster size, the delayer effect may still exist. This is because the computing power and channel conditions of the learning endpoints can vary significantly. Even within a single cluster, the training latency of the learning endpoints can vary considerably. The delayer effect refers to the edge server needing to wait for model updates from all learning endpoints in the cluster; however, learning endpoints with poor channel conditions or low computing power will severely slow down the entire training process. To mitigate the delayer effect, the number of learning endpoints in each second user cluster is obtained. The number of learning endpoints in each second user cluster is the number of endpoints. If the number of endpoints in each second user cluster is not equal, then multiple second user clusters are treated as multiple target user clusters.

[0119] Understandably, to alleviate the procrastinator effect and further reduce training latency, the step of obtaining the number of user terminals in each second user cluster when the comparison result shows that the target exploration probability is greater than the preset cluster variable further includes: when the number of user terminals in each second user cluster is the same, determining the procrastinating user terminals in each second user cluster based on the user communication information of each user terminal to be learned; randomly associating the procrastinating user terminals in each second user cluster with multiple second user clusters to determine multiple first association results and the first association latency of each first association result; comparing the first association latency of each first association result, and determining a first target result among multiple first association results based on the comparison result; determining multiple third user clusters based on the first target result; and determining multiple target user clusters based on the multiple third user clusters.

[0120] In practical implementation, when the number of user terminals in each second user cluster is the same, the training latency of each cluster can be obtained through calculation, letting... The process involves identifying user clients in each second user cluster that consume longer training times than the average. These user clients are considered "procrastinating" clients. These procrastinating user clients are then continuously associated with other second user clusters, resulting in multiple first association results. The training latency of each first association result (i.e., each associated second user cluster) is calculated, and this training latency is the first association latency. The first association result with the minimum training latency is identified as the first target result. Based on the first target result, the procrastinating user clients are unloaded into their corresponding second user clusters, resulting in multiple third user clusters. These third user clusters are then used to determine multiple target user clusters. For example, for a second user cluster... In this process, after the delayed user terminal 1 is identified, it is continuously associated with other second user clusters. The training latency of the clusters after the delayed user terminal is associated with other second user clusters is calculated, and the association between the delayed user terminal and the second user cluster is determined. The first association during the association process has the lowest latency, at which point the delayed user terminal 1 is assigned to the second user cluster. This results in multiple third-user clusters. It's worth noting that when delaying user client unloading, the original cluster where the delayed user client resides can also be the optimal cluster for achieving minimum latency.

[0121] It should be noted that, in order to determine the accurate target user cluster using multiple third user clusters, the step of determining multiple target user clusters based on multiple third user clusters further includes: determining the fast user terminals in each third user cluster based on the user communication information of each user terminal to be learned; randomly associating the fast user terminals in each third user cluster with multiple third user clusters to determine multiple second association results and the second association delay of each second association result; comparing the second association delay of each second association result, and determining the second target result among the multiple second association results based on the comparison result; and determining multiple target user clusters based on the second target result.

[0122] Understandably, the process involves identifying learning client terminals in each third user cluster that consume longer training time than the average training time, and learning client terminals in each third user cluster that consume shorter training time than the average training time as fast client terminals. These fast client terminals are then continuously associated with other third user clusters to obtain multiple second association results between fast client terminals in each third user cluster and other third user clusters. The training latency of each second association result (i.e., each associated third user cluster) is calculated, and the training latency of each second association result is the second association latency. The second association result corresponding to the minimum training latency is obtained, and this minimum training latency is the second target result. Based on the second target result, the fast client terminals are unloaded into the corresponding third user clusters, thus obtaining multiple target user clusters.

[0123] In the implementation, both fast and slow user clients are offloaded to the correct cluster, thereby reducing training latency and mitigating the negative impact of the procrastinator effect. Furthermore, a Gibbs sampling scheme is used to iteratively obtain multiple target user cluster strategies, adapting to the heterogeneity of the user clients to be learned.

[0124] In this embodiment, multiple first user clusters are obtained by dividing each learning user terminal into clusters according to a random aggregation strategy; a first training delay is determined based on the user communication information of the multiple first user clusters and each learning user terminal; a random sampling strategy is performed in each first user cluster to determine the exchange user terminal and the first user cluster in which the exchange user terminal is located; multiple second user clusters are obtained by randomly associating the interaction user terminal, the first user cluster in which the exchange user terminal is located, and the multiple first user clusters; a first training delay and a second training delay are determined based on the user communication information of the multiple second user clusters and each learning user terminal; and multiple target user clusters are determined based on the first training delay and the second training delay. Through the above method,

[0125] refer to Figure 11 , Figure 11 This is a flowchart illustrating the third embodiment of a cluster-based parallel segmentation learning method of the present invention.

[0126] Based on the above embodiments, this embodiment of the cluster-based parallel segmentation learning method further includes:

[0127] Step S01: Distribute the user-end model to each learning user terminal in the target user cluster, so that each learning user terminal in the target user cluster can perform data sampling in parallel, generate shredded data, and feed back the shredded data and sampling data labels according to the target spectrum resources of each learning user terminal in the target user cluster.

[0128] It should be noted that when the target user cluster is the first cluster to be trained in the preset cluster training order, the user-side model refers to the latest user-side model stored in the edge server. When the target user cluster is not the first cluster to be trained in the preset cluster training order, the user-side model refers to the aggregated user-side model of the previous target user cluster.

[0129] Understandably, the edge server determines the target user cluster for which segmentation learning will begin, and broadcasts the user-end model to each user terminal within that cluster. Each user terminal then concurrently extracts a small batch of data samples from its local dataset, performs forward propagation on the received user-end model, and generates shredded data at the model segmentation layer. Each user terminal, based on its allocated target spectrum resources, transmits the shredded data and the labels of the sampled small batch of data samples to the edge server. These labels are the sampling data labels.

[0130] Step S02: Determine the pulverization data gradient based on the pulverization data and the labels of the sampling data.

[0131] It should be noted that the edge server performs forward propagation of the server-side model based on all the shredded data fed back by each learning user in the target user cluster that has started segmentation learning, and calculates the loss function based on the backpropagation results and mini-batch data labels. The server-side model is updated using the gradient of the loss function and backpropagation. The gradient of the shredded data is calculated in the model segmentation layer, thereby determining the shredded data gradient corresponding to each learning user in the target user cluster that has started segmentation learning.

[0132] Step S03: Send the shredded data gradient to each learning user terminal in each target user cluster, so that each learning user terminal in each target user cluster updates the user terminal model based on the feedback of the shredded data gradient.

[0133] It should be noted that the crushed data gradients corresponding to each learning user terminal in the target user cluster that is currently starting segmentation learning are sent to the corresponding learning user terminal. After receiving the crushed data gradients, each learning user terminal in the target user cluster that is currently starting segmentation learning updates its own user terminal model through backpropagation, and transmits the updated user terminal model to the edge server according to the target spectrum resources of each learning user terminal. The updated user terminal model is the updated user terminal model.

[0134] Step S04: Determine the aggregated client model for each target user cluster based on each updated client model.

[0135] It should be noted that each user client in the target user cluster currently undergoing segmentation learning will submit an updated user client model, which can be represented as follows: Where K is the number of user terminals to be learned in the target user cluster. This is the index number for the user terminal to be learned.

[0136] Understandably, the edge server performs weighted aggregation on all updated client-side models to obtain the aggregated client-side model corresponding to the target user cluster that is currently starting segmentation learning.

[0137] In the specific implementation, in order to ensure the accuracy of the aggregation, the step of determining the aggregated user terminal model of each target user cluster based on each updated user terminal model includes: obtaining the number of user terminals in each target user cluster and the number of user terminal samples to be learned in each target user cluster; and performing weighted aggregation based on the number of user terminals in each target user cluster, the number of user terminal samples to be learned in each target user cluster, and each user update model to obtain the aggregated user terminal model of each target user cluster.

[0138] It should be noted that the number of user terminals refers to the number K of user terminals to be learned in the target user cluster, and the number of samples refers to the total number of samples contained in the local dataset of each user terminal to be learned. The aggregated user terminal model for each target user cluster is obtained by weighted aggregation based on the number of user terminals in each target user cluster, the number of samples of user terminals to be learned in each target user cluster, and the update model for each user. ,in, This indicates the first segment in the target user cluster where segmentation learning is currently beginning. The number of samples from each user terminal to be learned.

[0139] It is understandable that, such as Figure 12As shown, when a target cluster user begins segmentation learning, the edge server distributes the latest user-end model stored in its storage to all user terminals in that target cluster that are to be learned. When all user terminals in the target user cluster receive the user-end model distributed by the edge server, all user terminals in the target user cluster begin to extract a small batch of data samples from their local datasets in parallel, perform forward propagation on the received latest user-end model, and generate shredded data at the model segmentation layer. Each user terminal transmits its own shredded data at the model segmentation layer and the sampled small batch of data labels to the edge server according to the allocated target spectrum resources. The edge server performs forward propagation of the server-side model based on all shredded data sent by all learning clients in the target user cluster. It then calculates the loss function based on the backpropagation results and mini-batch data labels. The server-side model is updated using the gradient of the loss function and backpropagation. The gradient calculation for the shredded data is performed at the model segmentation layer. The edge server sends the gradient of the shredded data to the corresponding learning clients in the target cluster. Upon receiving the shredded data and its gradient, each learning client updates its own client model through backpropagation, thus outputting its updated client model. Each learning client transmits its updated client model to the edge server based on its target spectrum resources. The edge server performs weighted aggregation of all updated client models to obtain the aggregated client model corresponding to the target user cluster. The distribution of client models within the cluster, the forward propagation and shredded data transmission by each learning client, the transmission of shredded data gradients by the edge server, and the backpropagation and updated client model transmission by each learning client are all implemented in parallel.

[0140] In specific implementations, such as Figure 13As shown, upon receiving a model training or segmentation learning start command, the network controller in the edge server begins cluster partitioning, determines multiple target cluster strategies, and, according to a preset cluster training order, initiates parallel segmentation learning for each user terminal to be trained within a target user cluster. It allocates resources to each user terminal to be trained in the current target user cluster, determines the target spectrum resources corresponding to each user terminal to be trained within the current target user cluster, and uses these target spectrum resources to perform parallel segmentation learning within the cluster. After determining the aggregated user terminal model for the current target user cluster, it distributes the aggregated user terminal model to each user terminal to be trained in the next target user cluster for segmentation learning and initiates parallel segmentation learning within the cluster. Training is performed serially between clusters according to a preset cluster training order. Once the last target user cluster completes its corresponding intra-cluster training, the final aggregated target user terminal model is uploaded to the edge server and stored.

[0141] This embodiment distributes the user-end model to each learning user terminal in the target user cluster, enabling each learning user terminal in the target user cluster to perform data sampling in parallel, generating fragmented data, and feeding back the fragmented data and sampling data labels based on the target spectrum resources of each learning user terminal in the target user cluster; determines the fragmented data gradient based on the fragmented data and sampling data labels; sends the fragmented data gradient to each learning user terminal in the target user cluster, so that each learning user terminal in the target user cluster updates its user-end model based on the feedback from the fragmented data gradient; and determines the aggregated user-end model for each target user cluster based on the updated user-end models. Through this method, parallel segmentation learning is initiated within each target user cluster, greatly reducing training latency while ensuring the convergence and accuracy of subsequent models.

[0142] In addition, refer to Figure 14 This invention also proposes a cluster-based parallel segmentation learning device, which includes:

[0143] The acquisition module 10 is used to acquire multiple user terminals to be learned and user communication information of each user terminal to be learned.

[0144] The segmentation module 20 is used to segment each user terminal to be learned into clusters based on each user terminal to be learned and the user communication information of each user terminal to be learned, and to determine multiple target user clusters.

[0145] The learning module 30 is used to perform cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model. The aggregated user terminal model is obtained by performing parallel segmentation learning in each target user cluster based on the target spectrum resources of each user terminal to be learned.

[0146] This embodiment acquires multiple user terminals to be learned and their user communication information; it then divides the user terminals to be learned into clusters based on their communication information to determine multiple target user clusters; and performs serial segmentation learning on the clusters according to the aggregated user terminal model and each target user cluster to obtain the target user terminal model. The aggregated user terminal model is obtained through parallel segmentation learning within each target user cluster based on the target spectrum resources of each user terminal to be learned. By dividing the multiple user terminals to be learned into clusters using the above method, optimal target user clusters are obtained. Serial segmentation learning between clusters is then performed based on the aggregated user terminal model and each target user cluster to obtain the trained target user terminal model. The aggregated user terminal model is obtained through parallel segmentation learning within each target user cluster based on the target spectrum resources of each user terminal to be learned, thus effectively reducing the overall training latency during the segmentation learning process, shortening the overall training time, improving the efficiency of segmentation learning, suppressing the negative impacts of network heterogeneity and dynamism, and ensuring the convergence and accuracy of existing segmentation learning techniques.

[0147] In one embodiment, the partitioning module 20 is further configured to partition each user terminal to be learned into clusters according to a random aggregation strategy to obtain multiple first user clusters;

[0148] The first training delay is determined based on the user communication information of multiple first user clusters and each user terminal to be learned.

[0149] Randomly select from each first user cluster according to a random sampling strategy to determine the exchange user terminal and the first user cluster where the exchange user terminal is located.

[0150] Multiple second user clusters are obtained by randomly associating the interactive user terminal, the first user cluster to which the exchange user terminal is located, and multiple first user clusters.

[0151] The first training delay and the second training delay are determined based on the user communication information of multiple second user clusters and each user terminal to be learned.

[0152] Multiple target user clusters are determined based on the first training delay and the second training delay.

[0153] In one embodiment, the partitioning module 20 is further configured to calculate the exploration probability based on the first training delay and the second training delay, and determine the target exploration probability;

[0154] The target exploration probability is compared with a preset cluster variable to determine the comparison result;

[0155] When the comparison result shows that the target exploration probability is greater than the preset cluster variable, the number of user terminals in each second user cluster is determined.

[0156] When the number of users in each second user cluster is different, each second user cluster is treated as multiple target user clusters.

[0157] In one embodiment, the partitioning module 20 is further configured to determine the delaying user terminal in each second user cluster based on the user communication information of each user terminal to be learned when the number of user terminals in each second user cluster is the same.

[0158] Based on the delayed user terminals in each second user cluster and multiple second user clusters, random associations are performed to determine multiple first association results and the first association delay of each first association result;

[0159] The first association delays of each first association result are compared, and the first target result is determined from multiple first association results based on the comparison results;

[0160] Based on the results of the first objective, multiple third-user clusters are identified;

[0161] Multiple target user clusters are identified based on multiple third-party user clusters.

[0162] In one embodiment, the partitioning module 20 is further configured to determine the fast user terminal in each third user cluster based on the user communication information of each user terminal to be learned;

[0163] Based on the fast user terminals in each third user cluster and multiple third user clusters, random association is performed to determine multiple second association results and the second association delay of each second association result;

[0164] The second association delays of each second association result are compared, and the second target result is determined from multiple second association results based on the comparison results;

[0165] Based on the results of the second objective, multiple target user clusters are identified.

[0166] In one embodiment, the learning module 30 is further configured to determine a first target cluster and a second target cluster according to a preset cluster training order;

[0167] Determine the aggregated user terminal model based on the first target cluster;

[0168] The aggregated client model is sent to the second target cluster so that the second target cluster can feed back the target client model.

[0169] In one embodiment, the learning module 30 is further configured to distribute the user-end model to each user-end to be learned in the target user cluster, so that each user-end to be learned in the target user cluster can perform data sampling in parallel, generate shredded data, and feed back the shredded data and sampling data labels according to the target spectrum resources of each user-end to be learned in the target user cluster.

[0170] The pulverization data gradient is determined based on the pulverization data and the labels of the sampling data;

[0171] The shredded data gradient is sent to each learning user terminal in each target user cluster, so that each learning user terminal in each target user cluster updates its user terminal model based on the feedback of the shredded data gradient.

[0172] The aggregated client model for each target user cluster is determined based on each updated client model.

[0173] In one embodiment, the learning module 30 is further configured to obtain the number of user terminals in each target user cluster and the number of samples of user terminals to be learned in each target user cluster.

[0174] The aggregated user terminal model for each target user cluster is obtained by weighting and aggregating the number of user terminals in each target user cluster, the number of user terminals to be learned in each target user cluster, and the update model of each user.

[0175] In one embodiment, the learning module 30 is further configured to allocate spectrum resources to each user terminal to be learned according to a resource allocation strategy, and determine the initial resources of each user terminal to be learned.

[0176] According to the resource allocation strategy, spectrum resources are allocated to each learning user terminal in each target user cluster to determine the initial resources of each learning user terminal.

[0177] The user training latency of each user terminal to be learned is calculated based on the user communication information and the initial resources of each user terminal to be learned.

[0178] The training latency of each user terminal is sorted, and the target learning user terminal is determined based on the sorting results.

[0179] Spectrum resources are allocated to the target learning user terminal to determine the allocated resources for the target learning user terminal;

[0180] The remaining resource quantity is determined based on the initial resources of each learning user terminal and the allocated resources of the target learning user terminal;

[0181] When the remaining resources are equal to the preset resources, the initial resources of each learning user terminal are updated according to the allocated resources of the target learning user terminal, and the target spectrum resources of each learning user terminal in each target user cluster are determined.

[0182] Since this device adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0183] Furthermore, this embodiment of the invention also proposes a storage medium storing a cluster-based parallel segmentation learning program, which, when executed by a processor, implements the steps of the cluster-based parallel segmentation learning method described above.

[0184] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0185] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0186] In addition, for technical details not described in detail in this embodiment, please refer to the cluster-based parallel segmentation learning method provided in any embodiment of the present invention, which will not be repeated here.

[0187] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0188] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0189] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0190] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A cluster-based parallel segmentation learning method, characterized in that, The cluster-based parallel segmentation learning method includes: Acquire multiple client-side learning targets and user communication information for each client-side learning target; Based on the learning user terminals and their user communication information, the learning user terminals are divided into clusters to determine multiple target user clusters. Based on the aggregated user terminal model and each target user cluster, a cluster serial segmentation learning is performed to obtain the target user terminal model. The aggregated user terminal model is obtained by performing parallel segmentation learning in each target user cluster according to the target spectrum resources of each user terminal to be learned. The step of dividing the learning user terminals into clusters based on the learning user terminals and their user communication information, and determining multiple target user clusters, includes: The learning endpoints are clustered according to a random aggregation strategy to obtain multiple first user clusters; a first training delay is determined based on the user communication information of the multiple first user clusters and each learning endpoint; a random sampling strategy is performed in each first user cluster to determine the exchange endpoint and the first user cluster to which the exchange endpoint belongs; multiple second user clusters are randomly associated based on the exchange endpoint, the first user cluster to which the exchange endpoint belongs, and the multiple first user clusters; a second training delay is determined based on the user communication information of the multiple second user clusters and each learning endpoint. The exploration probability is calculated based on the first training delay and the second training delay to determine the target exploration probability; the target exploration probability is compared with a preset cluster variable to determine the comparison result; when the comparison result is that the target exploration probability is greater than the preset cluster variable, the number of user terminals in each second user cluster is determined. When the number of users in each second user cluster is different, each second user cluster is treated as multiple target user clusters. When the number of user terminals in each second user cluster is the same, the delayed user terminals in each second user cluster are determined based on the user communication information of each user terminal to be learned; the delayed user terminals in each second user cluster are randomly associated with multiple second user clusters to determine multiple first association results and the first association delay of each first association result; the first association delay of each first association result is compared, and a first target result is determined from multiple first association results based on the comparison result; multiple third user clusters are determined based on the first target result; and multiple target user clusters are determined based on the multiple third user clusters.

2. The cluster-based parallel segmentation learning method as described in claim 1, characterized in that, The step of determining multiple target user clusters based on multiple third-party user clusters includes: The fast user terminals in each third user cluster are determined based on the user communication information of each user terminal to be learned. Based on the fast user terminals in each third user cluster and multiple third user clusters, random association is performed to determine multiple second association results and the second association delay of each second association result; The second association delays of each second association result are compared, and the second target result is determined from multiple second association results based on the comparison results; Based on the results of the second objective, multiple target user clusters are identified.

3. The cluster-based parallel segmentation learning method as described in claim 1, characterized in that, The step of performing cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model includes: The first and second target clusters are determined according to the preset cluster training order; Determine the aggregated user terminal model based on the first target cluster; The aggregated client model is sent to the second target cluster so that the second target cluster can feed back the target client model.

4. The cluster-based parallel segmentation learning method as described in claim 1, characterized in that, The cluster-based parallel segmentation learning method further includes: The user-end model is distributed to each learning user terminal in the target user cluster so that each learning user terminal in the target user cluster can perform data sampling in parallel to generate fragmented data, and feed back the fragmented data and sampling data labels according to the target spectrum resources of each learning user terminal in the target user cluster. The pulverization data gradient is determined based on the pulverization data and the labels of the sampling data; The shredded data gradient is sent to each learning user terminal in each target user cluster, so that each learning user terminal in each target user cluster updates its user terminal model based on the feedback of the shredded data gradient. The aggregated client model for each target user cluster is determined based on each updated client model.

5. The cluster-based parallel segmentation learning method as described in claim 4, characterized in that, The step of determining the aggregated user client model for each target user cluster based on each updated user client model includes: Obtain the number of client terminals in each target user cluster and the number of client terminals to be learned in each target user cluster; The aggregated user terminal model for each target user cluster is obtained by weighting and aggregating the number of user terminals in each target user cluster, the number of user terminals to be learned in each target user cluster, and the update model of each user.

6. The cluster-based parallel segmentation learning method as described in any one of claims 1 to 5, characterized in that, Before obtaining the target user terminal model by performing cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster, the process also includes: According to the resource allocation strategy, spectrum resources are allocated to each learning user terminal in each target user cluster to determine the initial resources of each learning user terminal. The user training latency of each user terminal to be learned is calculated based on the user communication information and the initial resources of each user terminal to be learned. The training latency of each user terminal is sorted, and the target learning user terminal is determined based on the sorting results. Spectrum resources are allocated to the target learning user terminal to determine the allocated resources for the target learning user terminal; The remaining resource quantity is determined based on the initial resources of each learning user terminal and the allocated resources of the target learning user terminal; When the remaining resources are equal to the preset resources, the initial resources of each learning user terminal are updated according to the allocated resources of the target learning user terminal, and the target spectrum resources of each learning user terminal in each target user cluster are determined.

7. A cluster-based parallel segmentation learning device, characterized in that, The cluster-based parallel segmentation learning device includes: The acquisition module is used to acquire multiple user terminals to be learned and user communication information of each user terminal; The segmentation module is used to segment the learning user terminals into clusters based on each learning user terminal and the user communication information of each learning user terminal, and to determine multiple target user clusters. The learning module is used to perform cluster serial segmentation learning based on the aggregated user terminal model and each target user cluster to obtain the target user terminal model. The aggregated user terminal model is obtained by performing parallel segmentation learning in each target user cluster based on the target spectrum resources of each user terminal to be learned. The partitioning module is also used to partition each user terminal to be learned into clusters according to a random aggregation strategy, so as to obtain multiple first user clusters. The first training delay is determined based on the user communication information of multiple first user clusters and each user terminal to be learned. Randomly select from each first user cluster according to a random sampling strategy to determine the exchange user terminal and the first user cluster where the exchange user terminal is located. Multiple second user clusters are obtained by randomly associating the switching user terminal, the first user cluster to which the switching user terminal belongs, and multiple first user clusters. The first training delay and the second training delay are determined based on the user communication information of multiple second user clusters and each user terminal to be learned. The exploration probability is calculated based on the first training delay and the second training delay to determine the target exploration probability; The target exploration probability is compared with a preset cluster variable to determine the comparison result; When the comparison result shows that the target exploration probability is greater than the preset cluster variable, the number of user terminals in each second user cluster is determined. When the number of users in each second user cluster is different, each second user cluster is treated as multiple target user clusters. The partitioning module is also used to determine the delaying user terminal in each second user cluster based on the user communication information of each user terminal to be learned when the number of user terminals in each second user cluster is the same. Based on the delayed user terminals in each second user cluster and multiple second user clusters, random associations are performed to determine multiple first association results and the first association delay of each first association result; The first association delays of each first association result are compared, and the first target result is determined from multiple first association results based on the comparison results; Multiple third user clusters are determined based on the first target result; multiple target user clusters are determined based on the multiple third user clusters.

8. A cluster-based parallel segmentation learning device, characterized in that, The cluster-based parallel segmentation learning device includes: a memory, a processor, and a cluster-based parallel segmentation learning program stored in the memory and executable on the processor, wherein the cluster-based parallel segmentation learning program is configured to implement the cluster-based parallel segmentation learning method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a cluster-based parallel segmentation learning program, which, when executed by a processor, implements the cluster-based parallel segmentation learning method as described in any one of claims 1 to 6.

Images