A method, device and storage medium for optimizing federated split learning

Abstract

This application relates to an optimization method, device, and storage medium for federated segmentation learning. The method is used to optimize a pre-defined federated segmentation learning system, which includes a master server, a federated server, and at least one client. The method includes: constructing a client benefit function considering model performance utility, rental cost, and training latency; constructing an optimization problem based on the client benefit function with the goal of maximizing the total benefit of the federated segmentation learning system; solving the optimization problem; and achieving joint optimization of the model segmentation points and client participation in the federated segmentation learning system. Compared with existing technologies, this application effectively solves the key problem of balancing performance and economy in practical deployments, possesses good practicality and scalability, and provides a feasible path for the efficient and low-cost implementation of federated segmentation learning in real-world commercial scenarios.

Description

Technical Field

[0001] This application relates to the field of federated segmentation learning technology, and in particular to an optimization method, apparatus and storage medium for federated segmentation learning. Background Technology

[0002] Split-Federated Learning (SFL) is a privacy-preserving collaborative machine learning paradigm that combines the advantages of Federated Learning (FL) and Split Learning (SL). It splits the machine learning model into two parts: a client-side model and a server-side model. This allows the client to handle only the lightweight initial neural network layers, while offloading the computationally intensive deep networks to a central server. This significantly reduces the memory and computational burden on the client (especially resource-constrained institutions), and the original data and initial feature representations remain local, providing stronger privacy guarantees than centralized training.

[0003] However, in actual deployments, clients need to rent cloud server computing and communication resources for model training and aggregation on the server side, which brings high economic costs. Therefore, how to effectively control rental costs and training latency while ensuring model performance has become a key challenge for the implementation of federated segmentation learning.

[0004] Currently, optimization of federated segmentation learning systems mainly revolves around the following technical approaches: Model segmentation location optimization: This type of method adjusts the segmentation location of the model between the client and server to balance computational load, communication overhead, and model performance. For example, setting the segmentation point closer to the input (…). Smaller size can significantly reduce the computational and memory pressure on the client; while shifting the split point to the back ( While a larger split (larger split) might reduce communication volume for intermediate features, it increases computational and memory pressure on the client. In non-independent and identically distributed (Non-IID) data scenarios, different split positions can significantly impact model convergence performance and even lead to a decrease in model training accuracy. How to choose a suitable split position... The data distribution characteristics and system objectives need to be considered comprehensively. Such methods are usually based on empirical testing or theoretical convergence boundary analysis to select the optimal solution. .

[0005] Client participation or sampling strategy optimization: To alleviate resource pressure and improve training efficiency, a subset of clients can be selected to participate in model updates during each training session. Some solutions reduce communication overhead, balance device heterogeneity, or improve model convergence speed by dynamically selecting a subset of clients participating in training (i.e., client sampling) or controlling the participation level (participation probability) of each client. These strategies are often designed in conjunction with factors such as data distribution and device resource status.

[0006] Client participation incentive mechanisms: Considering that clients may be reluctant to actively participate due to resource consumption or privacy concerns, some studies have introduced client participation incentive mechanisms. For example, through monetary rewards or differentiated model service quality, clients are guided to decide their participation level or whether to participate based on their own cost-benefit trade-offs, thereby indirectly affecting the overall efficiency of the system and improving the probability of client participation and the final model performance.

[0007] System-level parameter joint optimization: Some more complex solutions attempt to adjust multiple system parameters simultaneously, such as co-optimizing model split points with communication bandwidth allocation, local computing frequency, aggregation period, etc., in order to minimize training latency or resource overhead under specific constraints (such as wireless resource limitations and energy consumption limits).

[0008] While the above-mentioned solutions have achieved some success in achieving their respective objectives, they still have fundamental limitations: (1) Fragmented objectives: All existing work focuses on a single optimization objective. For example, some schemes only focus on convergence performance, while others focus on resource allocation under economic incentives or minimizing latency. No scheme can simultaneously and explicitly incorporate model performance, rental costs and training latency into a unified optimization framework.

[0009] (2) Isolated decision variables: Existing solutions usually only optimize the split point. Or optimize only the participation / sampling strategy ignoring and Strong coupling between them. For example, one factor that leads to high latency... Choices may force clients to reduce... To control costs, this interaction effect was not modeled.

[0010] (3) Ignoring rental costs: Most existing work does not consider the rental costs associated with renting servers. In actual commercial deployments, clients need to pay for the use of servers, which is an economic factor that cannot be ignored, but current optimization methods generally lack modeling and consideration of this cost. Summary of the Invention

[0011] Therefore, it is necessary to provide an optimization method, device, and storage medium for federated segmentation learning that can effectively solve the problem of performance and economy being difficult to balance in actual deployment, in order to address the aforementioned technical problems of lack of multi-objective collaborative optimization capabilities, neglect of the coupling relationship of decision variables, and disregard for rental costs.

[0012] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: In a first aspect, the present invention provides an optimization method for federated segmentation learning, used to optimize a preset federated segmentation learning system, the federated segmentation learning system comprising a master server, a federated server, and at least one client, the method comprising: Construct a client benefit function that considers model performance utility, rental cost, and training latency; With the goal of maximizing the total revenue of the federated segmentation learning system, an optimization problem is constructed based on the client revenue function, and the optimization problem is solved to achieve joint optimization of the model segmentation points and client participation of the federated segmentation learning system.

[0013] Secondly, the present invention also provides an electronic device, including a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform steps in an optimization method for federated segmentation learning as described above.

[0014] Thirdly, the present invention also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement an optimization method for federated segmentation learning as described above.

[0015] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention addresses the shortcomings of existing federated segmentation learning optimization methods in multi-objective collaboration and cost modeling, proposing a joint optimization framework oriented towards maximizing system benefits. By constructing a unified benefit function that integrates model performance, rental costs, and training latency, and by achieving collaborative decision-making between model segmentation points and client participation, this invention effectively solves the key challenge of balancing performance and economy in practical deployments. It possesses good practicality and scalability, providing a feasible path for the efficient and low-cost implementation of federated segmentation learning in real-world commercial scenarios. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an optimization method for federated segmentation learning in some embodiments of this application; Figure 2This is a schematic diagram of the SFL-V1 architecture in the federated segmentation learning system architecture of some embodiments of this application; Figure 3 This is a schematic diagram of the SFL-V2 architecture in the federated segmentation learning system architecture of some embodiments of this application; Figure 4 This is a schematic diagram illustrating the optimization problem-solving process of a federated segmentation learning optimization method applicable to SFL-V1 in some embodiments of this application. Figure 5 This is a schematic diagram illustrating the optimization problem-solving process of a federated segmentation learning optimization method applicable to SFL-V2 in some embodiments of this application. Figure 6 The figure shows the comparative experimental results of an optimization method for federated segmentation learning in some embodiments of this application under SFL-V1. Figure 7 The figures show comparative experimental results of an optimization method for federated segmentation learning in some embodiments of this application under SFL-V2. Detailed Implementation

[0017] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses. The term "determine" broadly covers a wide variety of actions, including acquiring, calculating, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), probing, and similar actions; it may also include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), and similar actions; it may also include generating, creating, establishing, and similar actions; and parsing, selecting, choosing, and similar actions, etc. Definitions of other terms will be given in the following description.

[0018] It should be noted that when one element is considered to be "connected" to another element, it can be directly connected to the other element or connected to the other element through an intermediary element. Furthermore, in the following embodiments, "connection" should be understood as "electrical connection," "communication connection," etc., if there is transmission of electrical signals or data between the connected objects.

[0019] It should be emphasized that the acquisition, transmission, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.

[0020] In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0021] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] Example 1 This embodiment provides an optimization method for federated segmentation learning, used to optimize a preset federated segmentation learning system. The federated segmentation learning system includes a master server, a federated server, and at least one client. (See attached document.) Figure 1 The method includes: S1. Construct a client revenue function that considers model performance utility, rental cost, and training latency; S2. With the goal of maximizing the total revenue of the federated segmentation learning system, construct an optimization problem based on the client revenue function, solve the optimization problem, and achieve joint optimization of the model segmentation points and client participation of the federated segmentation learning system.

[0024] Compared to existing technologies, this invention addresses the shortcomings of current federated segmentation learning optimization methods in multi-objective collaboration and cost modeling by proposing a joint optimization framework oriented towards maximizing system benefits. By constructing a unified benefit function that integrates model performance, rental costs, and training latency, and by achieving collaborative decision-making between model segmentation points and client participation, this invention effectively solves the key challenge of balancing performance and economy in practical deployments. It possesses good practicality and scalability, providing a feasible path for the efficient and low-cost implementation of federated segmentation learning in real-world commercial scenarios.

[0025] In some preferred embodiments, a client benefit function is constructed that considers model performance utility, rental cost, and training latency, specifically including: In the federated segmentation learning system, the preset common The global model of the layer is in the first layer. The model is split at the layer level to obtain a client-side model and a server-side model; among them, This set represents the model split points. All possible values, The minimum value of the model segmentation point. This represents the maximum value of the model's segmentation point; No. Client Client engagement Each client participates in each round of training; the client-side reward function is the same for each client, expressed as:

[0026] in, For the client The benefits; Model performance utility is used to represent the value of the trained model to the client; The rental cost required to rent a server; Indicates the client The training latency is determined by the fact that each client trains in parallel and the next round of training cannot begin until all clients have finished their current training round. Therefore, the training time of the client with the longest training time is taken. This serves as the training latency for all clients.

[0027] In some preferred embodiments, the architecture of the federated segmentation learning system is specifically SFL-V1 or SFL-V2; In SFL-V1, the master server stores and independently updates a server-side model for each client, see [link / reference]. Figure 2 ; In SFL-V2, the master server stores and updates a shared server-side model based on data from all clients. (See [link to relevant documentation]). Figure 3 .

[0028] In some preferred embodiments, based on the theoretical convergence boundary of federated segmentation learning, the convergence metric is negative and multiplied by the client. Personal value assessment of the model As a model performance utility ,include: For SFL-V1, model performance utility Represented as:

[0029] For SFL-V2, model performance utility Represented as:

[0030] in, and , respectively, are the convergence metrics for SFL-V1 and SFL-V2, representing the time-weighted average of the expected square norm of the loss function gradient. This metric is an important standard for measuring the average convergence of the model throughout the training process; the smaller the value, the better the model convergence. Represented for all clients A set; Indicates the total number of training rounds; Indicates the first Learning rate during training rounds; For mathematical expectation; The square of the Euclidean norm; For gradient operators; Indicates the first During each training round, based on the current model parameters Calculated global loss function The gradient; This represents the mathematical expectation of the squared norm of the gradient mentioned above; Based on the cost of renting the main server Cost of federal servers Get Client rental costs ,include: For SFL-V1, rental costs Represented as:

[0031] For SFL-V2, rental costs Represented as:

[0032] in, and These are the rental fees per unit of computing power for the main server and the federation server, respectively. and These represent the computational requirements for the primary server and the federation server, respectively, in FLOPs. For clients under SFL-V1 Corresponding model segmentation points; For model split points in SFL-V2; client The required computation can be expressed in terms of participation. Multiply by computational cost and express; Based on client-side model training time Main server model training time Aggregation time with federated server model Get Client Training latency ,include: For SFL-V1, the client Training latency Represented as:

[0033] For SFL-V2, the client Training latency Represented as:

[0034] in, , and They are the master server, the federated server, and the client. The computational speed, measured in FLOPS; , and They are the master server, the federated server, and the client. The amount of computation required.

[0035] In some preferred embodiments, the optimization problem constructed based on the client-side revenue function under SFL-V1 specifically includes: For SFL-V1, each client It has an independent server-side model, so it can choose its own split points. and participation Client The goal is to maximize one's own benefits. A non-cooperative game model of the client is constructed to form the first optimization problem, which is expressed as:

[0036]

[0037] in, Represented as a set of integers; Because the profit functions of all clients are coupled together (through...) Affecting the convergence boundary, through (Affecting training latency), this problem constitutes a game between clients. In this game, each client makes independent decisions, and their strategy affects the payoffs of other clients. The equilibrium solution of this game satisfies a Nash equilibrium, meaning that in this state, no client can improve its payoff by unilaterally changing its own strategy.

[0038] The first optimization problem is simplified as follows: An approximate representation of the convergence boundary is given as follows:

[0039] in, This represents the optimal global loss value. Total number of clients; For the smoothness parameter of the client-side local loss function; For client-side model aggregation cycle; For server-side model aggregation cycle; ; For the client Aggregate weights; For the client The upper bound of the stochastic gradient variance represents the client The gradient estimate is affected by the noise level caused by random sampling; This is the upper bound of the deviation between the local gradient and the global gradient, representing the degree of difference in data distribution among clients; Because the objective function contains non-smooth terms Introducing auxiliary variables Remove the non-smooth terms from the objective function Convert to constraints To make the objective function of the first optimization problem differentiable, and by omitting constant terms in the objective function that are irrelevant to the decision variables, we obtain the simplified first optimization problem, which is expressed as:

[0040]

[0041]

[0042] in, For the simplified client The objective function value can represent the client's... The benefits, The smaller the value, the higher the return. ; For the set of all clients, .

[0043] In some preferred embodiments, the optimization problem constructed based on the client-side revenue function under SFL-V2 specifically includes: For SFL-V2, a server-dominated two-stage Stackelberg game model is constructed to form a second optimization problem, which is expressed as: Phase 1: Server selection Minimizing the global convergence metric can be represented as the following server optimization subproblem:

[0044]

[0045] in, The server objective function value represents the model's convergence. The smaller the value, the better the model convergence. Phase Two: Given Each client choose To maximize its own benefit, the following client optimization subproblem can be represented:

[0046]

[0047] The second optimization problem is simplified as follows: An approximate representation of the convergence boundary is given as follows:

[0048] Because the objective function contains non-smooth terms Introducing auxiliary variables Remove the non-smooth terms from the objective function Convert to constraints To make the objective function of the second optimization problem differentiable, and by omitting constant terms in the objective function that are irrelevant to the decision variables, we obtain the simplified second optimization problem, which is expressed as: The simplified server optimization subproblem is represented as:

[0049]

[0050] in, The simplified server objective function value can represent the model convergence. The smaller the value, the better the model convergence. The simplified client-side optimization subproblem is represented as:

[0051]

[0052] in, .

[0053] In some preferred embodiments, solving the optimization problem achieves joint optimization of the model segmentation points and client engagement in the federated segmentation learning system, thereby determining the optimal model segmentation point and optimal client engagement for each client. Specifically, this includes: Let the sum of the revenue from all clients equal the total revenue of the system. , A smaller value indicates a greater benefit; therefore, an optimization method based on gradient descent is used to update the value. ; In the In this iteration, the server uses the current and previous rounds of total system revenue to estimate the objective function with respect to... The gradient of is expressed as:

[0054] Use the optimizer to perform gradient descent updates, iterating continuously until... convergence; The discrete model segmentation points are processed using a traversal search method to obtain the optimal model segmentation point, specifically including: because For finite discrete variables, traversal is used. All possible values ​​can be obtained.

[0055] For SFL-V1, each client iterates on its local machine. All possible values, combined with the current and Choose what benefits you the most. ; For SFL-V2, the server iterates... All possible values, and based on feedback from all clients. Choose the one that minimizes the global convergence metric. ; Client engagement Derive the closed-form solution constrained by time delay, and use the closed-form solution to calculate the time-delay-constrained solution. The optimal client engagement degree under the constraint specifically includes: For SFL-V1, the closed-form solution is expressed as:

[0056]

[0057] For SFL-V2, the closed-form solution is expressed as:

[0058]

[0059] in, This represents the optimal solution to the objective function under no-delay constraints. To meet the time delay constraint Upper bound of participation under given conditions; The optimal client engagement is expressed as: .

[0060] In some preferred embodiments, the optimizer is specifically the Adam optimizer (or other suitable optimizer).

[0061] For the specific implementation process, please refer to the solution procedure for the SFL-V1 optimization problem. Figure 4 Specifically, it includes: In SFL-V1, each client has an independent server-side model and can independently choose its split points. and participation This constitutes a client-side game, as follows: Phase 1 (Initialization without latency constraints): Each client locally iterates through all possible split points. For each candidate split point, the client calculates the optimal participation degree without latency constraints and selects the strategy combination that maximizes its own benefit. The server collects the initial strategies of all clients and calculates the initial maximum training latency accordingly. This serves as the starting point for subsequent optimizations.

[0062] Phase Two (Iterative Optimization with Time Delay Constraints): Using the results obtained in Phase One Initialize the Adam optimizer. The server uses the Adam optimizer to update... and the latest The value is broadcast to all clients. Each client receives it. Then, re-traverse all split points locally; for each split point, combine the current... The optimal participation level, constrained by latency, is calculated, and the client's own benefit is evaluated to ultimately select the optimal strategy. The server collects the new strategies from all clients for use in the next round. The update process is iterated repeatedly until... convergence.

[0063] Phase 3 (Decision Determination): After the algorithm converges, each client no longer updates its strategy. The final strategy combination constitutes the Nash equilibrium of the client game, which can be directly used for subsequent SFL training and deployment.

[0064] For the solution process of the SFL-V2 optimization problem, please refer to... Figure 5 Specifically, it includes: In SFL-V2, all clients share a single server-side model, and the split point... The decision is made uniformly by the server; the client can only select the level of participation. This constitutes a Stackelberg game. A two-layer structure of "outer traversal + inner optimization" is used to solve it. Outer traversal (server decision space traversal): The server traverses every possible common partition point. .

[0065] Inner optimization (finding the best client response): For the current given... To perform a complete client response optimization process: First, each client calculates its initial participation degree without latency constraints, and the server initializes accordingly. Subsequently, the server underwent iterative updates. And broadcast it, so all clients can calculate and update their optimal participation level under latency constraints, until that... Below Convergence. This process yields the client's response at the... The best response.

[0066] Final decision and deployment: The server compares all candidates The optimal client response obtained during inner-layer optimization is selected to optimize the global objective. This serves as the final dividing point. Subsequently, the server will... The broadcast is sent to all clients, and each client directly uses its own... The pre-calculated optimal participation level , and complete joint optimization deployment.

[0067] This method, for the first time, unifies the modeling of three major objectives—model performance (based on convergence boundaries), server rental costs, and model training latency—and jointly optimizes two strongly coupled variables: model split points and client engagement. This not only overcomes the fundamental limitations of existing solutions that "optimize only a single objective" and "isolate decision variables," but also explicitly introduces the generally neglected rental costs. Thus, while ensuring model accuracy, it significantly improves the overall system benefit, providing a complete solution that combines theoretical rigor and economic feasibility for the practical deployment of federated segmentation learning in cost-sensitive cross-institutional scenarios such as healthcare and finance.

[0068] Example 2 To verify the effectiveness, feasibility, and technical advantages of the present invention, this embodiment further provides a comparative simulation experiment based on embodiment 1. A complete simulation experiment platform was built and the above method was implemented using the Python programming language, and multiple sets of comparative experiments were conducted in a typical federated segmentation learning system scenario.

[0069] Experimental scenarios involving 10 heterogeneous clients, 1 master server, and 1 federated server were simulated for SFL-V1 and SFL-V2 architectures respectively, with client data conforming to Non-IID.

[0070] There is currently no joint optimization algorithm for performance, cost, and latency in the existing technology. Therefore, a comparative experiment is conducted between the method of this invention and a comparative method. The comparative method is a random selection algorithm for split points and participation, and its policy generation method is as follows: First, a normal distribution with the same mean and variance as the output policy of this invention is constructed; then, random sampling is performed based on this distribution, and the obtained split points and participation are used as the final policy output. The experiment applies the policy obtained by the automatic decision-making of this invention and the policy obtained by the random selection algorithm for split points and participation to the federated segmentation learning system for model training. This experiment uses the MNIST dataset with LeNet-5 model and CIFAR-100 with ResNet-18 model under both SFL-V1 and SFL-V2 architectures, respectively. The experimental results are calculated from the total system benefit ( The proposed technical solution is evaluated using two dimensions: model accuracy and model performance. , A smaller value indicates a higher total system benefit, while a larger accuracy indicates better model performance. Experimental results are as follows: Figure 6 and Figure 7 As shown, under different architectures, datasets, and models, the total system reward of the federated segmentation learning system using the strategy of this invention is higher than that of the system using the random selection algorithm for segmentation points and participation, and the model performance is similar without significant degradation. This verifies the effectiveness, feasibility, and universality of the optimization method proposed in this invention.

[0071] It is understood that the simulation experiment in this embodiment utilizes the method of Embodiment 1 above, and the optional options in Embodiment 1 above are also applicable to this embodiment, so they will not be described again here.

[0072] Example 3 This embodiment provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor, causing the processor to perform some or all of the steps of the method provided in Embodiment 1 of this application.

[0073] It is understood that the storage medium can be transient or non-transient. Exemplarily, the storage medium includes, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] By way of example, the processor may be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0075] By way of example, the read-only memory includes, but is not limited to, MASK ROM, PROM, EPROM, EEPROM, Flash, etc.

[0076] By way of example, the random access memory includes, but is not limited to, DRAM, SRAM, SDRAM, DDR SDRAM, etc.

[0077] In some examples, a computer program product is provided, which can be implemented by hardware, software, or a combination thereof. As a non-limiting example, the computer program product can be embodied in the storage medium, or it can be embodied in a software product, such as an SDK (Software Development Kit).

[0078] As a non-limiting example, a computer program product is provided, comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium, and executes the computer-executable instructions, causing the electronic device to perform some or all of the steps of the method described in the embodiments of this application.

[0079] In some examples, a computer program is provided, including computer-readable code, wherein, when the computer-readable code is run in a computer device, a processor in the computer device performs some or all of the steps for implementing the method.

[0080] This embodiment also proposes an electronic device, including a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. When the processor executes the at least one instruction, at least one program, code set, or instruction set, it implements some or all of the steps of the method described in Embodiment 1.

[0081] In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory, and a communication interface; wherein the processor typically controls the overall operation of the electronic device; the communication interface is used to enable the electronic device to communicate with other terminals or servers via a network; the memory is configured to store instructions and applications executable by the processor, and may also cache data to be processed or already processed (including but not limited to image data, audio data, voice communication data, and video communication data) to be processed by the processor and various modules in the electronic device, and may be implemented using flash memory (FLASH), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or random access memory (RAM).

[0082] A processor may include one or more processing elements. Therefore, a processor may include one or more integrated circuits (ICs) configured to perform the functions of the processor. Furthermore, each integrated circuit may include circuitry (e.g., a first circuit, a second circuit, and other circuitry) configured to perform the functions of the processor.

[0083] Furthermore, data can be transferred between the processor, communication interface, and memory via a bus, which can include any number of interconnected buses and bridges, connecting various circuits of one or more processors and memories together.

[0084] It is understood that the options in Embodiment 1 above also apply to this embodiment, so they will not be described again here.

[0085] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this application. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0086] In different specific implementations, the methods or systems described in this application can be implemented in software, hardware, or a combination thereof. Furthermore, the order of the method steps can be changed, and various elements can be added, reordered, combined, omitted, or modified.

[0087] Obviously, the above embodiments of this application are merely examples for clearly illustrating this application, and are not intended to limit the implementation of this application, nor are they intended to limit this application. For those skilled in the art, other variations or modifications can be made based on the above description. The separate structural / functional modules or units can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. The structure and function of the separate components can be implemented as a combined structure or component. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of the claims of this application.

Claims

1. An optimization method for federated segmentation learning, used to optimize a preset federated segmentation learning system, the federated segmentation learning system comprising a master server, a federated server, and at least one client, characterized in that, include: Construct a client revenue function that considers model performance utility, rental costs, and training latency, specifically including: In the federated segmentation learning system, the preset common The global model of the layer is in the first layer. The model is split at the layer level to obtain a client-side model and a server-side model; among them, , The minimum value of the model segmentation point. This represents the maximum value of the model's segmentation point; No. Client Based on client engagement Each client participates in each round of training; the client-side reward function is the same for each client, expressed as: in, For the client The benefits; For model performance utility; For rental costs; Indicates client The training latency is determined by the training time of the client with the longest duration. As the training latency for all clients; With the goal of maximizing the total revenue of the federated segmentation learning system, an optimization problem is constructed based on the client revenue function, and the optimization problem is solved to achieve joint optimization of the model segmentation points and client participation of the federated segmentation learning system. Specifically, the architecture of the federated segmentation learning system is SFL-V1 or SFL-V2. In SFL-V1, the master server stores and updates a server-side model for each client independently; In SFL-V2, the master server stores and updates a shared server-side model based on data from all clients. Based on the theoretical convergence boundary of federated partitioning learning, the convergence metric is negative and multiplied by the client. Personal value assessment of the model As a model performance utility ,include: For SFL-V1, model performance utility Represented as: For SFL-V2, model performance utility Represented as: in, and These are the convergence metrics for SFL-V1 and SFL-V2, respectively. Represented for all clients A set; Indicates the total number of training rounds; Indicates the first Learning rate during training rounds; For mathematical expectation; The square of the Euclidean norm; For gradient operators; Indicates the first During each training round, based on the current model parameters Calculated global loss function The gradient; The mathematical expectation of the squared norm of the gradient; Based on the cost of renting the main server Cost of federal servers Get Client rental costs ,include: For SFL-V1, rental costs Represented as: For SFL-V2, rental costs Represented as: in, and These are the rental fees per unit of computing power for the main server and the federation server, respectively. and These represent the computational requirements for the primary server and the federation server, respectively, in FLOPs. For clients under SFL-V1 Corresponding model segmentation points; These are the model split points in SFL-V2; Based on client-side model training time Main server model training time Aggregation time with federated server model Get Client Training latency ,include: For SFL-V1, the client Training latency Represented as: For SFL-V2, the client Training latency Represented as: in, , and They are the master server, the federated server, and the client. The computational speed, measured in FLOPS; , and They are the master server, the federated server, and the client. The amount of computation required.

2. The optimization method for federated segmentation learning according to claim 1, characterized in that, The optimization problem based on the client-side revenue function in SFL-V1 specifically includes: For SFL-V1, a client-side non-cooperative game model is constructed to form the first optimization problem, which is expressed as: in, Represented as a set of integers; The first optimization problem is simplified as follows: An approximate representation of the convergence boundary is given as follows: in, This represents the optimal global loss value. Total number of clients; For the smoothness parameter of the client-side local loss function; For client-side model aggregation cycle; For server-side model aggregation cycle; ; For the client Aggregate weights; For the client The upper bound of the variance of the stochastic gradient; This is the upper bound of the deviation between the local gradient and the global gradient; By introducing auxiliary variables Making the objective function of the first optimization problem differentiable yields the simplified first optimization problem, which is expressed as: in, For the simplified client The objective function value; ； For the set of all clients, .

3. The optimization method for federated segmentation learning according to claim 2, characterized in that, The optimization problem based on the client-side revenue function in SFL-V2 specifically includes: For SFL-V2, a server-dominated two-stage Stackelberg game model is constructed to form a second optimization problem, which is expressed as: Phase 1: Server selection Minimizing the global convergence metric can be represented as the following server optimization subproblem: in, The server's objective function value; Phase Two: Given Each client choose To maximize its own benefit, the following client optimization subproblem can be represented: The second optimization problem is simplified as follows: An approximate representation of the convergence boundary is given as follows: By introducing auxiliary variables Making the objective function of the second optimization problem differentiable yields the simplified second optimization problem, which is expressed as: The simplified server optimization subproblem is represented as: in, The simplified server objective function value; The simplified client-side optimization subproblem is represented as: in, .

4. The optimization method for federated segmentation learning according to claim 3, characterized in that, Solving the optimization problem involves jointly optimizing the model split points and client engagement of the federated segmentation learning system, thereby determining the optimal model split point and optimal client engagement for each client. Specifically, this includes: Let the sum of the revenue from all clients equal the total revenue of the system. An optimization method based on gradient descent is used to update... ; In the In this iteration, the server uses the current and previous rounds of total system revenue to estimate the objective function with respect to... The gradient of is expressed as: Use the optimizer to perform gradient descent updates, iterating continuously until... convergence; The discrete model segmentation points are processed using a traversal search method to obtain the optimal model segmentation point, specifically including: For SFL-V1, each client iterates on its local machine. All possible values, combined with the current and Choose the option that maximizes your own benefit. ; For SFL-V2, the server iterates... All possible values, and based on feedback from all clients. Choose the one that minimizes the global convergence metric. ; Client engagement Derive the closed-form solution constrained by time delay, and use the closed-form solution to calculate the time-delay-constrained solution. The optimal client engagement degree under the constraint specifically includes: For SFL-V1, the closed-form solution is expressed as: For SFL-V2, the closed-form solution is expressed as: in, This represents the optimal solution to the objective function under no-delay constraints. To meet the time delay constraint Upper bound of participation under given conditions; The optimal client engagement is expressed as: 。 5. The optimization method for federated segmentation learning according to claim 4, characterized in that, The optimizer is specifically the Adam optimizer.

6. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform the steps of the method as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the method as described in any one of claims 1-5.

Images