Distributed learning method and system for generating variable-length action sequence

Abstract

The invention provides a distributed learning method and system for variable-length action sequence generation. A rescue node sends a rescue action request to a nearby edge server according to collected state information; then the edge server sends the received action request to the cloud server, the cloud server makes a decision by using a sequence pointer graph network model after receiving the action request from the edge server to generate an action sequence with a determined step length, and feeds back a result to the edge server; and the edge server allocates resources for each step in the action sequence after receiving the action sequence returned by the cloud server, generates a final rescue action scheme by synthesizing the action sequence and a resource allocation result, issues the final rescue action scheme to the rescue node, and stores the record as a model training data set. Through the scheme of the invention, the privacy of the user is protected, and the efficiency and reliability of task resource allocation are improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a distributed learning method and system for generating variable-length action sequences. Background Art

[0002] In the field of emergency rescue, effective mission planning and resource allocation are key to ensuring the success of rescue operations. In emergency rescue mission scenarios, we face emergencies caused by natural disasters, accidents, or other sudden events, and we need to respond quickly and coordinate rescue operations. In this case, multiple participants such as rescuers and medical personnel need to work closely together to perform a series of complex tasks to rescue the victims and reduce losses. Timely perception of the occurrence of events, determination of the specific steps and division of labor for the rescue operation, and timely notification of relevant parties to initiate the rescue process, followed by resource allocation and management, taking into account factors such as the urgency of the task, geographical location, and resource requirements, to ensure that various resources such as manpower, materials, and transportation can be effectively allocated to each task point, and flexible adjustments can be made according to actual conditions.

[0003] Traditionally, these task planning tasks are often performed by central servers, but this centralized approach can face problems such as network latency and bandwidth limitations. With the rise of edge computing technology, edge devices can process tasks locally, greatly improving task response speed and system robustness. However, many challenges still exist in this context: First, due to limited computing resources, task planning and model training performed solely on edge devices are inefficient, and model performance is difficult to achieve ideal levels. Second, transmitting data from edge devices to the cloud for joint training incurs significant network communication overhead, placing a heavy burden on the system. Finally, transmitting user data to the cloud for training may also bring the risk of user privacy leakage. Summary of the Invention

[0004] To address the above problems, the present invention combines hierarchical task planning and federated learning, and proposes a distributed learning method and system for generating variable-length action sequences. By centrally determining action sequences for tasks on cloud servers and allocating resource nodes to action sequences on the edge, hierarchical execution of tasks is achieved, thereby improving the efficiency and reliability of task resource allocation.

[0005] In one aspect, the present invention provides a distributed learning method for generating variable-length action sequences, the method comprising:

[0006] Based on the collected status information, the rescue node sends a rescue action request to the nearest edge server to quickly initiate the rescue operation. The edge server then sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server uses the sequence pointer graph network model to make decisions and generate an action sequence with a certain step length. The result is fed back to the edge server. After receiving the action sequence returned by the cloud server, the edge server allocates resources for each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan. It is then sent to the rescue node, and the edge server stores this record as a model training dataset.

[0007] A personalized local model is generated for each edge server, namely the resource pointer graph network model. The resource pointer graph network model and the sequence pointer graph network model are both graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.

[0008] Furthermore, the cloud server uses a sequence pointer graph network model to generate action sequences, including: building a knowledge graph based on existing data and domain knowledge, the knowledge graph contains various entities (such as locations, resources, tasks, etc.) and the relationships and attributes between them, training a sequence pointer graph network model based on the data of the knowledge graph, and then using the sequence pointer graph network model to generate action sequences.

[0009] Furthermore, the edge server uses the generated data to train and update the local model. For cases where data samples cannot be shared directly, the federated distillation method is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, the federated aggregation method is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge and updates the local model through the global model.

[0010] The present invention also provides a distributed learning system for generating variable-length action sequences. The system includes a rescue node, an edge server, and a cloud server. The rescue node, the edge server, and the cloud server respectively implement the distributed learning method described above.

[0011] In another aspect, the present invention provides a distributed learning method for generating variable-length action sequences, the method comprising:

[0012] The rescue node sends a rescue action request to the nearest edge server based on the collected status information, quickly initiating the rescue action. The edge server then sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server initiates a request to the third-party server to generate a variable-length action sequence. The third-party server generates a determined rescue action sequence based on the large model according to real-time data and task requirements, and feeds the result back to the cloud server. The cloud server transmits the generated action sequence to the edge server. The edge server then allocates resources for each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan, which is then sent to the rescue node. The edge server stores this record as a model training dataset.

[0013] A personalized local model is generated for each edge server, namely the resource pointer graph network model. The resource pointer graph network model and the sequence pointer graph network model are both graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.

[0014] Furthermore, GPT is used as a pre-trained large-scale language model to learn language knowledge and patterns through a large-scale corpus, and then through fine-tuning, the model parameters are adjusted to adapt to the task requirements for specific rescue mission areas.

[0015] Furthermore, the edge server uses the generated data to train and update the local model. For cases where data samples cannot be shared directly, the federated distillation method is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, the federated aggregation method is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge and updates the local model through the global model.

[0016] The present invention also provides a distributed learning system for generating variable-length action sequences. The system includes a rescue node, an edge server, a cloud server, and a third-party server. The rescue node, the edge server, the cloud server, and the third-party server respectively implement the distributed learning method described above.

[0017] The present invention is aimed at emergency rescue tasks in rescue scenarios. It combines hierarchical task planning and federated learning to construct a hierarchical distributed network architecture including rescue nodes, edge servers and cloud servers. By centrally determining the action sequence for the rescue task on the cloud server and allocating resource nodes to the action sequence on the edge, hierarchical execution of tasks is achieved, and the computing resources of the edge devices are used to improve execution efficiency. At the same time, federated aggregation and federated distillation technologies are used to update the local pointer graph network model, protecting user privacy while optimizing and updating the model, thereby improving the efficiency and reliability of task resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 is a flow chart of a distributed learning method for generating variable-length action sequences provided by an embodiment of the present invention;

[0020] Figure 2 is a signaling diagram of a distributed learning method for generating variable-length action sequences provided by an embodiment of the present invention;

[0021] Figure 3 is a flow chart of a distributed learning method for generating variable-length action sequences provided by an embodiment of the present invention;

[0022] Figure 4 is a signaling diagram of a distributed learning method for generating variable-length action sequences provided by an embodiment of the present invention;

[0023] Figure 5 This is a flow chart of the implementation of federated distillation provided by an embodiment of the present invention;

[0024] Figure 6 This is a signaling diagram for implementing federated distillation according to an embodiment of the present invention;

[0025] Figure 7 This is a flowchart of a federated aggregation implementation provided by an embodiment of the present invention;

[0026] Figure 8 This is a signaling diagram for federated aggregation implementation provided by an embodiment of the present invention;

[0027] Figure 9 This is an architectural diagram of a distributed learning system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0029] In order to improve the efficiency and reliability of rescue mission resource allocation, the present invention proposes a distributed learning method and system for variable-length action sequence generation.

[0030] On the one hand, if Figure 1 and Figure 2 As shown, the present invention provides a distributed learning method for generating variable-length action sequences, comprising:

[0031] Based on the collected status information, the rescue node sends a rescue action request to the nearest edge server to quickly initiate the rescue action; the edge server then sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server uses the sequence pointer graph network model to make decisions and generate an action sequence with a certain step length, and feeds the results back to the edge server. After receiving the action sequence returned by the cloud server, the edge server allocates resources to each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan, which is sent to the rescue node. The edge server stores this record as a model training dataset, where the status information includes information such as the location of the rescue node itself and the purpose of the action. The sequence pointer graph network model is a graph convolutional network.

[0032] The cloud server uses the sequence pointer graph network model to generate action sequences, including: building a knowledge graph based on existing data and domain knowledge. The knowledge graph contains various entities (such as locations, resources, tasks, etc.) and the relationships and attributes between them. A sequence pointer graph network model is trained based on the data of the knowledge graph, and then the sequence pointer graph network model is used to generate action sequences; the knowledge graph provides semantic information and constraints. In the rescue mission, the knowledge graph contains various entities and the relationships between them. By searching the knowledge graph for entities and relationships related to the input task, additional semantic guidance is provided for action sequence generation. If the task involves performing an action at a specific location, the location information in the knowledge graph can help the model generate a suitable action sequence. The knowledge graph converts the input data into a graph, and then the sequence pointer graph network model implements sequence generation based on the graph.

[0033] A personalized local model is generated for each edge server, namely the resource pointer graph network model. The resource pointer graph network model and the sequence pointer graph network model are both graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.

[0034] On the other hand, Figure 3 and Figure 4 As shown, the present invention provides a distributed learning method for generating variable-length action sequences, comprising:

[0035] The rescue node sends a rescue action request to the nearest edge server based on the collected status information to quickly initiate the rescue action; the edge server then sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server initiates a request to the third-party server to generate a variable-length action sequence. The third-party server generates a determined rescue action sequence based on the large model according to real-time data and task requirements, and feeds the results back to the cloud server. The cloud server transmits the generated action sequence to the edge server. The edge server then allocates resources for each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan, which is sent to the rescue node. The edge server stores this record as a model training data set, where the status information includes information such as the location of the rescue node itself and the purpose of the action.

[0036] A personalized local model is generated for each edge server, namely the resource pointer graph network model. The resource pointer graph network model and the sequence pointer graph network model are both graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.

[0037] Large models incorporate the ability to understand and plan complex scenarios, comprehensively considering global information to make more intelligent decisions. First, they use pre-trained large-scale language models, such as GPT, to learn language knowledge and patterns from a large corpus. Then, through fine-tuning, the model parameters are adjusted to suit specific rescue mission requirements. Such models can generate a corresponding action sequence, including possible action plans and response strategies, based on the input rescue mission description.

[0038] Furthermore, how to solve the data sharing or model sharing of edge nodes is an important issue in federated learning. Because in a distributed scenario, the data features in each edge server are inconsistent, it is impossible to directly share data samples to train the local model, and it is also difficult to adapt the directly shared model to the local model features of the edge server. To address this problem, the present invention proposes a local model generation method to generate a personalized model for each edge server, namely a resource pointer graph network model. The resource pointer graph network model is a graph convolutional network. The edge server uses the resource pointer graph network to select and schedule resource nodes. These local models may be customized according to the specific environment, resources and requirements of the server to ensure optimal performance and adaptability.

[0039] The edge server uses the generated data to train and update the local model. For the problem that data samples cannot be shared directly, the features of other data are learned through federated distillation. For models with the same data samples, global knowledge is learned through federated aggregation, which reduces data transmission and improves model accuracy.

[0040] Consider the following scenario: the data in the edge servers have the same data characteristics, the edge servers have sufficient storage and computing capabilities, and federated distillation is used to learn other model data. Figure 5 and Figure 6 As shown in the figure, in this scenario, the rescue node first initiates an action request to the edge server based on the collected status information. The edge server's resource pointer graph network model then makes a resource scheduling decision based on the status and sends it to the rescue node. Each edge server uses local data records to train and update its own resource pointer graph network model and shares the model parameters with other edge servers through federated distillation without sharing the original data. During the federated distillation process, only the gradients or features of the model parameters are exchanged between edge servers, thus protecting data privacy. In this way, edge servers can use federated distillation to update the model without sharing the original data, while ensuring data privacy.

[0041] In this embodiment, it is assumed that there is no trusted central cloud node between edge servers. First, the rescue node initiates an action request to the edge server based on the collected status information, and then initiates an action sequence decision request to the cloud server. The cloud server feeds back the action sequence decision result to the edge server. Then, the resource pointer graph network model of the edge server makes a resource scheduling decision based on the status information and sends it to the rescue node, and stores the record as a model training data set. In the model update phase, the global teacher model shares sampled data with the local decision model, and distills it according to the calculated loss. The local decision model then distills the local policy updated by the teacher model. After a period of training and updating, the teacher model is broadcast to other edge servers. The edge server uses the teacher model to update the local resource pointer graph network, uses the local resource pointer graph network to complete resource element allocation, and generates the final action plan.

[0042] In federated aggregation, each edge server can upload its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge and updates the local model through the global model. This approach can reduce the amount of data transmission while improving the overall performance and accuracy of the model.

[0043] Consider the following scenario: there is no trusted central cloud node between edge servers, the edge servers have sufficient storage and computing capabilities, and global knowledge is learned using federated aggregation. Figure 7 and Figure 8 As shown, in this scenario, the rescue node first initiates a guidance request to the edge server based on the collected status information. The edge server then uses local data records to train and update the local resource pointer graph network model. It then updates the global model and broadcasts it to other edge servers. After receiving the broadcasted global model, the other edge servers aggregate the model parameters to obtain a new global model. In this way, edge servers train and update models using local data, while simultaneously updating the local resource pointer graph network through aggregation, achieving resource scheduling and federated aggregated model updates.

[0044] In this embodiment, it is assumed that there is no trusted central cloud node between edge servers. First, the rescue node initiates an action request to the edge server based on the collected status information, and then initiates a sequence decision request to the cloud server. The cloud server feeds back the action sequence decision result to the edge server. Then, the local resource pointer graph network model of the edge server makes a resource scheduling decision based on the status information and sends it to the rescue node, and stores the record as a model training data set. During the model training phase, the local resource pointer graph network is trained and updated on the data set, and the local resource pointer graph network is passed on for weighted average aggregation to obtain a global model, and the updated global model is broadcast to other edge servers. Finally, the edge server updates the local resource pointer graph network model based on the global model, and at the same time updates the new global model. The local resource pointer graph network is used to complete the resource element allocation and generate the final action plan.

[0045] On the other hand, the present invention provides a hierarchical distributed network system jointly constructed by multiple edge servers and cloud servers, such as Figure 9 As shown in the figure, for large-scale model decision-making, the system includes rescue nodes, edge servers, cloud services, and third-party servers. For sequence pointer graph network decision-making, the system includes rescue nodes, edge servers, and cloud services. The system implements the distributed learning method described above by distributing functions such as task allocation, resource scheduling, and action sequence determination to different server layers. The cloud provides global planning and decision-making support, the edge is responsible for flexible task execution and resource scheduling, and the terminal device is the executor of the actual action. This layered design effectively solves the bottlenecks and limitations of traditional centralized systems, improves the response speed and efficiency of emergency rescue tasks, and provides more reliable support for emergency rescue work. This architecture can achieve efficient task execution and rational resource utilization, providing more reliable support for emergency rescue work.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed learning method for variable-length action sequence generation, characterized in that: The method comprises: The rescue node sends a rescue action request to the nearest edge server based on the collected status information to quickly start the rescue action; the edge server then sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server uses the sequence pointer graph network model to make decisions to generate an action sequence with a certain step length, and feeds the result back to the edge server. After receiving the action sequence returned by the cloud server, the edge server allocates resources for each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan, which is sent to the rescue node. The edge server stores the record as a model training data set. A personalized local model is generated for each edge server, namely the resource pointer graph network model. The resource pointer graph network model and the sequence pointer graph network model are both a kind of graph convolutional network. The edge server uses the resource pointer graph network to select and schedule resource nodes.

2. The distributed learning method according to claim 1, characterized in that: The cloud server uses the sequence pointer graph network model to generate action sequences, including: building a knowledge graph based on existing data and domain knowledge, the knowledge graph contains various entities and their relationships and attributes, training a sequence pointer graph network model based on the data of the knowledge graph, and then using the sequence pointer graph network model to generate action sequences.

3. The distributed learning method according to claim 1, characterized in that: The edge server uses the generated data to train and update the local model. In the case where data samples cannot be shared directly, the federated distillation method is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, the federated aggregation method is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge and updates the local model through the global model.

4. A distributed learning method for variable-length action sequence generation, characterized in that: The method comprises: The rescue node sends a rescue action request to the nearest edge server based on the collected status information to quickly start the rescue action; then the edge server sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server initiates a request to generate a variable-length action sequence to the third-party server. The third-party server generates a determined rescue action sequence based on the large model according to the real-time data and task requirements, and feeds the result back to the cloud server. The cloud server transmits the generated action sequence to the edge server. The edge server then allocates resources for each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan, which is sent to the rescue node. The edge server stores the record as a model training data set. A personalized local model is generated for each edge server, namely the resource pointer graph network model. The resource pointer graph network model and the sequence pointer graph network model are both a kind of graph convolutional network. The edge server uses the resource pointer graph network to select and schedule resource nodes.

5. The distributed learning method according to claim 4, characterized in that: GPT is used as a pre-trained large-scale language model to learn language knowledge and patterns through a large-scale corpus, and then through fine-tuning, the model parameters are adjusted to adapt to the task requirements for specific rescue mission areas.

6. The distributed learning method according to claim 4, characterized in that: The edge server uses the generated data to train and update the local model. In the case where data samples cannot be shared directly, the federated distillation method is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, the federated aggregation method is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge and updates the local model through the global model.

7. A distributed learning system for variable-length action sequence generation, characterized in that: The system includes a rescue node, an edge server and a cloud server, and the rescue node, the edge server and the cloud server respectively implement the distributed learning method as described in any one of claims 1-3.

8. A distributed learning system for variable-length action sequence generation, characterized in that: The system includes a rescue node, an edge server, a cloud server and a third-party server, and the rescue node, the edge server, the cloud server and the third-party server respectively implement the distributed learning method as described in any one of claims 4-6.

Images