A distributed learning method and system for variable length action sequence generation
Patent Information
- Application Number
- CN202411211727.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2026-08-18
- Estimated Expiration
- 2044-08-30
AI Technical Summary
然而在这种背景下依然存在许多挑战：首先，由于计算资源受限，单独在边缘设备上进行任务规划和模型训练效率较低，模型性能也难以达到理想水平；其次，将边缘设备上的数据传输到云端进行联合训练，会带来大量的网络通信开销，给系统带来沉重的负担；最后，将用户数据传输到云端进行训练，也可能会带来用户隐私泄露的风险
[0017]This invention addresses emergency rescue missions in rescue scenarios by combining hierarchical task planning and federated learning to construct a hierarchical distributed network architecture comprising rescue nodes, edge servers, and cloud servers. By centrally determining action sequences for rescue missions on the cloud servers and allocating resource nodes for these action sequences at the edge, hierarchical execution of tasks is achieved. This leverages the computing resources of edge devices to improve execution efficiency. Simultaneously, federated aggregation and federated distillation techniques are employed to update the local pointer graph network model, protecting user privacy while optimizing and updating the model, thereby improving the efficiency and reliability of task resource allocation.
Smart Images
Figure CN119180323B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, and in particular to a distributed learning method and system for generating variable-length action sequences. Background Technology
[0002] In the field of emergency rescue, effective mission planning and resource allocation are crucial to ensuring the success of rescue operations. Emergency rescue missions involve situations arising from natural disasters, accidents, or other unforeseen events, requiring rapid response and coordinated rescue efforts. In such cases, multiple stakeholders, such as rescue personnel and medical staff, need to work closely together to execute a series of complex tasks to rescue victims and minimize losses. This involves timely detection of the event, determining the specific steps and division of labor for the rescue operation, and promptly notifying relevant parties to initiate the rescue process. Following this is resource allocation and management, taking into account factors such as the urgency of the mission, geographical location, and resource requirements, ensuring that various resources, such as manpower, materials, and transportation, are effectively allocated to each mission location and flexibly adjusted according to the actual situation.
[0003] Traditionally, task planning is often performed by a central server, but this centralized approach can suffer from network latency and bandwidth limitations. With the rise of edge computing, edge devices can process tasks locally, significantly improving task response speed and system robustness. However, several challenges remain: First, due to limited computing resources, task planning and model training on edge devices alone are inefficient, and model performance is difficult to achieve ideal levels. Second, transmitting data from edge devices to the cloud for joint training incurs significant network communication overhead, placing a heavy burden on the system. Finally, transmitting user data to the cloud for training may also pose a risk of user privacy breaches. Summary of the Invention
[0004] To address the aforementioned issues, this invention combines hierarchical task planning and federated learning to propose a distributed learning method and system for generating variable-length action sequences. By centrally determining action sequences for tasks on cloud servers and allocating resource nodes for these action sequences at the edge, hierarchical execution of tasks is achieved, improving the efficiency and reliability of task resource allocation.
[0005] On one hand, the present invention provides a distributed learning method for generating variable-length action sequences, the method comprising:
[0006] The rescue node sends a rescue action request to the nearest edge server based on the collected status information to quickly initiate the rescue operation. Then, the edge server sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server uses a sequence pointer graph network model to make decisions and generate an action sequence with a certain step length, and feeds the results back to the edge server. After receiving the action sequence returned by the cloud server, the edge server allocates resources for each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan, which is then sent to the rescue node. The edge server stores this record as a model training dataset.
[0007] A personalized local model, namely the resource pointer graph network model, is generated for each edge server. Both the resource pointer graph network model and the sequence pointer graph network model are graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.
[0008] Furthermore, the cloud server uses a sequence pointer graph network model to generate action sequences, including: constructing a knowledge graph based on existing data and domain knowledge, the knowledge graph containing various entities (such as locations, resources, tasks, etc.) and the relationships and attributes between them; training a sequence pointer graph network model based on the data in the knowledge graph; and then using the sequence pointer graph network model to generate action sequences.
[0009] Furthermore, edge servers use the generated data to train and update their local models. For cases where data samples cannot be directly shared, federated distillation is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, federated aggregation is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge. The local model is then updated using the global model.
[0010] The present invention also provides a distributed learning system for generating variable-length action sequences. The system includes rescue nodes, edge servers, and cloud servers, and the rescue nodes, edge servers, and cloud servers respectively implement the distributed learning method described above.
[0011] On the other hand, the present invention provides a distributed learning method for generating variable-length action sequences, the method comprising:
[0012] The rescue node sends a rescue action request to the nearest edge server based on the collected status information to quickly initiate the rescue operation. Then, the edge server sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server initiates a request to a third-party server to generate a variable-length action sequence. The third-party server generates a determined rescue action sequence based on real-time data and task requirements, according to a large model, and feeds the results back to the cloud server. The cloud server transmits the generated action sequence to the edge server. The edge server then allocates resources for each step in the action sequence and combines the action sequence and resource allocation results to generate the final rescue action plan, which is then sent to the rescue node. The edge server stores this record as a model training dataset.
[0013] A personalized local model, namely the resource pointer graph network model, is generated for each edge server. Both the resource pointer graph network model and the sequence pointer graph network model are graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.
[0014] Furthermore, GPT is used as a pre-trained large-scale language model to learn language knowledge and patterns through a large-scale corpus. Then, through fine-tuning, the model parameters are adjusted to adapt to the mission requirements for specific rescue mission domains.
[0015] Furthermore, edge servers use the generated data to train and update their local models. For cases where data samples cannot be directly shared, federated distillation is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, federated aggregation is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge. The local model is then updated using the global model.
[0016] The present invention also provides a distributed learning system for generating variable-length action sequences. The system includes rescue nodes, edge servers, cloud servers, and third-party servers, and the rescue nodes, edge servers, cloud servers, and third-party servers respectively implement the distributed learning method described above.
[0017] This invention addresses emergency rescue missions in rescue scenarios by combining hierarchical task planning and federated learning to construct a hierarchical distributed network architecture comprising rescue nodes, edge servers, and cloud servers. By centrally determining action sequences for rescue missions on the cloud servers and allocating resource nodes for these action sequences at the edge, hierarchical execution of tasks is achieved. This leverages the computing resources of edge devices to improve execution efficiency. Simultaneously, federated aggregation and federated distillation techniques are employed to update the local pointer graph network model, protecting user privacy while optimizing and updating the model, thereby improving the efficiency and reliability of task resource allocation. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart of a distributed learning method for generating variable-length action sequences provided in an embodiment of the present invention;
[0020] Figure 2 This is a signaling diagram of a distributed learning method for generating variable-length action sequences provided in an embodiment of the present invention;
[0021] Figure 3 This is a flowchart of a distributed learning method for generating variable-length action sequences provided in an embodiment of the present invention;
[0022] Figure 4 This is a signaling diagram of a distributed learning method for generating variable-length action sequences provided in an embodiment of the present invention;
[0023] Figure 5 This is a flowchart of the federated distillation implementation provided in an embodiment of the present invention;
[0024] Figure 6 This is a signaling diagram of the federated distillation implementation provided in an embodiment of the present invention;
[0025] Figure 7 This is a flowchart of the federated aggregation implementation provided in the embodiments of the present invention;
[0026] Figure 8 This is a signaling diagram of the federated aggregation implementation provided in an embodiment of the present invention;
[0027] Figure 9 This is an architecture diagram of the distributed learning system provided in an embodiment of the present invention. Detailed Implementation
[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0029] To improve the efficiency and reliability of resource allocation in rescue missions, this invention proposes a distributed learning method and system for generating variable-length action sequences.
[0030] On the one hand, such as Figure 1 and Figure 2 As shown, the present invention provides a distributed learning method for generating variable-length action sequences, comprising:
[0031] The rescue node sends a rescue action request to the nearest edge server based on the collected status information, quickly initiating the rescue operation. The edge server then forwards the received action request to the cloud server. Upon receiving the action request from the edge server, the cloud server uses a sequence pointer graph network model to generate an action sequence of a certain step length and feeds the result back to the edge server. After receiving the action sequence returned by the cloud server, the edge server allocates resources for each step in the action sequence and combines the action sequence and resource allocation results to generate the final rescue action plan, which is then distributed to the rescue node. The edge server stores this record as a model training dataset. The status information includes the rescue node's own location and action objective, etc. The sequence pointer graph network model is a type of graph convolutional network.
[0032] The cloud server utilizes a sequence pointer graph network model to generate action sequences, including: constructing a knowledge graph based on existing data and domain knowledge. The knowledge graph contains various entities (such as locations, resources, tasks, etc.) and their relationships and attributes; training a sequence pointer graph network model based on the data from the knowledge graph; and then using the sequence pointer graph network model to generate action sequences. The knowledge graph provides semantic information and constraints. In rescue missions, the knowledge graph contains various entities and their relationships. By searching for entities and relationships related to the input task in the knowledge graph, additional semantic guidance is provided for action sequence generation. If the task involves performing an action at a specific location, the location information in the knowledge graph can help the model generate appropriate action sequences. The knowledge graph transforms the input data into a graph form, and then the sequence pointer graph network model generates sequences based on the graph.
[0033] A personalized local model, namely the resource pointer graph network model, is generated for each edge server. Both the resource pointer graph network model and the sequence pointer graph network model are graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.
[0034] On the other hand, such as Figure 3 and Figure 4 As shown, the present invention provides a distributed learning method for generating variable-length action sequences, comprising:
[0035] The rescue node sends a rescue action request to the nearest edge server based on the collected status information, quickly initiating the rescue operation. The edge server then forwards the received action request to the cloud server. Upon receiving the action request from the edge server, the cloud server initiates a request to a third-party server to generate a variable-length action sequence. The third-party server generates a determined rescue action sequence based on real-time data and task requirements using a large model, and feeds the results back to the cloud server. The cloud server then transmits the generated action sequence to the edge server. The edge server subsequently allocates resources for each step in the action sequence and, by combining the action sequence and resource allocation results, generates the final rescue action plan, which is then distributed to the rescue node. The edge server stores this record as a model training dataset, where the status information includes the rescue node's own location and action objective.
[0036] A personalized local model, namely the resource pointer graph network model, is generated for each edge server. Both the resource pointer graph network model and the sequence pointer graph network model are graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes.
[0037] Large-scale models possess the ability to understand and plan for complex scenarios, enabling them to make more intelligent decisions by comprehensively considering global information. First, pre-trained large-scale language models, such as GPT, learn language knowledge and patterns through a large-scale corpus. Then, through fine-tuning, the model parameters are adjusted to adapt to the specific rescue mission domain and its requirements. Such models can generate corresponding action sequences, including possible action plans and response strategies, based on the input rescue mission description.
[0038] Furthermore, how to solve the data sharing or model sharing problem among edge nodes is an important issue in federated learning. This is because in distributed scenarios, the data features of each edge server are inconsistent, making it impossible to directly share data samples to train local models. Moreover, directly sharing models is also difficult to adapt to the local model features of the edge server. To address this issue, this invention proposes a local model generation method to generate a personalized model for each edge server, namely a resource pointer graph network model. The resource pointer graph network model is a graph convolutional network. The edge server uses the resource pointer graph network to select and schedule resource nodes. These local models may be customized according to the specific environment, resources, and needs of the server to ensure optimal performance and adaptability.
[0039] Edge servers use the generated data to train and update local models. For the problem that data samples cannot be directly shared, features of other data are learned through federated distillation. For models with the same data samples, global knowledge is learned through federated aggregation, reducing data transfer and improving model accuracy.
[0040] Consider the following scenario: the data on edge servers share similar characteristics, and the edge servers have sufficient storage and computing power. Federated distillation is used to learn from other model data. For example... Figure 5 and Figure 6 As shown, in this scenario, the rescue node first initiates an action request to the edge server based on the collected status information. Then, the resource pointer graph network model of the edge server makes resource scheduling decisions based on the status and distributes them to the rescue node. Each edge server uses local data records to train and update its own resource pointer graph network model, and shares the model parameters with other edge servers through federated distillation, without sharing the original data. During federated distillation, edge servers only exchange gradients or features of model parameters, thus protecting data privacy. In this way, edge servers can update their models using federated distillation without sharing the original data, while ensuring data privacy protection.
[0041] In this embodiment, assuming there is no trusted central cloud node among the edge servers, the rescue node first initiates an action request to the edge server based on the collected status information, and then initiates an action sequence decision request to the cloud server. The cloud server feeds back the action sequence decision result to the edge server. Next, the resource pointer graph network model of the edge server makes a resource scheduling decision based on the status information and distributes it to the rescue node, storing this record as a model training dataset. During the model update phase, the global teacher model and the local decision model share sampled data and perform distillation based on the calculated loss. The local decision model then distills and transmits the updated local policy to the teacher model. After a period of training and updating, the teacher model is broadcast to other edge servers. The edge servers use the teacher model to update their local resource pointer graph network, use the local resource pointer graph network to complete resource element allocation, and generate the final action plan.
[0042] In federated aggregation, each edge server can upload its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge. The local model is then updated using the global model. This approach reduces data transmission volume while improving the overall performance and accuracy of the model.
[0043] Consider the following scenario: there is no trusted central cloud node among the edge servers, the edge servers have sufficient storage and computing power, and a federated aggregation approach is used to learn global knowledge. For example... Figure 7 and Figure 8 As shown, in this scenario, the rescue node first initiates a guidance request to the edge server based on the collected status information. Subsequently, the edge server uses local data records to train and update its local resource pointer graph network model, and then broadcasts the updated global model to other edge servers. After receiving the broadcast global model, the other edge servers aggregate the model parameters to obtain a new global model. In this way, edge servers train and update models using local data, and simultaneously update their local resource pointer graph networks through aggregation, achieving resource scheduling and federated aggregation updates of the model.
[0044] In this embodiment, it is assumed that there is no trusted central cloud node among the edge servers. First, the rescue node initiates an action request to the edge server based on the collected status information, and then initiates a sequence decision request to the cloud server. The cloud server feeds back the action sequence decision result to the edge server. Next, the local resource pointer graph network model of the edge server makes a resource scheduling decision based on the status information and distributes it to the rescue node, storing this record as a model training dataset. During the model training phase, the local resource pointer graph network is trained and updated on the dataset, and a weighted average aggregation is performed on the local resource pointer graph network to obtain the global model. The updated global model is then broadcast to other edge servers. Finally, the edge server updates its local resource pointer graph network model based on the global model, simultaneously obtaining a new global model. The local resource pointer graph network is then used to complete resource element allocation and generate the final action plan.
[0045] Furthermore, this invention provides a layered distributed network system jointly constructed by multiple edge servers and cloud servers, such as... Figure 9 As shown, for large-scale model decision-making, the system includes rescue nodes, edge servers, cloud services, and third-party servers; for sequence pointer graph network decision-making, the system also includes rescue nodes, edge servers, and cloud services. The system implements the distributed learning method described above, distributing functions such as task allocation, resource scheduling, and action sequence determination to different server layers. The cloud provides global planning and decision support, the edge is responsible for flexible task execution and resource scheduling, and the terminal devices are the actual executors of the actions. This layered design effectively solves the bottlenecks and limitations of traditional centralized systems, improves the response speed and efficiency of emergency rescue tasks, and provides more reliable support for emergency rescue work. This architecture enables efficient task execution and rational resource utilization, providing more reliable support for emergency rescue work.
[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A distributed learning method for generating variable-length action sequences, characterized in that, The method includes: The rescue node sends a rescue action request to the nearest edge server based on the collected status information to quickly initiate the rescue operation. Then, the edge server sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server uses a sequence pointer graph network model to make decisions and generate an action sequence with a certain step length, and feeds the results back to the edge server. After receiving the action sequence returned by the cloud server, the edge server allocates resources for each step in the action sequence, and combines the action sequence and resource allocation results to generate the final rescue action plan, which is then sent to the rescue node. The edge server stores this record as a model training dataset. A personalized local model is generated for each edge server, namely the resource pointer graph network model. Both the resource pointer graph network model and the sequence pointer graph network model are graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes. Edge servers use the generated data to train and update their local models. In cases where data samples cannot be directly shared, federated distillation is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, federated aggregation is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge. The local model is then updated using this global model.
2. The distributed learning method according to claim 1, characterized in that, The cloud server uses a sequence pointer graph network model to generate action sequences, which includes: building a knowledge graph based on existing data and domain knowledge, the knowledge graph containing various entities and the relationships and attributes between them; training a sequence pointer graph network model based on the data in the knowledge graph; and then using the sequence pointer graph network model to generate action sequences.
3. A distributed learning method for generating variable-length action sequences, characterized in that, The method includes: The rescue node sends a rescue action request to the nearest edge server based on the collected status information to quickly initiate the rescue operation. Then, the edge server sends the received action request to the cloud server. After receiving the action request from the edge server, the cloud server initiates a request to a third-party server to generate a variable-length action sequence. The third-party server generates a determined rescue action sequence based on real-time data and task requirements, according to a large model, and feeds the results back to the cloud server. The cloud server transmits the generated action sequence to the edge server. The edge server then allocates resources for each step in the action sequence and combines the action sequence and resource allocation results to generate the final rescue action plan, which is then sent to the rescue node. The edge server stores this record as a model training dataset. A personalized local model is generated for each edge server, namely the resource pointer graph network model. Both the resource pointer graph network model and the sequence pointer graph network model are graph convolutional networks. The edge server uses the resource pointer graph network to select and schedule resource nodes. Edge servers use the generated data to train and update their local models. In cases where data samples cannot be directly shared, federated distillation is used to learn the features of other data. In federated distillation, each edge server can learn the features of other data samples without sharing the original data. For models with the same data samples, federated aggregation is used to learn global knowledge. In federated aggregation, each edge server uploads its locally updated model parameters to the central cloud server, which then aggregates and integrates them to generate global model knowledge. The local model is then updated using this global model.
4. The distributed learning method according to claim 3, characterized in that, GPT is used as a pre-trained large-scale language model to learn language knowledge and patterns through a large-scale corpus. Then, through fine-tuning, the model parameters are adjusted to adapt to the needs of specific rescue mission domains.
5. A distributed learning system for generating variable-length action sequences, characterized in that, The system includes rescue nodes, edge servers, and cloud servers, each of which implements the distributed learning method as described in any one of claims 1-2.
6. A distributed learning system for generating variable-length action sequences, characterized in that, The system includes rescue nodes, edge servers, cloud servers, and third-party servers, wherein the rescue nodes, edge servers, cloud servers, and third-party servers respectively implement the distributed learning method as described in any one of claims 3-4.