An information processing method for an embedded edge computing system

Abstract

The application relates to the technical field of edge computing, and particularly discloses an information processing method for an embedded edge computing system, which comprises system state data collection, decision model construction, information life cycle analysis, reward function design and continuous optimization. The scheme adopts a layered multi-agent reinforcement learning architecture, decomposes a joint optimization problem into upper-layer global configuration optimization and lower-layer distributed resource allocation, realizes efficient cooperation of multi-agents through a centralized training and distributed execution framework, and designs an automatic device migration mechanism to ensure high availability of the system; a complete information life cycle analysis model is constructed to clearly define a starting point, an ending point, a stage composition, an average value calculation method and a retransmission extension mechanism of the information life cycle, and the information life cycle is fused with a reward function and constraint conditions of the decision model to realize accurate quantification of information freshness and system-level optimization control, and ensure focus and operability of analysis and optimization.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, specifically to an information processing method for embedded edge computing systems. Background Technology

[0002] Embedded edge computing, as a distributed computing paradigm that pushes computing power down to the network edge, can effectively reduce data transmission latency, reduce core network load, and protect user data privacy, and has become a key infrastructure supporting next-generation intelligent applications. However, existing embedded edge computing systems still have the following prominent problems in information processing: edge devices typically have limited uplink bandwidth and heterogeneous computing capabilities, and multi-agent collaboration in the system faces severe resource competition problems. When an edge node fails, the terminal devices under its jurisdiction often lose connection, leading to large-scale service interruptions. At the same time, in dynamic edge environments, due to resource competition, task queuing, and retransmission mechanisms among multiple agents, the overall information timeliness of the system is severely reduced, resulting in outdated information used for decision-making. Furthermore, in embedded edge computing scenarios, the system needs to strike a balance between multiple objectives such as information timeliness, energy efficiency, computational accuracy, and privacy protection. Summary of the Invention

[0003] To address the aforementioned issues and overcome the shortcomings of existing technologies, this invention provides an information processing method for embedded edge computing systems. Addressing the problems of resource competition among multiple agents and service interruptions caused by edge node failures, this solution employs a hierarchical multi-agent reinforcement learning architecture. It decomposes the joint optimization problem into upper-layer global configuration optimization and lower-layer distributed resource allocation. Efficient collaboration among multiple agents is achieved through centralized training and a distributed execution framework, and an automatic device migration mechanism is designed to ensure high system availability. Furthermore, addressing the issues of neglecting system-level evaluation in information processing and the unresolved challenges of multi-objective comprehensive optimization, this solution constructs a complete information lifecycle analysis model. This model clarifies the start point, end point, stage composition, average value calculation method, and retransmission extension mechanism of the information lifecycle. It integrates this model with the reward function and constraints of the decision-making model, achieving precise quantification of information freshness and system-level optimization control, ensuring the focus and operability of the analysis and optimization.

[0004] The technical solution adopted in this invention is as follows: This invention provides an information processing method for an embedded edge computing system, the method comprising the following steps:

[0005] Step S1: System status data acquisition. Collect status data of each terminal device in the edge computing system, edge environment data, and computing resource status data of edge nodes. Preprocess and extract features from all collected data to construct a system status dataset, in the following format: ;

[0006] In the formula, Indicates time, express The system state dataset at time 10:00. This represents a set of status data for the terminal device. Represents a set of edge environment data. A set of computing resource status data representing edge nodes;

[0007] Step S2: Decision model construction. A centralized training and distributed execution framework is adopted to divide the embedded edge computing system into three layers: terminal device layer, edge node layer, and edge server layer. The data types, data flow, and conversion mechanisms of each layer are defined.

[0008] Step S3: Information lifecycle analysis, establish a mathematical analysis model of the information lifecycle, and quantify the timeliness of the entire process from data packet generation to successful processing;

[0009] Step S4: Design the reward function, comprehensively optimize the data packet information lifecycle, equipment energy efficiency and system computing accuracy objectives, and design the reward function for the decision model;

[0010] Step S5: Continuous optimization. Regularly collect operational feedback from the edge computing system and iteratively optimize the parameters of the decision model and reward function.

[0011] Furthermore, in step S2, the construction of the decision model specifically includes the following steps:

[0012] Step S21: Define the three-tier architecture for data storage, including the terminal device layer, the edge node layer, and the edge server layer;

[0013] Step S22: Define the forward inference execution flow. Forward inference execution achieves low-latency millisecond-level response. The data path is from the edge server to the edge node to the terminal device.

[0014] Step S23: Reverse training update stream definition. Reverse training update achieves quasi-static second-level cycle response. The data path is from the terminal device to the edge node to the edge server, and then back to the terminal device.

[0015] Step S24: Data conversion, defining the format conversion operations for data transmission between different layers during the training phase;

[0016] Step S25: Data maintenance, limiting data storage location and update triggering conditions, restricting data access by terminal devices, and restricting data storage by edge nodes;

[0017] Step S26: Device migration. Perform heartbeat detection on edge nodes. When an edge node is detected to have failed, automatically migrate its subordinate terminal devices.

[0018] Furthermore, in step S21, the definition of the three-tier architecture data storage specifically includes the following steps:

[0019] Step S211: Terminal device layer storage definition. Each terminal device, as an independent intelligent agent, stores data related to its own local decision-making, including local observation history. Local policy network parameters Local experience replay pool ;

[0020] Step S212: Edge node layer storage definition. Edge nodes do not generate resource allocation decisions; they are used for data aggregation and temporary storage. The stored data includes a mapping table of managed devices. Local resource snapshot matrix Local gradient buffer ;

[0021] Step S213: Define the edge server layer storage, construct the upper-layer controller as the training center, and store the global dimensional data of the entire embedded edge computing system, including the actions of the upper-layer controller. Global resource allocation state matrix Global state space vector Global value network parameters With hybrid network parameters Global Experience Replay Pool .

[0022] Further, in step S22, the forward inference execution flow definition includes the following steps:

[0023] Step S221: Upper-layer action generation, the edge server generates the action based on the current global state space vector. and global value network Inferring the actions of the upper-level global controller The format is as follows: ;

[0024] In the formula, Indicates the terminal device index. Indicates that the upper-level global controller is in Actions at any moment This indicates the proportion of bandwidth allocated to each terminal device. Indicates the computational resource quota;

[0025] Step S222: Upper-layer action is sent down. The edge server sends the upper-layer action to all edge nodes in the form of a broadcast. After receiving the action, each edge node only extracts the bandwidth ratio and computing resource quota subset belonging to its subordinate terminal devices and updates the local resource snapshot matrix. The corresponding column in the data is in the following format: ;

[0026] In the formula, Represents the lower-level intelligent agent exist Actions at any moment Represents a local resource snapshot matrix;

[0027] Step S223: Quota Transmission. The edge node performs format conversion and protocol adaptation on the extracted bandwidth ratio and computing resource quota subset, and then transmits it to its subordinate terminal devices. After receiving the bandwidth ratio and computing resource quota subset, the terminal device combines it with its own local observation vector. Enter local policy network parameters Perform forward reasoning and output the actions of the lower-level agent. The format is as follows: ; ;

[0028] In the formula, Represents intelligent agents exist Transmission power at any given time Indicates the local calculation frequency. Indicates the percentage of tasks unloaded. Represents intelligent agents The local state data set Represents intelligent agents The set of observable edge node computing resource status data. Indicates the action at the previous moment. This indicates the reward received in the previous moment;

[0029] Step S224: Action execution. After forward reasoning is completed, the terminal device immediately applies the action to adjust its own physical parameters and caches the instantaneous state data after execution locally.

[0030] Further, in step S23, the definition of the reverse training update stream specifically includes the following steps:

[0031] Step S231: Local gradient calculation, the terminal device retrieves gradients from the local experience replay pool. Randomly sample small batches of global transfer samples and calculate the gradient of local policy network parameters. Simultaneously, the time difference error between the local Q value and the target Q value is calculated;

[0032] Step S232: Gradient upload. Each terminal device encapsulates the calculated gradient value, the most recent local observation, and the obtained reward value into an uplink data packet and sends it to the edge node to which it belongs.

[0033] Step S233: Edge node gradient aggregation. After receiving gradients uploaded by all terminal devices within its jurisdiction, the edge node performs a federated average aggregation operation to generate the average gradient of the edge node. The formula used is as follows: ;

[0034] In the formula, Indicates the edge node index. Represents edge nodes The average gradient after aggregation, Represents edge nodes The collection of terminal devices under its jurisdiction The size of the set;

[0035] Step S234: Local snapshot update, edge nodes update their average gradient and local resource snapshot matrix. The latest data is compressed and uploaded to the edge server, using incremental transmission to send only the data that has changed relative to the previous round.

[0036] Step S235: Global matrix update. The edge server receives the average gradient and local resource snapshot matrix from all edge nodes and updates the global resource allocation state matrix. The corresponding rows and columns are used to input the average gradient into the hybrid network, and the global value network parameters are updated by minimizing the global loss function. Simultaneously, the hybrid network parameters are updated using a soft update method. The soft update coefficient is denoted as ;

[0037] Step S236: Parameter synchronization and distribution; the edge server will send the updated global value network parameters. With hybrid network parameters The packaged broadcast is sent to each edge node, which then forwards it to all subordinate terminal devices. The terminal devices replace the old parameters in their local policy network with the new parameters.

[0038] Further, in step S24, the data conversion specifically includes the following steps:

[0039] Step S241: Downlink data conversion, the edge server sends the upper-layer controller action and global value network parameters. With hybrid network parameters At this time, the edge node quantizes the network floating-point weight into an 8-bit integer, encapsulates the bandwidth ratio and computing resource quota value of its subordinate terminal devices into a binary control frame, and sends it to the terminal device through a short-range wireless protocol. After receiving the quantized data, the terminal device dequantizes it locally to restore it to a floating-point number and updates the local policy network.

[0040] Step S242: Uplink data conversion. After calculating the local gradient, the terminal device performs sparsification and quantization before transmission, retaining only the top 10% of gradient elements with the largest absolute values. These floating-point numbers are then converted to 16-bit half-precision format. After receiving the sparsified and quantized gradients uploaded by all terminal devices, the edge node performs dequantization and desparsening operations to restore the complete floating-point gradient tensor. Federated averaging aggregation is then performed, and the edge node calculates the average gradient after aggregation. Quantized again to an 8-bit integer, along with the local resource snapshot matrix. The key fields in the data are encapsulated as uplink frames and sent to the edge server. After receiving the frames, the edge server performs dequantization and restores them as floating-point gradients to update the global value network.

[0041] Furthermore, in step S25, the data maintenance specifically involves: defining a global resource allocation state matrix. The entire data is stored on the edge server. Each time the edge server receives a local resource snapshot matrix reported by an edge node, it does so. Immediately update the global resource allocation state matrix. The corresponding row; restricting terminal devices to only obtaining a subset of the bandwidth ratio and computing resource quota related to themselves for forwarding from edge nodes, and preventing them from obtaining the global resource allocation state matrix. Data from any other terminal device in the dataset; limiting edge nodes to storing only local resource snapshot matrices of their governed subset. .

[0042] Further, in step S26, the device migration specifically includes the following steps:

[0043] Step S261: Heartbeat packet transmission. Each edge node sends a heartbeat packet to the edge server at fixed time intervals. The heartbeat packet contains the edge node's local resource snapshot matrix. Mapping table of managed devices ;

[0044] Step S262: Heartbeat timeout determination. The edge server sets a timeout threshold and maintains a heartbeat timer for each edge node. If a heartbeat packet is not received from an edge node before the timeout threshold is exceeded, the edge node is determined to be faulty.

[0045] Step S263: Device migration, the edge server reads the managed device mapping table of the faulty edge node. Based on the backup copy, identify the terminal devices that need to be migrated, migrate them to the edge node closest to the faulty edge node, and update the managed device mapping table of the target edge node. The terminal device re-establishes a connection with the target edge node.

[0046] Furthermore, in step S3, the information lifecycle analysis includes the following steps:

[0047] Step S31: Define the information lifecycle. The information lifecycle is the total time elapsed from the moment a data packet is generated at its online terminal device until the moment the data packet is successfully received and processed by the edge server, in the following form: ;

[0048] In the formula, Indicates terminal device exist The information lifecycle of the data packet corresponding to a given moment. Indicates the time when the data packet was generated;

[0049] Step S32: Decompose the information lifecycle into stages, including sending queuing delay, transmission delay, and edge processing delay.

[0050] Step S33: Information lifecycle evolution analysis. A first-come, first-served (FFS) scheduling strategy is adopted for the task queues of edge nodes or edge servers. Data packets enter the task queue sequentially according to their generation order, and the queue service rate is [missing information]. The data packet arrival rate is And the system satisfies the stability condition. The average information lifecycle of the system is calculated using the following steps:

[0051] Step S331: Define the total time a data packet stays in the system, i.e., the information lifecycle of the data packet, using the following formula: ;

[0052] In the formula, Indicates the packet index. Indicates the first The information lifecycle of a data packet Indicates the moment when the data packet was successfully received. Indicates the time when the data packet was generated;

[0053] Step S332: Define the time interval between two adjacent data packets that are successfully received, and this interval follows the mean value. The exponential distribution is given by the following formula: ;

[0054] In the formula, This indicates the time interval between two consecutive successfully received data packets;

[0055] Step S333: Under steady-state conditions, assuming the arrival process of data packets follows a Poisson process, calculate the average information lifetime of the system using the following formula: ; ;

[0056] In the formula, Represents the mathematical expectation. The second moment representing the arrival interval. Indicates the data packet arrival rate. Indicates the average information lifecycle. This indicates a coupling term that shows how the delivery time of the current data packet affects the waiting time of the next data packet;

[0057] Step S34: Retransmission detection. When a data packet transmission failure is detected, a retransmission is triggered. Let the probability of a single successful detection be... The formula used to calculate the expected lifespan of a data packet from its initial transmission to its final successful processing is as follows: ;

[0058] In the formula, This indicates the expected lifecycle of the data packet before it is successfully processed. This represents the expected lifespan of information transmitted in a single transaction. This indicates the success rate of a single test. This indicates the expected time for a single retransmission.

[0059] Furthermore, in step S4, the reward function design specifically involves: combining three sub-objectives—minimizing information lifecycle, minimizing energy consumption, and maximizing processing accuracy—to calculate the reward function for the terminal device. exist The reward at each moment is determined by the reward function. A negative reward function indicates a penalty for a longer information lifecycle, while a positive value indicates an incentive for high processing accuracy. The formula used is as follows: ;

[0060] In the formula, Indicates terminal device exist Rewards earned at any time Indicates the current information lifecycle. This represents the maximum information lifecycle threshold. Indicates terminal device exist Energy consumption during the execution of actions. This indicates the maximum energy consumption allowed in a single decision. Indicates the precision of task processing. , , The weighting coefficients are used to balance the three optimization objectives of information lifecycle, energy consumption, and processing accuracy.

[0061] The beneficial effects achieved by the present invention using the above solution are as follows:

[0062] (1) To address the issues of resource competition among multiple agents and service interruption caused by edge node failures, this solution adopts a hierarchical multi-agent reinforcement learning architecture, which decomposes the joint optimization problem into upper-layer global configuration optimization and lower-layer distributed resource allocation. It achieves efficient collaboration among multiple agents through centralized training and distributed execution framework, and designs an automatic device migration mechanism to ensure high availability of the system.

[0063] (2) In response to the problem that information processing neglects system-level evaluation and the problem of multi-objective comprehensive optimization has not been effectively solved, this solution constructs a complete information life cycle analysis model, clarifies the starting point, ending point, stage composition, average value calculation method and retransmission extension mechanism of the information life cycle, and integrates it with the reward function and constraints of the decision model to achieve accurate quantification and system-level optimization control of information freshness, and ensure the focus and operability of analysis and optimization. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating an information processing method for an embedded edge computing system proposed in this invention.

[0065] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0067] Example 1, see Figure 1 The present invention provides an information processing method for an embedded edge computing system, the method comprising the following steps:

[0068] Step S1: System status data acquisition. Through sensors and monitoring interfaces deployed in the embedded edge computing system, status data of each terminal device, edge environment data, and computing resource status data of edge nodes are collected. All collected data are preprocessed and feature extracted to construct a system status dataset.

[0069] Step S2: Decision model construction. A centralized training and distributed execution framework is adopted to divide the embedded edge computing system into three layers: terminal device layer, edge node layer, and edge server layer. The data types, data flow, and conversion mechanisms of each layer are defined.

[0070] Step S3: Information lifecycle analysis, establish a mathematical analysis model of the information lifecycle, and quantify the timeliness of the entire process from data packet generation to successful processing;

[0071] Step S4: Design the reward function, comprehensively optimize the data packet information lifecycle, equipment energy efficiency and system computing accuracy objectives, and design the reward function for the decision model;

[0072] Step S5: Continuous optimization. Regularly collect operational feedback from the edge computing system and iteratively optimize the parameters of the decision model and reward function.

[0073] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, the system status data acquisition specifically includes the following steps:

[0074] Step S11: Terminal device status acquisition. Collect computing resource status data, communication resource status data and energy status data of each terminal device. The computing resource status data includes processor load, memory utilization rate and remaining storage space. The communication resource status data includes uplink bandwidth, channel gain and signal-to-noise ratio. The energy status data includes remaining power and power consumption.

[0075] Step S12: Environmental data collection, collecting dynamic data of the edge environment where each terminal device is located: for mobile augmented reality scenarios, this includes video frame rate, resolution, and detection task queue length; for vehicle networking scenarios, this includes vehicle position, speed, and heading angle; for recommendation system scenarios, this includes user interaction records and content features.

[0076] Step S13: Edge node status collection. Collect computing resource status data of each edge node from the edge server, including computing resource allocation, task processing rate, and task queue waiting time.

[0077] Step S14: Data preprocessing. The collected data is filled with missing values ​​by interpolation, outliers are identified and corrected by statistical thresholding, and data with different dimensions and distribution characteristics are converted into a standard distribution with a mean of 0 and a standard deviation of 1 by Z-score standardization to obtain a normalized dataset.

[0078] Step S15: Feature extraction, including the following steps:

[0079] Step S151: Temporal feature extraction, extracting temporal features from the normalized dataset, including extracting long-term trends using the moving average method, extracting periodic features using the periodogram analysis method, and extracting rate of change features by calculating the first and second differences;

[0080] Step S152: Introduce lag features. Determine the lag order based on the temporal correlation of the data and add historical data as lag features.

[0081] Step S153: Integrate the temporal features and lag features into the original feature set;

[0082] Step S16: Feature selection and dimensionality reduction. Pearson correlation coefficient analysis is used to remove redundant features from the original feature set. Principal component analysis is used to reduce the feature dimensionality. The dimensionality-reduced features are then aligned according to a unified timestamp and stored in a structured format as a system state dataset, as follows: ;

[0083] In the formula, Indicates time, express The system state dataset at time 10:00. This represents a set of status data for the terminal device. Represents a set of edge environment data. A set of computing resource status data representing edge nodes.

[0084] Example 3, see Figure 1 This embodiment is based on the above embodiment. In step S2, the construction of the decision model specifically includes the following steps:

[0085] Step S21: Define the three-tier architecture for data storage, including the terminal device layer, the edge node layer, and the edge server layer;

[0086] Step S22: Define the forward inference execution flow. Forward inference execution achieves low-latency millisecond-level response. The data path is from the edge server to the edge node to the terminal device.

[0087] Step S23: Reverse training update stream definition. Reverse training update achieves quasi-static second-level cycle response. The data path is from the terminal device to the edge node to the edge server, and then back to the terminal device.

[0088] Step S24: Data conversion, defining the format conversion operations for data transmission between different layers during the training phase;

[0089] Step S25: Data maintenance, limiting data storage location and update triggering conditions, restricting data access by terminal devices, and restricting data storage by edge nodes;

[0090] Step S26: Device migration. Perform heartbeat detection on edge nodes. When an edge node is detected to have failed, automatically migrate its subordinate terminal devices.

[0091] Example 4, see Figure 1 This embodiment is based on the above embodiment. In step S21, the three-tier architecture data storage definition specifically includes the following steps:

[0092] Step S211: Terminal device layer storage definition. Each terminal device, as an independent intelligent agent, stores data related to its own local decision-making, specifically including:

[0093] Local observation history Stores its most recent state information, including remaining battery power, local CPU task queue length, actions executed in the previous moment, local channel quality indicator, and task buffer occupancy rate.

[0094] Local policy network parameters : Stores the weights of the deep recursive Q-network, which are used to map the optimal local resource allocation action based on the current local observation;

[0095] Local experience replay pool Experience tuples generated during the interaction with storage terminal devices are in the following form: ;

[0096] In the formula, express The local observation vector at time t, express Time-based intelligent agent The lower-level actions executed This indicates the reward obtained after execution. Represents the local observation vector at the next moment;

[0097] Step S212: Edge node layer storage definition. Edge nodes do not generate resource allocation decisions; they are used for data aggregation and temporary storage. The stored data specifically includes:

[0098] Governed Equipment Mapping Table : Store this edge node IDs and communication addresses of all terminal devices under its jurisdiction;

[0099] Local resource snapshot matrix : Store this edge node The received actions from the upper-level controller and the actions of the lower-level intelligent agents from all subordinate terminal devices;

[0100] Local gradient buffer Temporarily store the local policy network parameter gradient uploaded by subordinate terminal devices. ;

[0101] Step S213: Edge server layer storage definition, constructing an upper-layer controller as a training center to store global dimensional data of the entire embedded edge computing system, specifically including:

[0102] Upper-level controller actions The bandwidth ratio and computing resource quota allocated to each terminal device;

[0103] Global resource allocation state matrix Record all terminal devices in Actions at any given moment, and the task queue state vector ;

[0104] Global state space vector ;

[0105] Global Value Network Parameters With hybrid network parameters The former represents the primary network parameters, while the latter represents the target network parameters. The hybrid network nonlinearly and monotonically aggregates the local Q-values ​​of each terminal device to calculate the global joint value. ;

[0106] Global Experience Replay Pool : Stores small batches of global migration samples after aggregation at edge nodes. Each sample contains a global state space vector. Joint actions, joint rewards, and the global state in the next moment.

[0107] Example 5, see Figure 1 This embodiment is based on the above embodiment. In step S22, the definition of the forward inference execution flow specifically includes the following steps:

[0108] Step S221: Upper-layer action generation, the edge server generates the action based on the current global state space vector. and global value network Inferring the actions of the upper-level global controller The format is as follows: ;

[0109] In the formula, Indicates the terminal device index. Indicates that the upper-level global controller is in Actions at any moment This indicates the proportion of bandwidth allocated to each terminal device. Indicates the computational resource quota;

[0110] Step S222: Upper-layer action is sent down. The edge server sends the upper-layer action to all edge nodes in the form of a broadcast. After receiving the action, each edge node only extracts the bandwidth ratio and computing resource quota subset belonging to its subordinate terminal devices and updates the local resource snapshot matrix. The corresponding column in the data is in the following format: ;

[0111] In the formula, Represents the lower-level intelligent agent exist Actions at any moment Represents a local resource snapshot matrix;

[0112] Step S223: Quota Transmission. The edge node performs format conversion and protocol adaptation on the extracted bandwidth ratio and computing resource quota subset, and then transmits it to its subordinate terminal devices. After receiving the bandwidth ratio and computing resource quota subset, the terminal device combines it with its own local observation vector. Enter local policy network parameters Perform forward reasoning and output the actions of the lower-level agent. The format is as follows: ; ;

[0113] In the formula, Represents intelligent agents exist Transmission power at any given time Indicates the local calculation frequency. Indicates the percentage of tasks unloaded. Represents intelligent agents The local state data set Represents intelligent agents The set of observable edge node computing resource status data. Indicates the action at the previous moment. This indicates the reward received in the previous moment;

[0114] Step S224: Action execution. After forward reasoning is completed, the terminal device immediately applies the action to adjust its own physical parameters and caches the instantaneous state data after execution locally.

[0115] Example 6, see Figure 1 This embodiment is based on the above embodiment. In step S23, the definition of the reverse training update stream specifically includes the following steps:

[0116] Step S231: Local gradient calculation, the terminal device retrieves gradients from the local experience replay pool. Randomly sample small batches of global transfer samples and calculate the gradient of local policy network parameters. Simultaneously, the time difference error between the local Q value and the target Q value is calculated;

[0117] Step S232: Gradient upload. Each terminal device encapsulates the calculated gradient value, the most recent local observation, and the obtained reward value into an uplink data packet and sends it to the edge node to which it belongs.

[0118] Step S233: Edge node gradient aggregation. After receiving gradients uploaded by all terminal devices within its jurisdiction, the edge node performs a federated average aggregation operation to generate the average gradient of the edge node. The formula used is as follows: ;

[0119] In the formula, Indicates the edge node index. Represents edge nodes The average gradient after aggregation, Represents edge nodes The collection of terminal devices under its jurisdiction The size of the set;

[0120] Step S234: Local snapshot update, edge nodes update their average gradient and local resource snapshot matrix. The latest data is compressed and uploaded to the edge server, using incremental transmission to send only the data that has changed relative to the previous round.

[0121] Step S235: Global matrix update. The edge server receives the average gradient and local resource snapshot matrix from all edge nodes and updates the global resource allocation state matrix. The corresponding rows and columns are used to input the average gradient into the hybrid network, and the global value network parameters are updated by minimizing the global loss function. Simultaneously, the hybrid network parameters are updated using a soft update method. The soft update coefficient is denoted as ;

[0122] Step S236: Parameter synchronization and distribution; the edge server will send the updated global value network parameters. With hybrid network parameters The packaged broadcast is sent to each edge node, which then forwards it to all subordinate terminal devices. The terminal devices replace the old parameters in their local policy network with the new parameters.

[0123] Example 7, see Figure 1 This embodiment is based on the above embodiment. In step S24, the data conversion specifically includes the following steps:

[0124] Step S241: Downlink data conversion, the edge server sends the upper-layer controller action and global value network parameters. With hybrid network parameters At this time, the edge node quantizes the network floating-point weight into an 8-bit integer, encapsulates the bandwidth ratio and computing resource quota value of its subordinate terminal devices into a binary control frame, and sends it to the terminal device through a short-range wireless protocol. After receiving the quantized data, the terminal device dequantizes it locally to restore it to a floating-point number and updates the local policy network.

[0125] Step S242: Uplink data conversion. After calculating the local gradient, the terminal device performs sparsification and quantization before transmission, retaining only the top 10% of gradient elements with the largest absolute values. These floating-point numbers are then converted to 16-bit half-precision format. After receiving the sparsified and quantized gradients uploaded by all terminal devices, the edge node performs dequantization and desparsening operations to restore the complete floating-point gradient tensor. Federated averaging aggregation is then performed, and the edge node calculates the average gradient after aggregation. Quantized again to an 8-bit integer, along with the local resource snapshot matrix. The key fields in the data are encapsulated as uplink frames and sent to the edge server. After receiving the frames, the edge server performs dequantization and restores them as floating-point gradients to update the global value network.

[0126] Example 8, see Figure 1 This embodiment is based on the above embodiment. In step S25, the data maintenance specifically involves: defining the global resource allocation state matrix. The entire data is stored on the edge server. Each time the edge server receives a local resource snapshot matrix reported by an edge node, it does so. Immediately update the global resource allocation state matrix. The corresponding row; restricting terminal devices to only obtaining a subset of the bandwidth ratio and computing resource quota related to themselves for forwarding from edge nodes, and preventing them from obtaining the global resource allocation state matrix. Data from any other terminal device in the dataset; limiting edge nodes to storing only local resource snapshot matrices of their governed subset. .

[0127] Example 9, see Figure 1 This embodiment is based on the above embodiment. In step S26, the device migration includes the following steps:

[0128] Step S261: Heartbeat packet transmission. Each edge node sends a heartbeat packet to the edge server at fixed time intervals. The heartbeat packet contains the edge node's local resource snapshot matrix. Mapping table of managed devices ;

[0129] Step S262: Heartbeat timeout determination. The edge server sets a timeout threshold and maintains a heartbeat timer for each edge node. If a heartbeat packet is not received from an edge node before the timeout threshold is exceeded, the edge node is determined to be faulty.

[0130] Step S263: Device migration, the edge server reads the managed device mapping table of the faulty edge node. Based on the backup copy, identify the terminal devices that need to be migrated, migrate them to the edge node closest to the faulty edge node, and update the managed device mapping table of the target edge node. The terminal device re-establishes a connection with the target edge node.

[0131] By performing the above operations, this solution addresses the issues of resource competition among multiple agents and service interruptions caused by edge node failures. It adopts a hierarchical multi-agent reinforcement learning architecture, decomposing the joint optimization problem into upper-layer global configuration optimization and lower-layer distributed resource allocation. It achieves efficient collaboration among multiple agents through centralized training and a distributed execution framework, and designs an automatic device migration mechanism to ensure high system availability.

[0132] Example 10, see Figure 1 This embodiment is based on the above embodiment. In step S3, the information lifecycle analysis specifically includes the following steps:

[0133] Step S31: Define the information lifecycle. The information lifecycle is the total time elapsed from the moment a data packet is generated at its online terminal device until the moment the data packet is successfully received and processed by the edge server, in the following form: ;

[0134] In the formula, Indicates terminal device exist The information lifecycle of the data packet corresponding to a given moment. Indicates the time when the data packet was generated;

[0135] Step S32: Information lifecycle stage decomposition, which decomposes the information lifecycle into transmission queuing delay, transmission delay, and edge processing delay, specifically:

[0136] Transmission queuing delay represents the time consumed when data packets wait in the terminal device's transmit buffer for uplink transmission resources. This delay depends on the terminal device's current task queue length and uplink bandwidth ratio. and channel contention status;

[0137] Transmission latency refers to the time it takes for a data packet to travel from a terminal device to an edge node or edge server via a wireless link. This latency depends on the data packet size and transmission power. Channel gain and available bandwidth;

[0138] Edge processing latency refers to the time it takes for data packets to queue on an edge node or edge server, wait for computing resources to execute analysis tasks, and generate processing results. This latency depends on the computing resource quota allocated to the terminal device by the edge server. And the computational complexity of the task;

[0139] Step S33: Information lifecycle evolution analysis. A first-come, first-served (FFS) scheduling strategy is adopted for the task queues of edge nodes or edge servers. Data packets enter the task queue sequentially according to their generation order, and the queue service rate is [missing information]. The data packet arrival rate is And the system satisfies the stability condition. The average information lifecycle of the system is calculated using the following steps:

[0140] Step S331: Define the total time a data packet stays in the system, i.e., the information lifecycle of the data packet, using the following formula: ;

[0141] In the formula, Indicates the packet index. Indicates the first The information lifecycle of a data packet Indicates the moment when the data packet was successfully received. Indicates the time when the data packet was generated;

[0142] Step S332: Define the time interval between two adjacent data packets that are successfully received, and this interval follows the mean value. The exponential distribution is given by the following formula: ;

[0143] In the formula, This indicates the time interval between two consecutive successfully received data packets;

[0144] Step S333: Under steady-state conditions, assuming the arrival process of data packets follows a Poisson process, calculate the average information lifetime of the system using the following formula: ; ;

[0145] In the formula, Represents the mathematical expectation. The second moment representing the arrival interval. Indicates the data packet arrival rate. Indicates the average information lifecycle. This indicates a coupling term that shows how the delivery time of the current data packet affects the waiting time of the next data packet;

[0146] Step S34: Retransmission detection. When a data packet transmission failure is detected, a retransmission is triggered. Let the probability of a single successful detection be... The formula used to calculate the expected lifespan of a data packet from its initial transmission to its final successful processing is as follows: ;

[0147] In the formula, This indicates the expected lifecycle of the data packet before it is successfully processed. This represents the expected lifespan of information transmitted in a single transaction. This indicates the success rate of a single test. This indicates the expected time for a single retransmission.

[0148] Example 11, see Figure 1 This embodiment is based on the above embodiment. In step S4, the reward function design specifically involves: combining the three sub-objectives of minimizing information lifecycle, minimizing energy consumption, and maximizing processing accuracy to calculate the reward function of the terminal device. exist The reward at each moment is determined by the reward function. A negative reward function indicates a penalty for a longer information lifecycle, while a positive value indicates an incentive for high processing accuracy. The formula used is as follows: ;

[0149] In the formula, Indicates terminal device exist Rewards earned at any time Indicates the current information lifecycle. This represents the maximum information lifecycle threshold. Indicates terminal device exist Energy consumption during the execution of actions. This indicates the maximum energy consumption allowed in a single decision. Indicates the precision of task processing. , , The weighting coefficients are used to balance the three optimization objectives of information lifecycle, energy consumption, and processing accuracy.

[0150] By performing the above operations, this solution addresses the unresolved issues of neglecting system-level evaluation and multi-objective comprehensive optimization in information processing. It constructs a complete information lifecycle analysis model, clearly defining the start point, end point, stage composition, average value calculation method, and retransmission extension mechanism of the information lifecycle. This model is then integrated with the reward function and constraints of the decision-making model to achieve precise quantification and system-level optimization control of information freshness, ensuring the focus and operability of analysis and optimization.

[0151] Example 12, see Figure 1 This embodiment is based on the above embodiment. In step S5, the continuous optimization specifically includes the following steps:

[0152] Step S51: Incremental model update. Periodically collect system status data of the embedded edge computing system during actual operation, add the newly collected data to the training set, and update the decision model parameters using incremental learning.

[0153] Step S52: Establish a feedback mechanism, create a feedback loop between decision-making effectiveness and system performance, compare the actual observed information lifecycle, energy consumption, and processing accuracy indicators with the model predictions, calculate the deviation, and use it for parameter correction of the decision-making model.

[0154] Step S53: Iterative improvement of strategy. Evaluate the performance of the current strategy based on feedback data, and continuously improve the decision strategy by adjusting the decision model parameters, updating the reward function weights, and expanding the system state data.

[0155] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0156] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0157] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. An information processing method for an embedded edge computing system, characterized in that: The method includes the following steps: Step S1: System status data acquisition. Collect status data of each terminal device in the edge computing system, edge environment data, and computing resource status data of edge nodes. Preprocess and extract features from all collected data to construct a system status dataset. Step S2: Decision model construction. A centralized training and distributed execution framework is adopted to divide the embedded edge computing system into three layers: terminal device layer, edge node layer, and edge server layer. The data types, data flow, and conversion mechanisms of each layer are defined. Step S3: Information lifecycle analysis, establish a mathematical analysis model of the information lifecycle, and quantify the timeliness of the entire process from data packet generation to successful processing; Step S4: Design the reward function, comprehensively optimize the data packet information lifecycle, equipment energy efficiency and system computing accuracy objectives, and design the reward function for the decision model; Step S5: Continuous optimization. Regularly collect operational feedback from the edge computing system and iteratively optimize the parameters of the decision model and reward function.

2. The information processing method for an embedded edge computing system according to claim 1, characterized in that: In step S2, the construction of the decision model specifically includes the following steps: Step S21: Define the three-tier architecture for data storage, including the terminal device layer, the edge node layer, and the edge server layer; Step S22: Define the forward inference execution flow. Forward inference execution achieves low-latency millisecond-level response. The data path is from the edge server to the edge node to the terminal device. Step S23: Reverse training update stream definition. Reverse training update achieves quasi-static second-level cycle response. The data path is from the terminal device to the edge node to the edge server, and then back to the terminal device. Step S24: Data conversion, defining the format conversion operations for data transmission between different layers during the training phase; Step S25: Data maintenance, limiting data storage location and update triggering conditions, restricting data access by terminal devices, and restricting data storage by edge nodes; Step S26: Device migration. Perform heartbeat detection on edge nodes. When an edge node is detected to have failed, automatically migrate its subordinate terminal devices.

3. The information processing method for an embedded edge computing system according to claim 2, characterized in that: In step S21, the three-tier architecture data storage definition includes the following steps: Step S211: Terminal device layer storage definition. Each terminal device, as an independent intelligent agent, stores data related to its own local decision-making, including local observation history, local policy network parameters, and local experience replay pool. Step S212: Edge node layer storage definition. Edge nodes do not generate resource allocation decisions. They are used for data aggregation and temporary storage. The stored data includes the managed device mapping table, local resource snapshot matrix and local gradient cache area. Step S213: Edge server layer storage definition, construct the upper-layer controller as the training center, store the global dimension data of the entire embedded edge computing system, including upper-layer controller actions, global resource allocation state matrix, global state space vector, global value network parameters and hybrid network parameters, and global experience replay pool.

4. The information processing method for an embedded edge computing system according to claim 3, characterized in that: In step S22, the forward inference execution flow definition includes the following steps: Step S221: Upper-layer action generation, the edge server infers the action of the upper-layer global controller based on the current global state space vector and global value network; Step S222: Upper-layer action is sent down. The edge server sends the upper-layer action to all edge nodes in the form of broadcast. After receiving it, each edge node only extracts the bandwidth ratio and computing resource quota subset of its subordinate terminal devices and updates the corresponding column in the local resource snapshot matrix. Step S223: Quota Transmission. The edge node performs format conversion and protocol adaptation on the extracted bandwidth ratio and computing resource quota subset, and transmits them to its subordinate terminal devices. After receiving the bandwidth ratio and computing resource quota subset, the terminal device combines its own local observation vector, inputs local policy network parameters for forward inference, and outputs the lower-layer agent action. Step S224: Action execution. After forward reasoning is completed, the terminal device immediately applies the action to adjust its own physical parameters and caches the instantaneous state data after execution locally.

5. The information processing method for an embedded edge computing system according to claim 3, characterized in that: In step S23, the definition of the reverse training update stream includes the following steps: Step S231: Local gradient calculation. The terminal device randomly samples a small batch of samples from the local experience replay pool, calculates the gradient of the local policy network parameters, and calculates the time difference error between the local Q value and the target Q value. Step S232: Gradient upload. Each terminal device encapsulates the calculated gradient value, the most recent local observation, and the obtained reward value into an uplink data packet and sends it to the edge node to which it belongs. Step S233: Edge node gradient aggregation. After receiving the gradients uploaded by all terminal devices within its jurisdiction, the edge node performs a federated average aggregation operation to generate the average gradient of the edge node. Step S234: Local snapshot update. The edge node compresses its average gradient and the latest data in the local resource snapshot matrix and uploads them to the edge server. The incremental transmission method is used to send only the data that has changed relative to the previous round. Step S235: Global matrix update. The edge server receives the average gradient and local resource snapshot matrix from all edge nodes, updates the corresponding rows and columns in the global resource allocation state matrix, inputs the average gradient into the hybrid network, updates the global value network parameters by minimizing the global loss function, and simultaneously updates the hybrid network parameters using a soft update method. Step S236: Parameter synchronization and distribution. The edge server packages the updated global value network parameters and hybrid network parameters and broadcasts them to each edge node. The edge nodes forward them to all subordinate terminal devices, and the terminal devices replace the old parameters in their local policy network with the new parameters.

6. The information processing method for an embedded edge computing system according to claim 3, characterized in that: In step S24, the data transformation includes the following steps: Step S241: Downlink data conversion. When the edge server sends the upper-layer controller action, global value network parameters and hybrid network parameters, the edge node quantizes the network floating-point weight into an 8-bit integer, encapsulates the bandwidth ratio and computing resource quota value of its subordinate terminal devices into a binary control frame, and sends it to the terminal device through a short-range wireless protocol. After receiving the quantized data, the terminal device dequantizes it locally to restore it to a floating-point number and updates the local policy network. Step S242: Uplink data conversion. After the terminal device calculates the local gradient, it performs sparsification and quantization processing before sending, retaining only the top 10% of gradient elements with the largest absolute value, and converting these floating-point numbers into 16-bit half-precision format. After the edge node receives the sparsified and quantized gradients uploaded by all terminal devices, it performs dequantization and desparse operations to restore the complete floating-point gradient tensor, and performs federated average aggregation. The edge node quantizes the aggregated average gradient again into an 8-bit integer, and encapsulates it together with the key fields in the local resource snapshot matrix into an uplink frame and sends it to the edge server. After receiving it, the edge server performs dequantization to restore the floating-point gradient for updating the global value network.

7. The information processing method for an embedded edge computing system according to claim 3, characterized in that: In step S25, the data maintenance specifically involves: limiting the global resource allocation state matrix to be completely stored on the edge server; the edge server immediately updates the corresponding row of the global resource allocation state matrix whenever it receives a local resource snapshot matrix reported by an edge node; limiting terminal devices to only obtain a subset of bandwidth ratios and computing resource quotas related to themselves forwarded by edge nodes, and not to obtain data from any other terminal devices in the global resource allocation state matrix; and limiting edge nodes to only save the local resource snapshot matrix of their managed subset.

8. The information processing method for an embedded edge computing system according to claim 3, characterized in that: In step S26, the device migration includes the following steps: Step S261: Heartbeat packet sending. Each edge node sends a heartbeat packet to the edge server at fixed time intervals. The heartbeat packet contains the edge node's local resource snapshot matrix and managed device mapping table. Step S262: Heartbeat timeout determination. The edge server sets a timeout threshold and maintains a heartbeat timer for each edge node. If a heartbeat packet is not received from an edge node before the timeout threshold is exceeded, the edge node is determined to be faulty. Step S263: Device migration. The edge server reads a backup copy of the managed device mapping table of the faulty edge node, determines the terminal device that needs to be migrated, migrates it to the edge node closest to the faulty edge node, updates the managed device mapping table of the target edge node, and the terminal device re-establishes its connection with the target edge node.

9. The information processing method for an embedded edge computing system according to claim 1, characterized in that: In step S3, the information lifecycle analysis includes the following steps: Step S31: Definition of information lifecycle. The information lifecycle is the total time from the moment a data packet is generated at its online terminal device until the moment the data packet is successfully received and processed by the edge server. Step S32: Decompose the information lifecycle into stages, including sending queuing delay, transmission delay, and edge processing delay. Step S33: Information lifecycle evolution analysis. A first-come, first-served scheduling strategy is adopted for the task queues of edge nodes or edge servers. Data packets in the system enter the task queue in the order of their generation and the average information lifecycle of the system is calculated. Step S34: Retransmission detection. When a data packet transmission failure is detected, a retransmission is triggered. The expected information lifecycle of the data packet from its initial transmission to its final successful processing is calculated.

10. The information processing method for an embedded edge computing system according to claim 1, characterized in that: In step S4, the reward function is designed as follows: combining the three sub-objectives of minimizing information lifecycle, minimizing energy consumption, and maximizing processing accuracy, the reward of the terminal device at each moment is calculated. When the reward function is negative, it indicates a penalty for a longer information lifecycle, and a positive value indicates an incentive for high processing accuracy.

Images