A random access control system and method for a heterogeneous wireless network

Abstract

The application discloses a random access control system and method for a heterogeneous wireless network, and belongs to the technical field of communication technology and artificial intelligence, and comprises the following: a node agent module for controlling the transmission of control signals of a reflective multiple access node radio frequency circuit; an observation agent module for periodically collecting state interaction data of all nodes within multiple time slots, generating semantic observation feedback and transmission probability disturbance items; a reflection agent module for performing strategy reasoning through a fine-tuned large language model, generating semantic reflection including specific strategy adjustment suggestions; a strategy agent module for updating a global transmission strategy; and a memory module for storing state interaction data, semantic observation feedback, semantic reflection and historical global transmission strategies. The application greatly reduces the dependence of traditional deep reinforcement learning methods on GPU computing power, and provides reliable data transmission protection for high real-time industrial communication systems.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of communication technology and artificial intelligence, and particularly relates to a random access control system and method for a heterogeneous wireless network. BACKGROUND

[0002] With the rapid development of key fields such as industrial Internet of Things, intelligent transportation systems, and remote medical treatment, a large number of distributed nodes (such as environmental monitoring sensors, vehicle-mounted communication units, medical sensing devices, etc.) need to periodically report real-time state information to the central control node to support accurate monitoring and control decisions. In such systems, the timeliness of information is one of the core indicators for measuring system performance. To quantify the "freshness" of information updates, the information age (Age of Information, AoI) indicator is proposed and widely used, which directly reflects the time difference between the information held by the receiving end and the real state of the source end, and is particularly important in scenarios such as industrial control, vehicle platoon coordination, and intelligent power grid scheduling, which have extremely high real-time requirements.

[0003] In the above application scenarios, wireless communication networks often exhibit high heterogeneity, with multiple devices nodes using different access protocols coexisting in the same channel. For example, some nodes use Time Division Multiple Access (TDMA) protocol for transmission in fixed time slots, some nodes use slotted ALOHA nodes to randomly access with a fixed probability, and emerging intelligent nodes may use access strategies based on Deep Reinforcement Learning (DRL). This multi-protocol coexistence heterogeneous environment poses a serious scheduling challenge for limited wireless resource sharing.

[0004] Currently, the existing technologies deployed in actual systems mainly face the following bottlenecks:

[0005] Poor adaptability of traditional fixed strategy protocols: standardized protocols such as TDMA and ALOHA have static or semi-static access mechanisms, lacking the ability to respond to dynamic changes in network state. When the network load changes or the number of nodes increases, it is easy to cause channel congestion and data packet collision to intensify, thereby significantly degrading the average AoI performance of the system, making it difficult to meet the needs of high real-time applications.

[0006] There are inherent defects in intelligent access methods based on deep reinforcement learning: although DRL methods can dynamically adjust strategies, the decision-making process is like a "black box", making it difficult for operators to understand and diagnose, and it is difficult to debug when facing sudden changes in network state. More importantly, the DRL training process usually requires millions of environmental interaction iterations, and there is a "data and computing power hunger" problem, which is computationally intensive, energy-consuming, and has high deployment costs, making it difficult to be widely applied on edge devices with limited computing resources.

[0007] In recent years, although some researches have attempted to introduce large language model-based communication protocol design methods (such as CP-AgentNet, etc.), and have made improvements in policy explainability and adaptability, the optimization effect of the system average AoI in the actual complex heterogeneous network environment still needs to be improved.

[0008] Therefore, there is an urgent need in the art for a wireless random access control technology solution that not only efficiently optimizes information timeliness, but also has an explainable decision-making process, controllable computing overhead, and can adapt to the dynamic changes of heterogeneous networks. SUMMARY

[0009] To solve the above technical problems, the present application provides a random access control system for a heterogeneous wireless network, comprising: a node agent module, an observation agent module, a reflection agent module, a policy agent module, and a memory module.

[0010] The node agent module is configured to generate a transmission control signal for controlling a reflection-based multiple access node radio frequency circuit according to a global transmission strategy provided by the policy agent module and a transmission probability perturbation term provided by the observation agent module, so that the radio frequency circuit transmits a data packet to a shared wireless channel at a determined time slot in response to the transmission control signal.

[0011] The observation agent module is configured to periodically collect state interaction data of all nodes in the heterogeneous wireless network within a plurality of time slots, the state interaction data including data packet collision, transmission success and channel idle events, statistically analyze the state interaction data to obtain network performance indicators, and generate semantic observation feedback for fine-tuning the transmission strategy and the transmission probability perturbation term according to the network performance indicators.

[0012] The reflection agent module is configured to generate a semantic reflection including specific policy adjustment suggestions by performing policy reasoning through a fine-tuned large language model according to a plurality of semantic observation feedback generated by the observation agent module within a single reflection period and historical reflection content.

[0013] The policy agent module is configured to update the global transmission strategy according to the semantic reflection generated by the reflection agent module.

[0014] The memory module is configured to store the state interaction data, the semantic observation feedback, the semantic reflection, and the historical global transmission strategy.

[0015] The heterogeneous wireless network includes time division multiple access nodes, time slot ALOHA nodes, and reflection-based multiple access nodes, and all nodes share the same wireless channel.

[0016] Optionally, the process of the node agent module generating the transmission control signal comprises:

[0017] Summing the global transmission probability provided by the policy agent module and the transmission probability perturbation value provided by the observation agent module, and limiting the summation result within a preset probability range to calculate a final transmission probability, and generating the transmission control signal corresponding to the final transmission probability.

[0018] Optionally, the process of the observation agent module generating the semantic observation feedback and the transmission probability perturbation item comprises:

[0019] In each observation period, a preset number of time slot data are counted to obtain network performance indicators including average information age, data packet collision rate and channel idle rate;

[0020] The network performance indicators of the current observation period are compared and analyzed with the network performance indicators of the last observation period;

[0021] Based on the comparison and analysis result, a natural language text describing the network state change trend is generated as the semantic observation feedback, and a numerical adjustment amount is generated as the transmission probability perturbation item.

[0022] Optionally, the process of the reflection agent module performing policy reflection through the fine-tuned large language model comprises:

[0023] All semantic observation feedback generated in the current reflection period, the semantic reflection generated in the last round and the historical excellent strategy samples extracted from the memory module are combined into prompt information;

[0024] The prompt information is input into the locally deployed fine-tuned large language model;

[0025] The natural language text output by the large language model, including the evaluation of the current strategy and the suggestion for subsequent strategy adjustment, is obtained as the semantic reflection generated in the current round.

[0026] Optionally, the process of fine-tuning the large language model comprises:

[0027] Different versions of semantic reflection generated in the system running process for the same strategy input are collected;

[0028] According to the change amount of the system average information age or the node average information age after each semantic reflection is adopted, high-quality reflection and low-quality reflection are labeled to construct a reflection memory dataset;

[0029] The reflection memory dataset is used to fine-tune the initial large language model through reinforcement learning to generate the fine-tuned large language model.

[0030] Optionally, the process of updating the global transmission strategy by the strategy agent module comprises:

[0031] Parsing the numerical part of the policy adjustment suggestion included in the semantic reflection;

[0032] Operating the parsed numerical adjustment amount with the current global transmission strategy to obtain a new global transmission strategy;

[0033] Storing the historical global transmission strategy that brings about the performance improvement of the information age and the corresponding performance improvement amount into the policy memory library of the memory module.

[0034] Optionally, the memory module comprises:

[0035] A short-term memory unit for cyclically storing the state and action data of all nodes in the last preset number of time slots;

[0036] A long-term memory unit for storing all semantic observation feedbacks generated by the observation agent module in one reflection cycle;

[0037] A reflection memory unit for storing historical semantic reflections and the corresponding information age performance improvement amounts generated by the reflection agent module;

[0038] A policy memory unit for storing the global transmission strategies that have historically brought about the performance improvement of the information age recorded by the strategy agent module.

[0039] The present application also proposes a random access control method for a heterogeneous wireless network, based on the system, comprising:

[0040] Generating a transmission control signal for controlling the reflection-type multiple access node radio frequency circuit according to the global transmission strategy provided by the strategy agent module and the transmission probability perturbation term provided by the observation agent module;

[0041] In response to the transmission control signal, transmitting data packets through the radio frequency circuit to the shared wireless channel in a determined time slot;

[0042] Periodically collecting the state interaction data of all nodes in the heterogeneous wireless network in multiple time slots by the observation agent module, wherein the state interaction data includes data packet collision, transmission success and channel idle events;

[0043] Statistically analyzing the state interaction data to obtain network performance indicators, and generating semantic observation feedbacks for fine-tuning the transmission strategy and the transmission probability perturbation term according to the network performance indicators;

[0044] The reflection agent module generates semantic reflection according to the semantic observation feedback generated by the observation agent module in a single reflection cycle and historical reflection content, and performs strategy inference using a fine-tuned large language model to generate semantic reflection including specific strategy adjustment suggestions;

[0045] The strategy agent module updates the global transmission strategy according to the semantic reflection generated by the reflection agent module;

[0046] The memory module stores the state interaction data, the semantic observation feedback, the semantic reflection, and the historical global transmission strategy;

[0047] The method is applied to a heterogeneous wireless network including time division multiple access nodes, time slot ALOHA nodes, and reflection-based multiple access nodes.

[0048] Optionally, the process of generating a transmission probability perturbation term according to network performance indicators includes:

[0049] When the analysis finds that the data packet collision rate rises above a first threshold value, a negative transmission probability perturbation term is generated;

[0050] When the analysis finds that the channel idle rate is higher than a second threshold value and the average information age continues to rise, a positive transmission probability perturbation term is generated.

[0051] Optionally, the memory module provides data support for the reflection process, including:

[0052] All semantic observation feedback in the current reflection cycle is obtained from the long-term memory unit of the memory module;

[0053] The semantic reflection generated in the last round is obtained from the reflection memory unit of the memory module;

[0054] One or more historical global transmission strategy samples that improve the information age performance are obtained from the strategy memory unit of the memory module;

[0055] The above information is collectively used as the input context of strategy inference.

[0056] Compared with the prior art, the present application has the following advantages and technical effects:

[0057] The application introduces a reflection type access control framework based on multi-agent cooperation and an RMA protocol, solves the channel congestion problem caused by the fixed strategy of the traditional protocol in the heterogeneous network environment, and solves the channel congestion problem caused by the fixed strategy of the traditional protocol in the heterogeneous network environment. In practical application scenarios such as industrial Internet of Things production line state monitoring and real-time traffic information interaction of Internet of Vehicles. The node agent module directly controls the radio frequency circuit transceiver signal, the observation agent module observes the channel collision and idle state in real time, the reflection agent module generates an interpretable strategy adjustment suggestion by using semantic understanding, and finally the system realizes the improvement of communication success rate and the reduction of transmission delay in a dynamic environment. The application significantly reduces the average information timeliness in various heterogeneous scenarios, and because the lightweight local model fine-tuning mechanism is adopted, the strategy optimization can be completed in a few observation cycles in the edge computing device, greatly reducing the dependence of the traditional deep reinforcement learning method on GPU computing power, and providing reliable data transmission guarantee for high real-time industrial communication system. BRIEF DESCRIPTION OF DRAWINGS

[0058] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application, and their

[0059] Figure 1 is a schematic diagram of a heterogeneous wireless network of an embodiment of the application;

[0060] Figure 2 is a schematic diagram of the change of node AoI over time of an embodiment of the application;

[0061] Figure 3 is a schematic diagram of the system structure of an embodiment of the application. DETAILED DESCRIPTION

[0062] It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0063] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0064] Embodiment one

[0065] In this embodiment, a random access control system for a heterogeneous wireless network is provided, which comprises a node agent module, an observation agent module, a reflection agent module, a strategy agent module and a memory module.

[0066] The node agent module is configured to generate a transmission control signal for controlling a radio frequency circuit of a reflexive multiple access node according to a global transmission strategy provided by the strategy agent module and a transmission probability perturbation term provided by the observation agent module, so that the radio frequency circuit transmits a data packet to a shared wireless channel at a determined time slot in response to the transmission control signal.

[0067] The observation agent module is configured to periodically collect state interaction data of all nodes in the heterogeneous wireless network within a plurality of time slots, the state interaction data including data packet collision, transmission success and channel idle events, statistically analyze the state interaction data to obtain network performance indicators, and generate semantic observation feedback for fine-tuning the transmission strategy and the transmission probability perturbation term according to the network performance indicators.

[0068] The reflection agent module is configured to perform strategy reasoning through a fine-tuned large language model according to a plurality of semantic observation feedbacks generated by the observation agent module within a single reflection cycle and historical reflection content, and generate semantic reflection including specific strategy adjustment suggestions.

[0069] The strategy agent module is configured to update the global transmission strategy according to the semantic reflection generated by the reflection agent module.

[0070] The memory module is configured to store the state interaction data, the semantic observation feedback, the semantic reflection and historical global transmission strategies.

[0071] In the heterogeneous wireless network, there are time division multiple access nodes, time slot ALOHA nodes and reflexive multiple access nodes, and all nodes share the same wireless channel.

[0072] As a specific implementation, the specific implementation process of the time-slotted heterogeneous wireless network includes:

[0073] The system runs in a wireless channel environment that is divided into fixed-length time slots in time. As shown in Figure 1 The network includes multiple types of wireless nodes. It is assumed that all nodes can only initiate transmission at the beginning of a time slot and must complete transmission within the same time slot. If multiple nodes transmit at the same time, collision will occur, resulting in transmission failure. Nodes in the network can use different access protocols, including TDMA, ALOHA, and the RMA (Reflexive Multiple Access) protocol node proposed in this paper. The details of each type of node are as follows:

[0074] Time Division Multiple Access Node (TDMA Node): This type of node selects a fixed number of time slots (e.g., X=2) in each frame (e.g., Y=10 time slots) for repetitive data transmission, and its transmission pattern remains consistent from frame to frame.

[0075] Slot ALOHA Node (ALOHA Node): This type of node independently decides whether to send a data packet in each time slot with a fixed probability (e.g., q=0.3), and its decision-making process does not depend on historical state or other nodes.

[0076] Reflective Multiple Access Node (RMA Node): This type of node is the core object controlled by the present embodiment, and it uses the RMA (Reflexive Multiple Access) protocol based on the Reflex-AgentNet framework described below. This protocol dynamically optimizes the information age performance (AoI) of the system in a heterogeneous network through a closed-loop control mechanism of "observation-reflection-strategy-execution". The node intelligently decides the transmission strategy in each time slot based on environmental information and historical experience, achieving efficient random access control.

[0077] In the network model, it is assumed that each node has a data packet to be transmitted in each time slot, which simulates the scenario of continuous reporting of status by nodes in Internet of Things and other applications.

[0078] The design and calculation process of information age in the system includes:

[0079] In the heterogeneous wireless network of the present embodiment, time is divided into time slots of fixed length, and each node has a latest data packet available for transmission in each time slot, i.e., the "generate-at-will" assumption is established. Nodes send time-stamped data packets to the access point (AP) through a shared wireless channel, and the timestamp of each data packet is set to the start time of its transmission slot to reflect the freshness of information.

[0080] In the same time slot, if multiple nodes send data packets simultaneously, collision will occur, resulting in the unsuccessful decoding of these data packets by the receiving end. Only when a single node successfully transmits, its AoI is reset to 1 (representing a one-time delay), otherwise the node's AoI will continue to accumulate in the next time slot.

[0081] Let the node be in time slot with current AoI , then define

[0082] ;

[0083] where represents the latest time slot successfully received by the receiving end The timestamp of the latest data packet. The AoI increases linearly over time, but drops sharply to 1 when a successful transmission event occurs, such as... Figure 2 As shown.

[0084] Node-based slotted AoI can define nodes in front The average AoI for each time slot is:

[0085] ;

[0086] For the network For each node, the System Average AoI (SystemAoI) is the cumulative average of the AoI of all nodes in each time slot.

[0087] ;

[0088] The system's average AoI directly reflects the freshness level of information in the network and is an important indicator for measuring the overall network performance. This design ensures that each node has a transmission opportunity in every time slot, and the changes in AoI can accurately reflect the impact of collisions and successful transmissions on information freshness, providing an evaluation basis for optimizing the RMA node strategy in Reflex-AgentNet.

[0089] The feasible implementation process of this system using the Reflex-AgentNet framework includes:

[0090] The Reflex-AgentNet framework achieves intelligent and dynamic management of Information Timeliness (AoI) in heterogeneous networks through continuous learning, feedback, and optimization. This framework addresses key issues in the background technology through the following technical design:

[0091] 1. Solve the "black box" problem of traditional DRL: Transform complex network states into interpretable natural language descriptions through semantic understanding mechanisms (e.g., if the collision rate increases by 30%, it is recommended to reduce the transmission probability), enabling industrial operations and maintenance personnel to understand the system decision-making process and solving the problem of the uninterpretable decision-making process of traditional deep neural networks.

[0092] 2. Reduced hardware resource consumption: Employing a reflective incremental optimization mechanism replaces the traditional DRL training method that requires millions of environment iterations. Each reflective cycle only requires 3-5 observations to generate an optimization strategy, significantly reducing the dependence on CPU / GPU resources and enabling it to run on resource-constrained edge computing devices.

[0093] like Figure 3 As shown, for ease of understanding and implementation, this embodiment divides the framework functions into the following modules for detailed description.

[0094] The implementation process of the node agent module includes:

[0095] The module is the final execution unit of control instructions on the reflective multiple access node. Its implementation process includes: being responsible for updating the global strategy in each time slot According to the global sending strategy provided by the strategy agent module (strategy Agent), and combined with the disturbance term generated by the observation agent module (observation Agent), the sending probability is adaptively adjusted to realize efficient random access. The node The final sending probability in the time slot is the sum of the global strategy sending probability and the disturbance term generated by the observation Agent, and the action result is fed back to the environment.

[0096] ;

[0097] Where is the global strategy sending probability (the value range is 0.1-0.9, which is dynamically adjusted according to the network congestion degree), is the disturbance term generated by the observation Agent (the value range is-0.1 to +0.1, which is used to fine-tune the sending strategy), to ensure that the node remains adaptive to environmental changes. The node executes the sending and feeds back the action result (success, failure, collision) to the environment.

[0098] The implementation process of the observation agent module includes:

[0099] The observation Agent periodically collects short-term experience of node interaction with the environment, analyzes the statistical results and generates semantic observation feedback, providing key support for the decision-making process.

[0100] In each reflection cycle, the observation Agent performs observations (O=3-5 times, determined according to the computing power of the edge device). Each observation cycle includes T time slots (T=200-400 time slots, determined according to the network size). In each time slot, the observation Agent analyzes the recent trajectory data, extracts key indicators such as AoI change, sending success rate, and collision rate. For the observation, the disturbance term is generated as follows:

[0101] ;

[0102] Where represents the real-time observation feedback provided by the observation Agent, including the average AoI of the current window and the last window, the comparison of the current strategy and the updated strategy after the environment feedback, and the state distribution statistics. Based on the performance comparison of the current window and the last window, combined with the environment state distribution analysis, the observation Agent generates the disturbance term of the sending probability and stored into the long-term memory module. Through real-time data analysis, the following specific observation feedback is generated:

[0103] For example, when detecting that the current window collision rate rises from 15% to 25%, the observation agent generates a negative perturbation term Δp_i(n)=-0.05, suggesting reducing the sending probability to alleviate network congestion; when detecting that the idle rate is as high as 70% and the AoI continues to rise, a positive perturbation term Δp_i(n)=+0.03 is generated, suggesting increasing the sending probability to improve information freshness.

[0104] The specific implementation process of the reflection agent module (reflection agent) includes:

[0105] The reflection agent is the core agent of the entire Reflex-AgentNet framework, responsible for guiding the strategy optimization and reflection generation process. Combined with the observation results of multiple rounds and the last reflection, the reflection LLM deployed locally is used to generate strategy reflections to provide optimization suggestions for the strategy agent, realizing the closed-loop optimization of the strategy.

[0106] The reflection agent quantitatively evaluates the improvement of AoI. This optimization index is dynamically determined by the number of RMA nodes in the environment:

[0107] When there is only one RMA node in the system, the average AoI of the optimization system is:

[0108] ;

[0109] When there are multiple RMA nodes in the system, the average AoI of the optimization node is:

[0110] ;

[0111] where and are the average AoI of the system before and after the strategy update, and are the average AoI of the node before and after the strategy update. If , it means that the reflection brings performance improvement.

[0112] The reflection process not only summarizes the performance of the node under the current strategy, but also actively analyzes the reasons for poor strategy performance and proposes targeted improvement suggestions. Through language-based reflection, semantic-level strategy optimization is achieved, and the following specific reflection feedback is generated:

[0113] Comprehensive analysis within the reflection period shows that the sending strategy p=0.35 (node ​​sending probability 35%) performed well overall in this round, but there is still room for optimization. Specifically, the average AoI in this reflection period was 45.1, an improvement from 48.7 slots in the previous round, and the system average AoI improved by 3.6 slots, indicating that the strategy adjustment was effective. However, in-depth analysis of the state distribution of each observation window revealed insufficient channel utilization: the average successful transmission rate was only 9% (the proportion of slots in which nodes successfully transmitted), while the proportion of observing others' successful transmissions was as high as 21% (the proportion of slots in which the node remained silent while other nodes successfully transmitted), and the collision rate remained at a reasonable level of 10%. This indicates that the current strategy is too conservative; although it avoids too many collisions, it misses a large number of usable transmission opportunities. Considering that the 21% success rate of observing others is much higher than the normal level, indicating a large number of idle transmission opportunities, and the 10% collision rate indicates that the network load is relatively light, leaving ample room to increase the sending frequency, and the system average AoI improved by 3.6, the current optimization direction is correct, and the improvement trend should be maintained. Based on the above analysis, it is recommended to increase the sending probability by 0.05 to p=0.40.

[0114] This semantic output mechanism solves the "black box" problem of the incomprehensible decision-making process in traditional DRL methods, enabling industrial operations personnel to understand the system's decision-making logic, facilitating system maintenance and fault diagnosis. The generated reflection results are passed to the policy agent to guide subsequent policy generation. This mechanism provides a completely new approach to optimizing traditional communication protocols through continuous self-reflection and policy improvement.

[0115] The specific implementation process of the policy agent module includes:

[0116] The policy agent is responsible for receiving optimization suggestions from the reflection agent and updating the system's global sending policy accordingly. The policy update process can be represented as follows:

[0117] ;

[0118] in The global sending strategy generated for the kth time (value range 0.1-0.9). To map language reflection into a function of strategy adjustment, The learning rate is used to control the magnitude of policy updates.

[0119] The core task of this module is to transform the results of language reflection into executable policy representation. In addition, the policy agent maintains a policy memory bank to store historical policy samples that have performed well in the past for reference or inheritance in future policy generation. This policy memory bank provides a historical reference for the reflection process, ensuring that the language-based optimization results can be effectively translated into system performance improvement. For example: when receiving the reflection suggestion "suggest increasing the sending probability by 0.05 to p=0.40", the policy agent will transform it into a new policy =0.40; if this policy brings AoI improvement, it will be stored in the policy memory bank for future reference.

[0120] Through the above-mentioned collaborative work of multiple agents, the system can achieve continuous policy update and performance optimization in a dynamic heterogeneous environment.

[0121] The Reflex-AgentNet framework can be implemented using a closed-loop control mechanism of "observation-reflection-policy-execution". The system runs in a reflection cycle as the basic unit:

[0122] At the beginning of each reflection cycle, the policy agent outputs a new global sending policy. The node agent continuously interacts with the environment based on this policy, with each observation cycle being slots. The observation agent analyzes the data of each observation cycle and generates feedback. When the number of observations of the observation agent reaches the preset threshold in a reflection cycle, the reflection agent triggers reflection.

[0123] The reflection agent generates semantic optimization suggestions based on the long-term memory of all observations in this reflection cycle, combined with the historical experience in the policy memory bank. The policy agent receives these suggestions and generates a new policy for the next reflection cycle, while maintaining the policy memory bank. Then the system enters the next reflection cycle and starts a new round of "observation-reflection-policy-execution" cycle.

[0124] The specific implementation process of the memory module includes:

[0125] This module serves as the "memory hub" of the system, providing data persistence and historical support for the entire intelligent framework. Its specific implementation consists of four sub-modules:

[0126] Short-term memory unit: its function is to act as a fixed capacity sliding window buffer, cyclically storing the state information (such as information age value) and action information (such as sending decision results) of all nodes in the last T time slots.

[0127] Specifically, the last ​The node state information of each time slot provides data basis for performance analysis of the observation agent. If the current time slot is represented as T, the short-term memory can be represented as:

[0128]

[0129] wherein are the state and action information of time slot respectively.

[0130] The long-term memory unit: its function is to store all O times of semantic observation feedback generated by the observation agent module in a complete reflection period, forming a complete observation record set of the period.

[0131] Specifically, it is formed by the observation agent based on the statistical information of multiple observations, and is used to support the strategy reflection and optimization analysis of the reflection agent:

[0132]

[0133] wherein is the statistical information of the th observation, is the total number of observations in the reflection period.

[0134] The reflection memory unit: its function is to record the historical output of the reflection agent module, and store the semantic reflection content generated each time, the information age performance improvement amount brought by the reflection being adopted, and the system state context at the time of generating the reflection in the form of triplets.

[0135] Specifically, it is maintained by the reflection agent, stores the quality evaluation results of strategy reflection, and is used to support subsequent model fine-tuning:

[0136]

[0137] wherein is the content of the th reflection, is the corresponding AoI improvement amount, is the system state context at the time of reflection. These records provide historical references for subsequent reflection optimization and model fine-tuning.

[0138] The strategy memory unit: its function is to be maintained by the strategy agent module, and is specially used to store those global sending strategy samples that have been verified to be successful (i.e. bring positive information age performance improvement amount) and their corresponding performance improvement amount in history, constituting a “good strategy case library”.

[0139] ​​​​Specifically, the global sending strategy samples that bring AoI improvement in history are maintained and stored by the strategy Agent, which are used to guide the generation of new strategies:

[0140] ;

[0141] wherein is the th strategy, is the corresponding AoI improvement amount. Only strategies that bring AoI improvement (ΔAoI>0) are stored.

[0142] Further, to ensure the performance of the reflection proxy module, the large language model used therein needs to be optimized, and the process of fine-tuning the reflection large language model includes:

[0143] This process is an offline and continuous improvement cycle aimed at improving the model's ability to generate high-quality reflections.

[0144] Data collection and construction:

[0145] During system operation, the reflection Agent continuously generates different versions of reflection content. For each strategy input, the system saves two structured reflections that meet the requirements and calculates the system average AoI change amount it brings in the next stage of operation. If reflection A leads to a greater decrease in system AoI, it is marked as "high-quality reflection", and the other is "low-quality reflection". These paired samples constitute the reflection memory dataset for subsequent fine-tuning training.

[0146] When constructing the dataset, select the reflection with better effect as the high-quality reflection, and determine the optimal reflection by comparing the AoI improvement amount brought by different reflections.

[0147] Supervised fine-tuning (SFT) stage:

[0148] Select well-formatted reflection texts as supervised samples to fine-tune the original reflection LLM, so that the output style, logical structure and expression method generated by it are more in line with the needs of the AoI optimization task. This stage uses a standard supervised learning objective function to train the model.

[0149] Reward model training stage:

[0150] Based on the difference in system AoI change, train the reward model to use the "information timeliness decrease" as an implicit reward signal to learn to distinguish between high-quality and low-quality reflection content, thereby establishing a measure of reflection quality. Reward model parameters are optimized through the relationship between reflection effect and AoI improvement.

[0151] PPO fine-tuning stage:

[0152] ​In combination with the SFT model and the reward model, the PPO algorithm under the RLHF framework is used to perform reinforcement learning fine-tuning on the reflection LLM. The update range is limited by policy constraints, so that the model can gradually optimize the output content while maintaining stability, making it more inclined to generate high-quality reflections that can effectively reduce the system AoI. The standard PPO algorithm objective function is used in this stage, including key elements such as importance weights, advantage estimates, and clipping parameters.

[0153] After the above process, the fine-tuned reflection LLM has stronger strategic analysis and generation capabilities, and can output more reasonable and targeted reflections, thereby improving the quality of strategy updates and significantly reducing the average AoI of the system, ultimately achieving self-evolution optimization at the agent level.

[0154] Compared with the prior art, the embodiment proposes a Reflex-AgentNet framework based on existing heterogeneous network random access technology, and is designed for information age of interest (AoI) optimization in heterogeneous networks. The main technical effects are as follows:

[0155] Optimization goal: Unlike existing methods that mainly focus on throughput or conflict rate, the Reflex-AgentNet framework focuses on AoI as the optimization goal, realizing the transformation from traditional performance indicators to timeliness indicators, and better meeting the demand for real-time information transmission in modern heterogeneous networks. This framework enables the system to effectively solve the information timeliness problem and significantly improve the performance of real-time systems.

[0156] Theoretical level: For the first time, a multi-agent framework based on large language models is applied to the AoI optimization problem, opening up a new research direction of using LLM agents for wireless access optimization. By introducing semantic understanding and reflection reasoning mechanisms into the field of wireless access optimization, a paradigm shift from data-driven to semantic-driven is achieved, providing a new technical path for solving AoI challenges in heterogeneous networks.

[0157] Framework architecture: A complete Reflex-AgentNet framework is designed, which adopts a closed-loop control architecture of "observation-reflection-strategy-execution". Through the collaborative work of four agent modules (node Agent, observation Agent, reflection Agent, and strategy Agent) and four types of memory modules (short-term memory, long-term memory, strategy memory, and reflection memory), the framework realizes the explainable design of communication protocols and intelligent decision-making.

[0158] On the basis of the original node Agent, observation Agent, and strategy Agent, the embodiment introduces a reflection Agent and adjusts the operation process of multiple rounds of observation, reflection, and strategy feedback, to realize continuous iteration and closed-loop optimization of node strategy updating. Specifically, the observation Agent collects short-term experience of node interaction with the environment and forms long-term memory; the reflection Agent uses these experiences and the strategy generated by the last reflection to make improvement suggestions; and the strategy Agent generates a new global sending strategy by synthesizing the suggestions of the reflection Agent and the historical strategy, thereby improving the adaptive ability and overall performance of the system strategy.

[0159] Protocol level: The RMA (Reflexive Multiple Access) protocol is proposed, which is designed specifically for the AoI optimization needs in heterogeneous network environments. Through intelligent semantic understanding and reflection mechanisms, intelligent nodes can work collaboratively with traditional TDMA and ALOHA nodes in heterogeneous networks, dynamically adjust transmission strategies according to real-time network states, and effectively reduce the system average AoI.

[0160] Reflection LLM fine-tuning mechanism:

[0161] The embodiment proposes a strategy adaptive optimization method combining reflection memory and RLHF fine-tuning mechanism. The system continuously collects reflection memory during operation and uses the system average AoI change as a reward signal to fine-tune the local reflection LLM offline through PPO optimization, thereby guiding the model to generate strategy feedback that better meets the system average AoI optimization goal.

[0162] The embodiment realizes continuous evolution and performance improvement of strategy optimization, significantly enhancing the adaptive learning and robustness of the system. The RLHF fine-tuning based on reflection memory and the gradual strategy optimization mechanism realize autonomous strategy optimization, enabling LLM-based agents to autonomously learn and improve transmission strategies in dynamic network environments, significantly reducing the dependence on human intervention.

[0163] System implementation and experimental design:

[0164] The technical effects of the embodiment are reflected in two levels:

[0165] First, at the application level, the embodiment effectively solves the information timeliness optimization problem in heterogeneous wireless networks. Through the intelligent collaboration of the Reflex-AgentNet framework, it provides better AoI performance for heterogeneous wireless network environments. Experimental results show that in various heterogeneous network scenarios, the embodiment achieves a 10% to 21% improvement in system average AoI compared to the CP-AgentNet method, providing more reliable data transmission guarantees for application scenarios that require real-time information transmission (such as IoT monitoring, vehicle-to-vehicle communication, etc.).

[0166] Second, in terms of technical characteristics, the computing power requirement of the embodiment is significantly reduced: traditional DRL methods require training deep neural networks, usually requiring high-performance GPU support, and the training process is computationally intensive, time-consuming and memory-intensive. The embodiment uses a locally deployed open-source LLM model for fine-tuning, without training the neural network from scratch, significantly reducing the computing power requirement, enabling the algorithm to be deployed and run on ordinary computing devices.

[0167] Policy update logic is transparent: the policy update process of CP-AgentNet is relatively complex, and it is difficult to understand its decision logic. The embodiment makes the policy optimization process completely transparent through the semantic reflection feedback mechanism, and can generate specific policy adjustment explanations such as "the current policy is too conservative, suggest increasing the sending probability by 0.02", which facilitates understanding of "why" the system makes specific policy adjustments, and is beneficial to system development and maintenance.

[0168] On the other hand, the embodiment also proposes a random access control method for a heterogeneous wireless network, based on the system, comprising:

[0169] According to the global sending policy provided by the policy agent module and the sending probability perturbation term provided by the observation agent module, a sending control signal for controlling the reflection-type multiple access node radio frequency circuit is generated;

[0170] In response to the sending control signal, data packets are sent to the shared wireless channel through the radio frequency circuit at a determined time slot;

[0171] The observation agent module periodically collects state interaction data of all nodes in the heterogeneous wireless network within multiple time slots, and the state interaction data includes data packet collision, sending success and channel idle events;

[0172] Statistical analysis is performed on the state interaction data to obtain network performance indicators, and semantic observation feedback for fine-tuning the sending policy and the sending probability perturbation term are generated according to the network performance indicators;

[0173] Through the reflection agent module, according to the multiple semantic observation feedback generated by the observation agent module within a single reflection period and the historical reflection content, a large language model is used for policy reasoning to generate semantic reflection including specific policy adjustment suggestions;

[0174] Through the policy agent module, the global sending policy is updated according to the semantic reflection generated by the reflection agent module;

[0175] The state interaction data, the semantic observation feedback, the semantic reflection and the historical global sending policy are stored in the memory module;

[0176] The method is applied to a heterogeneous wireless network including time division multiple access nodes, time slot ALOHA nodes and reflective multiple access nodes.

[0177] Further, the process of generating the transmission probability perturbation item according to the network performance index comprises:

[0178] When the analysis finds that the data packet collision rate rises above a first threshold, a negative transmission probability perturbation item is generated; when the analysis finds that the channel idle rate is higher than a second threshold and the average information age continues to rise, a positive transmission probability perturbation item is generated.

[0179] Further, the memory module provides data support for the reflection process, which comprises:

[0180] The memory module provides data support for the reflection process, which comprises:

[0181] Experimental environment:

[0182] Five different network environment scenarios are set up in the experiment, and the main difference between each scenario is the type and number of nodes in the system, covering a variety of network structures from simple homogeneous to complex heterogeneous:

[0183] Scenario one: a simple heterogeneous environment composed of 1 ALOHA, 1 TDMA and 1 RMA node; Scenario two: a medium complexity heterogeneous environment composed of 3 ALOHA, 1 TDMA and 1 RMA node;

[0184] Scenario three: a homogeneous environment composed of 2 RMA nodes;

[0185] Scenario four: a complex heterogeneous environment composed of 1 ALOHA, 1 TDMA and 2 RMA nodes;

[0186] Scenario five: a highly competitive heterogeneous environment composed of 3 ALOHA, 1 TDMA and 2 RMA nodes.

[0187] Evaluation index setting:

[0188] According to the characteristics of different scenarios, differentiated optimization evaluation indexes are used to objectively measure the technical effect:

[0189] System-level optimization evaluation (Scenario 1, Scenario 2): The system average AoI is used as the main evaluation index, and the optimization effect of the information timeliness of the whole network is focused on, and the technical advantages of the application in improving the overall network performance are verified.

[0190] Node-level optimization evaluation (Scenario 3, Scenario 4, Scenario 5): The RMA node's own average AoI is used as the main evaluation index, and the individual performance of the RMA protocol in the competitive environment is focused on, and the self-adaptive ability in the complex network environment is verified.

[0191] Technical effect comparison and analysis:

[0192] System average AoI optimization effect:

[0193] In the scenario with system average AoI as the optimization target, the Reflex-AgentNet framework and the CP-AgentNet framework are compared in performance. The CP-AgentNet framework corresponds to the LLMA node, and the Reflex-AgentNet framework corresponds to the RMA node. According to the experimental results, in a simple heterogeneous environment (Scenario 1), the system average AoI of the present embodiment is reduced by about 10.3% compared with the prior art; in a complex environment with an increasing number of competitive nodes (Scenario 2), the performance improvement is more obvious, and the system average AoI is improved by 10.1%. This result shows that the reflection mechanism of the present embodiment has good adaptability and effectiveness in system-level optimization tasks of different complexity, and the technical advantage is more prominent as the network complexity increases.

[0194] Node average AoI optimization effect:

[0195] In the scenario with RMA node's own average AoI as the optimization target, Reflex-AgentNet also shows superior performance. According to the experimental results, in a homogeneous RMA environment (Scenario 3), an AoI reduction of 13.8% is achieved; in a medium-complexity heterogeneous environment (Scenario 4), the performance improvement reaches 14.7%; in a highly competitive complex environment (Scenario 5), the present embodiment achieves the greatest performance advantage, and the node average AoI is reduced by 14.9% compared with the prior art. This shows that the reflection mechanism has good adaptability and effectiveness in node-level optimization tasks of different complexity.

[0196] Comprehensive technical effect summary:

[0197] From the overall experimental verification results, it can be seen that the Reflex-AgentNet framework proposed in the present embodiment performs better than the prior art in all five scenarios. Especially in the scenarios of Scenario 2, Scenario 4 and Scenario 5 with higher complexity of the competitive environment, the performance improvement brought by the present embodiment is more significant.

[0198] The technical verification results show that:

[0199] Good adaptability: This embodiment can adapt to various network environments from simple homogeneity to complex heterogeneity;

[0200] Significant performance improvement: In different scenarios, the AoI performance can be improved by 10% to 15%;

[0201] Complexity advantage is obvious: With the increase of network complexity, the technical advantage is more prominent;

[0202] Strong robustness: In a highly competitive complex environment, it can still maintain superior performance.

[0203] The above experimental results fully verify the effectiveness and robustness of this embodiment in different complexity communication environments, and prove that the Reflex-AgentNet framework has significant technical progress and practical value compared with existing technologies.

[0204] Application prospect:

[0205] The embodiment proposes a heterogeneous network random access optimization framework based on a large language model, namely Reflex-AgentNet. By introducing a multi-agent collaboration mechanism and a reflection fine-tuning method, the framework realizes a semantic closed-loop control of "observation-reflection-strategy-execution" and exhibits significant information timeliness optimization potential in a heterogeneous wireless network environment. The core contributions of the embodiment include: in terms of theory, the multi-agent framework based on reflection is applied to the AoI optimization of heterogeneous wireless networks for the first time, providing a new paradigm for traditional communication protocol design; in terms of method, the complete Reflex-AgentNet framework is designed, including four modules of node agent, observation agent, reflection agent and strategy agent, and four memory modules of short-term memory, long-term memory, strategy memory and reflection memory, combined with the reflection fine-tuning mechanism based on RLHF, realizing intelligent decision-making at the semantic level; in terms of protocol, the RMA (Reflexive Multiple Access) protocol is proposed to realize intelligent random access control in a heterogeneous network; in terms of mechanism, an intelligent scheduling mechanism based on semantic understanding is designed, which realizes the adaptive optimization strategy that traditional reinforcement learning needs a large amount of training through the semantic reasoning ability of LLM. Experimental results show that the technical scheme can realize a performance improvement of 10% to 15% in different complexity environments, and the technical advantage is more prominent with the increase of network complexity, fully proving the effectiveness and robustness of the framework. The embodiment has broad application prospects and can be widely applied to important fields such as industrial Internet of Things, intelligent transportation system, remote medical treatment, smart city construction and 5G / 6G network application. With the continuous development of related technologies, the Reflex-AgentNet framework proposed in the embodiment provides an important technical foundation for the development of agent networks, and has important significance in terms of theoretical innovation and practical value.

[0206] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application can be easily thought of by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A random access control system for heterogeneous wireless networks, characterized in that, include: The module includes a node proxy module, an observation proxy module, a reflection proxy module, a strategy proxy module, and a memory module. The node proxy module is used to generate a transmission control signal for controlling the radio frequency circuit of the reflective multiple access node based on the global transmission strategy provided by the strategy proxy module and the transmission probability perturbation term provided by the observation proxy module, so that the radio frequency circuit transmits data packets to the shared wireless channel in a determined time slot in response to the transmission control signal. The observation agent module is used to periodically collect state interaction data of all nodes in the heterogeneous wireless network in multiple time slots. The state interaction data includes packet collision, transmission success and channel idle events. The module performs statistical analysis on the state interaction data to obtain network performance indicators, and generates semantic observation feedback and transmission probability perturbation terms for fine-tuning the transmission strategy based on the network performance indicators. The reflection agent module is used to generate semantic reflections, including specific strategy adjustment suggestions, based on multiple semantic observation feedbacks generated by the observation agent module within a single reflection cycle and historical reflection content, through a finely tuned large language model for strategy reasoning. The policy proxy module is used to update the global sending policy based on the semantic reflection generated by the reflection proxy module; The memory module is used to store the state interaction data, the semantic observation feedback, the semantic reflection, and the historical global sending strategy; The heterogeneous wireless network includes time-division multiple access nodes, time-slotted ALOHA nodes, and reflective multiple access nodes, with all nodes sharing the same wireless channel.

2. The system according to claim 1, characterized in that, The process by which the node proxy module generates and sends control signals includes: The global transmission probability provided by the strategy proxy module is summed with the transmission probability perturbation value provided by the observation proxy module, and the summation result is limited to a preset probability range. The calculation result is used as the final transmission probability to generate the corresponding transmission control signal.

3. The system according to claim 1, characterized in that, The process by which the observation agent module generates semantic observation feedback and transmission probability perturbation terms includes: Within each observation period, a preset number of time slot data are collected to obtain network performance indicators including average information age, packet collision rate, and channel idle rate. Compare and analyze the network performance metrics of the current observation period with those of the previous observation period; Based on the results of the comparative analysis, natural language text describing the trend of network state changes is generated as semantic observation feedback, and a numerical adjustment is generated as a perturbation term for the transmission probability.

4. The system according to claim 1, characterized in that, The process by which the reflexive agent module performs policy reasoning through a finely tuned large language model includes: All semantic observation feedback generated in the current reflection cycle, semantic reflection generated in the previous round, and historical best strategy samples extracted from the memory module are combined into a prompt message; The prompt message is input into the locally deployed, finely tuned large language model; The natural language text output by the large language model, including the evaluation of the current strategy and suggestions for subsequent strategy adjustments, is obtained as the semantic reflection generated in this round.

5. The system according to claim 4, characterized in that, The process of fine-tuning the large language model includes: Collect semantic reflections on different versions of the same policy input generated during system operation; Based on the change in the average information age of the system or the average information age of the node itself after each semantic reflection is adopted, high-quality reflections and low-quality reflections are labeled, and a reflection memory dataset is constructed. Using the aforementioned reflective memory dataset, the initial large language model is fine-tuned through reinforcement learning to generate a fine-tuned large language model.

6. The system according to claim 1, characterized in that, The process by which the policy proxy module updates the global sending policy includes: Analyze the numerical portion of the strategy adjustment suggestions included in the semantic reflection; The parsed numerical adjustment amount is calculated with the current global transmission strategy to obtain a new global transmission strategy; The historical global sending strategies that improve information age performance and their corresponding performance improvements are stored in the strategy memory bank of the memory module.

7. The system according to claim 1, characterized in that, The memory module includes: Short-term memory units are used to cyclically store the state and action data of all nodes within the most recent preset number of time slots; A long-term memory unit is used to store all semantic observation feedback generated by the observation agent module within a reflection cycle; A reflection memory unit is used to store historical semantic reflections generated by the reflection agent module and their corresponding information age performance improvement amounts; The strategy memory unit is used to store global transmission strategies that have historically improved information age performance, as recorded by the strategy agent module.

8. A random access control method for heterogeneous wireless networks, characterized in that, The system based on any one of claims 1-7 comprises: Based on the global transmission strategy provided by the strategy agent module and the transmission probability perturbation term provided by the observation agent module, a transmission control signal is generated to control the radio frequency circuit of the reflective multiple access node. In response to the transmission control signal, data packets are transmitted to the shared wireless channel via the radio frequency circuit in a determined time slot; The proxy module periodically collects state interaction data of all nodes in the heterogeneous wireless network in multiple time slots. The state interaction data includes packet collision, transmission success and channel idle events. Statistical analysis is performed on the state interaction data to obtain network performance indicators, and semantic observation feedback and the transmission probability perturbation term are generated based on the network performance indicators for fine-tuning the transmission strategy. Through the reflection agent module, based on the multiple semantic observation feedbacks generated by the observation agent module within a single reflection cycle and the historical reflection content, a finely tuned large language model is used to perform strategy reasoning to generate semantic reflections that include specific strategy adjustment suggestions. The global sending policy is updated by the policy proxy module based on the semantic reflection generated by the reflection proxy module. The memory module stores the state interaction data, the semantic observation feedback, the semantic reflection, and the historical global sending strategy. The method is applied to heterogeneous wireless networks including time-division multiple access nodes, time-slotted ALOHA nodes, and reflective multiple access nodes.

9. The method according to claim 8, characterized in that, The process of generating the transmission probability perturbation term includes: When the analysis finds that the packet collision rate has increased beyond the first threshold, a negative transmission probability perturbation term is generated. When the analysis finds that the channel idle rate is higher than the second threshold and the average information age continues to rise, a positive transmission probability perturbation term is generated.

10. The method according to claim 8, characterized in that, The process of providing data support for strategy reasoning based on the memory module includes: Retrieve all semantic observation feedback within the current reflection cycle from the long-term memory unit of the memory module; Retrieve the semantic reflection generated in the previous round from the reflective memory unit of the memory module; Obtain one or more global transmission strategy samples that have historically improved information age performance from the strategy memory unit of the memory module; The above information is used together as the input context for policy reasoning.

Images