Dual-mode communication method based on experience and data collaborative mutual verification

Abstract

The invention provides a dual-mode communication method based on experience and data collaborative mutual verification, which belongs to the technical field of communication, and comprises the following steps: updating a dual-mode communication experience library based on data result feedback, constructing an initial dual-mode communication expert experience library, and generating a sample to train a preliminary communication optimization model; testing is carried out, a specific scene with a poor model decision effect or with an optimization space is screened out, and a to-be-optimized test set is formed; complex comparison with judgment of an experience library is carried out. Once the decision of the model is confirmed to be better, the new data insight is utilized to correct or update the old experience with the strongest correlation but poor performance in the experience library. And a dual-mode communication optimization model updating mechanism based on an experience library is adopted. According to the method, the efficient, robust and adaptive decision making capability in a complex dynamic communication environment is realized, the over-fitting of the model to experience is avoided, and the robustness and adaptability of the method in an unknown scene are ensured.

Description

Technical Field

[0001] This invention provides a dual-mode communication method based on experience and data collaborative verification, belonging to the field of communication technology. Background Technology

[0002] With the intelligent development of communication networks and the rapid growth of multimodal data, heterogeneous network convergence has become crucial for ensuring a seamless, high-speed, and highly reliable communication experience for users. Dual-mode or multi-mode communication technologies aim to maximize spectrum resource utilization and system throughput by dynamically selecting and switching between communication channels of different standards or frequency bands. Faced with increasingly complex electromagnetic environments, changing service demands, and terminal energy limitations, how to formulate optimal channel selection and switching strategies has become a core technological challenge of common concern to academia and industry. Currently, intelligent optimization algorithms, especially those based on machine learning and reinforcement learning, are widely considered effective ways to solve such dynamic optimization problems due to their powerful environmental awareness and autonomous decision-making capabilities, and have become a research hotspot in this field.

[0003] However, existing intelligent optimization methods have significant limitations. On the one hand, systems based solely on expert rules, while possessing good interpretability, have static and finite knowledge bases that struggle to cover all potential communication scenarios. They also exhibit poor adaptability to sudden or unknown network conditions, leading to conservative decision-making. On the other hand, purely data-driven reinforcement learning models, while capable of discovering non-explicit policies through interaction with the environment, typically require massive amounts of sample data, have slow convergence speeds, and lack transparency in their decision-making processes, making it difficult to guarantee their reliability in critical tasks. More importantly, these two approaches are currently fragmented, lacking an effective closed-loop mechanism. This prevents expert experience from being iteratively updated from real-world data, and the model's discoveries cannot be systematically distilled into structured knowledge, thus limiting the continuous evolution of the system's overall performance. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a dual-mode communication method based on experience and data-driven collaborative verification. First, this invention constructs a bidirectional feedback mechanism between experience and data. High-performance decisions, superior to existing rules, discovered by the data-driven model in a simulation environment, are used to quantitatively correct and expand the expert experience base, solving the problem of the inability to dynamically update experiential knowledge. Second, this invention designs a phased and differentiated model update strategy. After the experience base is updated, the model is first rapidly updated based on the guidance strength, and then fine-tuned through real performance feedback, resolving the contradiction between model update efficiency and generalization performance. By constructing a closed loop of mutual verification and co-evolution between experience and data, this invention effectively integrates the prior advantages of expert knowledge with the exploratory capabilities of reinforcement learning, thereby achieving a more intelligent, efficient, and robust dual-mode communication decision-making system.

[0005] To address the problem that traditional expert experience bases suffer from rigid knowledge and are unable to learn new knowledge from the actual operation of data-driven models for self-iteration, resulting in insufficient adaptability in complex and dynamic environments, this invention proposes a dual-mode communication experience base update method based on data result feedback. First, an initial dual-mode communication expert experience base is constructed, and samples are generated based on this base to train a preliminary communication optimization model. Second, this model is tested in an independent simulation environment. Using preset performance criteria, specific scenarios where the model's decision-making performance is poor or where there is room for optimization are selected, forming a test set to be optimized. Finally, these scenarios are analyzed in depth, and efficient decision-making behaviors discovered by the model during exploration that are superior to existing experience base suggestions are compared with the judgments of the experience base. Once it is confirmed that the model's decision is better, this new data insight is used to correct or update the older experiences in the experience base that are most relevant but perform poorly, achieving feedback from data to experience.

[0006] To address the problem that global retraining is too costly when a model incorporates new knowledge, while simple fine-tuning struggles to balance knowledge absorption efficiency with final generalization performance, this invention proposes a dual-mode communication optimization model update mechanism based on an experience base. First, after the experience base is updated, the communication optimization model undergoes a first, focused guided update using the calculated guidance strength. Second, the previously selected test set to be optimized is given to this initially guided model for further decision-making, generating a new secondary validation sample set. Finally, the model undergoes a second update, completely abandoning any guidance from experience and relying solely on the actual performance in the secondary validation sample set, with the sole objective of achieving the optimal final decision-making effect, to fine-tune the model.

[0007] The specific technical solution is as follows:

[0008] S1: Update the dual-mode communication experience base based on data feedback; specifically including the following sub-steps:

[0009] S1.1 Construction of Dual-Mode Communication Experience Base and Initial Model Training

[0010] First, build a dual-mode communication experience base. , This represents the total number of experiences in the experience base. The [number]th experience in the experience base... experience This represents a rule. Based on an experience base. Generate a set of initial training samples Each sample is a tuple ,in, This is the current state; Actions to be taken; For instant rewards; This represents the state at the next time step. The samples are used to train an initial dual-mode communication optimization model, with the following network parameters: The training process aims to minimize the loss function. , is represented as:

[0011] (1)

[0012] In the formula, In the state Take action below The value function is given by the parameter . Neural network prediction. These are the parameters of the target network. As a discount factor, For the next state Possible actions to take. Instant rewards. The design takes into account the data transmission rate. and latency , is represented as:

[0013] (2)

[0014] In the formula, and These are the weighting coefficients for data transmission rate and latency, respectively.

[0015] S1.2 Optimize test set selection

[0016] The trained and optimized model is deployed in an independent simulation testing environment. Running Each time slot yields a series of decision history records. , . Indicates that the system is in The state at any given moment; Indicates in At any given moment, the model adjusts according to the state. The actions taken; Indicates in At any moment, perform the action Immediate rewards for environmental feedback; Indicates that the system is in The state at any given moment, i.e., after the action has been completed. The next state after that. To identify scenarios where the model makes poor decisions or has optimization potential, a performance judgment criterion is defined. When a criterion is triggered, it means that the current sample's performance has not met the standard, and the sample is identified as needing optimization, as shown below:

[0017] (3)

[0018] In the formula, This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. It is the average reward threshold. For window The variance of internal rewards This is the reward variance threshold. The criterion selects samples with low recent average rewards and high reward volatility. All samples meeting this criterion... samples This constitutes the test set that needs optimization. .

[0019] S1.3 Calculation of Decision Results Based on Experience Base

[0020] for Each sample state in Utilizing experience base Calculate the corresponding expert experience action and expected rewards This process involves weighted aggregation with the current state. The most relevant multiple experiences are implemented as follows:

[0021] (4)

[0022] In the formula, The current state With experience The state defined in The similarity function; Indicates based on experience Select Action The probability of.

[0023] S1.4 Revision and Update of the Experience Base

[0024] against For each sample, compare the model decisions. Decision-making with experience base When the model discovers a high-yield strategy not covered by the experience base, that is... Then it is believed that experience needs to be revised. To update the confidence boundary, Represented as:

[0025] (5)

[0026] In the formula, This represents the variance of the reward for decisions made in the experience base. If correction is needed, then... Most relevant dominant experience Update and generate new experiences. The update rules are as follows:

[0027] (6)

[0028] In the formula, It's the learning rate. It is the output of the model. Tuple conversion with experience Functions with compatible representations.

[0029] When the model training results are worse than expert experience, i.e. , To guide the confidence boundary, empirical samples should be used to further refine the model. Represented as:

[0030] (7)

[0031] Define guidance strength for At that time, experience The value of specifically correcting the decision-making flaws of a model is expressed as:

[0032] (8)

[0033] S2 The experience-based dual-mode communication optimization model update includes the following sub-steps:

[0034] S2.1 Initial Update of the Model Based on Guiding Strength

[0035] Using new empirical data, through a composite loss function that includes a guiding term. To update the model, represented as:

[0036] (9)

[0037] (10)

[0038] In the formula, Based on optimizing model parameters The first term is the mean squared error loss, and the second term is the guiding loss. The model policy was penalized. In state Deviating from experience-based suggested actions , behavior It is the coefficient of the leading term.

[0039] S2.2 Secondary Update of Performance-Driven Model Based on Secondary Validation

[0040] The test set that needs optimization selected in S1.2 As input, in the model updated in S2.1 Run the test again to obtain a secondary validation sample set. Then, based entirely on the actual performance of this new sample set, the standard reinforcement learning loss function is used. The final update to the model is represented as:

[0041] (11)

[0042] In the formula, the secondary verification sample set One of the samples Indicates for those from original state The updated model Actions were taken Received a new reward And moved to a new state. . The current network's value assessment of this decision, and For the target network to the next state The maximum value assessment, of which To and The corresponding target network parameters.

[0043] The technical effects of this invention are as follows:

[0044] 1. This invention proposes a dual-mode communication experience base update method based on data result feedback, constructing a framework for co-evolution and bi-directional verification between an experiential knowledge system and a data-driven model. The model can be systematically captured and used to feed back into and iterate the expert experience base. This cyclical mechanism of "data verifying and expanding experience, experience guiding and optimizing the model" ensures the dynamic completeness and timeliness of the knowledge base, achieving efficient, robust, and adaptive decision-making capabilities in complex and dynamic communication environments.

[0045] 2. This invention proposes a dual-mode communication optimization model update mechanism based on an experience base, constructing a phased and differentiated model update strategy aimed at balancing knowledge transfer efficiency and model generalization performance. The first phase utilizes the model defect dataset constructed in the previous phase to perform a weighted guided update of the network, with the goal of efficiently correcting identified specific decision errors. The second phase performs performance-driven fine-tuning, removing the hard constraints of prior knowledge and forcing the model to learn how to flexibly apply rule-based knowledge to changing environments through real-world interactive feedback. This ensures that the final decision is based solely on empirically optimal performance, thereby avoiding overfitting of the model to experience and guaranteeing its robustness and adaptability in unknown scenarios. Attached Figure Description

[0046] Figure 1 This is a flowchart of a dual-mode communication method based on experience and data collaborative verification according to the present invention. Detailed Implementation

[0047] The specific technical solution of the present invention will be described in conjunction with the accompanying drawings.

[0048] This invention proposes a dual-mode communication method based on experience and data-driven collaborative verification, comprising two parts: S1: a dual-mode communication experience base update method based on data result feedback; and S2: a dual-mode communication optimization model update mechanism based on the experience base. The flowchart is shown below. Figure 1 As shown.

[0049] S1: Update the dual-mode communication experience base based on data result feedback;

[0050] This invention establishes an initial expert experience base and uses the model's performance on the test set to back-verify the accuracy of the experience. If the model performs better than the experience judgment in a specific scenario, it indicates that the experience has limitations and the experience base needs to be corrected using model data, thus achieving "data feeding back into experience".

[0051] S1.1 Construction of Dual-Mode Communication Experience Base and Initial Model Training

[0052] First, build a dual-mode communication experience base. , This represents the total number of experiences in the experience base. The [number]th experience in the experience base... experience This can be expressed as, for example, "when the signal-to-noise ratio of channel A is below a threshold..." And the interference level of channel B is less than "When switching from A to B." (Based on an experience base) Generate a set of initial training samples Each sample is a tuple ,in, This is the current state; For actions to be taken (such as selecting or switching channels); For instant rewards; This represents the state at the next time step. This sample is used to train an initial dual-mode communication optimization model, with the following network parameters: The training process aims to minimize the loss function. , represented as

[0053] (1)

[0054] In the formula, In the state Take action below The value function is given by the parameter . Neural network prediction. These are the parameters of the target network. As a discount factor, For the next state Possible actions to take. Instant rewards. The design takes into account the data transmission rate. and latency , represented as

[0055] (2)

[0056] In the formula, and These are the weighting coefficients for data transmission rate and latency, respectively.

[0057] S1.2 Optimize test set selection

[0058] The trained optimization model (with parameters) Deployed in a separate simulation testing environment Running Each time slot yields a series of decision history records. , . Indicates that the system is in The state at any given moment; Indicates in At any given moment, the model adjusts according to the state. The actions taken; Indicates in At any moment, perform the action Immediate rewards for environmental feedback; Indicates that the system is in The state at any given moment, i.e., after the action has been completed. The next state after that. To identify scenarios where the model makes poor decisions or has optimization potential, a performance judgment criterion is defined. When a criterion is triggered, it means that the current sample's performance has not met the standard, and the sample is identified as needing optimization, as shown below:

[0059] (3)

[0060] In the formula, This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. It is the average reward threshold. For window The variance of internal rewards This is the reward variance threshold. This criterion filters out samples with low recent average rewards and high reward volatility. All samples that meet this criterion... samples This constitutes the test set that needs optimization. .

[0061] S1.3 Calculation of Decision Results Based on Experience Base

[0062] for Each sample state in Utilizing experience base Calculate the corresponding expert experience action and expected rewards This process involves weighted aggregation with the current state. The most relevant multiple experiences are implemented as follows:

[0063] (4)

[0064] In the formula, The current state With experience The state defined in Similarity functions, such as cosine similarity or radial basis functions. Indicates based on experience Select Action The probability of.

[0065] S1.4 Revision and Update of the Experience Base

[0066] against For each sample, compare the model decisions. Decision-making with experience base When the model discovers a high-yield strategy not covered by the experience base, that is... ( If the confidence boundary needs to be updated, then the experience is considered to need to be revised. Represented as:

[0067] (5)

[0068] In the formula, This represents the variance of the reward for decisions made in the experience base. If correction is needed, then... Most relevant dominant experience (Right now Update (maximum) to generate new experience. The update rules are as follows:

[0069] (6)

[0070] In the formula, It's the learning rate. It is the output of the model. Tuple conversion with experience Functions with compatible representations. This process means that the experience base incorporates new knowledge explored by the model in practice.

[0071] For example, there is an original rule in the experience base: "When the signal-to-noise ratio of channel A is lower than the threshold..." And the signal-to-noise ratio of channel B is higher than that of channel B. "When the time comes, immediately switch from channel A to channel B." This experience is a greedy strategy based on the current state, without considering the timing correlation of channel states and the overhead of the switching behavior itself. At any given moment, the system state This manifests as follows: the signal-to-noise ratio of channel A briefly drops below a certain level due to a sudden strong interference. The signal-to-noise ratio of channel B is just slightly higher than that of channel B. Based on prior experience, the system should perform a handover operation. However, the optimization model of this invention, through learning from historical data, has identified that channel A has stable overall performance, while channel B exhibits greater volatility, and channel handover itself introduces fixed delays and packet retransmission losses. Therefore, the model predicts that handover at this moment would do more harm than good, and thus chooses "no handover," continuing to remain on channel A. In the subsequent... In the timeline, interference on channel A is eliminated, and the signal-to-noise ratio (SNR) quickly returns to normal, while the SNR on channel B drops significantly. This demonstrates that the model's decisions yield higher practical rewards than empirical recommendations. Satisfying The update conditions. At this point, the system triggers an update to the experience, generating new experience. The updated experience is expressed as: "When in continuous..." Within a given time period, the average signal-to-noise ratio of channel A remains below the threshold. Furthermore, the average signal-to-noise ratio of channel B is consistently higher than that of channel B. "Only when the time is right will the switch from channel A to channel B be executed." By introducing the concepts of "continuous timing" and "average value," the new experience integrates the ideas of timing analysis, learns to ignore instantaneous channel fluctuations that are not worth the switching costs, and thus makes the decision-making logic more robust and far-sighted, achieving an effective transformation from data insights to knowledge rules.

[0072] When the model training results are worse than expert experience, i.e. ( To guide the confidence boundary, empirical samples should be used to further refine the model. Represented as:

[0073] (7)

[0074] Define guidance strength for At that time, experience The value of specifically correcting the decision-making flaws of a model is expressed as:

[0075] (8)

[0076] S2 Empirical Base-Based Dual-Mode Communication Optimization Model Update

[0077] To address the challenge of efficiently transferring knowledge to the optimized model after updating the experience base, this invention proposes a two-stage model update strategy. The first stage regenerates training data and updates model parameters based on the guidance strength; the second stage relies entirely on actual test performance for model fine-tuning, avoiding the accumulation of empirical biases. This phased update mechanism ensures both the effective transfer of empirical knowledge and the model's adaptability to real-world environments.

[0078] S2.1 Initial Update of the Model Based on Guiding Strength

[0079] Using new empirical data, through a composite loss function that includes a guiding term. To update the model, represented as:

[0080] (9)

[0081] (10)

[0082] In the formula, Based on optimizing model parameters The first term is the mean squared error loss, and the second term is the guiding loss. The model policy was penalized. In state Deviating from experience-based suggested actions , behavior This is the coefficient of the guiding term. The update at this stage utilizes proven and superior experience to efficiently and specifically address the shortcomings in its decision-making capabilities.

[0083] S2.2 Secondary Update of Performance-Driven Model Based on Secondary Validation

[0084] Experience-based guidance alone may cause the model to get stuck in local optima. To ensure the model's generalization performance and final decision-making effectiveness, a second-stage performance-driven update is needed. This involves updating the test set selected in S1.2 that requires optimization. As input, in the model updated in S2.1 Run the test again to obtain a secondary validation sample set. Then, based entirely on the actual performance of this new sample set, the standard reinforcement learning loss function is used. The final update to the model is represented as:

[0085] (11)

[0086] In the formula, the secondary verification sample set One of the samples Indicates for those from original state The updated model Actions were taken Received a new reward And moved to a new state. . The current network's value assessment of this decision, and For the target network to the next state The maximum value assessment, of which To and The corresponding target network parameters. The essential difference between this stage of the update and the first update is that it no longer relies on any symbolic experience guidance. Its purpose is to allow the model to respond to feedback from real interactions (i.e., The model undergoes "real-world testing" to smooth and generalize the empirical rules introduced during the first update, ensuring that the model is more adaptable to the environment. Ultimately, the sole optimization objective is to maximize actual performance (maximizing cumulative rewards). This two-step update strategy, from "experience-guided" to "practice-based fine-tuning," ensures that the model can quickly absorb new knowledge while maintaining robustness and optimality in new environments.

Claims

1. A dual-mode communication method based on experience and data collaborative verification, characterized in that, include: S1: Update the dual-mode communication experience base based on data result feedback; First, an initial dual-mode communication expert experience base was constructed, and samples were generated based on this base to train a preliminary communication optimization model. Second, this model was tested in an independent simulation environment. Through preset performance criteria, specific scenarios where the model's decision-making performance was poor or where there was room for optimization were selected, forming a test set to be optimized. Finally, these scenarios were analyzed in depth, and the efficient decision-making behaviors discovered by the model during exploration, which were superior to the suggestions of the existing experience base, were compared with the judgments of the experience base. Once it was confirmed that the model's decision-making was better, this new data insight was used to correct or update the old experiences in the experience base that were most relevant but performed poorly, realizing feedback from data to experience. S2: Dual-mode communication optimization model update based on experience base; First, after the experience base is updated, the communication optimization model is updated with a first, focused guided update using the calculated guidance strength. Second, the previously selected test set to be optimized is given to this initially guided model for further decision-making, thereby generating a brand new secondary validation sample set. Finally, the model is updated a second time. This update completely abandons any guidance from experience and is based entirely on the actual performance in the secondary validation sample set, with the sole goal of achieving the best final decision effect, to fine-tune the model.

2. The dual-mode communication method based on experience and data collaborative verification according to claim 1, characterized in that, S1 specifically includes the following sub-steps: S1.1 Construction of Dual-Mode Communication Experience Base and Initial Model Training First, build a dual-mode communication experience base. , The total number of experiences in the experience base; the first experience in the experience base. experience Represents a rule; based on an experience base. Generate a set of initial training samples Each sample is a tuple ,in, This is the current state; Actions to be taken; For instant rewards; This represents the state at the next time step; the sample is used to train an initial dual-mode communication optimization model, whose network parameters are... The training process aims to minimize the loss function. , is represented as: （1） In the formula, In the state Take action below The value function is given by the parameter . Neural network prediction; These are the parameters of the target network. As a discount factor, For the next state Possible actions to take; instant rewards The design takes into account the data transmission rate. and latency , is represented as: （2） In the formula, and These are the weighting coefficients for data transmission rate and latency, respectively. S1.2 Optimize test set selection The trained and optimized model is deployed in an independent simulation testing environment. Running Each time slot yields a series of decision history records. , ; Indicates that the system is in The state at any given moment; Indicates in At any given moment, the model adjusts according to the state. The actions taken; Indicates in At any moment, perform the action Immediate rewards for environmental feedback; Indicates that the system is in The state at any given moment, i.e., after the action has been completed. The next state after that; in order to identify scenarios where the model's decisions are poor or have optimization potential, a performance judgment criterion is defined. When a criterion is triggered, it means that the current sample's performance has not met the standard, and the sample is identified as needing optimization, as shown below: （3） In the formula, This is an indicator function that takes the value 1 when the condition is true and 0 otherwise. It is the average reward threshold. For window The variance of internal rewards It is the reward variance threshold; the criterion selects samples with low recent average rewards and large reward fluctuations; all samples that meet the criteria... samples This constitutes the test set that needs optimization. ; S1.3 Calculation of Decision Results Based on Experience Base for Each sample state in Utilizing experience base Calculate the corresponding expert experience action and expected rewards This process involves weighted aggregation with the current state. The most relevant multiple experiences are implemented as follows: （4） In the formula, The current state With experience The state defined in The similarity function; Indicates based on experience Select Action The probability of; S1.4 Revision and Update of the Experience Base against For each sample, compare the model decisions. Decision-making with experience base When the model discovers a high-yield strategy not covered by the experience base, i.e. Then it is believed that experience needs to be revised. To update the confidence boundary, Represented as: （5） In the formula, This represents the variance of the decision reward in the experience base; if correction is needed, then... Most relevant dominant experience Update and generate new experiences. The update rules are as follows: （6） In the formula, It's the learning rate. It is the output of the model. Tuple conversion with experience Functions with compatible representations; When the model training results are worse than expert experience, i.e. , To guide the confidence boundary, empirical samples should be used to further refine the model; Represented as: （7） Define guidance strength for At that time, experience The value of specifically correcting the decision-making flaws of a model is expressed as: （8）。 3. The dual-mode communication method based on experience and data collaborative verification according to claim 2, characterized in that, S2 includes the following sub-steps: S2.1 Initial Update of the Model Based on Guiding Strength Using new empirical data, through a composite loss function that includes a guiding term. To update the model, represented as: （9） （10） In the formula, Based on optimizing model parameters The first term is the mean squared error loss, and the second term is the guiding loss. The model policy was penalized. In state Deviating from experience-based suggested actions , behavior It is the coefficient of the leading term; S2.2 Secondary Update of Performance-Driven Model Based on Secondary Validation The test set that needs optimization selected in S1.2 As input, in the model updated in S2.1 Run the test again to obtain a secondary validation sample set. Then, based entirely on the actual performance of this new sample set, the standard reinforcement learning loss function is used. The final update to the model is represented as: （11） In the formula, the secondary verification sample set One of the samples Indicates for those from original state The updated model Actions were taken Received a new reward And moved to a new state. ; The current network's value assessment of this decision, and For the target network to the next state The maximum value assessment, of which To and The corresponding target network parameters.

Images