Method for determining radio frequency link resource optimization model of heterogeneous network and related equipment

Abstract

The invention provides a method for determining a radio frequency link resource optimization model of a heterogeneous network and related equipment. The method comprises the following steps of: performing uniform mathematical modeling representation on signal coexistence, interference, blockage, loss and time division multiplexing link budget processes of parallel signal links in a multi-band, multi-mode and multi-system reconfigurable radio frequency front end; based on the mathematical modeling representation, dynamically configuring a regulation and control system through a topological connection mode of a time division multiplexing reconfigurable radio frequency front-end resource pool and a signal chain parameter; and optimizing dynamic topology planning and link parameters of the time division multiplexing reconfigurable radio frequency front-end resource pool by using a multi-agent deep reinforcement learning method.

Description

Technical Field

[0001] The present application relates to the technical field of resource optimization, and in particular to a method for determining a radio frequency link resource optimization model for a heterogeneous network and related equipment. Background Art

[0002] With the rapid development of mobile communication technology and the growth of communication needs, traditional RF front-ends can no longer meet the requirements of flexible, efficient and adaptable configuration in multi-functional, multi-standard and multi-band environments.

[0003] System complexity is particularly evident in complex massive MIMO (Multiple-Input Multiple-Output) architectures, where the RF front-end topology with its numerous channels leads to significant mutual interference between multiple antennas and signal paths. This interference complicates link design and places higher demands on the transceiver architecture. Furthermore, managing signal strength, loss, and interference across numerous parallel signal paths poses challenges. Traditional solutions rely heavily on domain expertise and are resource-intensive, struggling to cope with the complexity of multi-band, multi-mode, heterogeneous communication links and parameter inference. Furthermore, the response evaluation process is often time-consuming, making timely optimization and implementation difficult. As the number of bands, channels, and modes supported by reconfigurable RF front-ends increases, the complexity of the transmit and receive channels, the number of components / devices / modules, the planning of component / device / module characteristics (e.g., linearity, selectivity, dynamic range), and the interconnection and parameter configuration between different components / devices / modules all increase accordingly. As circuit scale grows, the solution space for planning strategies grows exponentially. At the same time, the discontinuous nature of dynamic cross-coupling effects (such as interference, blocking, and spurious emissions) between multiplexed RF system links, as well as the non-convex constraints introduced by these interactions, further increase the design complexity. Therefore, optimizing and designing a multi-dimensional, reconfigurable RF front-end architecture to accommodate multi-mode, multi-band, and multi-standard requirements while minimizing the size of the system's RF links presents significant challenges in terms of circuit topology construction and link planning. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a method for determining a radio frequency link resource optimization model for a heterogeneous network and related equipment.

[0005] Based on the above objectives, the present application provides a method for determining a radio frequency link resource optimization model for a heterogeneous network, including:

[0006] Determine RF components and RF resource pools;

[0007] In response to determining that communication is to be performed through a heterogeneous communication network, determining, based on the radio frequency component and the radio frequency resource pool, a first interference signal, a second interference signal, a signal output response of any radio frequency front-end device, and communication link performance;

[0008] Determining an optimization function for the radio frequency devices and radio frequency modules in the radio frequency resource pool according to the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device, and the communication performance of the communication link;

[0009] Decomposing the optimization function by a Chebyshev aggregation method to determine multiple single-objective optimization subproblems, and solving the multiple single-objective optimization subproblems by a multi-intelligence reinforcement learning method;

[0010] The pre-built strategy model is trained based on the solution results to determine the RF link resource optimization model, and link resource optimization is performed based on the RF link resource optimization model.

[0011] Optionally, the first interference signal is an interference signal of a radio frequency front-end device, and the second interference signal is an interference signal of a communication channel.

[0012] Optionally, solving the multiple single-objective optimization sub-problems by a multi-intelligence reinforcement learning method includes:

[0013] Establish a multi-intelligence reinforcement learning model based on the multi-intelligence reinforcement learning method;

[0014] The multiple single-objective optimization sub-problems are solved through the multi-intelligent reinforcement learning model.

[0015] Optionally, the solution result includes: environmental status information;

[0016] The step of training a pre-built strategy model based on the solution result to determine a radio frequency link resource optimization model includes:

[0017] Inputting the environmental state information into a neural network model to determine an embedding vector;

[0018] The pre-built strategy model is trained according to the embedding vector to determine the radio frequency link resource optimization model.

[0019] Optionally, the strategy model is a deep reinforcement learning model.

[0020] Optionally, the neural network model is a convolutional neural network model.

[0021] Based on the same inventive concept, an embodiment of the present application further provides a radio frequency resource optimization device based on heterogeneous communication, including:

[0022] A first determining module is configured to determine a radio frequency component and a radio frequency resource pool;

[0023] A second determination module is configured to determine, in response to determining that communication is performed through a heterogeneous communication network, a first interference signal, a second interference signal, a signal output response of any RF front-end device, and communication link performance based on the RF component and the RF resource pool;

[0024] A module configured to determine an optimization function of the radio frequency devices and radio frequency modules in the radio frequency resource pool based on the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device, and the communication performance of the communication link;

[0025] a decomposition module configured to decompose the optimization function by using a Chebyshev aggregation method, determine a plurality of single-objective optimization subproblems, and solve the plurality of single-objective optimization subproblems by using a multi-intelligence reinforcement learning method;

[0026] The optimization module is configured to train a pre-built strategy model according to the solution result, determine the radio frequency link resource optimization model, and determine according to the radio frequency link resource optimization model.

[0027] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the method for determining the radio frequency link resource optimization model of a heterogeneous network as described in any one of the above items.

[0028] Based on the same inventive concept, an embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned methods for determining a radio frequency link resource optimization model for a heterogeneous network.

[0029] Based on the same inventive concept, an embodiment of the present application also provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes any of the above-mentioned methods for determining a radio frequency link resource optimization model for a heterogeneous network.

[0030] From the above, it can be seen that the present application provides a method for determining a radio frequency link resource optimization model for a heterogeneous network, including: determining a radio frequency component and a radio frequency resource pool; in response to determining to communicate through a heterogeneous communication network, determining a first interference signal, a second interference signal, a signal output response of any radio frequency front-end device, and a communication link performance according to the radio frequency component and the radio frequency resource pool; determining an optimization function of the radio frequency device and the radio frequency module in the radio frequency resource pool according to the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device, and the communication link performance; decomposing the optimization function by the Chebyshev aggregation method to determine multiple single-objective optimization sub-problems, and solving the multiple single-objective optimization sub-problems by a multi-intelligent reinforcement learning method; training a pre-built strategy model based on the solution results to determine a radio frequency link resource optimization model, and optimizing the link resources according to the radio frequency link resource optimization model. By analyzing the adaptive framework and method of the radio frequency front-end in the heterogeneous communication system, the resource requirements in multi-band, multi-channel, multi-architecture, and multi-standard heterogeneous communication systems are predicted and managed. Combined with the current scheduling capability limitations of the RF front-end resource pool, the problem of efficient resource management for diversified communication needs is transformed into a problem of dynamic resource scheduling optimization within the RF front-end resource pool, thereby improving the efficiency of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 A flowchart of a method for determining a radio frequency link resource optimization model for a heterogeneous network according to an embodiment of the present application;

[0033] Figure 2 This is a schematic diagram of the dynamic construction process of a multi-channel RF front-end link according to an embodiment of the present application;

[0034] Figure 3 Schematic diagram of the representation and embedding process of the system environment state and policy network architecture of the intelligent agent k in an embodiment of the present application;

[0035] Figure 4 Schematic diagram of the architecture of a multi-agent deep reinforcement learning model for a heterogeneous communication system based on adaptive management and control of a radio frequency front-end resource pool according to an embodiment of the present application;

[0036] Figure 5This is a structural diagram of a radio frequency resource optimization device based on heterogeneous communication according to an embodiment of the present application;

[0037] Figure 6 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0039] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0040] In order to make this solution clearer and easier to understand, the following is a Figures 1-6 ) A method for determining a radio frequency link resource optimization model for a heterogeneous network provided in an embodiment of the present disclosure is described in detail.

[0041] As described in the background technology section, with the rapid development of mobile communication technology and the growth of communication needs, traditional RF front-ends can no longer meet the requirements of flexible, efficient and adaptable configuration in multi-functional, multi-standard and multi-band environments.

[0042] System complexity is particularly evident in complex massive MIMO architectures, where the RF front-end topology with numerous channels leads to significant mutual interference between multiple antennas and signal paths. This interference complicates link design and places higher demands on transceiver architectures. Furthermore, managing signal strength, loss, and interference across numerous parallel signal paths poses challenges. Traditional solutions rely heavily on domain expertise and are resource-intensive, struggling to cope with the complexity of multi-band, multi-mode, heterogeneous communication links and parameter inference. Furthermore, the response evaluation process is often time-consuming, making timely optimization and implementation challenging. As the number of bands, channels, and modes supported by reconfigurable RF front-ends increases, the complexity of the transmit and receive channels, the number of components / devices / modules, the planning of their requirements (e.g., linearity, selectivity, dynamic range), and the interconnections and parameter configurations between different components / devices / modules all increase accordingly. As circuit scale grows, the solution space for planning strategies grows exponentially. Furthermore, the discontinuous nature of dynamic cross-coupling effects (e.g., interference, blocking, and spurious emissions) between multiplexed RF system links, along with the non-convex constraints introduced by these interactions, further complicates design. Therefore, in terms of circuit topology construction and link planning, optimizing and designing a multi-dimensional, reconfigurable RF front-end architecture to adapt to multi-mode, multi-band, and multi-standard requirements while minimizing the scale of the system's RF link faces significant challenges.

[0043] In view of this, the embodiments of the present application provide a method, device, electronic device, storage medium and program product for determining a radio frequency link resource optimization model for a heterogeneous network. It includes: determining a radio frequency component and a radio frequency resource pool; in response to determining to communicate through a heterogeneous communication network, determining a first interference signal, a second interference signal, a signal output response of any radio frequency front-end device and the communication link communication performance according to the radio frequency component and the radio frequency resource pool; determining the optimization function of the radio frequency device and the radio frequency module in the radio frequency resource pool according to the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device and the communication link communication performance; decomposing the optimization function by the Chebyshev aggregation method to determine multiple single-objective optimization sub-problems, and solving the multiple single-objective optimization sub-problems by a multi-intelligence reinforcement learning method; training a pre-built strategy model according to the solution results, determining a radio frequency link resource optimization model, and optimizing the link resources according to the radio frequency link resource optimization model. By analyzing the adaptive framework and method of the radio frequency front-end in the heterogeneous communication system, the resource requirements in the multi-band, multi-channel, multi-architecture and multi-standard heterogeneous communication system are predicted and managed. Combined with the current scheduling capability limitations of the RF front-end resource pool, the problem of efficient resource management for diversified communication needs is transformed into a problem of dynamic resource scheduling optimization within the RF front-end resource pool, thereby improving the efficiency of data transmission.

[0044] Specifically, this application provides a unified mathematical modeling representation of the signal coexistence, interference, blocking, loss, and time-division multiplexing link budget process of parallel signal links in multi-band, multi-mode, and multi-standard reconfigurable RF front-ends. Furthermore, an advanced topological connection method for the time-division multiplexing reconfigurable RF front-end resource pool and a dynamic configuration and control system for signal chain parameters are proposed. Finally, an advanced method using multi-agent deep reinforcement learning is proposed to solve the dynamic topology planning and link parameter optimization problems of the time-division multiplexing reconfigurable RF front-end resource pool with multiple objectives.

[0045] This application analyzes the adaptive framework and method of RF front-ends in heterogeneous communication systems to predict and manage resource requirements in multi-band, multi-channel, multi-architecture, and multi-standard heterogeneous communication systems. Combined with the current scheduling capacity limitations of RF front-end resource pools, the problem of efficient resource management for diverse communication needs is transformed into a problem of dynamic resource scheduling optimization within the RF front-end resource pool. The specific steps of the method are as follows:

[0046] The first step is mathematical modeling of the signal chain. This involves describing the signal processing process throughout the chain to computably determine multiple optimization objectives for the overall multi-band, multi-channel, and multi-mode reconfigurable RF front-end system. Specifically, the topology design and link planning issues involved in the dynamic management and control of multi-frequency and multi-channel high-reuse systems based on front-end radio resource pools in heterogeneous communication systems are formalized. The concept of resource pools is outlined, including active and passive RF components, and a three-dimensional multiplexing architecture is employed. The concepts of interference and heterogeneous coupling are described, along with how interfering and heterogeneous coupled signals affect signal reception, taking into account signal strength, frequency variation, noise figure, and nonlinear interference. Finally, a system overview framework is provided to address dynamic management and control techniques in multi-band and multi-channel high-reuse systems, providing a foundation for subsequent dynamic topology planning and link parameter optimization.

[0047] The second step involves decomposing the multi-objective optimization strategy. This involves using a Pareto-based decomposition strategy to decompose the complex multi-objective optimization problem into multiple single-objective optimization sub-problems, simplifying the problem and finding the optimal solution set. This decomposition strategy can be used to solve the Pareto optimal frontier for multi-objective programming in a system—that is, to find the optimal trade-off between multiple optimization objectives, thereby facilitating the design of subsequent problem-solving models.

[0048] The third step is link parameter optimization modeling: A multi-agent reinforcement learning model is used to solve a series (or set of) single-objective optimization problems after the aforementioned transformation, promoting the collaborative optimization of multi-dimensional multiplexing circuit topology, link structure, and parameters. By applying the multi-agent reinforcement learning model, the system's elastic management and control efficiency are enhanced, thereby improving the overall system performance and effectiveness. This includes the design of a multi-agent deep reinforcement learning model algorithm, in which each agent solves a series (or set of) single-objective optimization problems to determine the optimal RF link construction strategy for the current optimization problem.

[0049] like Figure 1 As shown, the method for determining the radio frequency link resource optimization model of the heterogeneous network includes: step S102, determining the radio frequency components and the radio frequency resource pool;.

[0050] For example, in a radio frequency system, various radio frequency components, such as amplifiers, filters, mixers, etc., are regarded as a collection of resources that can be shared and dynamically allocated.

[0051] In some embodiments, the radio frequency component refers to a hardware device or element used to process radio frequency band signals. These components include but are not limited to: Filter: used to select signals in a specific frequency range while suppressing unwanted frequencies. Amplifier: enhances signal strength and is divided into low noise amplifier (LNA) and power amplifier (PA). The low noise amplifier (LNA) is used in the receiving path to minimize noise increase, while the power amplifier (PA) is used in the transmitting path to enhance the signal. Mixer: converts the input signal into a different frequency, usually from high to low (down conversion) or from low to high (up conversion), for further processing. Oscillator: generates a stable frequency reference signal, which is crucial for the modulation and demodulation process. Antenna: used to send and receive radio frequency signals and is an indispensable part of wireless communication systems.

[0052] For example, the RF front-end resource pool system in this solution includes active and passive RF components and adopts a three-dimensional multiplexing architecture (advanced electronic device packaging technology). The RF front-end resource pool is composed of RF device module layers with different functions, such as filters, power amplifiers, mixers, local oscillators, and RF link control switches. The system uses configurable RF switches to connect links and set the disconnection and access status between different module layers and within module layers.

[0053] Specifically, the RF switch matrix is used to achieve flexible scheduling and control of the channel paths of the RF front-end resource pool system. The device parameters on the RF path can also be dynamically configured, which has high adaptability and RF device management efficiency.

[0054] Step S104: In response to determining to communicate through a heterogeneous communication network, determine the first interference signal, the second interference signal, the signal output response of any RF front-end device, and the communication link performance according to the RF component and the RF resource pool.

[0055] For example, complex interference signals severely impact the adequate signal reception of the RF front-end receiver. To ensure the accuracy of the RF link resource optimization model, this application determines the interference signal of the communication network, the signal output response of the RF front-end device, and the communication link performance based on the communication state of the heterogeneous communication network. Specifically, this application represents the interference signal, the signal output response of the RF front-end device, and the communication link performance using mathematical parameters.

[0056] Specifically, the interference signals include interference signals from the RF front-end equipment and interference signals from the communication channel. Among them, the interference signals from the RF front-end equipment include amplitude modulation (AM) noise, frequency modulation (FM) noise, tone interference, multi-tone interference, linear frequency modulation interference, and sawtooth scanning interference. This type of interference will reduce the signal-to-noise ratio of the input signal and may cause system blocking and nonlinear distortion due to excessive interference intensity. We integrate characteristics such as frequency, bandwidth, and intensity into our system signal quantization model to fully consider these interference signals. Among them, the functional expression of the interference signal of the RF front-end equipment is shown in the following formula (1):

[0057]

[0058] in, represents the strength (amplitude) of the jth interference signal, Represents the modulation function of the jth interference signal with respect to time, which is used to simulate the change of bandwidth and signal shape. It can be a parameter representing bandwidth, such as a rectangular window or Gaussian window, or any other function that describes the bandwidth and shape of the noise signal. represents the center frequency of the jth interference signal, and represents the initial phase of the j-th interference signal.

[0059] Furthermore, the interference signal of the communication channel is the interference caused by the signals of other channels in the radio resource pool on the current i-th link. That is, the inter-channel signal interference of the i-th signal channel refers to the adverse effect of the signals of other channels in the radio resource pool on the current i-th link, excluding the interference of the i-th channel itself. The main focus is on how the signals of other channels in the radio resource pool interfere with the current (i.e., i-th) channel signal. The inter-channel interference function is shown in the following formula (2):

[0060]

[0061] T represents the total number of channels in the radio resource pool. represents the signal power spectral density on the nth device on channel j, Indicates the isolation between the nth device on channel j and channel i.

[0062] For example, when constructing the output response of a signal after it passes through a radio frequency device, it is necessary to consider signal strength, frequency variation, noise figure, and nonlinear interference (including intermodulation interference). The functional expression of the output response after passing through the radio frequency device (the change in the output signal after interference) is shown in the following formula (3):

[0063]

[0064] in, Represents the output signal of the nth RF device, D n Indicates the type of RF device, such as amplifier, filter, mixer, oscillator, switch, balanced / unbalanced converter, etc. Indicates the nth D n Type of radio frequency device signal conversion function, Represents the input signal of the nth RF device. n (f) represents the sparse gain or attenuation of the nth RF device with respect to frequency, NF n is the noise figure of the nth RF device, OIP2 n and OIP3 n Respectively represent the second-order intercept point and third-order intercept point of the nth RF device, IL n is the insertion loss of the nth RF device. It is the frequency deviation signal generated after the input signal enters the RF device. n (f) refers to the suppression rate of the RF device to signals outside the transmission signal band.

[0065] Step S106: Determine the optimization function of the radio frequency devices and radio frequency modules in the radio frequency resource pool according to the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device, and the communication performance of the communication link.

[0066] For example, by calculating the quality, stability, and accuracy of the system's received signal based on the first interference signal, the second interference signal, the signal output response of any RF front-end device, and the communication link performance, the effectiveness of the heterogeneous communication network in ensuring optimal communication performance can be evaluated. Specifically, this can be achieved by cascading RF devices with various functions, such as filtering, power amplification, mixing, and oscillation, to construct a multi-dimensional structure for a high-multiplex RF circuit communication chain.

[0067] The effectiveness of this architecture in ensuring optimal communication performance can be evaluated by calculating the quality, stability, and accuracy of the system's received signal. This involves building a multidimensional structure of a high-multiplex RF circuit communication chain by cascading RF devices with various functions such as filtering, power amplification, mixing, and oscillation. The evaluation covers the overall performance of the link, including key aspects such as gain, noise figure, linearity, dynamic range, and signal leakage. The formula used to evaluate this performance is as follows:

[0068]

[0069] in, It represents the signal to interference to noise ratio in the ith RF channel link, in decibels (dB). represents the total gain in the i-th RF channel link, represents the total noise figure in the i-th RF channel link, represents the total linearity in the ith RF channel link, represents the total dynamic range in the i-th RF channel link, represents the total insertion loss in the ith RF channel link, Represents the total isolation of the i-th RF channel link unit.

[0070] The components / devices in the RF front-end resource pool can be dynamically allocated and released to different communication links as needed, providing a flexible and efficient way to manage RF resources in multi-link communication systems. Dynamic means dynamically controlling the links of different RF devices through RF switches according to different communication needs at different time periods, so as to meet different communication needs. Figure 2 The following shows the dynamic construction of the RF front-end link (dynamic link mode and parameter setting of RF devices).

[0071] Assume that there are T groups of RF devices for T communication links, denoted as Each device in the set includes parameter characteristics (De i ) and usage cycle characteristics ([E i ,F i ]). If two elements De i and De j have the same parameter characteristics, and their periodic characteristics do not overlap, then we can say that It can be described as follows:

[0072]

[0073] First, all the radio frequency devices in the T communication link set are organized into a comprehensive set, recorded as Then, we establish the equivalence relation in Included in If and only if Equivalent to Based on this equivalence relation, we can Classify the RF devices in the , thus obtaining a series of equivalence classes, recorded as Each equivalence class C k All A subset of C, including all equivalent k Then, we select any element from each equivalence class to represent the class, thus generating a set of reusable RF devices called Reusable collection of RF devices It can be described as follows.

[0074]

[0075] For typical application scenarios such as multi-band multi-channel mobile communications, a system with flexible scheduling and control of the system's transmit or receive channel type, number of devices, RF front-end resource pool indicator parameters, and topological connections between functional modules is implemented. While meeting multiple communication standards and link design constraints (including RF circuit power, signal-to-noise ratio, noise figure, signal sensitivity, impedance matching, signal bandwidth, frequency matching, signal morphology matching, etc.), the number of devices or modules in the RF resource pool is minimized, circuit reuse is maximized, and the overall multi-channel performance of the system is optimized. The above problems are described as follows:

[0076]

[0077] in, Indicates the number of radio devices in T radio communication links. Indicates the set of cascaded radio devices in the i-th link. Indicates a noise interference signal. Indicates the nth device on the i-th link. It is a device The xth performance indicator can include power limit, SNR, NF, BW, dynamic range and other aspects. and The input signal of the i-th link is The minimum and maximum values of system performance when

[0078] Furthermore, in order to achieve a system that can be flexibly scheduled and controlled, including the type of system transmit or receive channel, the number of devices, the index parameters of the RF front-end resource pool, and the topological connection between various functional modules, while meeting multiple communication standards and link design constraints (including RF circuit power, signal-to-noise ratio, noise figure, signal sensitivity, impedance matching, signal bandwidth, frequency matching, signal shape matching, etc.), the number of devices or modules in the RF resource pool is minimized, circuit reuse is maximized, and the overall multi-channel performance of the system is optimized. The problem description is shown in Formula 8:

[0079]

[0080] The original main optimization goal of this problem was to improve various indicators of each channel, such as signal-to-interference-and-noise ratio, noise figure, linearity, dynamic range, insertion loss, and isolation. However, when the problem covers T channels in the system, its main goal is transformed into evaluating the overall indicator situation. Our optimization goal changes from independently optimizing the indicators of each channel to optimizing the cumulative sum of all T channel indicators. This transformation is possible because the channel links are assumed to be independent and have no mutual dependencies. Therefore, the solutions to the two problems are considered equivalent - despite the nonlinear and complex interdependencies and mutual constraints between the channels. Attempting to optimize the performance of each channel separately may violate these constraints, especially in the presence of random or uncertain factors.

[0081] Furthermore, optimizing individual channel performance in isolation may overlook these factors, potentially compromising the overall system performance. Therefore, the problem is reframed as an overall optimization task, aiming to optimize the collective performance of all channels while ensuring that each channel meets essential communication requirements. The goal of the topology optimization process, which comprehensively addresses the interdependencies, mutual constraints, and nonlinear dynamics between the different RF channel links in the system, is to achieve the optimal configuration and planning of the system's overall channel links.

[0082] Step S108: Decompose the optimization function by using the Chebyshev aggregation method to determine multiple single-objective optimization sub-problems, and solve the multiple single-objective optimization sub-problems by using a multi-intelligence reinforcement learning method.

[0083] For example, the Chebyshev aggregation technique is used to transform multiple optimization objectives into a series of single-objective optimization sub-problems for solution, specifically including:

[0084] First, establish a uniformly distributed set of weight vectors, expressed as Each of these Denotes the weight coefficient assigned to each target. Specifically, let represents the weight of the first objective function in the k-th subproblem. In addition, we introduce As the reference point set or ideal point set, it is usually selected as the ideal value or expected minimum value of the target optimization.

[0085] The transformation process of the k-th sub-problem can be expressed by Formula 9:

[0086]

[0087] in, and represent the noise interference and inter-channel interference vectors respectively, Indicates the maximum number of reusable radio frequency devices used when building all channels. represents the maximum cumulative signal-to-interference-plus-noise ratio (SIN) over all channels. The original problem involves the optimization of multiple objectives, both maximizing and minimizing them. Here, these different optimization objectives are integrated into a unified framework for minimizing multiple objective functions. We transform the maximized objective function into its reciprocal form to facilitate the minimization process. Given the varying dimensions and magnitudes of the objective functions, objectives with larger magnitudes may mask those with smaller magnitudes during the optimization process. Therefore, we employ normalization techniques to mitigate these differences and make the different objectives comparable. This approach enhances the intuitiveness and simplicity of establishing reference points and weights.

[0088] Furthermore, multiple single-objective optimization sub-problems are solved through multi-intelligence reinforcement learning methods.

[0089] Specifically include: using a tuple To characterize the process of solving the multi-dimensional structure multiplexing RF circuit topology and link planning problem. N represents the set of "intelligent agents" constructed after decomposing the optimization sub-problems according to the multiple optimization objectives of the system. When optimizing the problem, It constitutes the finite set of states of the decision environment observed by all agents. k It represents the state observation set of the topological links and parameter selection between all channel link RF devices in the kth optimization subproblem of the intelligent agent k in the sequential decision interaction environment. represents the action set of multi-agent collaborative planning channel link decision in the process of solving each optimization sub-problem, a k Represents the topological links and parameter decision action set between the radio frequency devices of each channel link of the intelligent agent. P(S'|S,A) reflects the Markov state transition probability function. R k :s k ×a k ×s k →R represents the reward function of agent k, and R is the reward function of the joint action of multiple agents. The model aims to learn the sequential decision set It is used to optimize the joint planning and parameter selection of topological links between radio frequency devices corresponding to the sub-problems.

[0090] Because each agent handles a subproblem of the aforementioned multi-objective optimization, the subproblems assigned to each agent are independent of each other. Consequently, each agent operates independently within its scope, virtually unaffected by the influence of other agents. Given the availability of some local information, each agent possesses autonomous decision-making authority. Given this, this application employs an independent proximal policy optimization algorithm to train a multi-agent network model. The modeling process for the kth agent can be described as follows.

[0091] like Figure 3 As shown, agent k The state of the environment consists of several components. These include the parameter characteristics matrix of each RF device in the resource pool. environment Noise and interference distribution within the system, communication requirements for all RF channels And the signal information of the currently planned RF channel Radio frequency equipment D i The parameter matrix It is described in Equation 11. It covers the device properties at a fixed number of frequency points sampled uniformly across a specified spectrum, such as the center frequency bandwidth Gain Noise Figure Linearity Dynamic Range OIP2 OIP3 Insertion loss Isolation Device D i The frequency offset at the Sth frequency point In addition, the equipment occupancy period is also considered and As described in Equation 10, the environmental interference data contains characteristics related to multiple interference sources. These interference sources include single-tone and multi-tone interference, frequency modulation, and amplitude modulation, each with different frequency ranges and power levels.

[0092]

[0093]

[0094] Where S is the total number of frequency samples. is the power of the pth interference or noise signal at time t. They represent the maximum signal power, minimum signal power, signal-to-interference-and-noise ratio, link start time and end time that the RF link of the Tth channel must meet. t The communication requirement matrix represents the communication requirements of all communication links that the system needs to support at the current time t, including the signal frequency, bandwidth, minimum transmit power, maximum transmit power, signal to interference and noise ratio requirements, and communication start and end time. t The communication demand vector represents the communication demand of a radio frequency link currently being planned by the system at the current time t, including the signal frequency, bandwidth, minimum transmit power, maximum transmit power, signal-to-interference-and-noise ratio requirements, and the communication start and end times.

[0095] A matrix of noise and interference for all signal types. Each noise type occupies a row in the matrix and is described by frequency, bandwidth, and power, followed by the start and end times of the signal.

[0096] Step S110: train the pre-built strategy model according to the solution result, determine the radio frequency link resource optimization model, and optimize the link resources according to the radio frequency link resource optimization model.

[0097] For example, environmental state information includes numerous parameters and occupancy details for a large number of RF devices in the RF resource pool, covering a wide range of operating frequencies. It also contains comprehensive information about channel communication requirements and various forms of environmental interference. Considering establishing a unified feature vector that encapsulates all relevant state information as a representation of these features. However, adopting such an approach prevents us from effectively handling resource pools containing different types and numbers of RF components, as well as situations involving different communication requirements and interference information. Furthermore, the presence of a large number of high-dimensional feature vectors leads to a sharp increase in the size of the model's policy network, which hinders the ease and convergence of model training.

[0098] Therefore, if Figure 3 As shown in the figure, we first build a convolutional neural network (CNN) to effectively encode the state information of the environment and generate its embedding vector. and Input into different CNNs, respectively recorded as and We can derive their respective embedding vectors as follows:

[0099]

[0100] Furthermore, the action of agent k Indicates In the status, select RF device Accessing RF channel links. Agent k pairs represent all RF channel links in the system and select RF device link topology and device parameters from the resource pool based on the state vector of the current step. Each RF link starts with the initially selected RF device and terminates at the analog-to-digital converter (ADC) when signal reception constraints and ADC sampling frequency constraints are met. Furthermore, each RF link must adhere to performance standards, meeting the ADC's dynamic range limitations and the channel's performance criteria.

[0101] In order to fully explore all RF devices in the resource pool, such as Figure 4 As shown, the selection action of the RF module is performed according to the circuit topology of the RF channel. Specifically, all RF devices of different types and parameters in the resource pool are designated as the action selection space A of each step of agent k k If the action equal Indicates RF devices of the same type and parameters, but if their operating hours do not overlap, they are considered the same device. On the contrary, if their operating hours overlap, they are considered independent devices. Similar actions will be taken based on the environment status. and The difference in reward feedback.

[0102] When determining the topology links and device selection of multi-channel RF link devices, model serialization sets the probability of infeasible operations to zero to screen out RF device selections that do not meet the problem constraints (topology connections that do not meet the input signal bandwidth and frequency restrictions and do not meet the connection logic shown in Table 1) and inefficient device topology links (RF devices with the same parameters connected in a continuous topology). Relevant constraints include: ① The signal bandwidth and frequency of the connection between devices must match. ② Devices with differential signals and single-ended signals cannot be directly connected. These constraints apply to low-noise amplifiers, tunable filters, differential mixers, single-ended mixers, I / Q mixers, gain blocks, surface acoustic waves, low-pass filters, differential low-pass filters, differential amplifiers, differential variable gain amplifiers, single-ended variable gain amplifiers, baseband programmable VGA filters, balancers and other devices.

[0103] Furthermore, the connection constraints of RF (radio frequency) devices are shown in Table 1. In Table 1, 1 indicates that two RF devices can be cascaded, and 0 indicates that they cannot be directly cascaded.

[0104] Table 1 Topology connection constraints between RF front-end devices

[0105]

[0106] The specific meanings of RF front-end devices are as follows: LAN (Low Noise Amplifier), TF (Transformer), Mixer Diff (Mixer Differential), Mixer Single (Mixer Single-ended), I / Q Mixer (In-phase / Quadrature Mixer), Gain Block (Gain Block), SAW (Surface Acoustic Wave), FLP (Front-End Low Pass), FLP Diff (Front-End Low Pass Differential), Amp Diff (Amplifier Differential), VGA Diff (Variable Gain Amplifier Differential), VGA Single (Variable Gain Amplifier Single-ended), TLP VGA (Triple LowPass Variable Gain Amplifier), Balun (Balanced toUnbalanced, balanced-to-unbalanced converter).

[0107] In order to guide agent k in multi-channel RF circuit topology and link planning strategy π k In the state and action selection A reward function R is designed to match the multiple objectives that need to be optimized in the problem domain. Considering the different optimization objectives of the above agents, a three-layer reward function is carefully developed for the model. The immediate reward function of agent k (expressed as ) is activated after each action. Subsequently, the stage reward function It takes effect after the current RF channel link is completed. Finally, we introduce the overall reward function This function is implemented after all communication links in the system are completed.

[0108] if Then the immediate reward function of agent k can be described as follows.

[0109]

[0110] in, The number of radio frequency devices used in the current radio frequency link. max The maximum number of RF devices that can be used during the link design process for each RF channel. base The penalty points incurred each time a new radio device is added to the radio resource pool system. When no new equipment is added T represents the total number of channels that need to be planned within the existing radio resource pool framework.

[0111] like Then the immediate reward function can be described as:

[0112]

[0113] -R error When the number of radio frequency devices used in the radio frequency channel link exceeds the maximum number of radio frequency devices used Ψ max The penalty score faced when

[0114] Assuming that agent k is currently optimizing the design of the i-th communication link, the stage reward function It can be expressed as follows:

[0115]

[0116] Overall reward function It can be described as:

[0117]

[0118] in Same as Formula 9, it represents the weight coefficient assigned to each target. and Represents the basic reward scalar for successfully completing one RF link and successfully completing all RF links respectively.

[0119] (e) System topology design and link planning Agent k’s strategic network

[0120] like Figure 3 The policy network of agent k is shown. The network uses an embedding vector representing the current state of the environment As input, it outputs the RF device selection for the current link. Specifically, the agent selects RF device i, which includes all initial types of RF devices and the currently selected device. The nonlinear function constructed by the multi-layer neural network Calculate the probability of selecting radio device i.

[0121]

[0122] in, are the embedding vector of the ith RF device at time t, the communication requirements of all RF channels, the noise and interference of the channels, and the information of the current RF channel signal.

[0123] During the model training phase, each action step will receive a reward The model parameters are optimized over multiple episodes. The goal is to maximize the expected value of the model reward from the policy network’s output policy, as detailed in Formula 22.

[0124]

[0125] Here we use the proximal policy optimization algorithm to train the network model of a single agent, where the model loss function of agent k can be described as follows:

[0126]

[0127] Among them, the proximal ratio reduction loss L clip (θ k ), value loss L vf (θ k ) and policy entropy where η1 and η2 represent the three components of the k-th agent’s loss. ε is a hyperparameter used to control the magnitude of the reduction. η1 and η2 represent hyperparameters used to coordinate the weights of different loss functions. is the advantage function value of agent k, which is calculated as follows:

[0128]

[0129] Furthermore, this application proposes a multi-agent deep reinforcement learning approach to address the challenges of dynamic topology planning and link parameter optimization in a time-division multiplexed reconfigurable RF front-end resource pool with multiple objectives. Furthermore, through a designed three-stage continuous reward function, predefined weights are used to guide the deep reinforcement learning agent to adapt to different optimization objective weights during training. Once the model is trained, it can be applied to address new multi-channel heterogeneous communication challenges under various communication standards, frequency bands, number of channels, electromagnetic environments, and channel performance constraints.

[0130] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0131] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0132] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a radio frequency resource optimization device based on heterogeneous communication.

[0133] refer to Figure 5 , the radio frequency resource optimization device based on heterogeneous communication includes:

[0134] A first determining module 502 is configured to determine a radio frequency component and a radio frequency resource pool;

[0135] A second determining module 504 is configured to, in response to determining that communication is performed through a heterogeneous communication network, determine, based on the radio frequency component and the radio frequency resource pool, a first interference signal, a second interference signal, a signal output response of any radio frequency front-end device, and communication link performance;

[0136] A calculation module 506 is configured to determine an optimization function of the radio frequency devices and radio frequency modules in the radio frequency resource pool based on the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device, and the communication performance of the communication link;

[0137] a decomposition module 508 configured to decompose the optimization function using a Chebyshev aggregation method to determine a plurality of single-objective optimization subproblems, and solve the plurality of single-objective optimization subproblems using a multi-intelligence reinforcement learning method;

[0138] The optimization module 510 is configured to train a pre-built strategy model according to the solution result, determine the radio frequency link resource optimization model, and determine according to the radio frequency link resource optimization model.

[0139] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0140] The apparatus of the above embodiment is used to implement the corresponding method for determining the radio frequency link resource optimization model of the heterogeneous network in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0141] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the method for determining the radio frequency link resource optimization model of the heterogeneous network described in any of the above embodiments.

[0142] Figure 6 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0143] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0144] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0145] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0146] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0147] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0148] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0149] The electronic device of the above embodiment is used to implement the corresponding heterogeneous network radio frequency link resource optimization model determination method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0150] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for determining the radio frequency link resource optimization model of the heterogeneous network as described in any of the above embodiments.

[0151] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0152] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method for determining the radio frequency link resource optimization model of the heterogeneous network as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0153] Based on the same inventive concept, corresponding to the method for determining a heterogeneous network radio link resource optimization model described in any of the above embodiments, the present disclosure further provides a computer program product comprising computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to execute the method for determining a heterogeneous network radio link resource optimization model. The processors that execute the corresponding steps corresponding to the execution entities corresponding to the steps in each embodiment of the method for determining a heterogeneous network radio link resource optimization model may belong to the corresponding execution entity.

[0154] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the method for determining the radio frequency link resource optimization model of the heterogeneous network as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0155] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0156] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0157] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0158] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A method for determining a radio frequency link resource optimization model for a heterogeneous network, characterized in that: include: Determine RF components and RF resource pools; In response to determining that communication is to be performed through a heterogeneous communication network, determining, based on the radio frequency component and the radio frequency resource pool, a first interference signal, a second interference signal, a signal output response of any radio frequency front-end device, and communication link performance; Determining an optimization function for the radio frequency devices and radio frequency modules in the radio frequency resource pool according to the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device, and the communication performance of the communication link; Decomposing the optimization function by a Chebyshev aggregation method to determine multiple single-objective optimization subproblems, and solving the multiple single-objective optimization subproblems by a multi-intelligence reinforcement learning method; The pre-built strategy model is trained based on the solution results to determine the RF link resource optimization model, and link resource optimization is performed based on the RF link resource optimization model.

2. The method according to claim 1, characterized in that The first interference signal is an interference signal of a radio frequency front-end device, and the second interference signal is an interference signal of a communication channel.

3. The method according to claim 1, characterized in that Solving the multiple single-objective optimization sub-problems by a multi-intelligence reinforcement learning method includes: Establish a multi-intelligence reinforcement learning model based on the multi-intelligence reinforcement learning method; The multiple single-objective optimization sub-problems are solved through the multi-intelligent reinforcement learning model.

4. The method according to claim 1, wherein The solution results include: environmental status information; The step of training a pre-built strategy model based on the solution result to determine a radio frequency link resource optimization model includes: Inputting the environmental state information into a neural network model to determine an embedding vector; The pre-built strategy model is trained according to the embedding vector to determine the radio frequency link resource optimization model.

5. The method according to claim 4, characterized in that The strategy model is a deep reinforcement learning model.

6. The method according to claim 4, characterized in that The neural network model is a convolutional neural network model.

7. A radio frequency resource optimization device based on heterogeneous communication, characterized in that: include: A first determining module is configured to determine a radio frequency component and a radio frequency resource pool; A second determination module is configured to determine, in response to determining that communication is performed through a heterogeneous communication network, a first interference signal, a second interference signal, a signal output response of any RF front-end device, and communication link performance based on the RF component and the RF resource pool; a calculation module configured to determine an optimization function of the radio frequency devices and radio frequency modules in the radio frequency resource pool based on the first interference signal, the second interference signal, the signal output response of any radio frequency front-end device, and the communication performance of the communication link; a decomposition module configured to decompose the optimization function by using a Chebyshev aggregation method, determine a plurality of single-objective optimization subproblems, and solve the plurality of single-objective optimization subproblems by using a multi-intelligence reinforcement learning method; The optimization module is configured to train a pre-built strategy model according to the solution result, determine the radio frequency link resource optimization model, and determine according to the radio frequency link resource optimization model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program. 9 . A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to claim 1 .

10. A computer program product comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

Images