Method for operating user equipment and user equipment

Abstract

A method for operating a user device and a user device are provided. In some embodiments, the method includes: receiving a first state and a first state transition by a first neural network, the first state including: one or more identifiers for a plurality of available active ports, and a set of available connections between two or more circuit elements, each of the two or more circuit elements being one of: (1) a first circuit type, (2) a second circuit type, the second circuit type operably connecting a circuit element of the first circuit type to one of the plurality of available active ports, and (3) the plurality of available active ports; and generating a first estimated quality value for the first state transition by the first neural network.

Description

[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 253,392, filed on October 7, 2021, entitled “PORT ALLOCATION FOR CARRIER AGGREGATION AND MIMO BASED ON MACHINE LEARNING,” and U.S. Application No. 17 / 834,845, filed on June 7, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0002] One or more aspects of embodiments according to the present disclosure relate to configuring radio frequency (RF) circuits, and more particularly, to systems and methods for configuring RF circuits in user equipment using machine learning. Background Art

[0003] A user equipment (UE) operating as part of a wireless network may sometimes receive configuration instructions from a network node, instructing the user equipment to operate within a particular frequency band, in a carrier aggregation mode, or in a multiple-input multiple-output (MIMO) mode. To comply with such instructions, the UE may configure a radio frequency integrated circuit (RFIC) by making connections within the RFIC, between a local oscillator of the RFIC and a mixer of the RFIC, and between the mixer and an active port of the RFIC. However, not all connections may be available to be established; for example, it may not be feasible to connect a particular local oscillator to a particular mixer or to connect a particular mixer to a particular active port.

[0004] It is with respect to this general technical environment that various aspects of the present disclosure are relevant. Summary of the invention

[0005] According to an embodiment of the present disclosure, a method is provided, comprising: receiving a first state and a first state transition by a first neural network, the first state comprising: one or more identifiers for available active ports, and a set of available connections between two or more circuit elements, each of the circuit elements being one of the following: (1) a first circuit type, (2) a second circuit type, the second circuit type operably connecting a circuit element of the first circuit type to one of the available active ports, and (3) an available active port; and generating a first estimated quality value for the first state transition by the first neural network, the first estimated quality value corresponding to the likelihood that the first state transition is one of a series of transitions terminating in a terminating state, in which a connection is established to each of the available active ports, wherein: the first state transition is a transition from the first state to the second state, and the second state comprises a connection between two of the circuit elements that are not present in the first state.

[0006] In some embodiments, the method further comprises: feeding the first state and the first state transition to the first neural network; receiving a first estimated quality value from the first neural network; feeding the first state and the second state transition to the first neural network; and receiving a second estimated quality value from the first neural network, the second estimated quality value corresponding to the likelihood that the second state transition is one of a series of transitions terminating in a terminating state, in which a connection is established to each of the available active ports, wherein the second state transition is a transition from the first state to a third state, the third state comprising a connection between two of the circuit elements that are not present in the first state and that are not present in the second state.

[0007] In some embodiments, the method further comprises determining that the second estimated mass value is greater than the first estimated mass value.

[0008] In some embodiments, the method further comprises, in response to determining that the second estimated quality value is greater than the first estimated quality value, feeding a third state and a third state transition to the first neural network, wherein the third state transition is a transition from the third state to a fourth state, the fourth state comprising a connection between two of the circuit elements that are not present in the third state.

[0009] In some embodiments, the method further comprises: feeding the second state and a fourth state transition to the first neural network, wherein: the fourth state transition is a transition from the second state to a fifth state, and the fifth state comprises a connection between two of the circuit elements that are not present in the second state.

[0010] In some embodiments, the first circuit type is a local oscillator and the second circuit type is a mixer, and wherein the first estimated quality value further corresponds to a likelihood that the first state transition is one of a series of transitions terminating in a termination state, in which a connection is established to each of the available active ports and in which corresponding two connections are established from the local oscillator to two mixers.

[0011] In some embodiments, the method further comprises performing a feasibility test to check for an indication that a terminated state cannot be reached from the first state, in which a connection is established to each of the available active ports.

[0012] In some embodiments, a plurality of circuit elements of a first circuit type are available, a plurality of circuit elements of a second circuit type are available, and the feasibility test is based on the number of available circuit elements of the first circuit type and the number of available circuit elements of the second circuit type.

[0013] In some embodiments, the method further includes: generating a training data set using a Monte Carlo tree search to assign a quality value to each of a plurality of combinations of states and state transitions; training a neural network using the training data set to generate a network parameter set; and storing the network parameter set in the first neural network.

[0014] According to an embodiment of the present disclosure, a user device is provided, comprising: a processing circuit; and a memory connected to the processing circuit, the memory storing instructions, wherein when the instructions are executed by the processing circuit, the user device executes a method, the method comprising: receiving a first state and a first state transition through a first neural network, the first state comprising: one or more identifiers for available active ports, and a set of available connections between two or more circuit elements, each of the circuit elements being one of the following: (1) a first circuit type, (2) a second circuit type, the second circuit type operably connecting a circuit element of the first circuit type to one of the available active ports, and (3) an available active port; and generating a first estimated quality value for the first state transition through the first neural network, the first estimated quality value corresponding to the likelihood that the first state transition is one of a series of transitions terminating in a terminating state, in which a connection is established to each of the available active ports, wherein: the first state transition is a transition from the first state to the second state, and the second state comprises a connection between two of the circuit elements that are not present in the first state.

[0015] In some embodiments, the method further comprises: feeding the first state and the first state transition to the first neural network; receiving a first estimated quality value from the first neural network; feeding the first state and the second state transition to the first neural network; and receiving a second estimated quality value from the first neural network, the second estimated quality value corresponding to the likelihood that the second state transition is one of a series of transitions terminating in a terminating state, in which a connection is established to each of the available active ports, wherein the second state transition is a transition from the first state to a third state, the third state comprising a connection between two of the circuit elements that are not present in the first state and that are not present in the second state.

[0016] In some embodiments, the method further comprises determining that the second estimated mass value is greater than the first estimated mass value.

[0017] In some embodiments, the method further comprises, in response to determining that the second estimated quality value is greater than the first estimated quality value, feeding a third state and a third state transition to the first neural network, wherein the third state transition is a transition from the third state to a fourth state, the fourth state comprising a connection between two of the circuit elements that are not present in the third state.

[0018] In some embodiments, the method further comprises: feeding the second state and a fourth state transition to the first neural network, wherein: the fourth state transition is a transition from the second state to a fifth state, and the fifth state comprises a connection between two of the circuit elements that are not present in the second state.

[0019] In some embodiments, the first circuit type is a local oscillator and the second circuit type is a mixer, and wherein the first estimated quality value further corresponds to a likelihood that the first state transition is one of a series of transitions terminating in a termination state, in which a connection is established to each of the available active ports and in which corresponding two connections are established from the local oscillator to two mixers.

[0020] In some embodiments, the method further comprises performing a feasibility test to check for an indication that a terminated state cannot be reached from the first state, in which a connection is established to each of the available active ports.

[0021] In some embodiments, a plurality of circuit elements of a first circuit type are available, a plurality of circuit elements of a second circuit type are available, and the feasibility test is based on the number of available circuit elements of the first circuit type and the number of available circuit elements of the second circuit type.

[0022] In some embodiments, the method further includes: generating a training data set using a Monte Carlo tree search to assign a quality value to each of a plurality of combinations of states and state transitions; training a neural network using the training data set to generate a network parameter set; and storing the network parameter set in the first neural network.

[0023] According to an embodiment of the present disclosure, a user equipment is provided, comprising: a device for processing; and a memory connected to the device for processing, the memory storing instructions, wherein when the instructions are executed by the device for processing, the user equipment executes a method, the method comprising: receiving a first state and a first state transition through a first neural network, the first state comprising: one or more identifiers for available active ports, and a set of available connections between two or more circuit elements, each of the circuit elements being one of the following: (1) a first circuit type, (2) a second circuit type, the second circuit type operably connecting a circuit element of the first circuit type to one of the available active ports, and (3) an available active port; and generating a first estimated quality value for the first state transition through the first neural network, the first estimated quality value corresponding to the likelihood that the first state transition is one of a series of transitions terminating in a terminating state, in which a connection is established to each of the available active ports, wherein: the first state transition is a transition from the first state to the second state, and the second state comprises a connection between two of the circuit elements that are not present in the first state.

[0024] In some embodiments, the method further comprises: feeding the first state and the first state transition to the first neural network; receiving a first estimated quality value from the first neural network; feeding the first state and the second state transition to the first neural network; and receiving a second estimated quality value from the first neural network, the second estimated quality value corresponding to the likelihood that the second state transition is one of a series of transitions terminating in a terminating state, in which a connection is established to each of the available active ports, wherein the second state transition is a transition from the first state to a third state, the third state comprising a connection between two of the circuit elements that are not present in the first state and that are not present in the second state. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] These and other features and advantages of the present disclosure will be appreciated and understood with reference to the specification, claims and drawings, in which:

[0026] Figure 1 is a block diagram of a portion of a radio frequency integrated circuit (RFIC) according to an embodiment of the present disclosure;

[0027] Figure 2A is a data flow diagram according to an embodiment of the present disclosure;

[0028] Figure 2B is a data flow diagram according to an embodiment of the present disclosure;

[0029] Figure 2C is a data flow diagram according to an embodiment of the present disclosure;

[0030] Figure 3A is a state tree traversal diagram according to an embodiment of the present disclosure;

[0031] Figure 3B is a state tree traversal diagram according to an embodiment of the present disclosure;

[0032] Figure 3C is a state tree traversal diagram according to an embodiment of the present disclosure;

[0033] Figure 3D According to the embodiment of the present disclosure FIG. 3A to FIG. 3C Legend of;

[0034] Figure 4 is a flow chart according to an embodiment of the present disclosure; and

[0035] Figure 5 is a block diagram of a system for wireless communication according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The specific embodiments described below in conjunction with the accompanying drawings are intended as a description of exemplary embodiments of systems and methods for configuring RF networks based on machine learning provided in accordance with the present disclosure, and are not intended to represent the only form in which the present disclosure can be constructed or utilized. The description describes features of the present disclosure in conjunction with the illustrated embodiments. However, it will be understood that the same or equivalent functions and structures can be implemented by different embodiments that are also intended to be included within the scope of the disclosure. As indicated elsewhere herein, the same element numbers are intended to indicate the same elements or features.

[0037] Figure 1Schematic diagram of a portion of a radio frequency (RF) integrated circuit (RFIC) that can be used as part of a user device in a wireless (e.g., fifth generation (5G)) communication system. Carrier aggregation (CA) and MIMO techniques can be used in wireless networks to increase the data rate of a user by allocating multiple frequency blocks to the same user. Signals under multiple frequency blocks can be received on different ports 105 of the RFIC of the user equipment (UE). The active port (active port) can be assigned to the UE by the network (e.g., by a network node (gNB)). In the RFIC, each active port can be assigned a mixer 110 and a local oscillator (LO) 115 to demodulate and decode the signal. In order to simplify the hardware of the RFIC, the feasible connection paths between the port and the mixer and between the mixer and the LO are limited. Therefore, it is possible that any given port cannot be connected to all mixers, and any given mixer cannot be connected to all LOs. Due to such limitations, it may not be simple to assign mixers and LOs to active ports. As used herein, an "element" (except when referring to an element of a vector) is an active port, or a mixer, or a local oscillator, and thus configuring the RFIC in response to an active port being assigned to a UE involves establishing one or more connections between the elements.

[0038] Some methods for assigning mixers and LOs to active ports involve manually updating a connection table. In some embodiments, a machine learning-based approach is used instead to automate the assignment using a neural network. Once the hardware design of the RFIC is complete, this approach can be used to train a neural network to find the mixer and LO assignment for each active port at run time. FIG. 2A to FIG. 2C In some embodiments, connections may be selected in the RFIC using: (i) reinforcement learning using Monte-Carlo tree search (MCTS) Figure 2A , wherein MCTS205 is used to generate a training data set 210), (ii) neural network training ( Figure 2B , wherein the training data set 210 is used to perform neural network training at 215 to generate a network parameter set θ), and (iii) online inference ( Figure 2C , wherein the trained neural network 220 is used to generate recommendations for connections to be made (e.g., mixer and LO assignments for each active port).

[0039] The system model and objectives can be defined as follows. is the set of available port types (frequency bands) at the UE, with a total of N a Available port types. For example, N can be as follows a=9 port types:

[0040]

[0041] Where 1 to 9 represent ports 1 to 9, LB is low band, MB is mid band, HB is high band, UHB is ultra high band, NRU is unlicensed new radio (new radio), and the letter M in ports 6, 7, 8, and 9 represents MIMO. Therefore, in the above list, ports 1 to 5 are CA ports, and ports 6 to 9 are MIMO ports. The RFIC includes M mixers and L LOs. The hardware constraints depend on the RFIC design. The constraints can be described according to two matrices G and B. G is N a ×M matrix, where M is the number of mixers. The matrix G indicates whether there is a path between port n and mixer m. For example, the entries of G can be interpreted as follows:

[0042] If G(n,m)=0, there is no path from port n to mixer m, n∈{1,2,…,N a}, m∈{1, 2, …, M},

[0043] If G(n,m)=1, then there exists a path from port n to mixer m, n∈{1,2,…,N a}, m∈{1, 2, …, M} (2)

[0044] Similarly, B is an M×L binary matrix indicating whether a path exists between mixer m and LO l (where L is the number of local oscillators):

[0045] If B(m, l) = 1, then there is a path from mixer m to LO l,

[0046] If B(m, l) = 0, there is no path from mixer m to LO l (3)

[0047] Depending on the RFIC design, there may be more or fewer hardware constraints.

[0048] According to the list in equation (1), the active port The list is distributed to the UE by the network. The ports in can be repeated; for example, It can be as follows:

[0049]

[0050] The number of active ports in is N. In the above example, N = 7. CA Can be defined as The number of CA ports in the MIMO Can be defined as The number of MIMO ports in the equation (4). CA =5,N MIMO =2.

[0051] X and Y can be defined as binary matrices of size N×M and M×L, respectively, indicating the port to mixer m and mixer m to LO m, where i = 1, 2, 3, ... N and is a list The i-th entry in .

[0052] The goal can be Find mixer assignments and LO assignments for each port in while minimizing the number of LOs used. The constraints are as follows:

[0053] Constraint 1: Constraints imposed by G and B in equations (2) and (3).

[0054] Constraint 2: A mixer can be connected to only one port, and a port can be connected to only one mixer.

[0055] Constraint 3: The mixer connected to the port must be connected to LO.

[0056] Constraint 4: In addition to the fact that the same LO can be used for CA ports and MIMO ports of the same band, one LO can be connected to only one mixer and one mixer can be connected to only one LO, for example, MB and MBM can be connected to the same LO via different mixers.

[0057] Alternatively, the goal can be stated as finding The mixer allocation and LO allocation of all ports in , while maximizing the number of LOs shared between CA ports and MIMO ports. Mathematically, the optimization problem can be stated as follows. The objective function can be as follows:

[0058]

[0059] Among them, (i) is the square of the Frobenius norm of X, which is equal to the number of connected ports, (ii) is an indicator function, if (i.e., the number of connected ports), then Equal to 1, otherwise is equal to 0, and (iii) S L is the number of LOs shared between the CA ports and the MIMO ports.

[0060] The constraints of the problem (C1 to C6) can be stated as follows:

[0061] C1: Binary entries

[0062] C2: X, Y follow the connection limits set by A and B (7)

[0063] X⊙A=X, where ⊙ represents element-by-element multiplication,

[0064] Y⊙B=Y,

[0065] in, in, is the matrix G row, n=1, 2,…, N.

[0066] C3: Each port mapped to a mixer,

[0067] For n = 1, 2, ..., N, ∑ m X n，m ≤1 (8)

[0068] C4: Each mixer mapped to at most one port,

[0069] For m = 1, 2, ..., M, ∑ n X n，m ≤1 (9)

[0070] C5: If mixer m is connected to any port in X, then mixer m is connected to an LO in Y,

[0071] For m = 1, 2, ..., M, ∑ l Y m，l =∑ n X n，m (10)

[0072] C6: LO sharing constraint,

[0073] Where 0≤S L ≤N MIMO .

[0074] The solution can be obtained by connecting one port to the mixer and LO iteratively at a time, with the ultimate goal of obtaining valid connections of all active ports to the mixer and LO. The method can be described as a Markov Decision Process (MDP), where:

[0075] - State s includes {a set of unconnected ports, available port-mixer connections, available mixer-LO connections}.

[0076] is the set of valid actions in state s.

[0077] -Effective Action indicates {unconnected port index n, mixer index m, LO index l}. Taking action a means connecting port n to mixer m and mixer m to LO l. This is equivalent to setting X n，m =1 and Y m，l =1.

[0078] - When action a is taken in state s, the MDP moves to (makes a state transition to) state s'.

[0079] - Q(s,a) is the sum of taking action from state s The long-term reward associated with state s′, or equivalently, the long-term reward associated with state s′.

[0080] -s T is a terminal state, so that (where φ is the empty set), i.e., no valid action is feasible.

[0081] -Reward Δ is returned at the terminal state.

[0082] -The reward is back-propagated in the tree from the terminal state to the initial state s 0 , and Q(s, a) is updated.

[0083] Each action corresponds to a state transition, and as used herein, “action” and “state transition” are synonymous. Once Q(s i , the value of a) has been determined, then from the initial state s 0 Start, Action can be chosen at each step (i.e., at each intermediate state) so that for i=0, 1, 2, ..., To reach the terminal state s T In the Monte Carlo process, at each state, if the path (action) has not been explored before the current iteration, the action is randomly selected. Otherwise, the path (action) is selected with some probability of random selection based on the accumulated Q value before the current iteration. Q(s i , a) is the estimate calculated by the Monte Carlo process. If the Monte Carlo process were to run for an infinite number of iterations, the estimated Q value could be expected to converge to Q(s i , a) is a “true” value. In some embodiments, Q(s i , a) is obtained, so that the terminal state s TThe reward under s is maximized and T As mentioned above, in some embodiments, three main operations are employed to achieve this goal: (i) reinforcement learning, which can be employed to learn the Q-values ​​Q(s, a) for different state-action pairs using Monte Carlo Tree Search (MCTS), (ii) reward calculation and back propagation during tree search to achieve connection search, and (iii) connection search algorithm to find the mixer and LO assignment for each of the active ports.

[0084] A feasibility test can be employed to evaluate whether, for a given set of constraints and active ports, there exists a solution in which each active port is connected to a mixer that is connected to a LO. Four infeasibility conditions of A can be defined. If any one of the conditions is met, it is infeasible to connect all ports to the mixer (i.e., the terminal state cannot be reached from the first state, in which the connection is established to each available active port). These conditions are sufficient to prove infeasibility, but are not necessary. If any one of them is met, it is infeasible to connect all active ports to the mixer. If none of them is met, it may or may not be feasible to connect all active ports to the mixer.

[0085] Listing 1 is a pseudo-code listing of code that can be used to check for infeasibility. In this listing, the following definitions are used:

[0086] A CA , A MIMO , so that the equation

[0087] Among them, A CA It is the N before A. CA Rows, A MIMO It is the N after A. MIMO A line.

[0088] Listing 1

[0089] 1. If the number of ports N is greater than the number of available mixers M, it is not feasible (condition 1).

[0090] 2. If the number of active CA ports is greater than the number of mixers connected to those CA ports (N CA Greater than A CA ), or if the number of active MIMO ports is greater than the number of mixers connected to those MIMO ports (N MIMO Greater than A MIMO The number of non-zero columns in ), it is not feasible (condition 2).

[0091] 3. If the total number of active ports > the number of mixers connected to those active ports (N > the number of non-zero columns in A), then not feasible (condition 3).

[0092] 4. Set loop count i = 1, A (i) =A, A (0) =A

[0093] 5. Loop (WHILE) (1)

[0094] a. For each port-mixer connection row of the same port type, (i) If the number of repetitions is greater than the number of mixers it can be connected to, then it is not feasible (condition 4).

[0095] b. If A (i) If the number of rows in A is repeated = the number of mixers, then those mixers cannot be assigned to other ports. Update A (i) to remove the connection between such mixer and other ports.

[0096] c. If A (i) If any row (port) in has only one partial connection left, it is converted to a full connection.

[0097] d. If (IF) A (i) = = A (i-1) ,but

[0098] Break out of the loop;

[0099] ELSE

[0100] Increment loop counter i

[0101] A (i) =A (i-1)

[0102] End Condition (ENDIF)

[0103] 6. End the loop (ENDWHILE)

[0104] Reinforcement learning using Monte Carlo Tree Search (MCTS) can be performed as follows. Given a port combination Initial state or root node s 0 can be constructed. Starting from the initial state, the tree is constructed by taking actions until the terminal state is reached, calculating and backing up rewards, and calculating Q-values. At each state or node, a record of the number of times each action is taken and the corresponding Q-value of each state-action pair is kept. As mentioned above, the goal of reinforcement learning using MCTS is to learn the Q-value of each state-action pair.

[0105] In the MCTS algorithm, state transitions can occur as follows. The initial connection matrix can be defined as G 0 =G,B 0 =B. can be the initial list of ports remaining to connect to, and X 0 =0, Y 0 = 0 can be the initial connection matrix. Each state can be defined as a tuple In status i Next, the matrix X i , Y i Indicates the connections from the port to the mixer and from the mixer to the LO, respectively. i indicates a list of unconnected ports, and the matrix G i , B i Indicates the available port-mixer and mixer-LO connections, respectively.

[0106] Can be state s i The set of valid actions under The action {n, m, l} connects port n to mixer m and mixer m to L01. For a valid action, the following holds:

[0107] - If n is a CA port, then:

[0108] ○Mixer m makes G i (n, m)≠0.

[0109] ○LO l makes B i (m, l)≠0, and LO l is not connected to any other port.

[0110] - If n is a MIMO port, then:

[0111] ○ CA of the same frequency band is already connected.

[0112] ○Mixer m makes G i (n, m)≠0.

[0113] ○LO l makes B i (m, l)≠0, and LO l is not connected to any other port except the CA port of the same frequency band.

[0114] When the action {n, m, l} is changed from state s i When taken, the parameters are updated as follows.

[0115] -The connection matrix is ​​updated as follows:

[0116] ○X i+1 =X i , Y i+1 =Y i , X i+1 (n, m) = 1, Y i+1 (m, l) = 1

[0117] - The mixer and LO availability matrix is ​​updated as follows:

[0118] ○G i+1 =G i , B i+1 =B i ,

[0119] ○ Mixer m is disconnected from other ports: G i+1 (:, m) = 0,

[0120] ○ LO l is disconnected from mixer m: B i+1 (m, l) = 0,

[0121] -Port n is removed from the list of unconnected ports:

[0122] In this way, from state s i To status A state transition of occurs with action a. The number of times an action is taken from state s is denoted by N(s, a). This counter is updated during state transitions.

[0123] The Q-value: Q(s, a) can then be calculated for each state-action pair by the number of times N(s, a) action a is taken from state s, as the sum of the rewards Δ achieved in the terminal states reached from s′ in the collection of Monte Carlo trees.

[0124] The algorithm involves selection, expansion, simulation, and back-propagation. These can be included as shown in the algorithm of Listing 2. When the algorithm reaches the terminal state, reward calculation and back-propagation are implemented as described below.

[0125] The reward calculation and back propagation may be performed as follows. During back propagation, the reward Δ is returned from the terminal state. Two methods, referred to herein as Option-1 and Option-2, may be used for reward calculation.

[0126] In option-1, a non-zero reward is returned only when all active ports are connected to the mixer and LO (if any port is left unconnected in the terminated state, a reward of zero is returned):

[0127]

[0128] In option 2, the termination reward Δ is calculated as:

[0129]

[0130] Among them, I 1 and I 2 is an integer; item Provides a bonus for the number of ports connected and the number of LOs shared; if all ports are connected, then the item To emphasize the importance of connecting all ports by the number of LOs connected and shared, the integer I 1 and I 2 Can be selected to satisfy

[0131] The reward is back-propagated from the terminal state to the initial state s 0 , and the Q value is updated. Back propagation can be performed according to various methods including the following three methods (which can be referred to as option-1, option-2, and option-3). In option-1, which can be referred to as additive back propagation, Q(s, a) is updated as follows:

[0132] Q(s,a)←Q(s,a)+Δ (15)

[0133] In option-2, which can be called maximum backpropagation, Q(s,a) is updated as follows:

[0134] Q(s,a)←max(Q(s,a),N(s,a)×Δ) (16) In what can be called the maximum (max) option-3 with scaling, Q(s,a) is updated as follows:

[0135]

[0136] Any method for calculating the terminal reward Δ can be used with any option for backpropagation. The reward calculation and backpropagation algorithms are summarized in Listing 2.

[0137] Listing 2

[0138] 1. If (IF)s = terminal state

[0139] a. If (IF) Option 1 is awarded, then

[0140]

[0141] b. ELSEIF option 2 is awarded, then

[0142]

[0143] c. End condition (ENDIF)

[0144] d. Loop (WHILE) s ≠ NULL

[0145] i. Find s′, a′ such that (s′, a′) → s

[0146] ii.s←s′, a←a′

[0147] iii. N(s, a) = N(s, a) + 1

[0148] iv. If (IF) back propagation option 1, then

[0149] Q(s, a)←Q(s, a)+Δ

[0150] v. Else if (ELSEIF) back propagation option 2, then

[0151] Q(s,a)←max(Q(s,a),N(s,a)×Δ)

[0152] vi. Else if (ELSEIF) back propagation option 3, then

[0153]

[0154] vii. End condition (ENDIF)

[0155] e. End the loop (ENDWHILE)

[0156] 2. End condition (ENDIF)

[0157] In some embodiments, the Q(s, a) values ​​and N(s, a) values ​​generated during MCTS are used to train a neural network to obtain a set of network parameters θ. The input of the network is the state s, and the output is Because states higher up in the tree have larger values ​​of Q than states closer to the terminal state, normalization (dividing by N) may be employed. Normalization may help ensure that each state has equal weight in training.

[0158] Once the training dataset 210 has been generated (e.g., using the MCTS method described above), the values ​​Q(s, a) and N(s, a) can be used for network training, where a neural network (referred to as a Q-network) is trained to estimate the Q value for each state-action (s, a). Each training example corresponds to a state s and a valid action for that state. The input to the Q-network consists of the state vector s and the action vector a, which are obtained from the state s and the action a (as discussed in further detail below).

[0159] For state si , the state vector s i , i = 0, 1, 2... based on G i , B i and a vector indicating which ports remain connected is defined. The vector s i can be expressed as follows:

[0160]

[0161] Among them, G i (:) and B i (:) is N a Mx1 and MLx1 column vectors. Vector is defined as

[0162]

[0163] Because X i , Y i Not including those not present in G i , B i So even if the state tuple s i Include X i , Y i , these vectors do not need to be included in the state vector s i Therefore, in s i Including G i , B i is enough.

[0164] The action vector of action a={n,m,l} is expressed as follows:

[0165]

[0166] Among them, e 13 (n) is N a ×1 vector where the nth element = 1 and all other elements = 0.

[0167] In this way, the input to the Q-network is the vector In order to estimate the state-action pair (s i , a) Q value. The output of the network is The network can be trained using a quasi-Newton method to obtain a set of network parameters θ (e.g., weights of the neural network). The network can be trained to estimate

[0168] When the UE receives an assignment of active ports from the network in operation, the Q-network programmed with the network parameter set θ (e.g., with a copy of the network parameter set θ) may perform an inference operation to identify a set of connections to be made in the RFIC. For example, the network parameter set θ is used to estimate the active ports for a given state s and action a. Value. Estimated by the network during inference The value of can be expressed as The estimated quality value The estimated quality value may correspond to a likelihood or probability that the second state transition is one of a series of transitions that terminate in a successful termination state (eg, a termination state in which a connection is made to each available active port). It may also correspond to the likelihood that in the terminated state at least one local oscillator is shared (eg two respective connections are made from the local oscillator to the two mixers in the terminated state).

[0169] If the network training is ideal, so that the action identified by the neural network as the best action is actually the best action (i.e., for all states s, ), then the inference time algorithm can be the algorithm shown in Listing 3.

[0170] Listing 3

[0171] 1. Input: G, B, C, θ,

[0172] 2. Check feasibility using Algorithm 1. If not feasible, stop (STOP).

[0173] 3. Initialization: initial state s 0

[0174] 4. State s = s 0 ,

[0175] 5. Loop (WHILE) (1)

[0176] a. Get a set of valid actions for state s

[0177] b. Run the neural network with parameters θ to obtain

[0178] c. Select action a with the largest Q-value * ,

[0179] d. Get the next state: (s, a * )→s′

[0180] e. Update: s←s′

[0181] f. If (IF)s is a leaf node, then

[0182] i. Stop (STOP). / / Success, all ports are connected, all constraints C1 to C8 are satisfied

[0183] g. End condition (ENDIF)

[0184] 6. End the loop (ENDWHILE)

[0185] In practice, the network may not be able to accurately predict the best action for all states. In this case, the algorithm of Listing 3 may not find a successful connection even when a successful connection is possible. To increase the likelihood of success, the algorithm of Listing 3 can be modified to Value or third largest The modified method is described in Listing 4. In this algorithm (as in the algorithm of Listing 3), the UE (e.g., the processing circuit of the UE) may feed the neural network (which may also be implemented in the processing circuit) an initial state and each of the available state transitions from the first state, and obtain the quality value of each state transition from the neural network. The process may then be repeated (as in the algorithm of Listing 3), at each step (i) advancing to a new state according to the state transition with the highest quality value (as in the algorithm of Listing 3), or (ii) if the first terminal state found is not a successful terminal state, then at one or more states in the series of states, advancing to a new state according to a state transition with a quality value lower than the highest quality value (e.g., having a second high quality value or a third high quality value). The main aspects of the algorithm include the following:

[0186] - the algorithm includes iteration index: i, level: l and action index in level l: kl,

[0187] - The action selected at level l=0, 1, 2, ... is the one with the maximum Action l ,

[0188] -kl is calculated as Therefore, k l = kth in i l Numbers from the right +1,

[0189] ○When i=0, k 0 =k 1 =k 2 =…=1,

[0190] ○When i=1, k 0 =2, k 1 =k 2 =…=1,

[0191] ○When i=100, k 2 =2, k 0 =k 1 =k 3 =…=1,

[0192] -When the algorithm reaches a leaf node but does not find a successful connection, the index i is incremented.

[0193] The algorithm is FIG. 3A to FIG. 3D is shown in (where, Figure 3D yes FIG. 3A to FIG. 3C ). FIG. 3A to FIG. 3C For any state, the action on the left hand side has a higher For ease of explanation, FIG. 3A to FIG. 3C A simple state tree is shown; in some more complex state trees, for example, some of the parent nodes may have more than two child nodes. Figure 3A Because at the third level (corresponding to k 2 ), the estimated quality value generated by the neural network has sufficient error so that a 1 Given a 2 The estimated quality value is high, so the path (which is the path that would be followed if the algorithm of Listing 3 was used) ends at a termination node to which not all active ports are connected. When the algorithm of Listing 4 detects that the termination state does not correspond to a successful set of connections, it begins trying to take the next best action at each level, one level at a time. Figure 3B It shows that when the right-hand side action is at the first level (corresponding to k 0 ) is taken, this approach also initially results in failure, and Figure 3C It shows that when the right-hand side action is at the third level (corresponding to k 2 ) is taken, the method leads to success. It can be seen that when the index is i=100, the algorithm thus reaches a successful state. At i=100, the algorithm selects at levels 0, 1, 3 and 4 the highest The corresponding action, and the algorithm selects the second highest Corresponding Actions. Once a successful set of connections has been found, the UE may make those connections in the RFIC (eg, by writing corresponding values ​​to registers in the RFIC, causing RF switches in the RFIC to close to make the connections found by the algorithm).

[0194] Listing 4

[0195] 1. Input: G, B, θ,

[0196] 2. Check feasibility using Algorithm 1. If not feasible, stop (STOP).

[0197] 3. Initialization: initial state s 0 , initial loop index i 0 =0, maximum loop index: i max =199,

[0198] 4. For (FOR) i = i 0 :i max

[0199] a. State s = s 0 , l = 0,

[0200] b. Loop (WHILE) (1)

[0201] i. Calculate the action index in level l:

[0202] ii. Get the set of valid actions A for state s.

[0203] iii. If (IF) k l >The number of valid actions, then

[0204] Break out of the loop;

[0205] iv. End condition (ENDIF)

[0206] v. Run the neural network with parameters θ to obtain

[0207] vi. Select the kth l The action with the maximum Q-value a * .

[0208] vii. Get the next state: (s, a * )→s′.

[0209] viii. Update: s←s′, l←l+1,

[0210] ix. If (IF)s is a leaf node, then

[0211] Break out of the loop;

[0212] x. End condition (ENDIF)

[0213] c. End the loop (ENDWHILE)

[0214] d. Check connection validity (check that all constraints C1 to C8 are satisfied)

[0215] e. If (IF) all ports have valid connections in s, then

[0216] Stop (STOP) (Success)

[0217] f. End condition (ENDIF)

[0218] 5. End the loop (ENDFOR)

[0219] Figure 4 A flow chart of a method is shown. In some embodiments, the method includes: at 405, receiving, by a first neural network, a first state and a first state transition, the first state including: one or more identifiers of available active ports, and a set of available connections between two or more elements, each element being an active port or a mixer or a local oscillator; and at 410, generating, by the first neural network, a first estimated quality value for the first state transition, the first estimated quality value corresponding to the likelihood that the first state transition is one of a series of transitions terminating in a terminal state, a connection being made to each of the available active ports in the terminal state, wherein: the first state transition is a transition from the first state to a second state, and the second state includes a connection between two of the elements that is not present in the first state. Figure 5 A system including a UE 505 and a gNB 510 in communication with each other is shown. The UE may include a radio transceiver 515 and a processing circuit (or means for processing) 520, which may include or be connected to a memory 525 and may perform various methods disclosed herein (e.g., Figure 4 For example, the processing circuit 520 may receive a transmission from the network node (gNB) 510 via the radio transceiver device 515, and the processing circuit 520 may send a signal to the gNB 510 via the radio transceiver device 515.

[0220] In some examples of the embodiments described herein, a mixer is connected to a local oscillator and an available active port, but the disclosure is not limited to such circuits. For example, in some embodiments, connections between circuit elements are selected, each circuit element being (1) a first circuit type, (2) a second circuit type that operably connects an element of the first circuit type to one of the available active ports, and (3) an available active port, wherein the first circuit type may be, but is not limited to, a local oscillator, and the second circuit type may be, but is not limited to, a mixer.

[0221] As used herein, "a portion" of something means "at least some" of the thing, which may mean less than all of the thing or may mean all of the thing. Thus, as a special case, "a portion" of a thing includes the entire thing (i.e., an example of a portion of the thing being a portion of the thing). As used herein, when a second quantity is "within Y" of a first quantity X, this means that the second quantity is at least XY and the second quantity is at most X+Y. As used herein, when a second number is "within Y%" of a first number, this means that the second number is at least (1-Y / 100) times the first number and the second number is at most (1+Y / 100) times the first number. As used herein, the term "or" should be interpreted as "and / or", so that, for example, "A or B" means either "A" or "B" or "A and B".

[0222] Each of the terms "processing circuit" and "means for processing" is used herein to refer to any combination of hardware, firmware, and software that is employed to process data or digital signals. Processing circuit hardware may include, for example, an application specific integrated circuit (ASIC), a general or dedicated central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), and a programmable logic device (such as a field programmable gate array (FPGA)). In a processing circuit, as used herein, each function is performed by hardware that is configured (i.e., hardwired) to perform that function, or by more general hardware (such as a CPU) that is configured to execute instructions stored in a non-temporary storage medium. The processing circuit may be manufactured on a single printed circuit board (PCB) or distributed on several interconnected PCBs. The processing circuit may include other processing circuits, for example, the processing circuit may include two processing circuits (FPGA and CPU) interconnected on a PCB.

[0223] As used herein, the term "array" refers to an ordered collection of numbers, regardless of how they are stored (e.g., in consecutive memory locations or in a linked list). As used herein, when a method (e.g., an adjustment) or a first quantity (e.g., a first variable) is referred to as being "based on" a second quantity (e.g., a second variable), this means that the second quantity is an input to the method or affects the first quantity (e.g., the second quantity may be an input (e.g., the only input or one of several inputs) to a function that calculates the first quantity), or the first quantity may be equal to the second quantity, or the first quantity may be the same as the second quantity (e.g., stored in the same one or more locations in memory).

[0224] It will be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, without departing from the spirit and scope of the inventive concept, the first element, first component, first region, first layer or first part discussed herein may be referred to as the second element, second component, second region, second layer or second part.

[0225] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the inventive concept. As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation rather than terms of degree, and are intended to account for the inherent deviations in measurements or calculations that one of ordinary skill in the art would recognize.

[0226] As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. It will also be understood that the terms "including" and / or "comprising" when used in this specification, illustrate the existence of narrated features, wholes, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their groups. As used herein, the term "and / or" includes any combination and all combinations of one or more of the relevant listed items. Statements such as "at least one of..." modify the entire column of elements when located after a column of elements, without modifying the individual elements in the column. In addition, the use of "may" when describing the embodiments of the inventive concept represents "one or more embodiments of the present disclosure". In addition, the term "exemplary" is intended to represent an example or description. As used herein, the terms "use", "being used" and "being used" may be considered to be synonymous with the terms "utilize", "being utilized" and "being utilized".

[0227] It will be understood that when an element or layer is referred to as being “on,” “connected to,” “bound to,” or “adjacent to” another element or layer, the element or layer may be directly on, directly connected to, directly bound to, or directly adjacent to the other element or layer, or one or more intervening elements or layers may be present. In contrast, when an element or layer is referred to as being “directly on,” “directly connected to,” “directly bound to,” or “immediately adjacent to” another element or layer, there are no intervening elements or layers.

[0228] Any numerical range listed herein is intended to include all sub-ranges of the same numerical precision that fall within the listed range. For example, a range of "1.0 to 10.0" or "between 1.0 and 10.0" is intended to include all sub-ranges between (and including) the listed minimum value of 1.0 and the listed maximum value of 10.0 (that is, all sub-ranges having a minimum value equal to or greater than 1.0 and a maximum value equal to or less than 10.0 (such as, for example, 2.4 to 7.6)). Similarly, a range described as "within 35% of 10" is intended to include all subranges between (and including) the listed minimum value of 6.5 (i.e., (1-35 / 100)×10) and the listed maximum value of 13.5 (i.e., (1+35 / 100)×10) (that is, all subranges having a minimum value equal to or greater than 6.5 and a maximum value equal to or less than 13.5 (such as, for example, 7.4 to 10.6)). Any maximum numerical limitation listed herein is intended to include all lower numerical limitations subsumed therein, and any minimum numerical limitation listed in this specification is intended to include all higher numerical limitations subsumed therein.

[0229] Although exemplary embodiments of systems and methods for configuring RF networks based on machine learning have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Therefore, it will be understood that systems and methods for configuring RF networks based on machine learning constructed in accordance with the principles of the present disclosure may be implemented in ways other than those specifically described herein. The invention is also defined in the appended claims and their equivalents.

Claims

1. A method for operating a user device, comprising: receiving, by a first neural network, a first state and a first state transition, the first state comprising: one or more identifiers for available active ports, and a set of available connections between two or more circuit elements, each of the circuit elements being one of: (1) a first circuit type, (2) a second circuit type, the second circuit type operably connecting the circuit element of the first circuit type to one of the available active ports, and (3) the available active port; and generating, by the first neural network, a first estimated quality value for the first state transition, the first estimated quality value corresponding to a likelihood that the first state transition is one of a series of transitions terminating in a terminal state in which a connection is established to each of the available active ports, in: A first state transition is a transition from a first state to a second state, and The second state includes a connection between two of the circuit elements that was not present in the first state.

2. The method according to claim 1, further comprising: feeding the first state and the first state transition to a first neural network; receiving a first estimated quality value from a first neural network; feeding the first state and the second state transition to the first neural network; as well as receiving a second estimated quality value from the first neural network, the second estimated quality value corresponding to a likelihood that the second state transition is one of a series of transitions terminating in a terminal state in which a connection is established to each of the available active ports, The second state transition is a transition from the first state to a third state, the third state including a connection between two of the circuit elements that are not present in the first state and are not present in the second state.

3. The method according to claim 2, further comprising: It is determined that the second estimated mass value is greater than the first estimated mass value.

4. The method according to claim 3, further comprising: In response to determining that the second estimated quality value is greater than the first estimated quality value, a third state and a third state transition are fed to the first neural network, wherein the third state transition is a transition from the third state to a fourth state, the fourth state including a connection between two of the circuit elements that were not present in the third state.

5. The method according to claim 4, further comprising: Feed the second state and fourth state transitions to the first neural network, in: The fourth state transition is a transition from the second state to the fifth state, and The fifth state includes a connection between two of the circuit elements that were not present in the second state.

6. The method according to claim 1, wherein: The first circuit type is a local oscillator and the second circuit type is a mixer, and wherein the first estimated quality value further corresponds to a likelihood that the first state transition is one of a series of transitions terminating in a termination state in which a connection is established to each of the available active ports and in which respective two connections are established from the local oscillator to the two mixers.

7. The method according to claim 1, further comprising: A feasibility test is performed to check for an indication that a terminated state cannot be reached from the first state, in which a connection is established to each of the available active ports.

8. The method according to claim 7, wherein: A number of circuit elements of a first circuit type are available, a number of circuit elements of a second circuit type are available, and the feasibility test is based on the number of available circuit elements of the first circuit type and the number of available circuit elements of the second circuit type.

9. The method according to claim 1, further comprising: using a Monte Carlo tree search to generate a training data set to assign quality values ​​to each of multiple combinations of states and state transitions; Using the training data set to train the neural network to generate a network parameter set; as well as The set of network parameters is stored in the first neural network.

10. A user equipment, comprising: Processing circuit; as well as memory, connected to the processing circuitry, The memory stores instructions which, when executed by the processing circuit, cause the user equipment to perform a method, the method comprising: receiving, by a first neural network, a first state and a first state transition, the first state comprising: one or more identifiers for available active ports, and a set of available connections between two or more circuit elements, each of the circuit elements being one of: (1) a first circuit type, (2) a second circuit type, the second circuit type operably connecting the circuit element of the first circuit type to one of the available active ports, and (3) the available active port; and generating, by the first neural network, a first estimated quality value for the first state transition, the first estimated quality value corresponding to a likelihood that the first state transition is one of a series of transitions terminating in a terminal state in which a connection is established to each of the available active ports, in: A first state transition is a transition from a first state to a second state, and The second state includes a connection between two of the circuit elements that was not present in the first state.

11. The user equipment according to claim 10, wherein: The method further comprises: feeding the first state and the first state transition to a first neural network; receiving a first estimated quality value from a first neural network; feeding the first state and the second state transition to the first neural network; and receiving a second estimated quality value from the first neural network, the second estimated quality value corresponding to a likelihood that the second state transition is one of a series of transitions terminating in a terminal state in which a connection is established to each of the available active ports, The second state transition is a transition from the first state to a third state, the third state including a connection between two of the circuit elements that are not present in the first state and are not present in the second state.

12. The user equipment according to claim 11, wherein: The method also includes determining that the second estimated mass value is greater than the first estimated mass value.

13. The user equipment according to claim 12, wherein: The method also includes, in response to determining that the second estimated quality value is greater than the first estimated quality value, feeding a third state and a third state transition to the first neural network, wherein the third state transition is a transition from the third state to a fourth state, the fourth state including a connection between two of the circuit elements that are not present in the third state.

14. The user equipment according to claim 13, wherein: The method further comprises: feeding the second state and the fourth state transition to the first neural network, in: The fourth state transition is a transition from the second state to the fifth state, and The fifth state includes a connection between two of the circuit elements that were not present in the second state.

15. The user equipment according to claim 10, wherein: The first circuit type is a local oscillator and the second circuit type is a mixer, and wherein the first estimated quality value further corresponds to a likelihood that the first state transition is one of a series of transitions terminating in a termination state in which a connection is established to each of the available active ports and in which respective two connections are established from the local oscillator to the two mixers.

16. The user equipment according to claim 10, wherein: The method further comprises performing a feasibility test to check for an indication that a terminated state cannot be reached from the first state, in which a connection is established to each of the available active ports.

17. The user equipment according to claim 16, wherein: A number of circuit elements of a first circuit type are available, a number of circuit elements of a second circuit type are available, and the feasibility test is based on the number of available circuit elements of the first circuit type and the number of available circuit elements of the second circuit type.

18. The user equipment according to claim 10, wherein: The method further comprises: using a Monte Carlo tree search to generate a training data set to assign quality values ​​to each of multiple combinations of states and state transitions; Using the training data set to train the neural network to generate a set of network parameters; and The set of network parameters is stored in the first neural network.

19. A user equipment, comprising: Device for processing; as well as a memory, connected to the means for processing, The memory stores instructions which, when executed by the apparatus for processing, cause the user equipment to perform a method comprising: receiving, by a first neural network, a first state and a first state transition, the first state comprising: one or more identifiers for available active ports, and a set of available connections between two or more circuit elements, each of the circuit elements being one of: (1) a first circuit type, (2) a second circuit type, the second circuit type operably connecting the circuit element of the first circuit type to one of the available active ports, and (3) the available active port; and generating, by the first neural network, a first estimated quality value for the first state transition, the first estimated quality value corresponding to a likelihood that the first state transition is one of a series of transitions terminating in a terminal state in which a connection is established to each of the available active ports, in: A first state transition is a transition from a first state to a second state, and The second state includes a connection between two of the circuit elements that was not present in the first state.

20. The user equipment according to claim 19, wherein: The method further comprises: feeding the first state and the first state transition to a first neural network; receiving a first estimated quality value from a first neural network; feeding the first state and the second state transition to the first neural network; and receiving a second estimated quality value from the first neural network, the second estimated quality value corresponding to a likelihood that the second state transition is one of a series of transitions terminating in a terminal state in which a connection is established to each of the available active ports, The second state transition is a transition from the first state to a third state, the third state including a connection between two of the circuit elements that are not present in the first state and are not present in the second state.

Images