Variable quantification method for MIMO detector and computer equipment

Abstract

The invention relates to the technical field of communication, and discloses a variable quantization method for an MIMO detector and computer equipment, and the method comprises the steps: determining a target quantization variable and corresponding state information for each time step, and enabling the target quantization variable to belong to a to-be-quantized variable set; historical time sequence information of the time step is obtained, the historical time sequence information and the state information are processed by using a deep cycle intelligent agent to determine a target output result of the target quantization variable, and the deep cycle intelligent agent is constructed based on a deep cycle Q network; determining a target execution action of the target quantization variable according to the target output result, and updating a quantization parameter of the target quantization variable according to the target execution action; and determining a final quantization strategy of the to-be-quantized variable set based on the quantization parameter updating result of each time step. According to the technical scheme provided by the invention, the calculation complexity and the hardware resource consumption of the MIMO detector can be reduced.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a variable quantization method and computer device for a MIMO detector. Background Technology

[0002] Currently, massively multi-input multiple-output (MIMO) detectors have become a key technology for improving the capacity and spectral efficiency of wireless communication systems. With the continuous expansion of antenna array size and the widespread application of high-order modulation techniques, the computational complexity of MIMO detectors is increasing exponentially, posing severe challenges to the energy efficiency, throughput, and cost of hardware implementation.

[0003] To balance the performance and power consumption of MIMO detectors, related technologies typically employ quantization to convert high-precision floating-point operations into low-precision fixed-point operations. The choice of quantization precision directly impacts the system's energy efficiency, processing throughput, and storage resource requirements. Furthermore, the impact of bit-width variations in different functional modules on overall computational complexity varies significantly during quantization, making it difficult to efficiently utilize the hardware resources of the MIMO detector. Therefore, reducing the computational complexity and hardware resource consumption of MIMO detectors has become a pressing issue. Summary of the Invention

[0004] This application provides a variable quantization method and computer device for MIMO detectors, which can reduce the computational complexity and hardware resource consumption of MIMO detectors.

[0005] A first aspect of this application provides a variable quantization method for a MIMO detector. The method includes: for each time step, determining a target quantization variable and its corresponding state information, wherein the target quantization variable belongs to a set of variables to be quantized, and the number of time steps is greater than or equal to the number of variables in the set of variables to be quantized; acquiring historical time-series information of the time step, and processing the historical time-series information and the state information using a deep recurrent agent to determine a target output result of the target quantization variable, wherein the deep recurrent agent is constructed based on a deep recurrent Q-network; determining a target execution action of the target quantization variable based on the target output result, and updating the quantization parameters of the target quantization variable based on the target execution action; and determining a final quantization strategy for the set of variables to be quantized based on the quantization parameter update results of each time step, wherein the final quantization strategy is used to quantize the set of variables to be quantized in the MIMO detector.

[0006] In one implementation, the state information includes variable index, quantization parameters, and historical execution actions. For each time step, determining the target quantization variable and its corresponding state information includes: based on the state of the MIMO detector at the current time step, determining the target quantization variable from the set of variables to be quantized, the set of variables to be quantized including intermediate variables and iteration counts of the MIMO detector during signal processing; obtaining the variable index, quantization parameters, and historical execution actions of the target quantization variable, and determining the obtained variable index, quantization parameters, and historical execution actions as the state information of the target quantization variable; wherein, the quantization parameters include the iteration count and the bit width of the intermediate variables.

[0007] In one implementation, the historical time series information includes a historical hidden state vector and a historical cell state vector, wherein the historical hidden state vector represents the state information of the previous time step, and the historical cell state vector represents the association information of one or more previous time steps.

[0008] In one implementation, the historical hidden state vector and the historical cell state vector are determined as follows: if the time step is the initial time step, the historical hidden state vector and the historical cell state vector are initialized to zero vectors; if the time step is not the initial time step, the historical hidden state vector and the cell state are determined by the LSTM layer based on the state information of the previous time step.

[0009] In one embodiment, the deep recurrent Q-network includes an embedding network layer, a first fully connected layer, an LSTM network layer, and a second fully connected layer. Processing the historical time-series information and the state information using a deep recurrent agent to determine the target output result of the target quantization variable includes: inputting the state information into the embedding network layer for embedding processing to obtain a state embedding vector; inputting the state embedding vector into the first fully connected layer for nonlinear transformation; inputting the transformed state embedding vector into the LSTM network layer; the LSTM network layer analyzing the transformed state embedding vector and the historical time-series information to output a state feature vector to the second fully connected layer; and the second fully connected layer analyzing the state feature vector to output the target output result of the target quantization variable.

[0010] In one embodiment, the embedding network layer includes a first embedding layer, a second embedding layer, and a third embedding layer. The state information includes a variable index, a variable bit width, and a historical execution action. Inputting the state information into the embedding network layer for embedding processing to obtain a state embedding vector includes: inputting the variable index, the variable bit width, and the historical execution action into the first embedding layer, the second embedding layer, and the third embedding layer, respectively, to obtain a variable index vector, a variable bit width vector, and a historical execution action vector; and concatenating the variable index vector, the variable bit width vector, and the historical execution action vector to form the state embedding vector.

[0011] In one implementation, the deep recurrent agent and the MIMO detector undergo multi-round interactive training. The deep recurrent agent includes a replay buffer and a deep recurrent Q-network. The deep recurrent agent is trained according to the following steps: at each time step of each training round, a target action is determined based on the current state information, and the target action is executed to update the quantization policy; the updated quantization policy is applied to the MIMO detector, and the MIMO detector outputs reward feedback and next state information; based on the reward feedback, the next state information, the current state information, and the target action, interactive experiences are constructed as tuples, and the interactive experiences at each time step are stored as tuples in the replay buffer; multiple interactive experiences as tuples in the replay buffer are sampled, the time difference error is calculated based on the sampling results, and the network parameters of the deep recurrent Q-network are updated according to the time difference error until the current training round terminates.

[0012] In one embodiment, the deep recurrent agent further includes a target network; after updating the network parameters of the deep recurrent Q-network according to the time difference error, the method further includes: periodically synchronizing the network parameters of the deep recurrent Q-network to the target network, and updating the target network based on the periodically synchronized network parameters, wherein the deep recurrent Q-network is used for real-time decision-making, and the target network is used to provide the target output result during the network parameter update of the deep recurrent Q-network.

[0013] In one implementation, the target execution action is determined based on a predefined discrete action space, which includes multiple action values. The target execution action is used to increase or decrease quantization parameters, which include variable bit width and iteration count.

[0014] In one implementation, the reward feedback includes positive rewards and negative rewards; the termination condition of the training round is: when the deep recurrent agent obtains negative rewards in consecutive time steps within a preset termination step number, the current training round is terminated, wherein the preset termination step number is determined based on the memory length of the LSTM layer of the deep recurrent Q network, and the reward feedback is determined based on the bit error rate performance, average bit width, and computational complexity under the current quantization strategy.

[0015] A third aspect of this application provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the quantization method for a MIMO detector as described in the first aspect above.

[0016] Based on the above ideas, the technical solution provided in this embodiment of the application analyzes the quantization method for the state information of each target quantization variable using a deep recurrent agent, thereby achieving automatic optimization of the hardware resources of the MIMO detector. Specifically, by decomposing the quantization process into multiple time steps and dynamically quantizing the variables to be quantized one by one according to the time step, comprehensive coverage and precise adjustment of all variables to be quantized are achieved. Using a deep recurrent agent, the target output result is determined based on the state information of the target quantization variable and the historical time sequence information of the time step, and the target action to be executed on the target quantization variable is determined to dynamically adjust the quantization parameters of different variables. This reduces unnecessary high-precision calculations and captures the time dependence of signal processing, avoiding bit width allocation from getting trapped in local optima. This ensures the calculation accuracy of the MIMO detector and effectively reduces computational complexity and hardware resource consumption, thereby improving hardware operating efficiency.

[0017] It is evident that the technical solution provided in this application can reduce the computational complexity and hardware resource consumption of MIMO detectors, while improving the quantization efficiency and accuracy of MIMO detectors. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A schematic diagram illustrating the steps of a variable quantization method for a MIMO detector provided in an embodiment of this application; Figure 2A schematic diagram of the architecture of a deep recurrent Q-network provided in one embodiment of this application; Figure 3 A schematic diagram illustrating the training steps of a deep recurrent agent provided in one embodiment of this application; Figure 4 A schematic diagram of the architecture steps of a deep loop agent provided in one embodiment of this application; Figure 5 A schematic diagram of a variable quantization device for a MIMO detector provided in one embodiment of this application; Figure 6 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Furthermore, the use of terms such as "first," "second," etc., in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of embodiments in this application, unless otherwise stated, "multiple" means two or more. Additionally, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0022] With the evolution of multi-antenna technology, large-scale multiple-input multiple-output (MIMO) systems have become a key technology for improving the capacity and spectral efficiency of wireless communication systems. However, the continuous expansion of antenna array size and the widespread application of higher-order modulation techniques have led to an exponential increase in the computational complexity of MIMO detectors, posing severe challenges to the energy efficiency, throughput, and cost of hardware implementation. In hardware implementation, fixed-point arithmetic is typically used instead of floating-point arithmetic to control computational complexity and power consumption. The choice of quantization precision directly affects the system's energy efficiency, processing throughput, and storage resource requirements.

[0023] In related technologies, mixed-precision quantization methods are typically used to handle different variables generated during signal processing in MIMO detectors. For example, equal weighting strategies are employed to handle the bit width allocation of different variables. However, this approach ignores the significant differences in the impact of bit width changes on the overall computational complexity of different functional modules. In particular, for the core computational units in iterative detection algorithms, bit width adjustments can lead to a non-linear increase in computational complexity. Related technologies have failed to effectively model and optimize this, resulting in significant hardware resource consumption and difficulty in achieving high hardware efficiency. Furthermore, the time-dependent characteristics of the iterative execution of MIMO detection algorithms are not considered in related technologies, and their bit width optimization results often get trapped in local optima, leading to poor hardware computational efficiency and accuracy of MIMO detectors.

[0024] In view of this, one or more embodiments of this application provide a variable quantization method and computer device for a MIMO detector, which can solve the above problems and realize automatic, efficient and coordinated mixed-precision quantization of each intermediate variable in the MIMO detector. While ensuring MIMO detection performance, it reduces the complexity of MIMO hardware computation, thereby reducing MIMO hardware resource consumption and improving the hardware computation efficiency and accuracy of the MIMO detector.

[0025] Please see Figure 1 One embodiment of this application provides a variable quantization method for a MIMO detector, which may include the following steps: S1: For each time step, determine the target quantization variable and its corresponding state information, wherein the target quantization variable belongs to the set of variables to be quantized, and the number of time steps is greater than or equal to the number of variables in the set of variables to be quantized; S3: Obtain the historical time series information of the time step, and use a deep recurrent agent to process the historical time series information and the state information to determine the target output result of the target quantization variable, wherein the deep recurrent agent is constructed based on a deep recurrent Q network; S5: Determine the target execution action of the target quantized variable based on the target output result, and update the quantization parameter of the target quantized variable based on the target execution action; S7: Based on the quantization parameter update results at each time step, determine the final quantization strategy for the set of variables to be quantized, wherein the final quantization strategy is used to quantize the set of variables to be quantized in the MIMO detector.

[0026] The aforementioned time step can be understood as the smallest time unit for a complete interaction between the agent and the environment. The entire quantization process is divided into multiple time steps, each corresponding to a specific quantization operation. For example, within each time step, the agent adjusts only the bit width of one variable. The set of variables to be quantized includes multiple intermediate variables generated by the MIMO detector during signal processing and the number of iterations. Where there are K intermediate variables in the set, the number of iterations is considered the (K+1)th variable to be quantized. In each time step, a target quantization variable is selected from the set of variables to be quantized, considered as the variable requiring quantization adjustment in the current time step. By dividing the entire quantization process into multiple time steps and adjusting the quantization parameters of the target quantization variables one by one, dynamic quantization of the entire set of variables to be quantized is achieved.

[0027] For each time step, the target quantization variable and its corresponding state information are determined. This state information includes the variable index, quantization parameters, and historical execution actions, describing the current state of the target quantization variable. Subsequently, based on this state information, corresponding quantization parameter changes are specified for the target quantization variable, such as appropriate bit width allocation. Through comprehensive and accurate state information, the deep recurrent agent can adjust the quantization parameters more precisely. Furthermore, during signal processing in the MIMO detector, some variables may require multiple adjustments. To ensure that each variable is processed one by one, the number of time steps must be greater than or equal to the number of variables to be quantized. This guarantees that each variable can be processed in a certain time step, thereby achieving comprehensive quantization adjustment of the entire variable set.

[0028] In this embodiment, the aforementioned deep recurrent agent can be understood as an intelligent decision-making system based on deep learning and reinforcement learning, and is a key technical component for optimizing the quantization strategy of the MIMO detector. By interacting with the MIMO detector and combining the advantages of deep learning, reinforcement learning, and recurrent neural networks, the deep recurrent agent can dynamically and intelligently adjust the quantization parameters to reduce computational complexity and hardware resource consumption while ensuring the performance of the MIMO detector. The deep recurrent agent is built based on a deep recurrent Q-network, enabling it to handle both historical time-series information and state information.

[0029] Specifically, the aforementioned deep recurrent agent incorporates a Deep Recurrent Q-Network (DRQN), a reinforcement learning network structure combining deep learning and recurrent neural networks (RNNs). This network serves as the core decision network of the deep recurrent agent, evaluating the expected reward (i.e., Q-value) for each possible action. Specifically, historical time-series information and state information are input into the DRQN ​​for calculation and processing to output the Q-value of the target quantized variable. This Q-value is the DRQN's estimate of the expected reward for each possible action in a given state. The target output includes at least the aforementioned Q-value, which guides subsequent quantization operations, ensuring that each target quantized variable achieves a better quantization state at the current time step. The deep recurrent agent is pre-trained and can utilize its learned knowledge to process the current historical time-series and state information, thereby outputting a reasonable target output.

[0030] The aforementioned historical time-series information is used to describe the changes and dependencies of the target quantization variable at different time steps, enabling the agent to capture the time dependencies in the signal processing process, avoid the bit width allocation of the variable getting trapped in local optima, and thus improve the computational accuracy of the MIMO detector. Specifically, the deep recurrent agent captures historical time-series information through a deep recurrent Q-network. By acquiring historical time-series information from previous time steps, it introduces time-series dependencies during the agent's processing of state information. The agent can comprehensively consider the historical and current states of the target quantization variable to dynamically adjust the quantization parameters and optimize the quantization strategy, thereby reducing the hardware resource consumption of the MIMO detector and improving its hardware efficiency and accuracy.

[0031] In this embodiment, the type of quantization parameters corresponding to the action to be performed is determined based on the different types of target quantization variables. These quantization parameters determine the quantization accuracy of the target quantization variable, affecting computational complexity and detection performance. The target execution action can be understood as a specific action determined based on the target output result, which changes the quantization parameters of the target quantization variable. Specifically, the quantization parameters can be bit width or iteration count. If the target quantization variable is an intermediate variable generated by the MIMO detector during signal processing, the bit width of the target quantization variable is changed; if the target quantization variable is the number of iterations of the MIMO detector during signal processing, the specific value of the iteration count is changed.

[0032] In this embodiment, the specific execution content of the aforementioned target action is determined by the aforementioned target output result, such as increasing or decreasing quantization parameters (e.g., bit width or number of iterations). Specifically, based on the target output result of the deep recurrent agent, it is determined how to adjust the quantization parameters of the target quantization variable so that the update method of the quantization parameters can be characterized by the target action. For example, if the target action is to increase the bit width by 1, then the bit width of the target quantization variable increases from b... k Updated to b k +1. Based on the target output results of different target quantization variables, the quantization parameters of the target quantization variables are dynamically adjusted to ensure that each target quantization variable can achieve a good quantization state at different time steps, thereby reducing unnecessary high-precision calculations, reducing computational complexity, and avoiding hardware resource consumption.

[0033] In this embodiment, the final quantization scheme for the entire set of variables to be quantized is determined based on the quantization parameter update results of all time steps. The quantization process is broken down into multiple time steps, each time step corresponding to the quantization operation of a single variable. Each variable to be quantized is dynamically quantized step by step to update the quantization parameters of the variables to be quantized, including the bit width and iteration number of each intermediate variable. Finally, the parameter update results of all time steps are integrated to form the final quantization strategy for the set of variables to be quantized, which is then used for quantization processing in the MIMO detector.

[0034] Based on the above ideas, the technical solution provided in this embodiment of the application analyzes the quantization method for the state information of each target quantization variable using a deep recurrent agent, thereby achieving automatic optimization of the hardware resources of the MIMO detector. Specifically, by decomposing the quantization process into multiple time steps and dynamically quantizing the variables to be quantized one by one according to the time step, comprehensive coverage and precise adjustment of all variables to be quantized are achieved. Using a deep recurrent agent, the target output result is determined based on the state information of the target quantization variable and the historical time sequence information of the time step, and the target action to be executed on the target quantization variable is determined to dynamically adjust the quantization parameters of different variables. This reduces unnecessary high-precision calculations and captures the time dependence of signal processing, avoiding bit width allocation from getting trapped in local optima. This ensures the calculation accuracy of the MIMO detector and effectively reduces computational complexity and hardware resource consumption, thereby improving hardware operating efficiency.

[0035] In one implementation, based on step S1 above, the state information includes variable indexes, quantization parameters, and historical execution actions. The variable indexes uniquely identify each variable in the set of variables to be quantized, allowing the agent to quickly locate the variable currently being processed. The quantization parameters describe the specific numerical values ​​of the variable's quantization state, typically including bit width and iteration count, i.e., the number of decimal places retained after quantization (decimal bit width) and the number of times the variable participates in calculations within the iterative algorithm. The historical execution actions record the agent's quantization adjustment actions in the previous time step. Further, after obtaining the quantized decimal bit width and iteration count by executing the target execution actions, a dataset of all variables can be generated using a Monte Carlo simulation algorithm. A limiting operation is then performed on the dataset to obtain the integer bit width of each variable.

[0036] In this implementation, for each time step, the variable index, quantization parameters, and historical execution actions of the target quantization variable at the current time step are determined. Specifically, based on the state of the MIMO detector at the current time step, the target quantization variable is determined from the set of variables to be quantized. The agent determines the target quantization variable that needs adjustment by evaluating the processing state of the MIMO detector at the current time step, including bit error rate, computational complexity, hardware resource usage, and signal processing status. Further, the variable index, quantization parameters, and historical execution actions of the target quantization variable are obtained, and these are used as the state information of the target quantization variable.

[0037] The technical solution provided in this embodiment refines the specific content of the state information, offering the agent richer state information, including variable indexes, quantization parameters, and historical execution actions. This provides the agent with comprehensive decision-making basis, enabling it to produce more accurate target output results. It also allows each target quantization variable to take more suitable target execution actions to update the quantization parameters, thereby optimizing the quantization strategy of the MIMO detector. This method not only reduces computational complexity and hardware resource utilization but also provides historical references for updating quantization parameters through historical execution actions, thereby improving the computational accuracy and efficiency of the MIMO detector.

[0038] In one implementation, the aforementioned historical time-series information includes a historical hidden state vector and a historical cell state vector. The historical hidden state vector is the hidden state vector of the previous time step, used to carry the state information of the previous time step. The historical cell state vector is a vector maintained internally by the deep recurrent Q-network, representing the correlation information including one or more previous time steps, i.e., representing the long-term dependencies in the time series, to ensure that the agent does not lose important historical information when processing the time series. The aforementioned deep recurrent Q-network includes an LSTM network layer, in which the historical time-series information of the current time step is obtained.

[0039] Specifically, the aforementioned historical hidden state vector and historical cell state vector are obtained in the LSTM network layer. The LSTM network layer is used to capture the temporal dependence characteristics of the MIMO detector during signal processing. At each time step, the LSTM network layer calculates new hidden state vector and cell state vector based on the current input state information, the historical hidden state vector of the previous time step, and the historical cell state vector of one or more previous time steps, as historical temporal information for the next time step.

[0040] In this embodiment, the determination of the historical hidden state vector and the historical cell state vector includes: if the current time step is the initial time step, the historical hidden state vector and the historical cell state vector are initialized to zero vectors in advance, because the system has not yet started processing any input data, so there is no historical state information for reference; if the current time step is not the initial time step, the historical hidden state vector is determined by the LSTM network layer based on the state information of the previous time step, and the historical cell state vector is determined by the LSTM network layer based on the state information of one or more previous time steps.

[0041] The technical solution provided in this embodiment of the application utilizes an LSTM network layer to process historical time-series information, enabling a deep recurrent agent to effectively capture the temporal dependency characteristics of the MIMO detector during signal processing. Specifically, the LSTM network layer dynamically updates the hidden state vector and cell state vector based on the state information of the previous time step and historical time-series information. Through the hidden state vector and cell state vector, the LSTM network layer can capture short-term and long-term dependencies in the time series. This provides the agent with rich contextual information, helping it make more accurate quantification adjustment decisions.

[0042] In one embodiment, the aforementioned deep recurrent Q-network includes an embedding network layer, a first fully connected layer, an LSTM network layer, and a second fully connected layer. The deep recurrent agent processes the historical time-series information and the state information to determine the target output result of the target quantized variable, including: inputting the state information into the embedding network layer for embedding processing to obtain a state embedding vector; inputting the state embedding vector into the first fully connected layer for nonlinear transformation; inputting the transformed state embedding vector into the LSTM network layer; the LSTM network layer analyzes the transformed state embedding vector and the historical time-series information to output a state feature vector to the second fully connected layer; and analyzing the state feature vector through the second fully connected layer to obtain the target output result of the target quantized variable. The target output result includes the Q-value of the target quantized variable, a new hidden state vector, and a cell state vector.

[0043] In this embodiment, the aforementioned embedding network layer includes a first embedding layer, a second embedding layer, and a third embedding layer. The aforementioned state information includes variable index, variable bit width, and historical execution actions. The state information is input into the embedding network layer for embedding processing to obtain a state embedding vector. Specifically, the variable index, variable bit width, and historical execution actions are input into the first, second, and third embedding layers, respectively, to obtain a variable index vector, a variable bit width vector, and a historical execution action vector. The variable index vector, variable bit width vector, and historical execution action vector are then concatenated to form the state embedding vector. By processing different state information separately through three embedding layers to represent the state information as a vector, this processing method converts discrete information into a continuous vector representation, enabling the neural network to process this information more effectively. Through embedding processing and vector concatenation, the agent can more accurately understand the dynamic changes of the target quantized variable, thereby making more reasonable quantization adjustment decisions.

[0044] In one embodiment, see Figure 2 , Figure 2 The network structure diagram represents the deep recurrent Q-network. This deep recurrent Q-network includes a first embedding layer, a second embedding layer, a third embedding layer, a first fully connected layer, a second fully connected layer, and an LSTM network layer. The deep recurrent agent processes historical temporal information and current state information through its network structure to determine the optimal action for the target quantized variable. The first, second, and third embedding layers encode the input; the first and second fully connected layers perform feature transformation; and the LSTM network layer captures temporal dependencies. Finally, it outputs the Q-value, hidden state vector, and cell state vector to guide action selection.

[0045] Specifically, historical time-series information and state information are first acquired and input into a deep recurrent Q-network. The input state information, including variable indices, variable bit widths, and historical actions, is processed through a first, second, and third embedding layer, respectively, to convert discrete or high-dimensional input data into continuous, low-dimensional vector representations. The output vectors of the embedding layers are then concatenated to form a state embedding vector. This state embedding vector is then input into a first fully connected layer, where a linear transformation is performed using weights and biases, and a non-linear characteristic is introduced through an activation function. The output of the first fully connected layer is then passed to an LSTM network layer. The LSTM network layer performs temporal dimension processing on the state embedding vector processed by the fully connected layer based on the historical time-series information and updates the hidden states and cell states. The output of the LSTM network layer is then further processed through a second fully connected layer to generate the target output results, including Q-values, hidden state vectors, and cell state vectors.

[0046] The technical solution provided in this embodiment of the application, by combining embedded network layers and LSTM network layers, enables deep recurrent Q-networks to effectively process historical temporal information and state information, thereby determining the target output result of the target quantization variable. This method not only transforms discrete information into a continuous vector representation, improving the model's expressive power and processing efficiency, but also enhances the model's generalization ability by capturing temporal dependency characteristics. The agent can more accurately understand the dynamic changes of the target quantization variable, thereby making more reasonable quantization adjustment decisions, optimizing the quantization strategy of the entire system, improving detection performance, reducing the computational complexity and hardware resource consumption of the MIMO detector, and simultaneously improving the quantization efficiency and accuracy of the MIMO detector.

[0047] In one implementation, the deep recurrent agent undergoes multi-round interactive training with the MIMO detector. The deep recurrent agent includes a revisit buffer and a deep recurrent Q-network. The revisit buffer can be understood as a data structure storing the interaction experience between the agent and the MIMO detector, used to store the agent's historical experience of interacting with the environment, i.e., the interaction experience is represented as tuples for training the deep recurrent agent. For details, please refer to... Figure 3 The aforementioned deep recurrent agent is trained according to the following steps: S701: At each time step of each training round, determine the target action based on the current state information, and execute the target action to update the quantization strategy; S702: The updated quantization strategy is applied to the MIMO detector, and the MIMO detector outputs reward feedback and next state information; S703: Construct interaction experience as a tuple based on the reward feedback, the next state information, the current state information, and the target execution action, and store the interaction experience of each time step as a tuple in the replay buffer. S704: Sample multiple interaction experiences in the revisit buffer as tuples, calculate the temporal difference error based on the sampling results, and update the network parameters of the deep recurrent Q network according to the temporal difference error until the current training round terminates.

[0048] The reward feedback mentioned above is used to evaluate the effectiveness of the quantization strategy and is usually a numerical value. The next state information mentioned above is the new state of the MIMO detector after performing an action, used by the deep recurrent agent to make the next decision or optimization. The interaction experience mentioned above, in tuples, typically includes current state information, target action, reward feedback, and next state information. This storage method facilitates sampling and updating network parameters during subsequent training. The temporal difference error mentioned above is the error signal used to update network parameters in reinforcement learning, representing the difference between the predicted and actual values. By sampling multiple interaction experiences in tuples to calculate the temporal difference error, the deep recurrent agent can self-update the network parameters of the deep recurrent Q-network, optimize its decision-making strategy, and thus output a more accurate Q-value.

[0049] In this embodiment, the deep recurrent agent further includes a target network. The deep recurrent Q-network is used for real-time decision-making, calculating the Q-value of the target quantized variable based on current state information and historical time-series information, and selecting the optimal quantization action. The target network has the same structure as the deep recurrent Q-network, but its parameter update method differs. The target network provides the target output result during the parameter update of the deep recurrent Q-network. During training, the parameters of the DRQN ​​are frequently updated, which may lead to large fluctuations in the Q-value, thus affecting the stability of training. The parameters of the target network are not updated in real time, but periodically from the deep recurrent Q-network. The target network can provide a relatively stable target output value, reducing Q-value fluctuations and enabling the deep recurrent Q-network to converge to the optimal policy more quickly, thereby improving the stability during training.

[0050] Specifically, in step S704 above, after updating the network parameters of the deep recurrent Q-network using the temporal difference error, the network parameters of the deep recurrent Q-network are periodically synchronized to the target network. Based on the network parameters obtained from the periodic synchronization, the target network is updated. For example, gradient descent can be used to update the network parameters of the deep recurrent Q-network. By periodically updating the parameters of the target network, the target network can smoothly adapt to changes during training, reduce instability during training, accelerate convergence, and optimize the quantization strategy, thereby improving the hardware efficiency and quantization accuracy of the MIMO detector.

[0051] In one embodiment, an interactive experience tuple can be represented as (S t a t r t S t+1 ), where S t This is the current state information, a t It is the target to perform the action, r t It's a reward feedback, S t+1 This is the next state information. For example, time difference error. It can be represented as The above formula calculates the TD error, i.e., the target value y. t With the current Q value Q(s) t ,h t-1 The square of the difference between y, a; θ) is taken as the time series difference error, where y t =r t +γmaxQ(s′ t+1 ,h t ,a′;θ - ), representing the maximum future Q value, γ is the discount factor, maxQ(s′) t+1 ,h t ,a′;θ - ) represents the state s′ in the target network. t+1 and hidden state h t Choose the action a′ that maximizes the Q value.

[0052] In this embodiment, the reward feedback includes positive and negative rewards. The termination condition for each training round is: when the deep recurrent agent receives negative rewards in consecutive time steps within a preset termination step number, the current training round is terminated. The preset termination step number is determined based on the memory length of the LSTM layer of the deep recurrent Q network. Since the LSTM layer can capture long-term dependencies in the time series, the memory length determines how many time steps of historical information it can remember. The reward feedback is determined based on the bit error rate performance, average bit width, and computational complexity under the current quantization strategy. The bit error rate performance is used to measure the ratio of the number of erroneous bits to the total number of bits in data transmission or storage. The average bit width is the average bit width of each variable to be quantized. The computational complexity represents the amount of computational resources required to execute the algorithm.

[0053] In one embodiment, the above reward function can be defined as: in, b0 represents the initial baseline quantization strategy, and BER0 represents the bit error rate performance under the baseline quantization strategy. t This indicates the updated quantization strategy, BER. tThe bit error rate performance is represented by ε1, ε2, and ε3, which are adjustment parameters controlling the shape of the reward curve and the bit error rate performance loss. -ρ represents the penalty coefficient for invalid solutions, and the negative reward is determined based on the above penalty coefficient. This reflects the average bit width and complexity under the current quantization strategy. ζ∈(0,1) is used to balance the optimization objective weights, where the average bit width... The computational complexity is calculated by taking the average decimal places of all variables. This is obtained by statistically analyzing the number of bit operations in the MIMO detection algorithm under the current quantization strategy.

[0054] The technical solution provided in this embodiment optimizes the quantization strategy through multi-round interactive training with the MIMO detector using a deep recurrent agent. Specifically, at each time step, the agent determines the target action, applies the quantization strategy, obtains reward feedback and next state information, and then constructs these information into an interaction experience tuple stored in a revisit buffer. During training, the agent samples the interaction experience from the buffer, calculates the temporal difference error, and updates the network parameters accordingly. Furthermore, the agent includes a target network whose parameters are periodically synchronized from the deep recurrent Q-network to provide stable target output values, reduce Q-value fluctuations, improve training stability and convergence speed, optimize the quantization strategy, and thus improve the hardware efficiency and quantization accuracy of the MIMO detector.

[0055] In one implementation, the target action is determined based on a predefined discrete action space, which is a set of predefined possible actions, including multiple action values, each representing a different quantization adjustment operation. The target action is used to increase or decrease quantization parameters, which include the number of iterations and the bit width of intermediate variables, such as increasing or decreasing the bit width of the variable, or increasing or decreasing the number of iterations. Preferably, after executing the action, the new quantization parameters of the variable are restricted to a valid range by a pruning function.

[0056] In one embodiment, during the training of a deep recurrent agent, an action selection mechanism based on a greedy policy randomly selects the action value of the current quantized variable. Specifically, the deep recurrent agent randomly selects an action value with probability ∈ , and selects the current optimal action value with probability 1-∈ , which can be represented as follows: Where, argmax a Q(s t , a; θ) represents selecting the action that maximizes the Q value, and θ represents the network parameters of the deep recurrent network.

[0057] In one embodiment, the discrete action space described above can be represented as: Take the corresponding action value action a t Then, the bit width of the k-th variable is updated to b.k '←clip(b k +a t ;0,b max ), based on the selected action a t Update the bit width b of the k-th variable k For b k′ b k ∈{0, 1, ..., b max}, where clip(v; min, max) is a clipping function used to limit the amplitude of the input variable v within [min, max] to ensure that the updated bit width does not exceed the predefined valid range [0, b]. max ].

[0058] The technical solution provided in this embodiment of the application enables an intelligent agent to precisely control quantization parameters through a predefined discrete action space, achieving fine-tuning of the quantization process. Specifically, the predefined discrete action space provides flexibility to the intelligent agent, allowing it to adapt to different environmental changes and performance requirements, thus enhancing the system's adaptability. By selecting actions within the predefined discrete action space to adjust quantization parameters, the intelligent agent can precisely control the quantization process, improve hardware resource utilization efficiency, reduce hardware computational complexity, and increase the accuracy of quantization calculations.

[0059] In one embodiment, see Figure 4 , Figure 4 The network architecture and training process characterize the deep recurrent agent. Specifically, the agent receives the current state s. t And obtain the historical hidden state vector h from the previous time step in the LSTM layer. t-1 and historical cell state vector c t-1 The input is fed into a deep recurrent agent, which calculates the Q-value Q(s) for each possible action using a Q-network. t ,a), and select an action a based on an ε-greedy policy. t The selected action a t It is applied to the quantitative strategy to update the quantitative parameters, and the environment outputs a reward r based on the newly applied quantitative strategy. t and the next state s t+1 The agent stores the current state, action, reward, and next state as an interaction experience tuple (s). t a t r t s t+1 ), and save it in the playback buffer.

[0060] In this embodiment, a batch of interactive empirical tuples is randomly sampled from the playback buffer. For each sampled empirical tuple, the target Q-value is calculated using the target network. The estimated Q-value calculated by the current Q-network and the target Q-value calculated by the target network are used to calculate the temporal difference error. Using the calculated temporal difference error and the sampled empirical tuples, the network parameters θ of the Q-network are updated using gradient descent or other optimization algorithms. The network parameters θ of the Q-network are periodically copied to the network parameters θ of the target network. - Through the training process described above, the deep recurrent agent can learn strategies for selecting the optimal action under different states.

[0061] Please see Figure 5 This application also provides a variable quantization device for a MIMO detector, the device comprising: a state information acquisition unit 100, for determining a target quantization variable and corresponding state information for each time step, wherein the target quantization variable belongs to a set of variables to be quantized, and the number of time steps is greater than or equal to the number of variables in the set of variables to be quantized; The target Q-value output unit 200 is used to input the state information into the deep recurrent agent and obtain the historical time series information of the time step. The deep recurrent agent determines the target output result of the target quantization variable based on the historical time series information and the state information. The deep recurrent agent is constructed based on a deep recurrent Q network. The action determination unit 300 is used to determine the target action of the target quantized variable based on the target output result, and update the quantization parameters of the target quantized variable based on the target action. The quantization strategy determination unit 400 is used to determine the final quantization strategy of the set of variables to be quantized based on the quantization parameter update results at each time step, wherein the final quantization strategy is used to quantize the set of variables to be quantized in the MIMO detector.

[0062] In one embodiment, the state information acquisition unit 100 is specifically used to determine the target quantization variable in the set of variables to be quantized based on the state of the MIMO detector at the current time step. The set of variables to be quantized includes intermediate variables and iterations of the MIMO detector during signal processing. The unit acquires the variable index, quantization parameters, and historical execution actions of the target quantization variable, and determines the acquired variable index, quantization parameters, and historical execution actions as the state information of the target quantization variable.

[0063] In one embodiment, the target Q-value output unit 200 is specifically used to input the state information of the target quantized variable into the embedding network layer in the deep recurrent Q network for embedding processing to obtain a state embedding vector, and input the state embedding vector into the LSTM network layer in the deep recurrent Q network. The LSTM network layer analyzes the transformed state embedding vector and historical time series information to output the target output result of the target quantized variable, including the output Q-value of the target quantized variable, the hidden state vector, and the cell state vector.

[0064] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0065] In this application, a variable quantization device for a MIMO detector is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, or other devices that can provide the above-mentioned functions.

[0066] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application, such as... Figure 6 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 Take a processor 10 as an example.

[0067] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0068] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0069] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0070] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0071] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0072] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0073] The apparatus, module, or unit described in the above embodiments can be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0074] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0075] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, devices, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0080] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0081] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0082] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A variable quantization method for a MIMO detector, characterized in that, The method includes: For each time step, a target quantization variable and its corresponding state information are determined, wherein the target quantization variable belongs to a set of variables to be quantized, and the number of time steps is greater than or equal to the number of variables in the set of variables to be quantized; The historical time series information of the time step is obtained, and the historical time series information and the state information are processed by a deep recurrent agent to determine the target output result of the target quantization variable, wherein the deep recurrent agent is constructed based on a deep recurrent Q network; Based on the target output, determine the target execution action for the target quantized variable, and update the quantization parameters of the target quantized variable based on the target execution action; Based on the quantization parameter update results at each time step, the final quantization strategy for the set of variables to be quantized is determined, wherein the final quantization strategy is used to quantize the set of variables to be quantized in the MIMO detector.

2. The method according to claim 1, characterized in that, The state information includes variable indexes, quantization parameters, and historical execution actions; for each time step, the target quantization variable and its corresponding state information are determined as follows: Based on the state of the MIMO detector at the current time step, a target quantization variable is determined from the set of variables to be quantized, the set of variables to be quantized including intermediate variables and iteration number of the MIMO detector during signal processing; Obtain the variable index, quantization parameters, and historical execution actions of the target quantized variable, and determine the obtained variable index, quantization parameters, and historical execution actions as the state information; wherein, the quantization parameters include the number of iterations and the variable bit width of the intermediate variable.

3. The method according to claim 1, characterized in that, The historical time series information includes historical hidden state vectors and historical cell state vectors. The historical hidden state vectors represent the state information of the previous time step, and the historical cell state vectors represent the association information of one or more previous time steps.

4. The method according to claim 3, characterized in that, The methods for determining the historical hidden state vector and the historical cell state vector include: If the time step is the initial time step, the historical hidden state vector and the historical cell state vector are initialized to zero vectors; If the time step is not the initial time step, the historical hidden state vector is determined based on the state information of the previous time step, and the historical cell state vector is determined based on the state information of one or more previous time steps.

5. The method according to claim 1, characterized in that, The deep recurrent Q-network includes an embedded network layer, a first fully connected layer, an LSTM network layer, and a second fully connected layer. The target output result of the target quantization variable is determined by processing the historical time series information and the state information using a deep recurrent agent, including: The state information is input into the embedding network layer for embedding processing to obtain a state embedding vector, and the state embedding vector is input into the first fully connected layer for nonlinear transformation. The transformed state embedding vector is input into the LSTM network layer, which analyzes the transformed state embedding vector and the historical time series information to output a state feature vector to the second fully connected layer. The second fully connected layer performs analysis based on the state feature vector to output the target output result of the target quantized variable.

6. The method according to claim 5, characterized in that, The embedding network layer includes a first embedding layer, a second embedding layer, and a third embedding layer. The state information includes variable index, variable bit width, and historical execution actions. The state information is input into the embedding network layer for embedding processing to obtain a state embedding vector, including: The variable index, variable bit width, and historical execution action are respectively input into the first embedding layer, the second embedding layer, and the third embedding layer to obtain the variable index vector, the variable bit width vector, and the historical execution action vector; The variable index vector, the variable bit width vector, and the historical execution action vector are concatenated to form the state embedding vector.

7. The method according to claim 1, characterized in that, The deep recurrent agent undergoes multi-round interactive training with the MIMO detector. The deep recurrent agent includes a revisit buffer and a deep recurrent Q-network. The deep recurrent agent is trained according to the following steps: At each time step of each training round, the target action is determined based on the current state information, and the target action is executed to update the quantization strategy. The updated quantization strategy is applied to the MIMO detector, which outputs reward feedback and next state information. Based on the reward feedback, the next state information, the current state information, and the target execution action, the interaction experience is constructed as a tuple, and the interaction experience of each time step is stored as a tuple in the replay buffer. Multiple interaction experiences in the revisit buffer are sampled as tuples. The temporal difference error is calculated based on the sampling results, and the network parameters of the deep recurrent Q network are updated according to the temporal difference error until the current training round terminates.

8. The method according to claim 7, characterized in that, The deep recurrent agent further includes a target network; after updating the network parameters of the deep recurrent Q-network according to the time difference error, the method further includes: The network parameters of the deep recurrent Q-network are periodically synchronized to the target network. Based on the network parameters obtained from the periodic synchronization, the target network is updated. The deep recurrent Q-network is used for real-time decision-making, and the target network is used to provide the target output result during the network parameter update of the deep recurrent Q-network.

9. The method according to claim 1 or 7, characterized in that, The target execution action is determined based on a predefined discrete action space, which includes multiple action values. The target execution action is used to increase or decrease quantization parameters, which include variable bit width and iteration number.

10. The method according to claim 7, characterized in that, The reward feedback includes positive rewards and negative rewards; the termination condition for the training round is: When the deep recurrent agent receives negative rewards in consecutive time steps within a preset termination step number, the current training round is terminated. The preset termination step number is determined based on the memory length of the LSTM layer of the deep recurrent Q network, and the reward feedback is determined based on the bit error rate performance, average bit width, and computational complexity under the current quantization strategy.

11. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the quantization method for a MIMO detector as described in any one of claims 1 to 10.

Images