A communication resource joint allocation method, device, apparatus and storage medium

Abstract

The application relates to the technical field of wireless communication, and discloses a communication resource joint allocation method, device, equipment and storage medium, the method comprising the following steps: based on a first constraint condition, determining a beamforming coefficient set and a power control coefficient set through a first machine learning model configured by each wireless access point in M wireless access points, the beamforming coefficient set being a set of K beamforming coefficients that maximize a total transmission rate under the first constraint condition, the power control coefficient set being a set of K power control coefficients that maximize the total transmission rate under the first constraint condition, and the total transmission rate being the sum of the transmission rates of signals received by each user equipment in K user equipments from a corresponding wireless access point; and based on the beamforming coefficient set and the power control coefficient set, controlling the M wireless access points to respectively allocate communication resources to corresponding user equipments. The application can reasonably allocate communication resources of a cell-free massive MIMO system.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and specifically to a method, apparatus, device, and storage medium for joint allocation of communication resources. Background Technology

[0002] Currently, non-cellular massive MIMO (Multiple-Input Multiple-Output) systems are a novel network architecture that aligns with the trend of decellularization in access networks. This system combines the characteristics of massive MIMO and distributed antenna systems, enabling it to simultaneously acquire the gains of both systems. Based on simple signal processing techniques, it provides users within the service area with considerable performance and wide coverage, making it a promising network architecture.

[0003] However, in cellular-free massive MIMO systems, the number of user equipment served by the access point (AP) is large. Due to the limited coherence time of the channel, the orthogonal pilot resources that the system can provide are limited. Consequently, if communication resources (such as transmit power and beamforming vectors) are not allocated reasonably, it will affect the transmission rate of the signal received by the user equipment, and thus affect the transmission quality of the signal. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, device and storage medium for joint allocation of communication resources to solve the problem that the transmission rate of a non-cellular massive MIMO system is affected by the inability to allocate communication resources reasonably.

[0005] In a first aspect, the present invention provides a method for joint allocation of communication resources, applied to the control module of a non-cellular massive MIMO system. The non-cellular massive MIMO system includes a control module, M wireless access points, and K user equipments, wherein the control module is connected to the M wireless access points, M and K are both integers, and M≥2, K≥2. The method includes: based on a first constraint, determining a beamforming coefficient set and a power control coefficient set using a first machine learning model configured for each of the M wireless access points, wherein the beamforming coefficient set is the set of K beamforming coefficients that maximize the total transmission rate under the first constraint. The power control coefficient set is the set of K power control coefficients that maximize the total transmission rate under the first constraint. Both the K beamforming coefficients and the K power control coefficients correspond one-to-one with the K user equipments. The total transmission rate is the sum of the transmission rates of the signals received by each of the K user equipments from their respective wireless access points. Based on the beamforming coefficient set and the power control coefficient set, M wireless access points are controlled to allocate communication resources to their corresponding user equipments. These communication resources include transmit power and beamforming vectors. The transmit power is determined by the corresponding power control coefficients, and the beamforming vectors are determined by the corresponding beamforming coefficients.

[0006] The communication resource joint allocation method provided in this embodiment, based on a first constraint, optimizes the beamforming coefficients and power control coefficients allocated to each user equipment using a first machine learning model configured in each of the M wireless access points. This results in a set of beamforming coefficients and a set of power control coefficients that maximize the total transmission rate. Then, based on these sets, each wireless access point allocates communication resources to its corresponding user equipment. This method, by jointly optimizing two coupled resource factors—beamforming vector and transmit power—using a machine learning model, maximizes the total transmission rate, thereby improving data transmission quality and system capacity. Simultaneously, joint optimization using a machine learning model reduces the complexity of resource optimization in cellular-free massive MIMO systems, improves optimization efficiency, and reduces the utilization of computing resources and time costs.

[0007] In an optional implementation, the method further includes: determining the effective power of the desired signal of the k-th user equipment based on the beamforming coefficient and power control coefficient corresponding to the k-th user equipment, and determining the effective power of the interference signal between the k-th user equipment and other user equipment, where k = 1, 2, ..., K, and the other user equipment refers to user equipment other than the k-th user equipment among the K user equipment; determining the signal-to-interference-plus-noise ratio (SIR) based on the effective power of the desired signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power; and determining the transmission rate at which the k-th user equipment receives the signal from the corresponding wireless access point based on the subcarrier control coefficient, the signal bandwidth, and the SIR.

[0008] The communication resource joint allocation method provided in this embodiment determines the effective power of the desired signal of the k-th user equipment (UE) based on the beamforming coefficient and power control coefficient corresponding to the k-th UE, and also determines the effective power of the interference signal between the k-th UE and other UEs. Based on the effective power of the desired signal of the k-th UE, the effective power of the interference signal between the k-th UE and other UEs, and the white noise power, the signal-to-interference-plus-noise ratio (SINR) is determined. Then, based on the subcarrier control coefficient, signal bandwidth, and SINR, the transmission rate of the signal received by the k-th UE from the corresponding wireless access point is determined. This allows for the correspondence between the beamforming coefficient, power control coefficient, and transmission rate, facilitating the accurate determination of the total transmission power corresponding to different sets of beamforming coefficients and power control coefficients by the first machine learning model configured for each wireless access point.

[0009] In one optional implementation, determining the signal-to-interference-plus-noise ratio (SIR) based on the effective power of the desired signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power includes: determining the SIR based on the effective power of the desired signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power, in the following manner:

[0010]

[0011] in, Indicates the signal-to-interference-plus-noise ratio, y v，k This represents the signal received by the k-th user on the v-th subcarrier, where v is an integer. This represents the effective power of the expected signal of the k-th user equipment. σ represents the effective power of the interference signal between the k-th user equipment and other user equipment. 2 α represents the power of white noise. m,kα represents the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment, where m = 1, 2, ..., M. m,k, p represents the wireless access point selection coefficient between the m-th wireless access point and other user equipment. m,k p represents the power control coefficient between the m-th wireless access point and the k-th user equipment. m,k, This represents the power control coefficient between the m-th wireless access point and other user equipment. This represents the channel vector between the m-th wireless access point and the k-th user equipment. This represents the channel vector between other wireless access points and the k-th user equipment, where other wireless access points are the M wireless access points excluding the m-th wireless access point. m,k w represents the beamforming coefficient between the m-th wireless access point and the k-th user equipment. m,k, This represents the beamforming coefficient between the m-th wireless access point and other user equipment.

[0012] In this embodiment, the signal-to-interference-plus-noise ratio can be determined more accurately using a formula, thereby improving the accuracy of the determined transmission rate.

[0013] In one optional implementation, determining the transmission rate of the signal received by the k-th user equipment from the corresponding wireless access point based on the subcarrier control coefficients, signal bandwidth, and signal-to-interference-plus-noise ratio (SINNR) includes: determining the transmission rate based on the subcarrier control coefficients, signal bandwidth, and SINNR in the following manner:

[0014]

[0015] Where, r k Let v represent the transmission rate at which the k-th user equipment receives the signal from the corresponding wireless access point, v represent the v-th subcarrier, V represent the set of subcarriers, and v and V are both integers. ρ v,k Let W represent the subcarrier control coefficient of the k-th user equipment on the v-th subcarrier, and let W represent the signal bandwidth. Indicates the signal-to-interference-plus-noise ratio, y v，k This represents the signal received by the k-th user on the v-th subcarrier.

[0016] In this embodiment, the correspondence between beamforming coefficients, power control coefficients, and transmission rates determined by the formula takes into account the mutual influence between user equipment and wireless access points. This can improve the accuracy of the first machine learning model configured for each wireless access point in optimizing the initial beamforming coefficients and initial power control coefficients allocated to the corresponding user equipment with the goal of maximizing the total transmission power. In turn, it can improve the accuracy of determining the set of beamforming coefficients and the set of power control coefficients.

[0017] In one optional implementation, the first constraint includes one or more of the following conditions: the transmission rate corresponding to the k-th user equipment is greater than or equal to a preset minimum transmission rate, where k = 1, 2, ..., K; the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment is 0 or 1; the number of user equipments connected to the m-th wireless access point is greater than or equal to 1, and the number of user equipments connected to the m-th wireless access point is less than or equal to the corresponding first maximum number of connections; the number of wireless access points connected to the k-th user equipment is greater than or equal to 1, and the number of wireless access points connected to the k-th user equipment is less than or equal to the corresponding second maximum number of connections; the power control coefficient corresponding to the k-th user equipment is greater than or equal to 0, and the power control coefficient of the k-th user equipment is less than or equal to a preset maximum power control coefficient; the total transmit power of the m-th wireless access point is less than or equal to a preset total transmit power; the subcarrier control coefficient of the k-th user equipment is 0 or 1; and the sum of the subcarrier control coefficients of the K user equipments is 1.

[0018] In this embodiment, by setting a first constraint, the set of beamforming coefficients and the set of power control coefficients that maximize the total transmission rate can be obtained more quickly and accurately.

[0019] In one alternative implementation, the control module is configured with an experience replayer that stores multiple training samples; the method further includes: obtaining network parameters of a first machine learning model; updating the network parameters based on the multiple training samples obtained from the experience replayer to obtain updated network parameters; and updating the first machine learning model configured for each wireless access point according to the updated network parameters.

[0020] The communication resource joint allocation method provided in this embodiment, after controlling each wireless access point to allocate communication resources to the corresponding user equipment based on the beamforming coefficient set and the power control coefficient set, updates the network parameters of the first machine learning model through training samples. This can further improve the accuracy of the first machine learning model configured for each wireless access point in optimizing the initial beamforming coefficients and initial power control coefficients allocated to the corresponding user equipment with the goal of maximizing the total transmission power.

[0021] In one optional implementation, the training samples stored in the experience replay unit come from each wireless access point; wherein each wireless access point is used to acquire the current state, which includes the current beamforming coefficient set, the current power control coefficient set, and the total transmission rate corresponding to the current beamforming coefficient set and the current power control coefficient set; each wireless access point is used to determine the current action corresponding to the current state based on an exploration strategy, the exploration strategy including: if the acquired random number is less than the current exploration rate, randomly selecting an action from the action set as the current action corresponding to the current state; if the random number is greater than or equal to the current exploration rate, determining the action corresponding to the current state as the current action. The optimal action is determined as the current action corresponding to the current state. The action set includes multiple preset beamforming coefficients and multiple preset power control coefficients. Each wireless access point is used to determine the reward value based on the current action. The reward value is the total transmission rate corresponding to the current state. Each wireless access point is used to determine whether the current state has reached the termination condition. The termination condition is that the number of cycles reaches a preset number of cycles, or the termination condition is that the reward value corresponding to the current state reaches the maximum reward value. Each wireless access point is used to determine the next state based on the current action. Each wireless access point is used to store the current state, current action, reward value, next state, and whether the termination condition has been reached as a training sample in the experience replay unit.

[0022] In this embodiment, updating the network parameters of the first machine learning model using training samples from each wireless access point can improve the generalization ability of the first machine learning model.

[0023] Secondly, the present invention provides a communication resource joint allocation apparatus, comprising: a first determining module, configured to determine a beamforming coefficient set and a power control coefficient set based on a first constraint condition and a first machine learning model configured for each of M wireless access points, wherein the beamforming coefficient set is the set of K beamforming coefficients that maximize the total transmission rate under the first constraint condition, and the power control coefficient set is the set of K power control coefficients that maximize the total transmission rate under the first constraint condition, wherein the K beamforming coefficients and K power control coefficients each correspond one-to-one with K user equipments, and the total transmission rate is the sum of the transmission rates of signals received by each of the K user equipments from the corresponding wireless access points, wherein M and K are both integers, and M≥2, K≥2; and an allocation module, configured to control the M wireless access points to allocate communication resources to the corresponding user equipments based on the beamforming coefficient set and the power control coefficient set, wherein the communication resources include transmit power and beamforming vector, the transmit power being determined by the corresponding power control coefficient, and the beamforming vector being determined by the corresponding beamforming coefficient.

[0024] In an optional embodiment, the apparatus further includes: a second determining module, configured to determine the effective power of the desired signal of the k-th user equipment based on the beamforming coefficient and power control coefficient corresponding to the k-th user equipment, and to determine the effective power of the interference signal between the k-th user equipment and other user equipment, wherein k = 1, 2, ..., K, and the other user equipment refers to user equipment other than the k-th user equipment among the K user equipment; a third determining module, configured to determine the signal-to-interference-plus-noise ratio (SIR) based on the effective power of the desired signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power; and a fourth determining module, configured to determine the transmission rate at which the k-th user equipment receives the signal from the corresponding wireless access point based on the subcarrier control coefficient, the signal bandwidth, and the SIR.

[0025] In an optional implementation, the third determining module includes: a first determining unit, configured to determine the signal-to-interference-plus-noise ratio (SIR) based on the effective power of the desired signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power, in the following manner:

[0026]

[0027] in, Indicates the signal-to-interference-plus-noise ratio, y v，k This represents the signal received by the k-th user on the v-th subcarrier, where v is an integer. This represents the effective power of the expected signal of the k-th user equipment. σ represents the effective power of the interference signal between the k-th user equipment and other user equipment. 2 α represents the power of white noise. m,k α represents the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment, where m = 1, 2, ..., M. m,k, p represents the wireless access point selection coefficient between the m-th wireless access point and other user equipment. m,k p represents the power control coefficient between the m-th wireless access point and the k-th user equipment. m,k, This represents the power control coefficient between the m-th wireless access point and other user equipment. This represents the channel vector between the m-th wireless access point and the k-th user equipment. This represents the channel vector between other wireless access points and the k-th user equipment, where other wireless access points are the M wireless access points excluding the m-th wireless access point. m,k w represents the beamforming coefficient between the m-th wireless access point and the k-th user equipment. m,k,This represents the beamforming coefficient between the m-th wireless access point and other user equipment.

[0028] In one optional implementation, the fourth determining module includes: a second determining unit, configured to determine the transmission rate based on subcarrier control coefficients, signal bandwidth, and signal-to-interference-plus-noise ratio (SINR) in the following manner:

[0029]

[0030] Where, r k Let v represent the transmission rate at which the k-th user equipment receives the signal from the corresponding wireless access point, v represent the v-th subcarrier, V represent the set of subcarriers, and v and V are both integers. ρ v,k Let W represent the subcarrier control coefficient of the k-th user equipment on the v-th subcarrier, and let W represent the signal bandwidth. Indicates the signal-to-interference-plus-noise ratio, y v，k This represents the signal received by the k-th user on the v-th subcarrier.

[0031] In one optional implementation, the first constraint includes one or more of the following conditions: the transmission rate corresponding to the k-th user equipment is greater than or equal to a preset minimum transmission rate, where k = 1, 2, ..., K; the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment is 0 or 1; the number of user equipments connected to the m-th wireless access point is greater than or equal to 1, and the number of user equipments connected to the m-th wireless access point is less than or equal to the corresponding first maximum number of connections; the number of wireless access points connected to the k-th user equipment is greater than or equal to 1, and the number of wireless access points connected to the k-th user equipment is less than or equal to the corresponding second maximum number of connections; the power control coefficient corresponding to the k-th user equipment is greater than or equal to 0, and the power control coefficient of the k-th user equipment is less than or equal to a preset maximum power control coefficient; the total transmit power of the m-th wireless access point is less than or equal to a preset total transmit power; the subcarrier control coefficient of the k-th user equipment is 0 or 1; and the sum of the subcarrier control coefficients of the K user equipments is 1.

[0032] In one optional implementation, the control module is configured with an experience replayer that stores multiple training samples; the device further includes: an acquisition module for acquiring network parameters of a first machine learning model; a first update module for updating the network parameters based on the multiple training samples acquired from the experience replayer to obtain updated network parameters; and a second update module for updating the first machine learning model configured for each wireless access point according to the updated network parameters.

[0033] In one optional implementation, the training samples stored in the experience replay unit come from each wireless access point; wherein each wireless access point is used to acquire the current state, which includes the current beamforming coefficient set, the current power control coefficient set, and the total transmission rate corresponding to the current beamforming coefficient set and the current power control coefficient set; each wireless access point is used to determine the current action corresponding to the current state based on an exploration strategy, the exploration strategy including: if the acquired random number is less than the current exploration rate, randomly selecting an action from the action set as the current action corresponding to the current state; if the random number is greater than or equal to the current exploration rate, determining the action corresponding to the current state as the current action. The optimal action is determined as the current action corresponding to the current state. The action set includes multiple preset beamforming coefficients and multiple preset power control coefficients. Each wireless access point is used to determine the reward value based on the current action. The reward value is the total transmission rate corresponding to the current state. Each wireless access point is used to determine whether the current state has reached the termination condition. The termination condition is that the number of cycles reaches a preset number of cycles, or the termination condition is that the reward value corresponding to the current state reaches the maximum reward value. Each wireless access point is used to determine the next state based on the current action. Each wireless access point is used to store the current state, current action, reward value, next state, and whether the termination condition has been reached as a training sample in the experience replay unit.

[0034] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0035] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of the present invention, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the structure of a cellular-free massive MIMO system according to an embodiment of the present invention;

[0038] Figure 2 This is a flowchart illustrating a method for joint allocation of communication resources according to an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram illustrating the determination of the first beamforming coefficient set and the first power control coefficient according to an embodiment of the present invention;

[0040] Figure 4 This is a flowchart illustrating another method for joint allocation of communication resources according to an embodiment of the present invention;

[0041] Figure 5 This is a flowchart illustrating another method for joint allocation of communication resources according to an embodiment of the present invention;

[0042] Figure 6 This is a schematic diagram of the process for determining training samples using a wireless access point according to an embodiment of the present invention;

[0043] Figure 7 This is a structural block diagram of a communication resource joint allocation device according to an embodiment of the present invention;

[0044] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] For 5G and subsequent mobile communication systems, cellular-free MIMO systems have garnered significant attention due to their superior spectral efficiency compared to cellular MIMO systems. Specifically, the higher spectral efficiency of cellular-free MIMO systems stems from the fact that a large number of access points (APs) distributed across various geographical locations utilize the same time-frequency resources in a non-orthogonal manner to provide services to a smaller number of user devices. Cellular-free MIMO systems can fully leverage diversity gain, reducing cross-cell interference inherent in cellular MIMO. Therefore, the higher the AP deployment density, the more significant the improvement in spectral efficiency for cellular-free MIMO.

[0047] However, in a cellular-free massive MIMO system, the AP serves a large number of user equipment. Due to the limited coherence time of the channel, the orthogonal pilot resources that the system can provide are limited. Consequently, if communication resources (such as transmit power and beamforming vectors) are not allocated reasonably, it will affect the transmission rate of the signal received by the user equipment (i.e., the receiving rate of the user equipment), and thus affect the signal transmission quality.

[0048] In view of this, the present invention provides a joint allocation method for communication resources, which uses a machine learning model to solve the problem of communication resource allocation in a non-cellular massive MIMO system, and achieves joint optimization of beamforming vector and transmit power allocation to maximize the total transmission rate.

[0049] For ease of understanding, this invention first combines... Figure 1 This paper describes a downlink non-cellular massive MIMO system operating in Time Division Duplex (TDD) mode.

[0050] Time division duplex (TDD) is a duplex mode in a communication system that separates the receive channel and the transmit channel (or uplink and downlink) based on time. In other words, in a TDD communication system, the receive channel and the transmit channel are on the same frequency channel, and the receive channel and the transmit channel are separated by guaranteed time (different time slots of the carrier).

[0051] Specifically, such as Figure 1 As shown, the cellular-free massive MIMO system 100 includes a control module 110, K user equipments 120 and M wireless access points 130. The user equipment 120 has a single antenna, and the wireless access points 130 include N antennas. K, M and N are all integers, and K≥2, M≥2, N≥2, M>>K, N>>K.

[0052] For example, the control module 110 can be a central processing unit (CPU). The M wireless access points 130 and K user equipment 120 can be randomly distributed within the service area covered by the system. Each of the M wireless access points 130 is connected to the control module 110 through a backhaul link to exchange information. Each wireless access point has a maximum number of users it can serve. Wireless access points serving the same user can cooperate with each other to provide services to the connected user under the same time and frequency resources. Each user also has a maximum number of wireless access points it can connect to.

[0053] The communication resource joint allocation method provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0054] According to an embodiment of the present invention, a method for joint allocation of communication resources is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in the control module of a non-cellular massive MIMO system. Furthermore, although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0055] This embodiment provides a method for joint allocation of communication resources, which can be used in the control module 110 of the aforementioned non-cellular massive MIMO system. The control module can be a CPU, computer, or server, etc. Figure 2 This is a flowchart illustrating a communication resource joint allocation method according to an embodiment of the present invention, as shown below. Figure 2 As shown, the method includes the following steps:

[0056] Step S201: Based on the first constraint, determine the beamforming coefficient set and the power control coefficient set by using the first machine learning model configured for each of the M wireless access points.

[0057] The beamforming coefficient set is the set of K beamforming coefficients that maximize the total transmission rate under the first constraint, and the power control coefficient set is the set of K power control coefficients that maximize the total transmission rate under the first constraint. Both the K beamforming coefficients and the K power control coefficients correspond one-to-one with the K user equipments. The total transmission rate is the sum of the transmission rates of the signals received by each of the K user equipments from the corresponding wireless access point.

[0058] Specifically, research on resource allocation in cellular-free massive MIMO systems mainly focuses on univariate and bivariate resource allocation. Joint optimization of resources, comprehensively considering beamforming vectors and transmit power, can improve data transmission robustness and user equipment reception rate (the rate at which the signal received by the user equipment is transmitted). The inner product of the beamforming vectors characterizes the beamforming gain, which is the beamforming coefficient; that is, the beamforming vector is determined by the beamforming coefficients. Transmit power can be determined by power control coefficients; for example, the transmit power can be the product of the power control coefficients and a preset transmit power.

[0059] It should be understood that beamforming improves channel utilization by controlling the phase and amplitude of the signal transmitted through the transmitting antenna, focusing the signal into a narrow beam in space. After beamforming, the signal is transmitted only in the direction of the receiving antenna, thus reducing channel interference and energy loss.

[0060] For example, the first machine learning model is a pre-trained machine learning model. This invention does not limit the type of machine learning model, as long as it can determine the set of beamforming coefficients and the set of power control coefficients that maximize the total transmission rate under the first constraint. For example, the machine learning model can be a reinforcement learning model, a neural network model, or other learning models.

[0061] Specifically, machine learning models have shown potential in maximizing the total transmission rate under constraints (i.e., the first constraint of this invention), especially reinforcement learning models, which are machine learning models that optimize decisions by interacting with the environment and can be applied to the communication resource allocation problem in non-cellular massive MIMO systems.

[0062] In related research, in millimeter-wave networks, the power control problem was solved by using a deep reinforcement learning model, which improved the performance of non-line-of-sight transmission. In dynamically correlated multichannel access environments, a strategy to maximize successful transmission can be obtained using a deep reinforcement learning model. In cognitive radio with low signal-to-interference-plus-noise ratio, the automatic identification of modulation can be enhanced by using a deep convolutional neural network.

[0063] Specifically, there is a correspondence between the power control coefficient, beamforming coefficient, and transmission rate. After obtaining the initial beamforming coefficient and initial power control coefficient allocated by the wireless access point to each user equipment, the control module can determine the transmission rate corresponding to the initial beamforming coefficient and initial power control coefficient of each user equipment based on the correspondence. For example, the correspondence can be pre-configured in the control module.

[0064] like Figure 3 As shown, the control module can send (input) initial beamforming coefficients, initial power control coefficients, and initial transmission rates allocated to multiple first user equipments to the agent (i.e., the first machine learning model) of the first wireless access point. After receiving these initial beamforming coefficients, power control coefficients, and transmission rates, the agent optimizes them to maximize the sum of the transmission rates. Then, the agent outputs multiple optimized initial beamforming coefficients (i.e., the first beamforming coefficient set) and multiple optimized initial power control coefficients (i.e., the first power control coefficient set). Here, the first wireless access point is one of M wireless access points, and the first user equipment is one of K user equipments connected to the first wireless access point. The control module can repeat the above process in each wireless access point to obtain the optimized initial beamforming coefficients and optimized initial power control coefficients for each user equipment in the cellular-free massive MIMO system, i.e., to obtain the beamforming coefficient set and the power control coefficient set.

[0065] For example, Figure 3 The codebook shown can be a set of multiple preset beamforming coefficients. The agent can determine the codebook by cyclically increasing or decreasing the preset beamforming coefficients. The continuous power discretization set can be a set of multiple preset power control coefficients. The agent can optimize multiple initial beamforming coefficients and multiple initial power control coefficients based on the codebook and the continuous power discretization set with the goal of maximizing the total transmission rate.

[0066] Step S202: Based on the beamforming coefficient set and the power control coefficient set, control the M wireless access points to allocate communication resources to the corresponding user equipment.

[0067] The communication resources include transmit power and beamforming vector. The transmit power is determined by the corresponding power control coefficient, and the beamforming vector is determined by the corresponding beamforming coefficient.

[0068] Specifically, after the control module determines the beamforming coefficient set and the power control coefficient set, it can send the beamforming coefficient set and the power control coefficient set to each wireless access point. After receiving the beamforming coefficient set and the power control coefficient set, the wireless access point adjusts the transmit power and beamforming vector when sending signals to the user equipment based on the corresponding beamforming coefficient and power control coefficient.

[0069] For example, the control module can use a series of bits as an action register to perform joint optimization of transmit power and beamforming vector.

[0070] The communication resource joint allocation method provided in this embodiment, based on a first constraint, optimizes the beamforming coefficients and power control coefficients allocated to each user equipment using a first machine learning model configured in each of the M wireless access points. This results in a set of beamforming coefficients and a set of power control coefficients that maximize the total transmission rate. Then, based on these sets, each wireless access point allocates communication resources to its corresponding user equipment. This method, by jointly optimizing two coupled resource factors—beamforming vector and transmit power—using a machine learning model, maximizes the total transmission rate, thereby improving data transmission quality and system capacity. Simultaneously, joint optimization using a machine learning model reduces the complexity of resource optimization in cellular-free massive MIMO systems, improves optimization efficiency, and reduces the utilization of computing resources and time costs.

[0071] In one alternative implementation, the intelligent agent can determine multiple preset power control coefficients based on the maximum transmit power of the corresponding wireless access point, and can determine the transmit power allocated to the user equipment based on the power control coefficients and the maximum transmit power.

[0072] Specifically, the maximum transmit power can be reduced according to a preset gradient to obtain multiple preset power control coefficients. Then, the maximum transmit power is normalized, and the product of the power control coefficients and the normalized maximum transmit power is determined as the transmit power allocated to the user. For example, if the maximum transmit power is 10W and the preset gradient is 1W, then {0, 1, 2, ..., 9, 10} can be used as multiple preset power control coefficients. If the normalized value of the maximum transmit power is 0.6 and the power control coefficient is 8, then the transmit power can be 4.8W. In this embodiment, determining the transmit power in the above manner can avoid the transmit power allocated to the user exceeding the carrying capacity of the wireless access point.

[0073] This embodiment provides a method for joint allocation of communication resources, which can be used in the control module 110 of the aforementioned non-cellular massive MIMO system. Figure 4 This is a flowchart of another communication resource joint allocation method according to an embodiment of the present invention, such as... Figure 4 As shown, the method includes the following steps:

[0074] Step S401: Based on the beamforming coefficient and power control coefficient corresponding to the kth user equipment, determine the effective power of the desired signal of the kth user equipment, and determine the effective power of the interference signal between the kth user equipment and other user equipment.

[0075] Where k = 1, 2, ..., K, and other user equipment refers to user equipment other than the kth user equipment among the K user equipment.

[0076] Specifically, based on the beamforming coefficient and power control coefficient corresponding to the kth user equipment, the effective power of the desired signal of the kth user equipment can be determined by the following formula (1):

[0077]

[0078] Where A represents the effective power of the desired signal of the k-th user equipment, α m,k p represents the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment, where m = 1, 2, ..., M. m,k This represents the power control coefficient between the m-th wireless access point and the k-th user equipment. w represents the channel vector between the m-th wireless access point and the k-th user equipment. m,kx represents the beamforming coefficient between the m-th wireless access point and the k-th user equipment. k This represents the data signal sent to the k-th user device.

[0079] Since the non-cellular massive MIMO system operates in time-frequency duplex mode, it can utilize the channel hardening conditions of massive MIMO, eliminating the need for downlink channel estimation. Therefore, in the non-cellular massive MIMO system of this invention, the channel vector between the m-th radio access point and the k-th user equipment... It can be determined by the following formula (2):

[0080]

[0081] Where, β m,k denoted by h, the large-scale fading coefficient between the m-th wireless access point and the k-th user equipment. The large-scale fading coefficient reflects the effects of path loss and shadow fading. m,k Let represent an N×1 vector that models small-scale fading between the m-th wireless access point and the k-th user equipment. Its elements are independent and identically distributed random variables that follow a cyclic symmetric complex Gaussian distribution.

[0082] Specifically, in the downlink of a cellular-free massive MIMO system, there is no pilot training and the system relies on channel reciprocity. During the downlink data transmission phase, each radio access point can use the channel estimation information obtained from the uplink pilot training in the previous phase to precode the signal sent to the user equipment. The signal sent by the m-th radio access point to all connected user equipment can be determined by the following formula (3):

[0083]

[0084] Where, q k This represents the signal sent by the m-th wireless access point to all connected user devices.

[0085] Specifically, based on the beamforming coefficient and power control coefficient corresponding to the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment can be determined by the following formula (4):

[0086]

[0087] Where B represents the effective power of the interference signal between the k-th user equipment and other user equipment, k represents other user equipment, and α m,k, p represents the wireless access point selection coefficient between the m-th wireless access point and other user equipment. m,k, This represents the power control coefficient between the m-th wireless access point and other user equipment. This represents the channel vector between other wireless access points and the k-th user equipment, where other wireless access points are the M wireless access points excluding the m-th wireless access point. m,k, x represents the beamforming coefficient between the m-th wireless access point and other user equipment. k, This indicates a data signal sent to other user devices.

[0088] Step S402: Determine the signal-to-interference-plus-noise ratio (SIR) based on the effective power of the expected signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power.

[0089] For example, the signal-to-interference-plus-noise ratio (SIR / NOT ratio) can be determined using the following formula (5) based on the effective power of the expected signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power:

[0090]

[0091] in, Indicates the signal-to-interference-plus-noise ratio, y v，k This represents the signal received by the k-th user on the v-th subcarrier, where v is an integer. This represents the effective power of the expected signal of the k-th user equipment. σ represents the effective power of the interference signal between the k-th user equipment and other user equipment. 2 This represents the power of white noise.

[0092] Specifically, the signal y received by the k-th user on the v-th subcarrier can be determined by the following formula (6). v，k :

[0093]

[0094] Among them, z k This represents the quantization noise of the k-th user device.

[0095] Step S403: Determine the transmission rate of the signal received by the k-th user equipment from the corresponding wireless access point based on the subcarrier control coefficient, signal bandwidth, and signal-to-interference-plus-noise ratio.

[0096] For example, based on the subcarrier control coefficients, signal bandwidth, and signal-to-interference-plus-noise ratio, the transmission rate of the signal received by the k-th user equipment from the corresponding wireless access point can be determined by the following formula (7):

[0097]

[0098] Where, r kLet v represent the transmission rate at which the k-th user equipment receives the signal from the corresponding wireless access point, v represent the v-th subcarrier, V represent the set of subcarriers, and v and V are both integers. ρ v,k Let W represent the subcarrier control coefficient of the k-th user equipment on the v-th subcarrier, and let W represent the signal bandwidth.

[0099] Specifically, the relationship between the power control coefficient, beamforming coefficient, and transmission power can be expressed by formula (7). The control module can determine the transmission rate of the signal received by each user equipment from the corresponding wireless access point through formula (7), and thus obtain the total transmission power ∑. k∈K r k .

[0100] Step S404: Based on the first constraint, determine the beamforming coefficient set and the power control coefficient set using the first machine learning model configured for each of the M wireless access points.

[0101] For example, the first constraint (St) can be represented by the following formula (8):

[0102]

[0103] Where, r min Indicates the preset minimum transmission rate, r k ≥r min α indicates that the transmission rate corresponding to the k-th user equipment is greater than or equal to the preset minimum transmission rate. m,k ={0,1} indicates that the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment is 0 or 1, U m This represents the first maximum number of connections, 1 ≤ ∑ k α m,k ≤U m This indicates that the number of user devices connected to the m-th wireless access point is greater than or equal to 1, and the number of user devices connected to the m-th wireless access point is less than or equal to the corresponding first maximum connection number. Universal quantifiers represent the entirety of a specified range or the whole of that range. (C) k This represents the second maximum number of connections, 1 ≤ ∑ m α m,k ≤C k This indicates that the number of wireless access points connected to the k-th user equipment is greater than or equal to 1, and the number of wireless access points connected to the k-th user equipment is less than or equal to the corresponding second maximum number of connections, p. max This represents the preset maximum power control coefficient, 0 ≤ p m,k ≤p maxThis indicates that the power control coefficient corresponding to the k-th user equipment is greater than or equal to 0, and the power control coefficient of the k-th user equipment is less than or equal to the preset maximum power control coefficient, p total Represents the preset total transmit power, ∑ v∈V ∑ k ρ v，k p v,m,k ≤p total This indicates that the total transmit power of the m-th wireless access point is less than or equal to the preset total transmit power, ρ v，k ∈{0,1} indicates that the subcarrier control coefficient of the k-th user equipment is 0 or 1, ∑ k ρ v，k =1 indicates that the sum of the subcarrier control coefficients of the K user equipments is 1.

[0104] That is, the first constraint includes one or more of the following conditions: the transmission rate corresponding to the k-th user equipment is greater than or equal to the preset minimum transmission rate; the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment is 0 or 1; the number of user equipment connected to the m-th wireless access point is greater than or equal to 1, and the number of user equipment connected to the m-th wireless access point is less than or equal to the corresponding first maximum number of connections; the number of wireless access points connected to the k-th user equipment is greater than or equal to 1, and the number of wireless access points connected to the k-th user equipment is less than or equal to the corresponding second maximum number of connections; the power control coefficient corresponding to the k-th user equipment is greater than or equal to 0, and the power control coefficient of the k-th user equipment is less than or equal to the preset maximum power control coefficient; the total transmit power of the m-th wireless access point is less than or equal to the preset total transmit power; the subcarrier control coefficient of the k-th user equipment is 0 or 1; and the sum of the subcarrier control coefficients of the K user equipment is 1.

[0105] Specifically, under the conditions of the above formula (8), the first machine learning model configured for each wireless access point is max (α,ρ,p,w) ∑ k∈K r k To achieve this, the initial beamforming coefficients and initial power control coefficients allocated to the corresponding user equipment are optimized, thereby enabling the control module to determine the set of beamforming coefficients and the set of power control coefficients.

[0106] In this embodiment, the correspondence between beamforming coefficients, power control coefficients, and transmission rates determined by the formula takes into account the mutual influence between user equipment and wireless access points. This can improve the accuracy of the first machine learning model configured for each wireless access point in optimizing the initial beamforming coefficients and initial power control coefficients allocated to the corresponding user equipment with the goal of maximizing the total transmission power. In turn, it can improve the accuracy of determining the set of beamforming coefficients and the set of power control coefficients.

[0107] Step S405: Based on the beamforming coefficient set and the power control coefficient set, control the M wireless access points to allocate communication resources to the corresponding user equipment.

[0108] Please see details Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0109] The communication resource joint allocation method provided in this embodiment determines the effective power of the desired signal of the k-th user equipment (UE) based on the beamforming coefficient and power control coefficient corresponding to the k-th UE, and also determines the effective power of the interference signal between the k-th UE and other UEs. Based on the effective power of the desired signal of the k-th UE, the effective power of the interference signal between the k-th UE and other UEs, and the white noise power, the signal-to-interference-plus-noise ratio (SINR) is determined. Then, based on the subcarrier control coefficient, signal bandwidth, and SINR, the transmission rate of the signal received by the k-th UE from the corresponding wireless access point is determined. This allows for the correspondence between the beamforming coefficient, power control coefficient, and transmission rate, facilitating the accurate determination of the total transmission power corresponding to different sets of beamforming coefficients and power control coefficients by the first machine learning model configured for each wireless access point.

[0110] This embodiment provides a method for joint allocation of communication resources, which can be used in the control module 110 of the aforementioned non-cellular massive MIMO system. Figure 5 This is a flowchart illustrating another communication resource joint allocation method according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps:

[0111] Step S501: Based on the first constraint, determine the beamforming coefficient set and the power control coefficient set by using the first machine learning model configured for each of the M wireless access points.

[0112] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0113] Step S502: Based on the beamforming coefficient set and the power control coefficient set, control the M wireless access points to allocate communication resources to the corresponding user equipment.

[0114] Please see details Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0115] Step S503: Obtain the network parameters of the first machine learning model.

[0116] Specifically, the first machine learning model configured in the M wireless access points is identical, and the network parameters can be the learning rate, the weight parameters of the neural network, or a discount factor, etc. The discount factor is used to balance the reward value (i.e., the total transmission rate mentioned above).

[0117] Step S504: Based on multiple training samples obtained from the experience replayer, update the network parameters to obtain the updated network parameters.

[0118] Specifically, the control module is equipped with an experience replay unit, which stores multiple training samples. These training samples can be configured by staff according to their needs, or they can come from each of the M wireless access points.

[0119] The process of acquiring training samples for each wireless access point is explained below.

[0120] Specifically, the process of each wireless access point acquiring training samples includes steps a1 to a6:

[0121] Step a1: Obtain the current state.

[0122] The current state includes the current set of beamforming coefficients, the current set of power control coefficients, and the total transmission rate corresponding to the current set of beamforming coefficients and the current set of power control coefficients. After determining the current state, the loop count is incremented by 1 and recorded.

[0123] For example, a counter can be used to record the number of loops.

[0124] Step a2: Based on the exploration strategy, determine the current action corresponding to the current state.

[0125] Specifically, the wireless access point is configured with an exploration rate parameter ξ, which controls the randomness of action selection. The exploration strategy includes: when the acquired random number r is less than the exploration rate ξ, an action randomly selected from the action set is determined as the current action corresponding to the current state; when the random number r is greater than or equal to the exploration rate ξ, the optimal action corresponding to the current state is determined as the current action corresponding to the current state. The action set includes multiple preset beamforming coefficients and multiple preset power control coefficients.

[0126] For example, it can be based on the policy network Q π Given the action value function and the current state, determine the optimal action corresponding to the current state. Specifically, the current state can be input into the policy network, and then the action output by the policy network can be determined as the optimal action. The optimal action is the action in the action set that maximizes the reward value (total transmission rate).

[0127] Step a3: Determine the reward value based on the current action.

[0128] The reward value is the total transmission rate corresponding to the current state.

[0129] Specifically, after determining the current action, the transmission rate of the signal received by each user equipment from the corresponding wireless access point can be determined according to the above formula (7) and the current action (current beamforming coefficient and power control coefficient), thereby obtaining the total transmission rate (i.e. the reward value corresponding to the current action).

[0130] For example, if the obtained reward value γ[t] is higher than the first preset target threshold (γtarget), the current state, current action, and reward value are gradually increased to promote the learning of the machine learning model. If the obtained reward value is lower than the second preset target threshold (γtarget), the reward value is lowered. min If the condition is met, the reward value will be forcibly set to a smaller preset threshold, and the current training round will end.

[0131] Step a4: Determine whether the current state meets the termination condition.

[0132] The termination condition is either that the number of loops reaches a preset number of loops, or that the reward value corresponding to the current state reaches the maximum reward value. That is, if the number of loops is less than the preset number of loops, or if the reward value corresponding to the current state is less than the maximum reward value, step a5 is executed; if the number of loops is greater than the preset number of loops, or if the reward value corresponding to the current state is the maximum reward value, step a6 is executed and the process ends.

[0133] Specifically, after exploring all explorable actions (all actions in the action set) in the current state, the reward value will no longer increase, that is, the reward value will reach its maximum.

[0134] Step a5: Determine the next state based on the current action.

[0135] Specifically, the next state is the next set of beamforming coefficients, the next set of power control coefficients, and the total transmission rate corresponding to the next set of beamforming coefficients and the next set of power control coefficients.

[0136] For example, after determining the next state, steps a1 to a5 are re-executed as the new current state. Specifically, when the current state is obtained for the first time, the current state is a preset state.

[0137] Step a6: Store the current state, current action, reward value, next state, and whether the termination condition has been met as a training sample in the experience replay unit.

[0138] Specifically, the process by which a wireless access point acquires training samples can be as follows: Figure 6 As shown, the wireless access point determines its current state by receiving the initial beamforming coefficient and initial power control coefficient from the control module, i.e., executing step S601. Then, it increments the loop count by 1 and determines whether the loop count is greater than a preset loop count, i.e., executing step S602. If the loop count is greater than the preset loop count, the process ends. If the loop count is less than or equal to the preset loop count, it determines the current action corresponding to the current state based on the exploration strategy, determines the reward value based on the current action, and determines whether the reward value has reached the maximum reward value, i.e., executing steps S603 (step a2 above), S604 (step a3 above), and S605 (step a4 above). If the reward value reaches the maximum reward value, the process ends. If the reward value has not reached the maximum reward value, it determines the next state based on the current action, i.e., executing step S606, and then uses the next state as the new current state to re-execute steps S601 to S605.

[0139] Specifically, such as Figure 3 As shown, the wireless access point stores the training samples in the empirical replay unit (i.e., Figure 3 The control module obtains training samples from the experience replay unit to train and update the network parameters of the first machine learning model (using the dataset in the dataset).

[0140] For example, after acquiring the network parameters and training samples, the control module determines the target reward value and the predicted reward value (the reward value in the training samples). Then, it updates the network parameters θ using an optimization algorithm, such as Stochastic Gradient Descent (SGD), with the objective of minimizing the Mean Square Error (MSE) of the loss function. The loss function is determined based on the target reward value and the predicted reward value.

[0141] Alternatively, the target reward value can be determined using formula (9):

[0142] y j = γ j +β maxQ π (s j+1 ,a,θ t (9)

[0143] Among them, y j γ represents the target reward value. j s represents the current reward value, β represents the discount factor, and s represents the current reward value. j+1 Let 'a' represent the next state, 'a' represent the next action, and 'θ' represent the next action. t Q represents the current network parameters. π (s j+1 ,a,θt ) indicates that at θ t The state s of action a under parameter 2 j+1 The maximum reward value.

[0144] Step S505: Update the first machine learning model configured for each wireless access point based on the updated network parameters.

[0145] Specifically, such as Figure 3 As shown, after determining the updated network parameters, the control module sends the network parameters to each wireless access point. After receiving the updated network parameters, each wireless access point can update the network parameters of the first machine learning model according to the updated network parameters.

[0146] The communication resource joint allocation method provided in this embodiment, after controlling each wireless access point to allocate communication resources to the corresponding user equipment based on the beamforming coefficient set and the power control coefficient set, updates the network parameters of the first machine learning model through training samples. This can further improve the accuracy of the first machine learning model configured for each wireless access point in optimizing the initial beamforming coefficients and initial power control coefficients allocated to the corresponding user equipment with the goal of maximizing the total transmission power.

[0147] This embodiment also provides a communication resource joint allocation device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0148] This embodiment provides a communication resource joint allocation device (i.e., the aforementioned control module), such as Figure 7 As shown, it includes:

[0149] The first determining module 701 is used to determine the beamforming coefficient set and the power control coefficient set based on the first constraint condition and through the first machine learning model configured for each of the M wireless access points. The beamforming coefficient set is the set of K beamforming coefficients that maximize the total transmission rate under the first constraint condition, and the power control coefficient set is the set of K power control coefficients that maximize the total transmission rate under the first constraint condition. The K beamforming coefficients and K power control coefficients are each one-to-one with the K user equipments. The total transmission rate is the sum of the transmission rates of the signals received by each of the K user equipments from the corresponding wireless access points. M and K are both integers, and M≥2, K≥2.

[0150] Allocation module 702 is used to control M wireless access points to allocate communication resources to corresponding user equipment based on a beamforming coefficient set and a power control coefficient set. The communication resources include transmit power and beamforming vectors. The transmit power is determined by the corresponding power control coefficients, and the beamforming vectors are determined by the corresponding beamforming coefficients.

[0151] In some alternative embodiments, the apparatus further includes:

[0152] The second determining module is used to determine the effective power of the desired signal of the kth user equipment and the effective power of the interference signal between the kth user equipment and other user equipment based on the beamforming coefficient and power control coefficient corresponding to the kth user equipment, where k = 1, 2, ..., K, and other user equipment refers to user equipment other than the kth user equipment among the K user equipment.

[0153] The third determining module is used to determine the signal-to-interference-plus-noise ratio (SINR) based on the effective power of the expected signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power.

[0154] The fourth determining module is used to determine the transmission rate of the signal received by the k-th user equipment from the corresponding wireless access point based on the subcarrier control coefficient, signal bandwidth, and signal-to-interference-plus-noise ratio.

[0155] In some alternative implementations, the third determining module includes:

[0156] The first determining unit is configured to determine the signal-to-interference-plus-noise ratio (SIR) based on the effective power of the desired signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power, in the following manner:

[0157]

[0158] in, Indicates the signal-to-interference-plus-noise ratio, y v，k This represents the signal received by the k-th user on the v-th subcarrier, where v is an integer. This represents the effective power of the expected signal of the k-th user equipment. σ represents the effective power of the interference signal between the k-th user equipment and other user equipment. 2 α represents the power of white noise. m,k α represents the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment, where m = 1, 2, ..., M. m,k, p represents the wireless access point selection coefficient between the m-th wireless access point and other user equipment. m,kp represents the power control coefficient between the m-th wireless access point and the k-th user equipment. m,k, This represents the power control coefficient between the m-th wireless access point and other user equipment. This represents the channel vector between the m-th wireless access point and the k-th user equipment. This represents the channel vector between other wireless access points and the k-th user equipment, where other wireless access points are the M wireless access points excluding the m-th wireless access point. m,k w represents the beamforming coefficient between the m-th wireless access point and the k-th user equipment. m,k, This represents the beamforming coefficient between the m-th wireless access point and other user equipment.

[0159] In some alternative implementations, the fourth determining module includes:

[0160] The second determining unit is used to determine the transmission rate based on the subcarrier control coefficients, signal bandwidth, and signal-to-interference-plus-noise ratio in the following manner:

[0161]

[0162] Where, r k Let v represent the transmission rate at which the k-th user equipment receives the signal from the corresponding wireless access point, v represent the v-th subcarrier, V represent the set of subcarriers, and v and V are both integers. ρ v,k Let W represent the subcarrier control coefficient of the k-th user equipment on the v-th subcarrier, and let W represent the signal bandwidth. Indicates the signal-to-interference-plus-noise ratio, y v，k This represents the signal received by the k-th user on the v-th subcarrier.

[0163] In some optional implementations, the first constraint includes one or more of the following conditions: the transmission rate corresponding to the k-th user equipment is greater than or equal to a preset minimum transmission rate, where k = 1, 2, ..., K; the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment is 0 or 1; the number of user equipments connected to the m-th wireless access point is greater than or equal to 1, and the number of user equipments connected to the m-th wireless access point is less than or equal to the corresponding first maximum number of connections; the number of wireless access points connected to the k-th user equipment is greater than or equal to 1, and the number of wireless access points connected to the k-th user equipment is less than or equal to the corresponding second maximum number of connections; the power control coefficient corresponding to the k-th user equipment is greater than or equal to 0, and the power control coefficient of the k-th user equipment is less than or equal to a preset maximum power control coefficient; the total transmit power of the m-th wireless access point is less than or equal to a preset total transmit power; the subcarrier control coefficient of the k-th user equipment is 0 or 1; and the sum of the subcarrier control coefficients of the K user equipments is 1.

[0164] In some optional implementations, the control module is configured with an experience replayer that stores multiple training samples; the device also includes:

[0165] The acquisition module is used to acquire the network parameters of the first machine learning model.

[0166] The first update module is used to update the network parameters based on multiple training samples obtained from the experience replay unit, and obtain the updated network parameters.

[0167] The second update module is used to update the first machine learning model configured for each wireless access point based on the updated network parameters.

[0168] In some optional implementations, the training samples stored in the experience replay unit come from each wireless access point; wherein each wireless access point is used to acquire the current state, which includes the current beamforming coefficient set, the current power control coefficient set, and the total transmission rate corresponding to the current beamforming coefficient set and the current power control coefficient set; each wireless access point is used to determine the current action corresponding to the current state based on an exploration strategy, the exploration strategy including: if the acquired random number is less than the current exploration rate, randomly selecting an action from the action set as the current action corresponding to the current state; if the random number is greater than or equal to the current exploration rate, determining the action corresponding to the current state as the current action. The optimal action is determined as the current action corresponding to the current state. The action set includes multiple preset beamforming coefficients and multiple preset power control coefficients. Each wireless access point is used to determine the reward value based on the current action. The reward value is the total transmission rate corresponding to the current state. Each wireless access point is used to determine whether the current state has reached the termination condition. The termination condition is that the number of cycles reaches a preset number of cycles, or the termination condition is that the reward value corresponding to the current state reaches the maximum reward value. Each wireless access point is used to determine the next state based on the current action. Each wireless access point is used to store the current state, current action, reward value, next state, and whether the termination condition has been reached as a training sample in the experience replay unit.

[0169] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0170] The communication resource allocation device in this embodiment is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0171] This invention also provides a computer device having the above-described features. Figure 7 The communication resource joint allocation device shown.

[0172] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8 As shown, the computer device includes one or more processors 810, memory 820, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take the 810 processor as an example.

[0173] The processor 810 may be a central processing unit, a network processor, or a combination thereof. The processor 810 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPA), or any combination thereof.

[0174] The memory 820 stores instructions executable by at least one processor 810 to cause the at least one processor 810 to perform the method shown in the above embodiments.

[0175] The memory 820 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 820 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 820 may optionally include memory remotely located relative to the processor 810, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0176] The memory 820 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 820 may also include a combination of the above types of memory.

[0177] The computer device also includes a communication interface 830 for communicating with other devices or communication networks.

[0178] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0179] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for joint allocation of communication resources, characterized in that, A control module for a non-cellular massive MIMO system, the non-cellular massive MIMO system including the control module, M wireless access points and K user equipment, wherein the control module is connected to the M wireless access points, M and K are both integers, and M≥2, K≥2, the method includes: Based on the first constraint, a beamforming coefficient set and a power control coefficient set are determined using the first machine learning model configured for each of the M wireless access points. The beamforming coefficient set is the set of K beamforming coefficients that maximize the total transmission rate under the first constraint. The power control coefficient set is the set of K power control coefficients that maximize the total transmission rate under the first constraint. Both the K beamforming coefficients and the K power control coefficients correspond one-to-one with the K user equipments. The total transmission rate is the sum of the transmission rates of the signals received by each of the K user equipments from their respective wireless access points. Based on the set of beamforming coefficients and the set of power control coefficients, the M wireless access points are controlled to allocate communication resources to their respective user equipment. The communication resources include transmit power and beamforming vector. The transmit power is determined by the corresponding power control coefficients, and the beamforming vector is determined by the corresponding beamforming coefficients.

2. The method according to claim 1, characterized in that, The method further includes: Based on the beamforming coefficient and power control coefficient corresponding to the kth user equipment, the effective power of the desired signal of the kth user equipment is determined, as well as the effective power of the interference signal between the kth user equipment and other user equipment is determined, where k = 1, 2, ..., K, and the other user equipment refers to the user equipment other than the kth user equipment among the K user equipment. The signal-to-interference-plus-noise ratio (SINR) is determined based on the effective power of the desired signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power. The transmission rate at which the k-th user equipment receives signals from the corresponding wireless access point is determined based on the subcarrier control coefficient, signal bandwidth, and the signal-to-interference-plus-noise ratio.

3. The method according to claim 2, characterized in that, The step of determining the signal-to-interference-plus-noise ratio (SIR) based on the effective power of the expected signal of the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power includes: The signal-to-interference-plus-noise ratio (SIR) is determined based on the effective power of the desired signal from the k-th user equipment, the effective power of the interference signal between the k-th user equipment and other user equipment, and the white noise power, as follows: in, The signal-to-interference-plus-noise ratio (SIR) is represented by y. v，k This represents the signal received by the k-th user on the v-th subcarrier, where v is an integer. This represents the effective power of the expected signal of the k-th user equipment. σ represents the effective power of the interference signal between the k-th user equipment and other user equipment. 2 α represents the power of the white noise. m,k α represents the wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment, where m = 1, 2, ..., M. m,k, p represents the wireless access point selection coefficient between the m-th wireless access point and the other user equipment. m,k p represents the power control coefficient between the m-th wireless access point and the k-th user equipment. m,k, This represents the power control coefficient between the m-th wireless access point and the other user equipment. This represents the channel vector between the m-th wireless access point and the k-th user equipment. This represents the channel vector between other wireless access points and the k-th user equipment, where the other wireless access points are all wireless access points among the M wireless access points except for the m-th wireless access point. m,k w represents the beamforming coefficient between the m-th wireless access point and the k-th user equipment. m,k, This represents the beamforming coefficient between the m-th wireless access point and the other user equipment.

4. The method according to claim 2, characterized in that, Determining the transmission rate of the signal received by the k-th user equipment from the corresponding wireless access point based on the subcarrier control coefficient, signal bandwidth, and signal-to-interference-plus-noise ratio includes: The transmission rate is determined based on the subcarrier control coefficients, the signal bandwidth, and the signal-to-interference-plus-noise ratio (SINR) in the following manner: Where, r k This represents the transmission rate at which the k-th user equipment receives signals from the corresponding wireless access point, where v represents the v-th subcarrier, V represents the set of subcarriers, and both v and V are integers. ρ v,k Let W represent the subcarrier control coefficient of the k-th user equipment on the v-th subcarrier, and let W represent the signal bandwidth. The signal-to-interference-plus-noise ratio (SIR) is represented by y. v，k This represents the signal received by the k-th user on the v-th subcarrier.

5. The method according to any one of claims 1 to 4, characterized in that, The first constraint includes one or more of the following conditions: The transmission rate corresponding to the kth user equipment is greater than or equal to the preset minimum transmission rate, where k = 1, 2, ..., K; The wireless access point selection coefficient between the m-th wireless access point and the k-th user equipment is 0 or 1, where m = 1, 2, ..., M; The number of user devices connected to the m-th wireless access point is greater than or equal to 1, and the number of user devices connected to the m-th wireless access point is less than or equal to the corresponding first maximum number of connections. The number of wireless access points connected to the kth user equipment is greater than or equal to 1, and the number of wireless access points connected to the kth user equipment is less than or equal to the corresponding second maximum number of connections. The power control coefficient corresponding to the kth user equipment is greater than or equal to 0, and the power control coefficient of the kth user equipment is less than or equal to the preset maximum power control coefficient; The total transmission power of the m-th wireless access point is less than or equal to the preset total transmission power; The subcarrier control coefficient of the kth user equipment is 0 or 1; The sum of the subcarrier control coefficients of the K user equipments is 1.

6. The method according to any one of claims 1 to 4, characterized in that, The control module is configured with an experience replay unit, which stores multiple training samples; the method further includes: Obtain the network parameters of the first machine learning model; Based on multiple training samples obtained from the experience replayer, the network parameters are updated to obtain the updated network parameters; The first machine learning model configured for each wireless access point is updated based on the updated network parameters.

7. The method according to claim 6, characterized in that, The training samples stored in the experience replay unit come from each of the wireless access points; Each wireless access point is used to obtain the current state, which includes the current beamforming coefficient set, the current power control coefficient set, and the total transmission rate corresponding to the current beamforming coefficient set and the current power control coefficient set. Each wireless access point is used to determine the current action corresponding to the current state based on an exploration strategy. The exploration strategy includes: when the obtained random number is less than the current exploration rate, determining the action randomly selected from the action set as the current action corresponding to the current state; when the random number is greater than or equal to the current exploration rate, determining the optimal action corresponding to the current state as the current action corresponding to the current state. The action set includes a preset set of multiple beamforming coefficients and a preset set of multiple power control coefficients. Each wireless access point is used to determine a reward value based on the current action, where the reward value is the total transmission rate corresponding to the current state. Each wireless access point is used to determine whether the current state has reached a termination condition. The termination condition is that the number of cycles reaches a preset number of cycles, or the termination condition is that the reward value corresponding to the current state reaches the maximum reward value. Each wireless access point is used to determine the next state based on the current action; Each wireless access point is used to store the current state, current action, reward value, next state, and whether the termination condition has been met as a training sample in the experience replay unit.

8. A communication resource joint allocation device, characterized in that, The device includes: The first determining module is configured to determine a beamforming coefficient set and a power control coefficient set based on a first constraint condition and using a first machine learning model configured for each of the M wireless access points. The beamforming coefficient set is the set of K beamforming coefficients that maximize the total transmission rate under the first constraint condition. The power control coefficient set is the set of K power control coefficients that maximize the total transmission rate under the first constraint condition. Both the K beamforming coefficients and the K power control coefficients correspond one-to-one with the K user equipments. The total transmission rate is the sum of the transmission rates of the signals received by each of the K user equipments from their corresponding wireless access points. M and K are both integers, and M≥2, K≥2. The allocation module is used to control M wireless access points to allocate communication resources to corresponding user equipment based on the beamforming coefficient set and the power control coefficient set. The communication resources include transmit power and beamforming vector. The transmit power is determined by the corresponding power control coefficient, and the beamforming vector is determined by the corresponding beamforming coefficient.

9. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 7.

Images