Terminal matching method, device, electronic device and computer readable storage medium
By performing measurement information processing and parameter optimization on the uplink terminal and downlink terminal in the mobile communication system, dynamic scheduling of terminals in the full duplex cell is realized, and the problems of low spectrum utilization and serious interference between terminals are solved, which improves the cell Capacity and spectrum efficiency.
Patent Information
- Application Number
- CN202311205103.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-18
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-09-18
AI Technical Summary
In existing mobile communication systems, the frequency division duplex and time division duplex modes lead to low spectrum utilization, especially in the networking scenarios of full-duplex base stations and half-duplex terminals, the interference between terminals is severe, resulting in a decrease in cell capacity.
By measuring information between the uplink terminal and the downlink terminal in the cell, the measurement weighted value is calculated based on the current measurement information and cell parameters, the Hungarian maximum weight matching algorithm and reinforcement learning optimization parameters are used to realize dynamic scheduling of terminal pairs to reduce interference. Improve spectrum utilization.
The spectrum utilization rate of full-duplex cells is improved, interference between terminals is reduced, cell capacity is improved, complex requirements for terminal channel gain measurement is avoided, and the scheduling process of the base station is simplified.
Smart Images

Figure CN117177250B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of communication technology, and in particular to a terminal matching method, a terminal matching device, an electronic device, and a computer-readable storage medium. Background Art
[0002] In the existing mobile communication system, frequency division duplexing (FDD) or time division duplexing (TDD) is generally used for communication. In the mobile communication system of FDD mode, different frequencies are used for uplink and downlink communication, and signals are received and sent on two separate symmetrical frequency channels. When the mobile communication system adopts FDD mode to support asymmetric services, the spectrum utilization rate will be greatly reduced; and the mobile communication system adopting FDD mode cannot flexibly configure the frequency, and it is not easy to use scattered frequency bands, which also leads to low spectrum utilization rate. In the mobile communication system of TDD mode, reception and transmission are carried out in different time slots of the same frequency channel, and different time slots are used to separate the receiving and transmitting channels. Since the transceiver channel of the mobile communication system adopting TDD mode uses the same frequency channel, a large guard band needs to be reserved, which affects the overall spectrum utilization efficiency. That is, for the FDD mode and TDD mode often used in the existing mobile communication system, the signal can only be transmitted unidirectionally within a certain time or on a specific frequency band, resulting in very low spectrum utilization rate. Summary of the invention
[0003] An embodiment of the present disclosure provides a terminal matching method, wherein a cell includes an uplink terminal and a downlink terminal, and the method includes: obtaining current measurement information of the uplink terminal and the downlink terminal to determine a current state according to the current measurement information; determining a current maximum weight of the current state according to current parameters of the cell and the current measurement information; determining a current reward based on the current maximum weight; adjusting the current parameters according to the current state and the current reward to obtain a target parameter; determining a terminal pair set of the uplink terminal and the downlink terminal according to the current measurement information and the target parameter, wherein the terminal pair set includes a terminal pair, and the uplink terminal and the downlink terminal in the terminal pair share the same frequency resource.
[0004] An embodiment of the present disclosure provides a terminal matching device, which includes an uplink terminal and a downlink terminal in a cell, and the device includes: a receiving unit, used to obtain current measurement information of the uplink terminal and the downlink terminal to determine a current state according to the current measurement information; a processing unit, used to determine a current maximum weight of the current state according to current parameters of the cell and the current measurement information; the processing unit is also used to determine a current reward based on the current maximum weight; the processing unit is also used to adjust the current parameters according to the current state and the current reward to obtain a target parameter; the processing unit is also used to determine a terminal pair set of the uplink terminal and the downlink terminal according to the current measurement information and the target parameter, wherein the terminal pair set includes a terminal pair, and the uplink terminal and the downlink terminal in the terminal pair share the same frequency resource.
[0005] An embodiment of the present disclosure provides an electronic device, including a processor, a memory, and an input / output interface; the processor is connected to the memory and the input / output interface, respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the electronic device including the processor executes the terminal matching method in any embodiment of the present disclosure.
[0006] An embodiment of the present disclosure provides a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded and executed by a processor, so that an electronic device having the processor executes the terminal matching method in any embodiment of the present disclosure.
[0007] The embodiments of the present disclosure provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method provided in various optional modes in any embodiment of the present disclosure.
[0008] An embodiment of the present disclosure provides a communication device, including a processor, which is coupled to a memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions in the memory, so that the method provided by any possible implementation of the present disclosure is executed.
[0009] Possibly, the memory coupled to the processor may be located outside the communication device, or may be integrated with the processor.
[0010] In a possible implementation, the communication device further includes a transceiver for implementing a transceiver function of the communication device.
[0011] In a possible implementation, the communication device includes a network device.
[0012] In a possible implementation, the network device includes a base station.
[0013] In a possible implementation, the base station includes a full-duplex base station.
[0014] The embodiment of the present disclosure provides a communication device, including: an input interface, a logic circuit and an output interface. The logic circuit is used to execute the method provided by any possible implementation of the present disclosure.
[0015] An embodiment of the present disclosure provides a chip, including a processor, where the processor is used to call and run a computer program from a memory to execute a method in any possible implementation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0017] Figure 1 It is a method flow chart of a terminal matching method provided by an embodiment of the present disclosure.
[0018] Figure 2 It is a network interaction architecture diagram of a terminal matching method provided by an embodiment of the present disclosure.
[0019] Figure 3 It is a method flow chart of another terminal matching method provided by an embodiment of the present disclosure.
[0020] Figure 4 Schematic diagram of a reinforcement learning network used in the method provided in an embodiment of the present disclosure.
[0021] Figure 5 It is a schematic diagram of a reinforcement learning network and a recurrent neural network applied in the method provided in an embodiment of the present disclosure.
[0022] Figure 6 It is a schematic diagram of a terminal matching device provided in an embodiment of the present disclosure.
[0023] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0025] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.
[0026] The terms “first”, “second”, etc. in the description and claims of the embodiments of the present disclosure are used to distinguish different objects rather than to describe a specific order of the objects.
[0027] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0028] In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.
[0029] It should be noted that, in the embodiments of the present disclosure, "uplink" may refer to a signal transmission direction for signal reception for a wireless access device and a signal transmission direction for signal sending for a terminal; "downlink" may refer to a signal transmission direction for signal sending for a wireless access device and a signal transmission direction for signal reception for a terminal; an "uplink terminal" refers to a terminal in an uplink service, that is, a terminal that sends an uplink signal to a wireless access device; a "downlink terminal" refers to a terminal in a downlink service, that is, a terminal that receives a downlink signal from a wireless access device; of course, other variations or substitutions based on this should all fall within the protection scope of the present disclosure.
[0030] The technical solutions in the embodiments of the present disclosure can be applied to communication systems including but not limited to: Narrow Band-Internet of Things (NB-IoT) system, Long Term Evolution (LTE) system, three major application scenarios of 5G mobile communication system: Enhanced Mobile Broadband (eMBB), Ultra-reliable and Low Latency Communications (URLLC) and Massive Machine Type Communication (mMTC) or new communication systems such as 6G.
[0031] Various embodiments of the present disclosure relate to terminals (including uplink terminals and downlink terminals). Terminals may refer to user equipment (UE), access terminals, subscriber units, user stations, mobile stations, mobile stations, remote stations, remote terminals, mobile devices, user terminals (TE), terminals, wireless communication devices, user agents or user devices. The access terminal may be a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA), a tablet computer (pad), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network or a network after 5G, etc.; with the development of wireless communication technology, devices that can access a communication system, communicate with the network side of a communication system, or communicate with other objects through a communication system can all be terminals in the embodiments of the present disclosure, for example, terminals and cars in intelligent transportation, household appliances in smart homes, power meter reading instruments, voltage monitoring instruments, environmental monitoring instruments in smart grids, video surveillance instruments in intelligent security networks, cash registers, machine type communication (English: Machine Type Communication, abbreviated as: MTC) terminals, etc., and the present disclosure does not limit this.
[0032] The various embodiments of the present disclosure also relate to network devices. The network device may be a device for communicating with a terminal, for example, a base station (Base Transceiver Station, BTS) in a GSM system or CDMA, a base station (NodeB, NB) in a WCDMA system, an evolutionary base station (Evolutional Node B, eNB or eNodeB) in an LTE system, a next generation base station (next generation nodeB, gNB) in a 5G system, a transmission reception point (TRP), a relay node (relay node), an access point (access point, AP), or the network device may be a relay station, an access point, an on-board device, a wearable device, and a network side device in a future network system, etc., which is not limited by the present disclosure.
[0033] The network device of the embodiment of the present disclosure may be a network device of a cell, a base station at the cell level, or a network device with functions similar to a base station. The network device may be a network device that provides wireless access and communication services for mobile or fixed terminal devices in the cell.
[0034] In communication systems, the duplexing method is usually frequency division duplex FDD, time division duplex TDD or a combination of the two. However, the above duplexing methods can only utilize one of the spectrum resources and time resources. In order to alleviate the problem of increasingly scarce wireless spectrum resources as wireless devices increase exponentially, the communication field adopts co-time co-frequency full duplex (CCFD, hereinafter referred to as "full duplex") technology to improve the utilization rate of wireless spectrum resources. CCFD wireless communication technology utilizes both spectrum resources and time resources, so it nearly doubles the spectrum utilization rate compared to the half-duplex (HD) mode (such as frequency division duplex mode, time division duplex mode).
[0035] CCFD base stations (full-duplex base stations / simultaneous and same-frequency full-duplex base stations) have the ability to transmit and receive simultaneously in the same frequency band and at the same time. With the development of technology, the self-interference suppression capability of CCFD base stations has been greatly improved, and they have the ability to be deployed in a single station. The scenario closest to commercial use is the single-station networking scenario of full-duplex base stations + half-duplex terminals, but the most difficult problem in this scenario is that the uplink terminal interferes with the downlink terminal, resulting in a serious decrease in the overall capacity of the cell. Therefore, in the networking scenario of full-duplex base stations and half-duplex terminals, a method of terminal coordinated scheduling is urgently needed to alleviate interference between terminals, thereby increasing cell capacity.
[0036] In the related technology, the interference coordination management of the full-duplex base station single-cell network and the half-duplex terminal networking scenario is coordinated and scheduled through methods such as user matching and power control. However, this method needs to calculate the rate when each uplink terminal is matched with each downlink terminal. In this calculation process, it is necessary to know the channel gain from each uplink terminal to each downlink terminal and report it to the base station. According to the current terminal detection capabilities, coupled with the fact that the terminal's location is constantly moving, this is almost impossible in engineering implementation.
[0037] In summary, CCFD base stations have the ability to transmit and receive simultaneously in the same frequency band. In the scenario where CCFD base stations are networked with half-duplex terminals, the uplink terminals in the cell will interfere with the downlink terminals using the same frequency. Under severe terminal interference, the spectrum efficiency of CCFD base stations is limited, and even lower than that of half-duplex base stations under extreme conditions. Therefore, in the scenario where full-duplex base stations are networked with half-duplex terminals, a terminal coordinated scheduling method (i.e., a terminal matching method) is needed to alleviate interference between terminals, thereby increasing cell capacity.
[0038] Figure 1 The method flow chart of a terminal matching method provided by the embodiment of the present disclosure is as follows. In the embodiment of the present disclosure, the cell includes uplink terminals and downlink terminals.
[0039] For example, the scenario of full-duplex base station and half-duplex terminal is as follows Figure 2 As shown, it is assumed that there are N uplink terminals in the cell 200 (for example, a full-duplex cell), denoted as U = {u1, u2…u N}, there are M downlink terminals, denoted as D = {d1, d2…d M}, N and M are both positive integers greater than 1. Assume that the Nth uplink terminal is paired with the Mth downlink terminal (pairing means sharing the same frequency resource, i.e. u N With d M Sharing the same spectrum resource and communicating with the full-duplex base station 210 at the same time, the uplink terminal u N For downlink terminal d M Cause interference.
[0040] In the related art, it is necessary to know the channel gain of the interference (for example, g NM = represents the channel gain from the Nth uplink terminal to the Mth downlink terminal), the Shannon formula can be used to calculate the uplink and downlink terminal rates at this time and then report them to the base station. However, current terminals basically do not have this channel measurement capability. Therefore, in actual engineering, it is impossible to complete all pairing situations {(u i ,d j )|u i ∈U,d j ∈D} under the terminal rate measurement or calculation.
[0041] like Figure 1 As shown, the method provided by the embodiment of the present disclosure may include the following steps.
[0042] In S110, current measurement information of the uplink terminal and the downlink terminal is obtained to determine a current state according to the current measurement information.
[0043] In the disclosed embodiment, the current measurement information of the uplink terminal and the downlink terminal in the cell refers to any parameters obtained by measuring the uplink terminal and / or the downlink terminal in the cell at the current moment (for example, moment k, where k is a positive integer greater than or equal to 1), and / or any information related to the pairing of the uplink terminal and the downlink terminal in the cell, such as the current coordinates of the uplink terminal and / or the downlink terminal at the current moment, the current distance between the uplink terminal and the downlink terminal, the area, terrain, building distribution, and building height of the cell, the obstruction of the uplink terminal and the downlink terminal by the buildings in the cell, and the like.
[0044] In an exemplary embodiment, the current measurement information of the uplink terminal and the downlink terminal includes at least one of the following:
[0045] The current distance L between the i-th uplink terminal and the j-th downlink terminal ij ;
[0046] Current signal interference plus noise ratio of the jth downlink terminal
[0047] Current channel quality indicator of the jth downlink terminal
[0048] Current reference signal received power of the i-th uplink terminal
[0049] Current reference signal received power of the jth downlink terminal
[0050] Among them, i is a positive integer greater than or equal to 1 and less than or equal to N, j is a positive integer greater than or equal to 1 and less than or equal to M, N is the number of uplink terminals in the cell, and M is the number of downlink terminals in the cell.
[0051] In an exemplary embodiment, the cell also includes network equipment (eg Figure 2 The network device measures and obtains the current measurement information. That is, in some embodiments, user pairing and resource allocation can be completed for the terminal of the full-duplex cell only by using the base station measurement. By bypassing the terminal measurement, the uplink and downlink terminals are uniformly scheduled only by the base station measurement, and interference coordination is performed on the uplink and downlink terminals, thereby reducing terminal interference and improving the capacity of the full-duplex cell.
[0052] In other embodiments, each uplink terminal and each downlink terminal may also obtain their own current measurement information, and then report it to the network equipment in the cell respectively, which is not limited in the present disclosure.
[0053] In S120, a current maximum weight of the current state is determined according to the current parameters of the cell and the current measurement information.
[0054] In the disclosed embodiment, the current parameters of the cell refer to parameters used to adjust current measurement information and are related to factors such as the distribution of uplink and downlink terminals in the cell and the topography of the cell. The current parameters of the cell will affect the pairing of uplink and downlink terminals in the cell. By optimizing the current parameters of the cell, a terminal pairing that better matches the cell can be obtained, thereby reducing interference between terminals.
[0055] In an exemplary embodiment, the current parameter includes at least one of the following:
[0056] Uplink and downlink terminal distance parameter a;
[0057] Signal-to-interference-plus-noise ratio parameter b;
[0058] Channel quality indicator parameter c;
[0059] Reference signal received power parameter d.
[0060] In some embodiments, when the i-th uplink terminal u i With the jth downlink terminal d j When pairing, the measurement weighting equation is defined as follows:
[0061]
[0062] where h ij represents the i-th uplink terminal u i With the jth downlink terminal d j The measurement weight when pairing. L ij represents the i-th uplink terminal u i With the jth downlink terminal d j The current distance between the two, in the embodiment of the present disclosure, it is considered that L ij The larger the value, the higher the i-th uplink terminal u i With the jth downlink terminal d j The more suitable for pairing. Represents the i-th downlink terminal d jThe current SINR (Signal to Interference plus Noise Ratio) and the current CQI (Channel Quanlity Indicator) of the present disclosure are considered to be The larger the value, the higher the jth downlink terminal d j The better the channel conditions, the stronger the anti-interference ability, and the more suitable it is for pairing with the uplink terminal. Represents the i-th uplink terminal u i With the jth downlink terminal d j RSRP (Reference Signal Receiving Power), the embodiment of the present disclosure believes that the lower the RSRP, the more obstructions there are around the corresponding terminal, the higher the isolation, and the more suitable it is for pairing.
[0063] In the above formula, a, b, c, and d are all adjustable parameters, which can be adjusted and optimized according to factors such as user distribution and topography and architecture in the community. The user distribution and topography and architecture in the community will be reflected in the above measurement parameter L ij , a, b, c, d are all real numbers greater than 0.
[0064] In an exemplary embodiment, the base station measures all terminals in the cell to obtain the above measurement parameter L ij , And calculate all terminal pairings in the base station background {(u i ,d j )|u i ∈U,d j ∈D}, forming a measurement weighted graph matrix, which also represents a bipartite graph with weights:
[0065]
[0066] The measurement weighted graph matrix H NM Processing is performed (for example, using the Hungarian maximum weight matching algorithm, but the present disclosure is not limited thereto), and the measurement weighted graph matrix H is returned NM The corresponding pairing sequence set P (i.e., terminal pair set) and the current maximum weight h sum , which is used by the base station to complete terminal matching and resource allocation.
[0067] In S130 , a current reward is determined based on the current maximum weight.
[0068] In an exemplary embodiment, determining the current reward based on the current maximum weight includes: obtaining the historical maximum weights of the n states before the current state, where n is a positive integer greater than or equal to 1; and determining the current reward based on the current maximum weight of the current state and the historical maximum weights of the n states before the current state.
[0069] In an exemplary embodiment, the current reward is determined based on the current maximum weight of the current state and the historical maximum weights of the previous n states, including: obtaining the maximum weight difference value of the current state based on the current maximum weight of the current state, the historical maximum weights of the previous n states, and a discount factor; and determining the current reward based on the maximum weight difference value of the current state.
[0070] In an exemplary embodiment, n is a positive integer greater than or equal to 1 and less than or equal to 3.
[0071] In S140, the current parameter is adjusted according to the current state and the current reward to obtain a target parameter.
[0072] In an exemplary embodiment, the current parameter is adjusted according to the current state and the current reward to obtain a target parameter, including: obtaining a predicted reward for m future states of the current state according to the current state and the current reward, where m is a positive integer greater than or equal to 2; changing the current parameter within an adjustment range according to a preset step size, and determining that the current parameter that maximizes the expected sum of the predicted rewards for the m future states is the target parameter.
[0073] In an exemplary embodiment, the predicted rewards for the future m states of the current state are obtained based on the current state and the current reward, including: obtaining a current state reward time series of the cell, the current state reward time series including the current reward of the current state; and obtaining the predicted rewards for the future m states based on the current state reward time series.
[0074] In an exemplary embodiment, m is a positive integer greater than or equal to 2 and less than or equal to 5.
[0075] In S150, a terminal pair set of the uplink terminal and the downlink terminal is determined according to the current measurement information and the target parameter, wherein the terminal pair set includes a terminal pair, and the uplink terminal and the downlink terminal in the terminal pair share the same frequency resource.
[0076] The terminal matching method provided by the embodiment of the present disclosure determines the current state according to the current measurement information of the uplink terminal and the downlink terminal, determines the current reward according to the current parameters of the cell and the current measurement information, and then adjusts the current parameters based on the current state and the current reward to obtain the adjusted target parameters, thereby achieving online dynamic parameter adjustment based on newly generated data when measuring the current measurement information of the uplink and downlink terminals in the cell and allocating resources to the uplink and downlink terminals, so that the obtained target parameters can change with the changes in factors such as the user distribution and the topographical buildings in the cell, thereby improving the accuracy of terminal pairing, reducing interference between terminals, and improving the communication capacity of the cell.
[0077] Combine the following Figures 3 to 5 The method provided in the embodiment of the present disclosure is illustrated by way of example, but the present disclosure is not limited thereto.
[0078] Figure 3 is a method flow chart of another terminal matching method provided by an embodiment of the present disclosure. Figure 3 As shown, the method provided by the embodiment of the present disclosure may include the following steps.
[0079] In S310, SINR, RSRP, CQI and coordinates of the terminals in the cell are measured.
[0080] For example, the CCFD base station obtains the coordinates of each uplink terminal and downlink terminal in the cell by measuring, and according to the i-th uplink terminal u i The coordinates of the jth downlink terminal d j The coordinates of the i-th uplink terminal u are calculated i With the jth downlink terminal d j The current distance L between ij The CCFD base station also measures and obtains the i-th downlink terminal d j of and The i-th uplink terminal u i of With the jth downlink terminal d j of
[0081] In S320, the CCFD base station calculates the i-th uplink terminal u according to the designed measurement weight equation (such as the above formula (1)). i With the jth downlink terminal d j The measurement weight h for pairing ij , get the measurement weighted graph matrix H NM (As above formula (2)).
[0082] In S330, H is calculated using, for example, the Hungarian maximum weight matching algorithm.NM , and returns the pairing sequence set P.
[0083] In some embodiments, the embodiments of the present disclosure may return the pairing sequence set P obtained in S330 to all terminals in the cell, and pair uplink terminals with downlink terminals according to the pairing sequence set P.
[0084] In some other embodiments, S340 may be further included, in which a reinforcement learning method is used to maximize the reward h Q As the goal, optimize the adjustable parameters a, c, c, d.
[0085] According to the above formulas (1) and (2), H NM The value of the element in is affected by the adjustable parameters a, b, c, d, so different a, b, c, d parameters will return different terminal pairing sequence sets P and maximum weights h sum Therefore, the embodiment of the present disclosure can further reduce the interference between terminals by optimizing the a, b, c, and d parameters through a reinforcement learning algorithm.
[0086] In the disclosed embodiment, the main elements of reinforcement learning are designed as follows:
[0087] State: The user's measurement parameters and the user's distribution (e.g., L in the above formula (1) ij , and and ), it is defined that a measurement is performed every predetermined time period (for example, 10ms, but the present disclosure is not limited thereto and can be set according to actual needs), and the measurement result is a state.
[0088] Action: Change the adjustable parameters (a, b, c, d) with a small preset step size. For example, assume that the preset step size of a is Δa, the preset step size of b is Δb, the preset step size of c is Δc, and the preset step size of d is Δd. The preset step sizes of different parameters can be different or the same, and the present disclosure does not limit this. After adjustment, new parameters (a+Δa, b+Δb, c+Δc, d+Δd) are obtained. The adjustment range of the parameters can be set, that is, the minimum and maximum values of each parameter. When changing the parameters, each parameter is not allowed to be within its corresponding adjustment range. Different parameters can be set with different adjustment ranges. The values of the preset step size and the adjustment range can be determined according to actual needs. For example, it can be designed according to the specific situation of a specific cell. The smaller the preset step size is designed, the more likely it is to obtain a better solution, but the convergence time will be longer.
[0089] Reward: Current reward h Q The current maximum weight of the current state and the historical maximum weight of the previous n states () (respectively represented as ) jointly determined, n∈[1,3]. If the value of n is greater than 3, there are too many states associated with the current reward, and the computational complexity will increase greatly. If the value of n is less than 1, the accuracy of terminal pairing will decrease. n is within the positive integer range of 1 to 3, which can maintain a good balance between computational complexity and pairing accuracy. Where γ is a discount factor, which is greater than 0 and less than 1. n and γ are determined according to the actual scenario. In the disclosed embodiment, the current reward can be determined by the following formula:
[0090]
[0091]
[0092] In the above formula, Δh sum Indicates the maximum weight difference value of the current state.
[0093] Value strategy: The value is the reward h for the next m states Q The expectation of the sum, m∈[2,5]. If the value of m is greater than 5, there are too many value-associated states, and the computational complexity will increase greatly. If the value of m is less than 2, the accuracy of terminal pairing will decrease. m is within the positive integer range of 2 to 5, which can maintain a good balance between computational complexity and pairing accuracy. Adjust parameters a, b, c, d in the direction of maximizing value. The predicted reward for the future m states can be implemented by any algorithm or model that has a prediction function for time series data. In the following embodiments, recurrent neural networks (such as LSTM (Long Short-Term Memory), GRU (Gate Recurrent Unit, gated recurrent unit), etc.) are used as examples, but the present disclosure is not limited to this. Other algorithms can also be used for calculation, which can be determined according to actual needs.
[0094] In S350, H under new parameters a, b, c, d (target parameters) is calculated. NM , and use the Hungarian maximum weight matching algorithm to return the optimized pairing sequence set P.
[0095] For example, the target parameters a, b, c, and d optimized by the reinforcement learning network are substituted into the above formula (1), and the L obtained by the above measurement is ij , and and You can get a new H NM , for this new H NMThe Hungarian maximum weight matching algorithm is used to process to obtain an optimized pairing sequence set P, and the optimized pairing sequence set P is used as a terminal pair set returned to the terminals in the cell.
[0096] In S360, the pairing sequence set P is sent to all terminals, pairing is performed according to the sequence P and resource allocation is completed, and the paired uplink and downlink terminals share the same frequency resources.
[0097] In S370, the base station performs an active measurement once in each radio frame (eg, 10 ms), and then returns to the above S310 to execute S310 to S360 in a loop.
[0098] The terminal matching method provided by the embodiment of the present disclosure, on the one hand, by constructing a measurement weighted equation, the parameters of the measurement weighted equation (such as a, b, c, d) can be trained online through reinforcement learning when the base station measures and allocates resources, and the online reinforcement learning training is continuously performed according to the newly generated data (current measurement information), so that the parameters are more in line with the characteristics of the cell, that is, the integration of reinforcement learning in the maximum weight matching of graph theory can be dynamically optimized according to the characteristics of the cell, so that the cell capacity is further optimized. On the other hand, by designing the measurement weighted equation and the measurement weighted graph matrix, the terminal measurement is bypassed, and the uplink terminal is not required to know the channel gain to all downlink terminals, which reduces the terminal's measurement capability requirements for the channel. The embodiment of the present disclosure only constructs the measurement weighted graph matrix through base station measurement, and selects the maximum weight matching of the measurement weighted graph matrix through the Hungarian maximum weight matching algorithm in graph theory. The terminal pair set returned by the algorithm is the best pairing scheme for the uplink and downlink terminals. That is, it can be realized to rely on the measurement of the base station to uniformly schedule the uplink and downlink terminals, and to coordinate interference between the uplink and downlink terminals, thereby reducing terminal interference and improving the capacity of the full-duplex cell. Compared with the related art, the method provided by the embodiment of the present disclosure does not need to calculate the rate through the signal-to-noise ratio of all terminals, but only requires base station measurement.
[0099] like Figure 4 As shown, the reinforcement learning network 400 includes an agent 410 and an environment 420. Define a set of states (hereinafter referred to as s), actions (hereinafter referred to as a) and rewards (hereinafter referred to as h) Q A practical framework for solving decision problems in uncertain environments. Figure 4 As shown, each action changes the parameters a, b, c, d, thereby changing the environment 420 and generating a new state and a new reward. The new state and the new reward are input into the agent 410 to determine the parameters a, b, c, d at the next moment, that is, to determine the action at the next moment.
[0100] like Figure 4As shown, the environment 420 may include factors such as the distribution of terminals in the cell, the topography of the cell, etc., and the L ij , and and To indicate. Agent 410 can be a neural network, that is, the output of agent 410 can be a new parameter after adjustment, and the new parameter is obtained from the adjustment range corresponding to each parameter. Agent 410 will perceive the current state, take actions on environment 420 according to the strategy, thereby changing the state of environment 420 and obtaining rewards, and the accumulation of rewards over time is called return. Solve the agent strategy that maximizes the return.
[0101] like Figure 5 As shown, the agent 410 may further include a strategy network 411 and a value network 412. The state s describes each specific environmental state of the system. The current state at time k may include the L obtained above. ij , and and For example, the fully connected layer FC 421, the fully connected layer FC 422, the fully connected layer FC 423, and the fully connected layer FC 424 are used to encode L ij , and The encoding obtains distance coding information 431, signal interference plus noise ratio coding information 432, channel quality indicator coding information 433, and reference signal received power coding information 434. The distance coding information 431, signal interference plus noise ratio coding information 432, channel quality indicator coding information 433, and reference signal received power coding information 434 are concatenated to obtain state embedding 430, and the state embedding 430 is input into the strategy network 411.
[0102] Reward r is the feedback for each action. Reward r is a value used to measure the goodness of an action in reinforcement learning. Maximizing the expectation of the sum of the predicted rewards for the future m states to be achieved by the final strategy can be accomplished through the training process of reinforcement learning.
[0103] The strategy π defines the transition probability distribution between states. It can provide guidance for the next operation (action) based on the current information (state, reward). Through repeated events (consisting of a sequence of states, actions, and rewards), a strategy π is trained using the neural network in the agent 410. θ, learning to take the best action for a specific state. The trained π is the best strategy under ideal conditions. The parameters θ of the strategy can be trained. The strategy refers to the conditional probability of determining the parameters a, b, c, and d at the next moment k+1 under the premise of the state and reward at the current moment k. It can give the probability distribution of the next action based on the current state.
[0104] Specifically, the policy network 411 includes a fully connected layer ( Figure 5 FC 4111 in the example) and a deconvolution network structure 4112. The state embedding 430 is first input into FC 4111 to convert the state embedding 430 into a format suitable for processing by the deconvolution network structure 4112. The deconvolution network structure 4112 can generate a two-dimensional probability distribution in the action space. The value network 412 can include an MLP network 4121, which can be used to calculate the expectation of the sum of the predicted rewards for the future m states. The policy network 411 is then trained and optimized to maximize the expectation of the sum of the predicted rewards for the future m states output by the value network 412.
[0105] like Figure 5 As shown, the current state reward time series can be processed by the recurrent neural network 500 to obtain the predicted rewards for the next m states. The current state reward time series refers to the sequence formed by sorting the current reward and the historical rewards of the previous p states of the current state (p is a positive integer greater than or equal to 1, which can be set according to actual needs) in chronological order. The current state reward time series is input into the trained recurrent neural network 500 to predict the predicted rewards for the next m states.
[0106] In the disclosed embodiment, the recurrent neural network 500, the policy network 411 and the value network 412 can be jointly trained so that the network parameters of the recurrent neural network 500 are trained together with the parameters of the policy network 411, that is, the recurrent neural network 500 is connected to the reinforcement learning network, so during the training process, the rewards collected by the reinforcement learning network can be fed back to the update of all network parameters, including the network parameters of the recurrent neural network 500 and the parameters of the policy network.
[0107] It should be noted that the network structures of the recurrent neural network, policy network and value network in the embodiments of the present disclosure are not limited to the above examples. For example, the deconvolution network structure of the policy network can also be replaced by MLP (multi-layer perceptron).
[0108] The terminal matching method provided in the embodiment of the present disclosure uses a reinforcement learning algorithm to dynamically optimize adjustable parameters, so that the method provided by the present disclosure can dynamically adapt to cells in different scenarios, making the scope of application of the method provided by the present disclosure wider. It can be applied to the uplink and downlink terminal resource allocation in the networking scenario of simultaneous co-frequency full-duplex cells. In this scenario, the method provided by the embodiment of the present disclosure is used to perform terminal pairing and resource allocation, which can improve the spectrum utilization rate of simultaneous co-frequency full-duplex cells, reduce the interference between uplink terminals and downlink terminals in the scenario of single-station networking of CCFD base stations, and thus improve the cell capacity.
[0109] Figure 6 is a schematic diagram of a terminal matching device provided in an embodiment of the present disclosure. The terminal matching device may be a computer program (including program code, etc.) running in an electronic device; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present disclosure. The cell includes an uplink terminal and a downlink terminal. Figure 6 As shown, the terminal matching device 600 can be used in the electronic device in the corresponding embodiment. Specifically, the device can include: a receiving unit 610 and a processing unit 620.
[0110] The receiving unit 610 is used to obtain the current measurement information of the uplink terminal and the downlink terminal to determine the current state according to the current measurement information. The processing unit 620 is used to determine the current maximum weight of the current state according to the current parameters of the cell and the current measurement information. The processing unit 620 is also used to determine the current reward based on the current maximum weight. The processing unit 620 is also used to adjust the current parameters according to the current state and the current reward to obtain the target parameters. The processing unit 620 is also used to determine the terminal pair set of the uplink terminal and the downlink terminal according to the current measurement information and the target parameters, wherein the terminal pair set includes terminal pairs, and the uplink terminal and the downlink terminal in the terminal pair share the same frequency resources.
[0111] In an exemplary embodiment, the processing unit 620 is also used to: obtain the historical maximum weight of the n states before the current state, where n is a positive integer greater than or equal to 1; and determine the current reward based on the current maximum weight of the current state and the historical maximum weights of the n states before the current state.
[0112] In an exemplary embodiment, the processing unit 620 is also used to: obtain the maximum weight difference value of the current state according to the current maximum weight of the current state, the historical maximum weights of the previous n states and the discount factor; and determine the current reward based on the maximum weight difference value of the current state.
[0113] In an exemplary embodiment, the processing unit 620 is also used to: obtain the predicted rewards of the future m states of the current state based on the current state and the current reward, where m is a positive integer greater than or equal to 2; change the current parameter within the adjustment range according to a preset step size, and determine that the current parameter that maximizes the expected sum of the predicted rewards of the future m states is the target parameter.
[0114] In an exemplary embodiment, the processing unit 620 is further used to: obtain a current state reward time series of the cell, wherein the current state reward time series includes the current reward of the current state; and obtain predicted rewards of the future m states based on the current state reward time series.
[0115] The method and device provided in the embodiments of the present disclosure are conceived based on the same application. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.
[0116] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure.
[0117] See also Figure 7 , Figure 7 Schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 7 As shown, the electronic device in the embodiment of the present disclosure may include: one or more processors 701, a memory 702, and an input-output interface 703. The processor 701, the memory 702, and the input-output interface 703 are connected via a bus 704. The memory 702 is used to store a computer program, which includes program instructions, and the input-output interface 703 is used to receive data and output data, such as for data interaction between a host machine and an electronic device, or for data interaction between various virtual machines in a host machine; the processor 701 is used to execute the program instructions stored in the memory 702.
[0118] Among them, the processor 701 can perform the following operations: obtain current measurement information of the uplink terminal and the downlink terminal to determine the current state according to the current measurement information; determine the current maximum weight of the current state according to the current parameters of the cell and the current measurement information; determine the current reward based on the current maximum weight; adjust the current parameters according to the current state and the current reward to obtain the target parameters; determine the terminal pair set of the uplink terminal and the downlink terminal according to the current measurement information and the target parameters, wherein the terminal pair set includes a terminal pair, and the uplink terminal and the downlink terminal in the terminal pair share the same frequency resources.
[0119] In some feasible implementations, the processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
[0120] The memory 702 may include a read-only memory and a random access memory, and provides instructions and data to the processor 701 and the input / output interface 703. A portion of the memory 702 may also include a nonvolatile random access memory.
[0121] In a specific implementation, the electronic device can execute the implementation methods provided in the steps in the above figure through its built-in functional modules. For details, please refer to the implementation methods provided in the steps in the above figure, which will not be repeated here.
[0122] The embodiment of the present disclosure provides an electronic device, including: a processor, an input / output interface, and a memory. The processor obtains a computer program in the memory to execute each step of the method shown in the above figure.
[0123] The embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by the processor and executing the terminal matching method provided by each step in the above figure. For details, please refer to the implementation method provided by each step in the above figure, which will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in the present disclosure, please refer to the description of the method embodiment of the present disclosure. As an example, the computer program can be deployed to be executed on an electronic device, or on multiple electronic devices located in one place, or on multiple electronic devices distributed in multiple locations and interconnected by a communication network.
[0124] The computer-readable storage medium may be a terminal matching device provided in any of the foregoing embodiments or an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the electronic device. Further, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
[0125] The present disclosure also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of an electronic device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the electronic device executes the method provided in the various optional modes in the above figure.
[0126] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.
[0127] The method and related apparatus provided by the embodiments of the present disclosure are described with reference to the method flow chart and / or structural diagram provided by the embodiments of the present disclosure. Specifically, each process and / or block in the method flow chart and / or structural diagram, as well as the combination of the processes and / or blocks in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable search device to generate a machine, so that the instructions executed by the processor of the computer or other programmable search device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable search device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including the instruction device, which implements the function specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable search device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.
[0128] The above disclosure is only a preferred embodiment of the present disclosure, which certainly cannot be used to limit the scope of rights of the present disclosure. Therefore, equivalent changes made according to the claims of the present disclosure are still within the scope of the present disclosure.
Claims
1. A terminal matching method, characterized in that: The cell includes an uplink terminal and a downlink terminal, and the method includes: Obtaining current measurement information of the uplink terminal and the downlink terminal to determine a current state according to the current measurement information; Determining a current maximum weight of the current state according to current parameters of the cell and the current measurement information, wherein the current parameters of the cell are parameters related to the distribution of uplink terminals and downlink terminals in the cell and topographic factors of the cell; determining a current reward based on the current maximum weight; The current parameter is adjusted according to the current state and the current reward to obtain a target parameter, so that the obtained target parameter changes with the distribution of uplink terminals and downlink terminals in the cell and the changes of the cell topography and geomorphology factors; A terminal pair set of the uplink terminal and the downlink terminal is determined according to the current measurement information and the target parameter, wherein the terminal pair set includes a terminal pair, and the uplink terminal and the downlink terminal in the terminal pair share the same frequency resource.
2. The method according to claim 1, characterized in that Determining a current reward based on the current maximum weight includes: Obtain the historical maximum weight of the n states before the current state, where n is a positive integer greater than or equal to 1; The current reward is determined according to the current maximum weight of the current state and the historical maximum weights of the previous n states.
3. The method according to claim 2, characterized in that Determining the current reward according to the current maximum weight of the current state and the historical maximum weights of the previous n states includes: Obtaining a maximum weight difference value of the current state according to the current maximum weight of the current state, the historical maximum weights of the previous n states, and a discount factor; The current reward is determined based on a maximum weight difference value of the current state.
4. The method according to claim 2, characterized in that n is a positive integer greater than or equal to 1 and less than or equal to 3.
5. The method according to claim 1, characterized in that The current parameter is adjusted according to the current state and the current reward to obtain a target parameter, including: Obtain predicted rewards for m future states of the current state according to the current state and the current reward, where m is a positive integer greater than or equal to 2; The current parameter is changed within an adjustment range according to a preset step size, and the current parameter that maximizes the expected sum of the predicted rewards of the future m states is determined as the target parameter.
6. The method according to claim 5, characterized in that Obtaining predicted rewards for m future states of the current state according to the current state and the current reward, including: Obtaining a current state reward time series of the cell, wherein the current state reward time series includes the current reward of the current state; The predicted rewards for the future m states are obtained according to the current state reward time series.
7. The method according to claim 5, characterized in that m is a positive integer greater than or equal to 2 and less than or equal to 5.
8. The method according to any one of claims 1 to 7, characterized in that The current measurement information of the uplink terminal and the downlink terminal includes at least one of the following: The current distance between the i-th uplink terminal and the j-th downlink terminal; The current signal-to-interference-plus-noise ratio of the jth downlink terminal; The current channel quality indicator of the j-th downlink terminal; The current reference signal received power of the i-th uplink terminal; The current reference signal received power of the j-th downlink terminal; Among them, i is a positive integer greater than or equal to 1 and less than or equal to N, j is a positive integer greater than or equal to 1 and less than or equal to M, N and M are positive integers greater than 1, N is the number of uplink terminals in the cell, and M is the number of downlink terminals in the cell.
9. The method according to any one of claims 1 to 7, characterized in that: The cell also includes a network device, and the network device obtains the current measurement information through measurement.
10. The method according to any one of claims 1 to 7, characterized in that: The current parameters include at least one of the following: Uplink and downlink terminal distance parameters; Signal-to-interference-plus-noise ratio parameter; Channel quality indicator parameters; Reference signal received power parameter.
11. A terminal matching device, characterized in that: The cell includes an uplink terminal and a downlink terminal, and the device includes: a receiving unit, configured to obtain current measurement information of the uplink terminal and the downlink terminal, so as to determine a current state according to the current measurement information; a processing unit, configured to determine a current maximum weight of the current state according to current parameters of the cell and the current measurement information, wherein the current parameters of the cell are parameters related to the distribution of uplink terminals and downlink terminals in the cell and topographic factors of the cell; The processing unit is further configured to determine a current reward based on the current maximum weight; The processing unit is further configured to adjust the current parameter according to the current state and the current reward to obtain a target parameter, so that the obtained target parameter changes with changes in the distribution of uplink terminals and downlink terminals in the cell and changes in the cell topography and geomorphology; The processing unit is further used to determine a terminal pair set of the uplink terminal and the downlink terminal according to the current measurement information and the target parameter, wherein the terminal pair set includes a terminal pair, and the uplink terminal and the downlink terminal in the terminal pair share the same frequency resource.
12. An electronic device, characterized in that: Includes processor, memory, input and output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the electronic device executes the method described in any one of claims 1-10.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that an electronic device having the processor executes the method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Spectrum resource allocation method, device and system, electronic device and storage medium
CN113225820A
