Electronic device and method for wireless communication

Abstract

The present disclosure provides an electronic device and method for wireless communication, the electronic device comprising: a processing circuit configured to: determine a set of cooperative access points for users within a predetermined range based on a wireless network topology structure of a wireless network as a state; and re-determine a set of cooperative access points for users in response to a change in the wireless network topology structure, wherein the wireless network topology structure includes a distribution of users and a distribution of access points.

Description

[0001] This application claims the priority of Chinese patent application filed with the China Patent Office on October 25, 2017, with application number 201711009075.6 and invention name “Electronic device and method for wireless communication”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] Embodiments of the present invention generally relate to the field of wireless communications, and more particularly to resource management in a User Centric Network (UCN), and more particularly to electronic devices and methods for wireless communications. Background Art

[0003] With the rapid development of communication networks, the demand for user mobile data rates has increased exponentially. In addition, the mobility and flexible configuration of devices have also brought challenges to future wireless networks. Ultra-dense networks (UDN) deploy miniaturized small base stations, which has become an effective technical method to meet the growing demand for mobile data rates. The dense and flexible configuration of small base stations makes it possible to realize user-centric networks (UCN) to support effective communication of massive mobile users and devices. UCN allows each user to jointly select multiple access points such as base stations for collaborative transmission in order to meet the service quality requirements of all users with the greatest probability. Therefore, user-centric ultra-dense networks (UUDN) will become the main trend of future networks.

[0004] In addition, with the rise of artificial intelligence and the Internet of Things, artificial intelligence methods such as machine learning are also one of the hot topics of recent research. Wireless networks simulate human thinking to make resource management and other aspects more intelligent. Summary of the invention

[0005] A brief overview of the present application is provided below in order to provide a basic understanding of certain aspects of the present application. It should be understood that this overview is not an exhaustive overview of the present application. It is not intended to identify the key or important parts of the present application, nor is it intended to limit the scope of the present application. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0006] According to one aspect of the present application, an electronic device for wireless communication is provided, comprising: a processing circuit configured to: determine a set of cooperative access points for users within a predetermined range based on a wireless network topology structure of a wireless network as a state; and re-determine a set of cooperative access points for users in response to a change in the wireless network topology structure, wherein the wireless network topology structure includes a distribution of users and a distribution of access points.

[0007] According to another aspect of the present application, a method for wireless communication is provided, comprising: determining a set of cooperative access points for users within a predetermined range based on a wireless network topology structure of a wireless network as a state; and re-determining a set of cooperative access points for users in response to a change in the wireless network topology structure, wherein the wireless network topology structure includes a distribution of users and a distribution of access points.

[0008] According to other aspects of the present application, a computer program code and a computer program product for implementing the above method and a computer-readable storage medium having the computer program code for implementing the above method recorded thereon are also provided.

[0009] The electronic device and method according to the present application can realize the selection of a dynamic collaborative access point group (APG) to better meet the communication requirements of all users.

[0010] These and other advantages of the present application will become more apparent through the following detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to further illustrate the above and other advantages and features of the present invention, the specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. The accompanying drawings together with the following detailed description are included in this specification and form a part of this specification. Elements with the same function and structure are represented by the same reference numerals. It should be understood that these drawings only describe typical examples of the present invention and should not be regarded as limiting the scope of the present invention. In the accompanying drawings:

[0012] Figure 1 A schematic diagram of a scenario of UUDN is shown;

[0013] Figure 2 A functional module block diagram of an electronic device for wireless communication according to an embodiment of the present application is shown;

[0014] Figure 3 A graph showing an example of a utility function;

[0015] Figure 4A functional module block diagram of an electronic device for wireless communication according to an embodiment of the present application is shown;

[0016] Figure 5 A functional module block diagram of an electronic device for wireless communication according to another embodiment of the present application is shown;

[0017] Figure 6 A functional module block diagram of an electronic device for wireless communication according to another embodiment of the present application is shown;

[0018] Figure 7 A functional module block diagram of an electronic device for wireless communication according to another embodiment of the present application is shown;

[0019] Figure 8 A schematic diagram showing the information flow between a user, an access point and a spectrum management device;

[0020] Fig. 9 A schematic diagram of a simulation scenario of a simulation example is shown;

[0021] Fig.10 A diagram showing an example of a behavior matrix and a Q-value matrix;

[0022] Fig.11 A schematic diagram showing the result of performing the determined behavior;

[0023] Fig.12 A diagram showing another example of a behavior matrix and a Q-value matrix;

[0024] Fig.13 A schematic diagram showing the result of performing the determined behavior;

[0025] Fig.14 A schematic diagram of a simulation scenario 1 of another simulation example is shown;

[0026] Fig.15 A schematic diagram of a simulation scenario 2 of another simulation example is shown;

[0027] Fig.16 A comparison chart of the cumulative distribution function (CDF) of user satisfaction rates obtained based on simulation scenario 1 is shown;

[0028] Fig.17 A curve diagram showing the proportion of user communication quality requirements being met at different numbers of circles when the user moves along a rectangular track in simulation scenario 2;

[0029] Fig.18 A flow chart of a method for wireless communication according to an embodiment of the present application is shown;

[0030] Fig.19 is a block diagram showing an example of a schematic configuration of a server 700 to which the technology of the present disclosure may be applied; and

[0031] Fig. 20 is a block diagram of an exemplary structure of a general personal computer in which the method and / or apparatus and / or system according to the embodiments of the present invention can be implemented. DETAILED DESCRIPTION

[0032] Exemplary embodiments of the present invention will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of actual implementations are described in the specification. However, it should be understood that many implementation-specific decisions must be made in the process of developing any such actual implementation in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and these constraints may vary from implementation to implementation. In addition, it should be understood that although the development work may be very complex and time-consuming, it is only a routine task for those skilled in the art who benefit from the present disclosure.

[0033] It is also necessary to explain here that, in order to avoid obscuring the present invention due to unnecessary details, only the device structure and / or processing steps closely related to the solution according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0034] <First Embodiment>

[0035] Figure 1 A scenario diagram of UUDN is shown. There are multiple access points (AP) around the user equipment (UE, also referred to as user below), and the UE performs collaborative transmission by using different APs. In addition, each AP is communicatively connected to a spectrum management device such as a spectrum coordinator (SC). The SC determines a collaborative APG for the UE within its management range. The collaborative APG is a set of APs that have a collaborative relationship with the corresponding UE, that is, that provide communication access services to the UE. In addition, the local SC and the adjacent SC can also communicate appropriately to exchange information. It can be seen that compared with the traditional cellular network architecture, Figure 1 The characteristic of the network architecture is that the number of APs is large, even more than the number of UEs.

[0036] The AP described herein may be any node that provides network communication services, such as a base station, a small base station, etc. The base station may be implemented as any type of evolved Node B (eNB), such as a macro eNB and a small eNB. A small eNB may be an eNB that covers a cell smaller than a macro cell, such as a pico eNB, a micro eNB, and a home (femto) eNB. Alternatively, the base station may be implemented as any other type of base station, such as a NodeB and a base transceiver station (BTS). The base station may include: a main body (also referred to as a base station device) configured to control wireless communications; and one or more remote radio heads (RRHs) arranged in a place different from the main body. In addition, various types of terminals may work as a base station by temporarily or semi-persistently performing base station functions.

[0037] UE or user can be any terminal device or wireless communication device providing services. For example, the terminal device can be implemented as a mobile terminal (such as a smart phone, a tablet personal computer (PC), a notebook PC, a portable game terminal, a portable / dongle type mobile router and a digital camera) or a vehicle terminal (such as a car navigation device). The terminal device can also be implemented as a terminal that performs machine-to-machine (M2M) communication (also called a machine type communication (MTC) terminal). In addition, the terminal device can be a wireless communication module (such as an integrated circuit module including a single chip) installed on each of the above-mentioned terminals.

[0038] also, Figure 1 The SC shown in the figure is only an example of a spectrum management device, and other forms of spectrum management devices may also be used, such as a spectrum access system (Spectrum Access System, SAS), etc., which are not restrictive.

[0039] exist Figure 1 In the scenario shown, the user and even the access point may be in a mobile state, so the dynamic selection of the user's cooperative APG helps to maintain stable and high-quality communication. To this end, this embodiment provides an electronic device 100 for wireless communication, such as Figure 2 As shown, the electronic device 100 includes: a determination unit 101, configured to determine a cooperative access point set (APG) for users within a predetermined range based on the wireless network topology structure of the wireless network as a state; and an update unit 102, configured to re-determine the cooperative access point set for the user in response to a change in the wireless network topology structure.

[0040] The determining unit 101 and the updating unit 102 may be implemented by one or more processing circuits, which may be implemented as a chip, for example. The electronic device 100 may be located at Figure 1On the spectrum management device (such as SC or SAS) shown in, or communicatively connected to the spectrum management device.

[0041] In this embodiment, the electronic device 100 may use a reinforcement learning algorithm to determine a cooperative APG for users within a predetermined range, wherein the predetermined range may be, for example, at least a portion of a management range of a spectrum management device where the electronic device is located.

[0042] The reinforcement learning algorithm regards learning as a trial and evaluation process, learning the mapping from environmental state to behavior, so that the selected behavior can obtain the maximum reward from the environment, that is, the external environment's evaluation of the learning system in a certain sense (or the operating performance of the entire system) is optimal. The reinforcement learning algorithm used in this article can include, for example, the Q-learning algorithm, the differential learning algorithm, etc., in which the wireless network topology can be used as a state.

[0043] In one example, the wireless network topology includes the distribution of users and the distribution of access points. In other words, when users and / or access points move, or when the switch status of specific users and / or access points changes, the wireless network topology changes. Figure 1 As shown in FIG. 1 , when the UE moves from bottom to top along the direction of the black dotted arrow, the wireless network topology changes, for example, corresponding to the state S shown in the figure. t , S t+1 , and S t+2 In this case, the cooperative APG of the user determined in the previous state may no longer be applicable in the new state, for example, it cannot meet the communication needs of the user. Therefore, the updating unit 102 re-determines the cooperative APG for the user in response to this change, thereby providing the user with stable and continuous communication services.

[0044] In one example, the change in the wireless network topology includes a change in the user's location, which is detected by the user. When the user detects the change, the user reports the change to the electronic device 100 and requests the electronic device 100 to redefine the collaborative APG for the user. In other examples, for example, the change in the network topology also includes a change in the location of the access point, and the access point also reports the change in its location to the electronic device 100. Accordingly, the electronic device 100 can also redefine the user's collaborative APG based on the change.

[0045] For example, the determination unit 101 may use the cooperative relationship between the user and the access point as a behavior in the reinforcement learning algorithm, and for each behavior, the evaluation of the behavior is calculated based on the degree of satisfaction of the user's communication quality requirements when the behavior is performed and the network overhead caused. Generally, the user has specific requirements for its communication quality. When a certain behavior is implemented, the degree of satisfaction of the user's communication quality requirements indicates one aspect of the evaluation of the behavior. The user's communication quality requirements can be represented by the quality of service (Quality of Service) required by the user, and specifically, as described below, it can be represented by a signal to interference and noise ratio (SINR) threshold. However, it should be understood that this is only an example and is not restrictive.

[0046] In addition, when changing from the previous state to the current state, the behavior also changes accordingly, for example, changing from a behavior determined in the previous state to a certain behavior, the change in behavior means a change in the cooperative APG of the UE, so an AP switching operation will occur, causing network overhead. In terms of the evaluation of the behavior, it is expected that the network overhead is as small as possible, so the network overhead indicates another aspect of the evaluation of the behavior.

[0047] In one example, the determination unit 101 determines the cooperative APG of the user in the current state based on the behavior with the highest evaluation. In other words, the determination unit 101 determines the behavior with the highest evaluation as the behavior to be implemented, thereby determining the cooperative APG of each user. For example, the behavior with the highest evaluation is the behavior that satisfies the user's communication quality requirements to the highest degree and brings the least network overhead when performing the behavior compared with other behaviors.

[0048] In the following, for ease of understanding, various aspects of the embodiment will be described using the Q-learning algorithm as an example. However, it should be understood that this is not restrictive, and other reinforcement learning algorithms may also be applicable to the present application.

[0049] Assuming that there are N users and M APs within a predetermined range, the cooperative relationship between the users and the access points, that is, the behaviors (also referred to as individuals) in the reinforcement learning algorithm, can be represented by the following matrix.

[0050]

[0051] Among them, a n，m (n=1, ..., N; m=1, ..., M) represents the cooperative relationship between the nth user and the mth AP. For example, when a n，m When it is 1, it means that there is a cooperative relationship between the nth user and the mth AP. n，m When it is 0, it indicates that there is no cooperative relationship between the nth user and the mth AP.

[0052] For ease of operation, (1) can also be changed into a vector form as shown in formula (2).

[0053] A i =[a 11 a 12 …a 1M a 21 a 22 …a 2M …a N1 a N2 …a NM ] 1×NM (2)

[0054] That is, the rows of equation (1) are rearranged into one row. When there are multiple behaviors, each behavior is regarded as a row to form a behavior matrix.

[0055] First, for a certain state, such as state S t , multiple behaviors can be generated initially, that is, multiple A with different values ​​can be generated i For example, a predetermined condition may be set to limit the generated behavior, and the predetermined condition may include one or more of the following: the generated behavior should make the communication quality of each user meet its communication quality requirements; the network overhead relative to the behavior determined in the previous state does not exceed a predetermined overhead threshold. For example, the communication quality requirement may be represented by an SINR threshold.

[0056] As mentioned above, the degree to which each user's communication quality requirements are met and the resulting network overhead when each behavior is implemented in this state is used as the evaluation of the behavior. In the Q-learning algorithm, the evaluation of the behavior is represented by Q-value, and the evaluation of each behavior constitutes a Q-value matrix.

[0057] Exemplarily, the determination unit 101 may calculate the degree of satisfaction of the communication quality requirements of the user by using the SINR threshold of each user and the estimated SINR of the corresponding user. When the estimated SINR of the corresponding user is closer to the SINR threshold of the user, the degree of satisfaction of the communication quality requirements of the user is higher. For example, the determination unit 101 may comprehensively consider the degree of satisfaction of the communication quality requirements of each user.

[0058] In one example, the degree of satisfaction of the communication quality requirements of the user includes the utility values ​​of all users and the cost value of the SINR of the unsatisfied user, wherein the utility value of the user is calculated by a utility function, the utility function is a nonlinear function of the ratio of the estimated SINR of the user to the SINR threshold of the user, and the cost value depends on the difference between the SINR threshold of the corresponding user and the estimated SINR. The utility value is used to indicate the degree of satisfaction of the SINR of the corresponding user relative to the SINR threshold, and the cost value is used to indicate the degree of deficiency of the SINR of the corresponding user relative to the SINR threshold.

[0059] For example, in state S t Implement Behavior A i When the user's communication quality requirements are met, R(S t , A i ) can be calculated using the following formula (3):

[0060]

[0061] Among them, U n is the utility value of the nth user, which is calculated by the user's utility function, for example, it can be calculated by the following formula (4); σ is the cost factor, is the SINR threshold of the nth user, SINR n is the estimated SINR of the nth user.

[0062]

[0063] Wherein, tanh() is the hyperbolic tangent function, ξ is the expansion factor (for example, 3.5834), and η is the symmetry center (for example, 0.8064). Figure 3 The curve of this utility function is shown as an example. Figure 3 As shown, when the SINR of a user exceeds the SINR threshold, the utility function curve changes relatively slowly and approaches 1, so as to avoid a user's SINR being too high and causing an excessively large R value. It should be understood that the utility function is not limited to the form shown in equation (4), but can be modified appropriately.

[0064] In the above calculation For example, it can be provided by the user, and SINR n Various communication system models can be used to estimate the obtained. As an example, the SINR can be calculated as follows: n :

[0065]

[0066] Among them, p j and p kare the power of the jth AP and the kth AP respectively, d nj and d nk are the distances from the nth user to the jth AP and the kth AP respectively, α is the path loss factor, Φ c(n) is the collaborative APG of the nth user, Φ I(n) is the interference APG of the nth user, n0 is the noise power at the user receiver, and the interference APG refers to the set of APs that provide communication access services to other users and thus cause interference to the nth user in question.

[0067] As shown in the above formulas (3) to (5), the determination unit 101 calculates the satisfaction degree of the user's communication quality requirements, which is equivalent to the reward value in the Q-learning algorithm. In the above calculation process, the user's location information, the location information and transmission power of the access point, and the user's communication quality requirements such as the SINR threshold are used.

[0068] In addition, the determination unit 101 may also be configured to use, for each behavior, the difference between the behavior and the behavior determined in the previous state as the network overhead caused by the behavior. For example, when the determination unit 101 determines the behavior with the highest evaluation as the behavior to be implemented, the behavior determined in the previous state is the behavior with the highest evaluation in the previous state. When the current state is the initial state, that is, when there is no previous state, the network overhead may be set to 0.

[0069] In an example, the determining unit 101 may take the amount of network switching operations to be performed when a certain behavior is implemented, compared with the behavior determined in the previous state, as the network overhead caused by the behavior.

[0070] As mentioned above, the behavior can be represented by a binary matrix of the cooperative relationship. In this case, the network overhead can be represented by the Hamming distance between behaviors, as shown in the following equation (6). In fact, when the behavior consists of 0 or 1, the physical meaning of the Hamming distance between behaviors is the number of cooperative AP switches between two APG selection schemes. In the Q-learning algorithm, this network overhead is equivalent to the cost value.

[0071]

[0072] in, is the previous state S t-1 The behavior determined by the implementation, σ is the cost factor, D ham () indicates the calculation of Hamming distance. As mentioned above, when the state S t When it is in the initial state, PH(S t , A i ) is set to 0.

[0073] In another example, the network overhead may be taken into consideration only when the network overhead caused by the behavior exceeds a predetermined overhead threshold. In this case, the following formula (7) may be used to calculate the network overhead:

[0074]

[0075] Among them, T d is the predetermined network overhead threshold, that is, the predetermined Hamming distance threshold. As shown in formula (7), only in A i and The Hamming distance between them is greater than T d The network cost is calculated only when the AP reaches 0, otherwise the network cost is regarded as 0. In the calculation, a predetermined network cost threshold is used, which can be provided by the AP.

[0076] Combining the above equations (3) and (7), the evaluation of the behavior can be calculated as follows to obtain the state S t The Q-value matrix Q(S t ), Q(S t ) is calculated as follows.

[0077] Q(S t , A i )=R(S t , A i )+PH(S t , A i ) (8)

[0078] Among them, the Q-value matrix Q(S t ) is a T×1 dimensional matrix, where T is the number of behaviors. t ), for example, we can choose the behavior corresponding to the largest Q-value, that is, the behavior with the highest evaluation, as the state S t In this case, the communication quality requirements of each user can be met as much as possible, while reducing the network overhead caused by AP switching.

[0079] It should be understood that the calculation process of the APG selection can be performed online in real time, or offline, or a combination of the two.

[0080] like Figure 4 As shown, the electronic device 100 may further include: a storage unit 103 configured to store, for each state, each behavior in the state in association with the evaluation calculated for the behavior as an evaluation matrix.

[0081] The storage unit 103 may be implemented by various memories. The evaluation may include, for example, the satisfaction degree of the communication quality requirements of the aforementioned user (for example, R(S t , A i )) and the network overhead caused by the execution behavior (for example, PH(S t , A i ))Two aspects.

[0082] It can be understood that after establishing such an evaluation matrix, the update unit 102 can be configured to determine the behavior to be adopted in the changed state based on the content of the evaluation matrix when the state changes and in the case where there is an evaluation matrix for the changed state. Specifically, the appropriate behavior in the state, such as the behavior with the highest evaluation, can be selected according to the current state. After the behavior is selected, the cooperative relationship between the UE and the AP is determined accordingly. In this way, the calculation load can be reduced, the processing speed can be improved, and fast and stable APG switching can be achieved when the user is moving.

[0083] On the other hand, when there is no evaluation matrix for the changed state, an evaluation matrix is ​​established for the changed state as described above.

[0084] In addition, the updating unit 102 can also be configured to update the evaluation of the behavior performed in the previous state stored in the storage unit 103 using information on the actual communication quality of the user when performing the determined behavior in the previous state when the state changes. The actual communication quality of the user is obtained by the user's measurement.

[0085] For example, the updating unit 102 may replace the stored estimated communication quality requirement satisfaction degree with the communication quality requirement satisfaction degree calculated based on the user's actual communication quality. i Change to S t+1 And in state S t The behavior determined below is A i In the case of, the updating unit 102 may replace the stored R(S) with the following equation (9), for example: t , A i ):

[0086]

[0087] in, is the actual SINR of the nth user, and in the calculation of U in formula (9) n Also used For example, when using equation (4) to calculate U n When the numerator of the tanh function is

[0088] By using the information of actual communication quality to update the evaluation matrix, when the actual communication quality corresponding to the behavior determined in a certain state is poor, if the state is subsequently returned to, the previously selected behavior will not be selected, which helps to improve the communication quality.

[0089] In another example, when updating the evaluation matrix, the correlation between the changed state, i.e., the current state and the previous state, may also be considered. For example, the updating unit 102 is configured to replace the part of the satisfaction degree of the user's communication quality requirements in the evaluation of the behavior performed in the previous state with the value calculated as follows: the weighted sum of the actual satisfaction degree of the user's communication quality requirements in the previous state and the highest satisfaction degree of the user's communication quality requirements estimated in the current state.

[0090] For example, in the state S t Change to S t+1 And in state S t The behavior determined below is A i In the case of, the updating unit 102 may replace the stored R(S) with the following equation (10), for example: t , A i ):

[0091]

[0092] Among them, R t+1 As shown in formula (9), For state S t+1 Find an action A that makes R(S t+1 , A) has the largest R value among all behaviors; γ is the discount factor, which indicates the correlation between the previous state and the current state; if γ = 0, it means that the R value is only related to the R value of the previous state.

[0093] In addition, more generally, the wireless network topology structure as a state may also include other variable parameters, such as one or more of the following: the communication quality requirement of the UE, the maximum transmit power of the AP, the predetermined network overhead threshold of the AP, etc. In other words, the change of these parameters may also cause the updating unit 102 to re-determine the APG, or update the stored evaluation of the behavior performed in the previous state.

[0094] In summary, the electronic device 100 according to this embodiment uses a reinforcement learning algorithm to determine the collaborative APG for different states, thereby enabling dynamic APG selection to better meet the communication quality requirements of all users. In addition, although the reinforcement learning algorithm is used as an example for description above, it is not limited to this, and other algorithms can also be used to determine the collaborative APG.

[0095] <Second Embodiment>

[0096] Figure 5 A functional module block diagram of an electronic device 200 for wireless communication according to another embodiment of the present application is shown. Figure 2 In addition to the various units shown, the electronic device 200 further includes a grouping unit 201 configured to obtain a behavior in each state by grouping access points based on users and selecting a cooperative APG for a corresponding user in the user's group.

[0097] Similarly, the grouping unit 201 may be implemented by one or more processing circuits, which may be implemented as a chip, for example. Figure 5 , but the electronic device 200 may also include a reference Figure 4 The storage unit 103 is shown.

[0098] For example, the grouping unit 201 can group users according to the Euclidean distance between the users and the access points. The following formula (11) shows the membership parameter value of the access point to the user calculated using the Euclidean distance:

[0099]

[0100] Among them, u j represents the jth UE, x i represents the i-th AP. In different states, the positions of AP and UE in the wireless network are different, and their membership parameter values ​​are also different. The smaller the Euclidean distance from AP to UE, the larger the membership parameter value. If the membership parameter value of AP to a UE is larger, the AP is assigned to the UE. In this way, groups of UEs are established.

[0101] The determination unit 101 randomly selects a set of cooperative access points for the corresponding user in the user group and takes the cooperative relationship between the user and the access point that meets the predetermined condition as a behavior. Similar to the first embodiment, the predetermined condition may include one or more of the following: the communication quality of each user meets its communication quality requirement; when the cooperative relationship is adopted, the network overhead relative to the behavior determined in the previous state does not exceed a predetermined overhead threshold.

[0102] The difference between this embodiment and the first embodiment is that the behavior is generated differently. For example, when a binary matrix is ​​used to represent the behavior, in this embodiment, the bits corresponding to APs other than the UE's group are all set to values ​​that do not have a cooperative relationship (eg, 0).

[0103] Therefore, the electronic device 200 according to this embodiment can narrow the selection range of the user's cooperative AP by including the grouping unit 201, thereby easily obtaining more reasonable behavior, improving the selection accuracy and reducing the calculation load.

[0104] <Third Embodiment>

[0105] Figure 6 A functional module block diagram of an electronic device 300 for wireless communication according to another embodiment of the present application is shown. Figure 2 In addition to the various units shown, the electronic device 300 further includes: an estimating unit 301 configured to estimate a new behavior based on the initially obtained behavior for each state.

[0106] Similarly, the estimation unit 301 may be implemented by one or more processing circuits, which may be implemented as a chip, for example. Figure 6 , but the electronic device 300 may also include a reference Figure 4 The storage unit 103 shown, referring to Figure 5 The grouping unit 201, etc.

[0107] In the first embodiment and the second embodiment, the behavior is initially generated by randomly selecting APs for the user. In this embodiment, in order to improve efficiency, a new behavior can be further estimated based on the initially obtained behavior.

[0108] For example, the estimation unit 301 may use a genetic algorithm (GA) to estimate the new behavior.

[0109] Specifically, the estimation unit 301 may select N from the initially obtained behaviors. p The behaviors with better R values ​​constitute the initial population of the genetic algorithm. The network fitness matrix of the initial population is calculated. The network fitness matrix of the population is obtained according to the Q-value of each behavior, as shown in the following formula (12).

[0110]

[0111] Among them, P i is the i-th individual in the population, i.e., the i-th behavior, Δ is a value close to 0, Q(S t , P i ) is in S t P in state i The corresponding Q-value.

[0112] Next, a selection operation is performed, such as the roulette wheel selection method, to calculate the probability of each individual appearing in the offspring based on the network fitness value of the individuals in the initial population, and randomly select N according to this probability. p individuals constitute the offspring population, with probability p i As shown in the following formula (13):

[0113]

[0114] Then perform a crossover operation and randomly select two individuals A from the resulting offspring population. m and A n , randomly select multiple points for multi-point crossover to generate new individuals or populations. For example, the mth individual A m and the nth individual A n The crossover operation at position i is shown in equation (14):

[0115]

[0116] It should be understood that the individuals in formula (14) are merely illustrative and do not constitute a limitation to the present application.

[0117] Next, a mutation operation is performed. An individual is randomly selected from the population obtained after the crossover operation, and a point in the individual is randomly selected for mutation to produce a better individual. Since the chromosome of an individual is 0 or 1, the mutation operation is to mutate chromosome 0 to 1, or 1 to 0. In this way, a new individual, that is, a new behavior, is obtained.

[0118] The estimation unit 301 may repeatedly perform the selection operation, the execution operation and the mutation operation to generate a plurality of new behaviors. For example, the number of repeated operations may be preset.

[0119] In one example, the estimation unit 301 is further configured to use the behavior estimated by the genetic algorithm as a new behavior only when the behavior meets a predetermined condition. Similarly, the predetermined condition may include one or more of the following: the communication quality of each user meets its communication quality requirements; when the behavior is adopted, the network overhead relative to the behavior determined in the previous state does not exceed a predetermined overhead threshold.

[0120] The new behaviors obtained above are added to the behaviors obtained initially to form a new behavior set. The determination unit 101 uses a reinforcement learning algorithm to determine the evaluation of the behavior (for example, the Q-value described in the first embodiment), thereby selecting the behavior with the highest evaluation as the behavior to be implemented in the current state to determine the collaborative APG of each user.

[0121] The electronic device 300 according to the present embodiment obtains new behaviors through an estimation method such as a genetic algorithm, thereby expanding the behavior set, so that the optimal cooperative APG can be determined more accurately.

[0122] <Fourth Embodiment>

[0123] Figure 7 A functional module block diagram of an electronic device 400 for wireless communication according to another embodiment of the present application is shown. Figure 2 In addition to the various units shown, the electronic device 400 also includes: a transceiver unit 401, which is configured to receive the user's location information and one or more of the communication quality requirements, receive the location information of the access point, the maximum transmission power information and one or more of the predetermined network overhead thresholds, and send the determined cooperative access point set information to the access point.

[0124] The transceiver unit 401 may be implemented, for example, by a communication interface. The communication interface may include, for example, a network interface, or an antenna and a transceiver circuit. Figure 7 , but the electronic device 400 may also include a reference Figure 4 The storage unit 103 shown, referring to Figure 5 The grouping unit 201, reference Figure 6 The described estimation unit 301 etc.

[0125] The above information received by the transceiver unit 401 is used to determine and update the user's cooperative APG. For example, when the wireless network topology structure as a state changes, the transceiver unit 401 will re-acquire the above information.

[0126] In addition, the transceiver unit 401 is also configured to receive information about the user's actual communication quality. For example, when the state changes, the user reports to the electronic device 400 the actual communication quality such as the actual SINR and utility value obtained by performing the determined behavior in the state before the change.

[0127] The user's location information and communication quality requirements may be provided to the transceiver unit 401 via an access point, or may be provided directly to the transceiver unit 401 .

[0128] For ease of understanding, Figure 8 The schematic diagram shows the information flow between the user (UE), the access point (AP) and the spectrum management device when the electronic device 400 is set on the spectrum management device (such as SC or SAS).

[0129] First, the UE requests the AP for cooperative communication from the spectrum management device, and reports its location information and information on communication quality requirements, such as the SINR threshold. The AP reports its location information, maximum transmit power information, and a predetermined network overhead threshold to the spectrum management device. When the AP is located at a fixed position, the AP can report its location information only when the system is initialized. As mentioned above, the UE can report relevant information directly to the spectrum management device, or it can report relevant information via the AP. In the latter case, the information reported by the AP also includes the user's location information and information on communication quality requirements.

[0130] After acquiring the above-mentioned various information, the spectrum management device selects the user's cooperative APG. Specifically, the spectrum management device can use the Q-learning reinforcement learning algorithm specifically described in the first embodiment to select the behavior with the largest Q-value, thereby determining the cooperative APG of each user. It should be noted that in the case where the evaluation matrix for multiple states is stored in the spectrum management device, if the current state is also included in the stored state, the stored evaluation matrix can be used to select the behavior without having to repeatedly execute the reinforcement learning algorithm.

[0131] Next, the spectrum management device sends the information of the determined cooperative APG to the AP, so that the AP can cooperate with the UE based on the information.

[0132] exist Figure 8 In the example, it is assumed that only the location of the UE changes. Therefore, the UE, for example, periodically determines whether its location has changed. When the location changes or changes to a certain extent, it means that the wireless network topology has changed, and the UE needs to re-request the collaborative APG. At this time, the UE provides the spectrum management device with its changed location information. In addition, the UE also provides the spectrum management device with the actual utility value and SINR value when it performs the determined behavior in the state before the change. The spectrum management device updates the Q-value of the behavior determined in the previous state based on the actual utility value and SINR value provided by the UE. In addition, the spectrum management device also reselects the behavior to be performed in the current state based on the current location information of the UE. For example, the selection can be made by executing the Q-learning reinforcement learning algorithm as described above. Alternatively, in the case where an evaluation matrix for the current state is stored in the spectrum management device, the behavior to be performed can be selected by searching the evaluation matrix. Similarly, the spectrum management device sends the information of the determined collaborative APG to the AP so that the AP can collaborate with the UE based on the information.

[0133] It should be understood that Figure 8 The information flows shown are illustrative only and are not intended to be limiting.

[0134] Below, in order to further illustrate the details and effects of the technology of the present application, two simulation examples of applying the technology of the present application will be given. Figures 9 to 13 Describe the first simulation example. Fig. 9 A schematic diagram of a simulation scenario of the simulation example is shown, wherein a triangle represents a UE, a square represents an AP, and a dotted line and an arrow indicate a movement trajectory of one of the UEs. Fig. 9 Four different positions of the UE are shown in FIG. 1 , representing states S1 , S2 , S3 and S4 , respectively.

[0135] The parameters used in the simulation are listed as follows: operating frequency, 3.5GHz; channel bandwidth, 10MHz; number of UEs, 3; transmit power, 0dBm; number of APs, 16; UE SINR threshold, 7dB; noise coefficient at UE receiver, 5dB; number of population evolutions in the genetic algorithm, 10; crossover ratio, 0.7; mutation ratio, 0.1; number of individuals, 10; Hamming distance threshold, 5.

[0136] In state S1, the UE uploads the location information and communication quality requirement information to the spectrum management device, and the AP uploads the location information, maximum transmit power information, and Hamming distance threshold to the spectrum management device. The spectrum management device generates some preliminary behaviors and uses a genetic algorithm to generate new behaviors based on these preliminary behaviors. The preliminary behaviors and new behaviors constitute a behavior matrix Fig.10 The left side shows the behavior matrix in state S1 In the example of , each row represents a behavior, that is, a cooperative relationship between AP and UE, and there are 18 behaviors in total. Each behavior is represented by a 48-bit binary sequence, where there are M (in this example, M=16) APs, and bits 1 to M represent the cooperative relationship between user 1 and AP, bits M+1 to 2M represent the cooperative relationship between user 2 and AP, and so on.

[0137] The spectrum management matrix is ​​generated using the aforementioned Q-learning algorithm. The corresponding Q-value matrix like Fig.10 The Q-value matrix is ​​calculated using the above equations (3) to (5) and (7) to (8). In this simulation example, state S1 is the initial state. is a zero matrix.

[0138] The spectrum management device selects the behavior corresponding to the maximum value in the Q-value matrix, such as behavior 15, and informs the AP to cooperate with the UE based on the behavior. Fig.11 The schematic diagram shows the result of executing behavior 15. The UE and AP circled by the same line type have a cooperative relationship.

[0139] Next, due to the movement of the UE, the state changes to S2. The UE uploads the new location information, the actual SINR and utility value obtained by executing behavior 15 in state S1 to the spectrum management device. The spectrum management device calculates the actual communication quality satisfaction degree obtained by executing behavior 15 based on this information using formula (9), and updates as shown in formula (10) The value of R_15 in . In formula (10), γ is set to 0.

[0140] The spectrum management device uses a genetic algorithm to update the behavior matrix in state S1 to obtain the behavior matrix in state S2 like Fig.12 Similarly, the spectrum management device generates the same The corresponding Q-value matrix like Fig.12 The Q-value matrix is ​​calculated using the above equations (3) to (5) and (7) to (8).

[0141] The spectrum management device selects the behavior corresponding to the maximum value in the Q-value matrix, such as behavior 11, and informs the AP to cooperate with the UE based on the behavior. Fig.13 The schematic diagram shows the result of executing behavior 11. The UE and AP circled by the same line type have a cooperative relationship.

[0142] When the state continues to change to state S3 and S4, the spectrum management device performs operations similar to those in state S2, which will not be described in detail here.

[0143] Refer to the following Figures 14 to 17 The second simulation example is described. Fig.14 and Fig.15 Two simulation scenarios of the simulation example are shown, wherein the dotted line and arrows show the movement trajectory of one of the UEs. Fig.14 In the simulation scenario 1 shown, the UE moves back and forth along the dotted line, so the state changes from S1→S9→S1. Fig.15 In the simulation scenario 2 shown, the UE moves in a circular motion along the rectangle formed by the dotted line. In both scenarios, it is assumed that the positions of the remaining UEs and APs remain unchanged. The initial state is S1 at t=0, and the remaining states are deduced in turn according to the positions of the UEs.

[0144] The parameters used in the simulation are listed as follows: operating frequency, 28 GHz; channel bandwidth, 10 MHz; number of UEs, 6; transmit power, 0 dBm; number of APs, 60; UE SINR threshold, 7 dB; noise factor at UE receiver, 5 dB; number of population evolutions in the genetic algorithm, 10; crossover ratio, 0.7; mutation ratio, 0.1; number of individuals, 10; beam width, π / 4; the Hamming distance threshold is 5 in simulation scenario 1 and 10 in simulation scenario 2.

[0145] In addition to the APG selection based on the reinforcement learning algorithm proposed in this application, for comparison, a simulation of APG selection of the following comparison algorithm is also performed for simulation scenario 1: a genetic algorithm is used to obtain new behaviors, but the behavior is determined only based on the switching threshold reselected by the APG, i.e., the Hamming distance threshold Td. Fig.16The figure shows a comparison of the cumulative distribution function (CDF) of the user satisfaction rate obtained based on simulation scenario 1. The solid line is the CDF curve corresponding to the reinforcement learning algorithm, and the two curves from top to bottom are the CDF curve corresponding to the comparison algorithm with a Hamming distance threshold of 5 and the CDF curve corresponding to the comparison algorithm with a Hamming distance threshold of 20. It can be seen that the performance of the algorithm based on reinforcement learning is better than that of the comparison algorithm.

[0146] Fig.17 The figure shows the proportion of UEs that meet user communication quality requirements, such as QoS requirements, under different numbers of circles when the UE moves along a rectangular track in simulation scenario 2. It can be seen that as the number of circles increases, the UE satisfaction rate also increases, that is, the effect of the reinforcement learning algorithm becomes more and more significant over time.

[0147] <Fifth Embodiment>

[0148] In the process of describing the electronic device for wireless communication in the above embodiments, it is obvious that some processes or methods are also disclosed. Below, an overview of these methods is given without repeating some of the details discussed above, but it should be noted that although these methods are disclosed in the process of describing the electronic device for wireless communication, these methods do not necessarily use the components described or are not necessarily performed by those components. For example, the embodiments of the electronic device for wireless communication can be partially or completely implemented using hardware and / or firmware, and the methods for wireless communication discussed below can be completely implemented by a computer executable program, although these methods can also use the hardware and / or firmware of the electronic device for wireless communication.

[0149] Fig.18 A flow chart of a method for wireless communication according to an embodiment of the present application is shown. Fig.18 As shown, the method includes: taking the wireless network topology structure of the wireless network as a state, determining a cooperative APG for a user within a predetermined range (S12); and re-determining a cooperative APG for the user in response to a change in the wireless network topology structure (S17), wherein the wireless network topology structure may include the distribution of users and the distribution of access points. Exemplarily, a reinforcement learning algorithm may be used to determine the cooperative APG in step S12.

[0150] In step S12, the cooperative relationship between the user and the access point is used as a behavior in the reinforcement learning algorithm, and for each behavior, the evaluation of the behavior is calculated based on the satisfaction degree of the user's communication quality requirements when the behavior is performed and the resulting network overhead. For example, the cooperative APG of the user in the current state is determined based on the behavior with the highest evaluation. The behavior with the highest evaluation can be the behavior that, compared with other behaviors, satisfies the user's communication quality requirements to the highest degree and has the lowest network overhead when the behavior is performed.

[0151] In one example, the degree of satisfaction of the user's communication quality requirements is calculated using the signal interference and noise ratio threshold of each user and the estimated signal interference and noise ratio of the corresponding user. The degree of satisfaction of the user's communication quality requirements may include the utility values ​​of all users and the cost value of the signal interference and noise ratio of the user that does not meet the requirements, wherein the user's utility value is calculated by a utility function, the utility function is a nonlinear function of the ratio of the estimated signal interference and noise ratio of the user and the signal interference and noise ratio threshold of the corresponding user, and the cost value depends on the difference between the signal interference and noise ratio threshold of the corresponding user and the estimated signal interference and noise ratio of the user.

[0152] In addition, for each behavior, the difference between the behavior and the behavior determined in the previous state can be used as the network overhead caused by the behavior. The behavior can be represented by a binary matrix of the cooperation relationship, and the network overhead can be represented by the Hamming distance between behaviors. The network overhead can be taken into consideration only when the network overhead caused by performing the behavior exceeds a predetermined overhead threshold.

[0153] In addition, if Fig.18 As shown in the dotted box in , the method may further include the following steps: receiving one or more of the user's location information and the communication quality requirement, and one or more of the access point's location information, maximum transmission power information, and a predetermined network overhead threshold (S11), and sending the determined cooperative access point set information to the access point (S13). The information received in step S11 is used for the calculation in step S12.

[0154] The method may further include step S14: for each state, storing each behavior in the state in association with the evaluation calculated for the behavior as an evaluation matrix. In this way, when the state changes, if there is an evaluation matrix for the changed state, the behavior to be adopted in the changed state is determined based on the content of the evaluation matrix.

[0155] In addition, the method further comprises step S15: receiving information about the actual communication quality of the user when the state changes. Further, the method further comprises step S16: using the information about the actual communication quality of the user when performing the determined behavior in the previous state to update the stored evaluation of the behavior performed in the previous state, that is, to update the content of the evaluation matrix.

[0156] For example, the part concerning the degree of satisfaction of the user's communication quality requirements in the evaluation of the behavior performed in the previous state can be replaced by the value calculated as follows: the weighted sum of the actual degree of satisfaction of the user's communication quality requirements in the previous state and the maximum degree of satisfaction of the user's communication quality requirements estimated in the current state.

[0157] In addition, although Fig.18Although not shown in the figure, the above method may further include the following steps: in each state, the behavior is obtained by grouping the access points with the user as the center and selecting a set of cooperative access points for the corresponding user in the user's group. For example, the grouping may be performed according to the Euclidean distance between the user and the access point. In this case, a set of cooperative access points is randomly selected for the corresponding user in the user's group and the cooperative relationship between the user and the access point that meets the predetermined condition is used as the behavior. The predetermined condition may include, for example, one or more of the following: the communication quality of each user meets its communication quality requirement; when the cooperative relationship is adopted, the network overhead relative to the behavior determined in the previous state does not exceed a predetermined overhead threshold.

[0158] In addition, when acquiring behaviors, new behaviors can be estimated for each state based on the behaviors initially acquired. For example, a genetic algorithm can be used to estimate new behaviors. It is also possible to use the behaviors estimated by the genetic algorithm as new behaviors only when the behaviors meet the above-mentioned predetermined conditions.

[0159] Note that the details of the above method have been described in detail in the first to fourth embodiments and will not be repeated here.

[0160] The technology of the present disclosure can be applied to various products. For example, the electronic devices 100 to 400 can be implemented as any type of server, such as a tower server, a rack server, and a blade server. The electronic devices 100 to 400 can be a control module installed on the server (such as an integrated circuit module including a single chip, and a card or blade inserted into a slot of a blade server).

[0161] [Application examples about servers]

[0162] Fig.19 700 is a block diagram showing an example of a schematic configuration of a server 700 to which the technology of the present disclosure can be applied. The server 700 includes a processor 701 , a memory 702 , a storage device 703 , a network interface 704 , and a bus 706 .

[0163] The processor 701 may be, for example, a central processing unit (CPU) or a digital signal processor (DSP), and controls functions of the server 700. The memory 702 includes a random access memory (RAM) and a read-only memory (ROM), and stores data and programs executed by the processor 701. The storage device 703 may include a storage medium such as a semiconductor memory and a hard disk.

[0164] The network interface 704 is a communication interface for connecting the server 700 to the communication network 705. The communication network 705 may be a core network such as an evolved packet core (EPC) or a packet data network (PDN) such as the Internet.

[0165] The bus 706 connects the processor 701, the memory 702, the storage device 703, and the network interface 704 to each other. The bus 706 may include two or more buses each having a different speed (such as a high-speed bus and a low-speed bus).

[0166] exist Fig.19 In the server 700 shown, refer to Figure 2 , Figure 5 and Figure 6 The determination unit 101, the updating unit 102, the grouping unit 201, the estimation unit 301, etc. described above may be implemented by the processor 701. Figure 4 The storage unit 103 described above can be implemented by, for example, the memory 702 or the storage device 703, see Figure 7 The transceiver unit 401 described may be implemented by the network interface 704, for example, and part of its functions may also be implemented by the processor 701. For example, the processor 701 may perform the selection and update of the cooperative APG by executing the functions of the determination unit 101, the update unit 102, and the like.

[0167] The basic principles of the present invention are described above in conjunction with specific embodiments. However, it should be pointed out that for those skilled in the art, it is understandable that all or any steps or components of the method and device of the present invention can be implemented in any computing device (including a processor, storage medium, etc.) or a network of computing devices in the form of hardware, firmware, software, or a combination thereof. This can be achieved by those skilled in the art using their basic circuit design knowledge or basic programming skills after reading the description of the present invention.

[0168] Furthermore, the present invention also provides a program product storing machine-readable instruction codes. When the instruction codes are read and executed by a machine, the method according to the embodiment of the present invention can be executed.

[0169] Accordingly, the storage medium for carrying the program product storing the machine-readable instruction code is also included in the disclosure of the present invention, including but not limited to a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

[0170] When the present invention is implemented by software or firmware, the data is transferred from a storage medium or a network to a computer (eg, Fig. 20 The general-purpose computer 2000 shown in the figure is installed with the programs constituting the software, and when the various programs are installed, the computer can execute various functions, etc.

[0171] exist Fig. 20In the embodiment of the present invention, a central processing unit (CPU) 2001 performs various processes according to a program stored in a read-only memory (ROM) 2002 or a program loaded from a storage section 2008 to a random access memory (RAM) 2003. In the RAM 2003, data required when the CPU 2001 performs various processes and the like is also stored as needed. The CPU 2001, the ROM 2002, and the RAM 2003 are connected to each other via a bus 2004. An input / output interface 2005 is also connected to the bus 2004.

[0172] The following components are connected to the input / output interface 2005: an input section 2006 (including a keyboard, a mouse, etc.), an output section 2007 (including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), a storage section 2008 (including a hard disk, etc.), a communication section 2009 (including a network interface card such as a LAN card, a modem, etc.). The communication section 2009 performs communication processing via a network such as the Internet. A drive 2010 may also be connected to the input / output interface 2005 as needed. A removable medium 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive 2010 as needed, so that a computer program read therefrom is installed in the storage section 2008 as needed.

[0173] In the case where the above-described series of processing is realized by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 2011 .

[0174] It should be understood by those skilled in the art that such storage media is not limited to Fig. 20 The removable medium 2011 shown has a program stored therein and is distributed separately from the device to provide the program to the user. Examples of the removable medium 2011 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including minidiscs (MD) (registered trademark)), and semiconductor memories. Alternatively, the storage medium may be a ROM 2002, a hard disk included in the storage section 2008, or the like, in which the program is stored and distributed to the user together with the device containing them.

[0175] It should also be noted that in the apparatus, method and system of the present invention, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present invention. In addition, the steps of performing the above series of processes can naturally be performed in chronological order according to the order of description, but do not necessarily need to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0176] Finally, it should be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or device. In addition, in the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or device that includes the elements.

[0177] Although the embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, it should be understood that the above-described embodiments are only used to illustrate the present invention and do not constitute a limitation of the present invention. For those skilled in the art, various modifications and changes can be made to the above-mentioned embodiments without departing from the essence and scope of the present invention. Therefore, the scope of the present invention is limited only by the attached claims and their equivalent meanings.

Claims

1. An electronic device for wireless communication, comprising: The processing circuit is configured to: Determining a set of cooperative access points for users within a predetermined range using a wireless network topology of the wireless network as a state; as well as In response to the change of the wireless network topology, re-determine a set of cooperative access points for the user, The wireless network topology structure includes the distribution of users and the distribution of access points. The processing circuit is configured to, for a plurality of cooperation relationships between the user and each access point in the distribution of the access points, take each cooperation relationship as a corresponding behavior, and for each behavior, calculate an evaluation of the behavior based on the satisfaction degree of the communication quality requirement of the user when the behavior is performed and the network overhead caused, and the processing circuit is configured to determine the cooperation access point set of the user in the current state based on the behavior with the highest evaluation, The electronic device further includes a memory configured to store, for each state, each behavior in the state in association with the evaluation calculated for the behavior as a corresponding evaluation matrix, and The processing circuit is configured to update the evaluation of the behavior performed in the previous state stored in the memory by using information on the actual communication quality of the user when the determined behavior is performed in the previous state when the state changes.

2. The electronic device according to claim 1, wherein: The behavior with the highest evaluation is the behavior that satisfies the user's communication quality requirements to the highest degree and brings the least network overhead when the behavior is performed, compared with other behaviors.

3. The electronic device according to claim 1, wherein: The processing circuit is configured to determine the satisfaction degree of the communication quality requirement of each user by using the signal to interference plus noise ratio threshold value of the user and the estimated signal to interference plus noise ratio of the user.

4. The electronic device according to claim 3, wherein: The satisfaction degree of the communication quality requirement of the user includes the utility value of all users and the cost value of the signal to interference plus noise ratio of the user that does not meet the requirement, wherein the utility value of the user is calculated by a utility function, the utility function is a nonlinear function of the ratio of the estimated signal to interference plus noise ratio and the signal to interference plus noise ratio threshold, and the cost value depends on the difference between the signal to interference plus noise ratio threshold of the corresponding user and the estimated signal to interference plus noise ratio of the user.

5. The electronic device according to claim 1, wherein: The processing circuit is configured to use, for each behavior, a difference between the behavior and a behavior determined in a previous state as a network cost caused by the behavior, wherein the network cost is represented by a Hamming distance between behaviors.

6. The electronic device according to claim 1, wherein: The network cost of performing the action is taken into consideration when the network cost exceeds a predetermined cost threshold.

7. The electronic device according to claim 1, wherein: The processing circuit is further configured to, when the state changes, determine a behavior to be adopted in the changed state based on content of the evaluation matrix if there is an evaluation matrix for the changed state.

8. The electronic device according to claim 1, wherein: The processing circuit is configured to replace the part of the evaluation of the behavior performed in the previous state regarding the degree of satisfaction of the user's communication quality requirements with a value calculated as follows: the weighted sum of the actual degree of satisfaction of the user's communication quality requirements in the previous state and the maximum degree of satisfaction of the user's communication quality requirements estimated in the current state.

9. The electronic device according to claim 1, wherein: The processing circuit is configured to: in a state, obtain the behavior by grouping the access points based on users and selecting a set of cooperative access points for corresponding users within the user group.

10. The electronic device according to claim 9, wherein: The processing circuit is configured to perform the grouping according to the Euclidean distance between the users and the access points.

11. The electronic device according to claim 9, wherein: The processing circuit is configured to randomly select a set of cooperative access points for a corresponding user in a user group and use a cooperative relationship between a user and an access point that meets a predetermined condition as the behavior.

12. The electronic device according to claim 11, wherein: The predetermined conditions include one or more of the following: the communication quality of each user meets its communication quality requirements; when the cooperative relationship is adopted, the network overhead relative to the behavior determined in the previous state does not exceed a predetermined overhead threshold.

13. The electronic device according to claim 1, wherein: The processing circuit is further configured to estimate a new behavior for a state based on the initially obtained behavior.

14. The electronic device according to claim 13, wherein: The processing circuit is configured to estimate a new behavior using a genetic algorithm, and the processing circuit is configured to use the behavior estimated by the genetic algorithm as the new behavior only when the behavior satisfies a predetermined condition.

15. The electronic device according to claim 1, further comprising: The transceiver unit is configured to receive one or more of the user's location information and communication quality requirements, and one or more of the access point's location information, maximum transmit power information, and a predetermined network overhead threshold, and to send information about the determined collaborative access point set to the access point.

16. The electronic device according to claim 15, wherein: The transceiver unit is further configured to receive information about actual communication quality of the user.

17. A method for wireless communication, comprising: Determining a set of cooperative access points for users within a predetermined range using a wireless network topology of the wireless network as a state; as well as In response to the change of the wireless network topology, re-determine a set of cooperative access points for the user, The wireless network topology structure includes the distribution of users and the distribution of access points. The step of determining the cooperative access point set for the user comprises: for a plurality of cooperative relationships between the user and each access point in the distribution of the access points, taking each cooperative relationship as a corresponding behavior; for each behavior, calculating an evaluation of the behavior based on the satisfaction degree of the communication quality requirement of the user when the behavior is performed and the resulting network overhead; and determining the cooperative access point set of the user in the current state based on the behavior with the highest evaluation, The method also includes: for each state, storing each behavior in the state in association with the evaluation calculated for the behavior in a memory as a corresponding evaluation matrix; when the state changes, using information on the actual communication quality of the user when the determined behavior is performed in the previous state to update the evaluation of the behavior performed in the previous state stored in the memory.

18. A computer-readable storage medium comprising computer-executable instructions which, when executed by a processor, cause the processor to perform the method of claim 17.

19. A computer program product comprising a computer program / instructions, wherein: When the computer program / instructions are executed by a processor, the steps of the method of claim 17 are implemented.

Images