Beam scanning method and communication device

Abstract

The invention provides a beam scanning method and a communication device, and the method comprises the steps: determining a second codebook set from a first codebook set based on a beam selection model, and then carrying out the beam scanning according to a beam corresponding to the second codebook set. The beam selection model assists the network device in executing beam scanning, so that the network device can complete coverage of all terminal devices in the coverage area within the preset time range, and the beam scanning efficiency is improved. Wherein the beam scanning model can be a reinforcement learning model, the reinforcement learning model can be used for learning to obtain beams with good coverage performance, and the flexibility of beam scanning is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technology, and in particular to a beam scanning method and a communication device. Background Art

[0002] The propagation of wireless signals will reduce the signal coverage due to the existence of path loss. In order to increase the signal coverage, beamforming technology is introduced. Beamforming technology can focus the beam in a precise spatial direction, thereby effectively combating path loss. In the new radio (NR) system, the initial access (IA) process includes the process of beam scanning using a synchronization signal block (SSB). The network device can scan a beam in one direction in each time slot. Each beam contains an SSB. The terminal device can measure the reference signal received power (RSRP) of each beam and determine the beam pair used for subsequent random access and data transmission based on the measurement results. However, the current method of beam scanning is an exhaustive scanning method, that is, in the IA process, the network device will perform beam scanning within the full beam range in chronological order. This beam scanning method is inefficient. Therefore, how to efficiently perform beam scanning is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a beam scanning method and a communication device, which can perform beam scanning based on a beam selection model to achieve coverage of all terminal devices in the coverage area within a preset time range, which is beneficial to improving beam scanning efficiency.

[0004] In a first aspect, an embodiment of the present application provides a beam scanning method, which can be executed by a network device, or by a device matched with the network device, such as a processor, a chip, or a chip system. The method may include: based on a beam selection model, determining a second codebook set from a first codebook set, the beam corresponding to the second codebook set performs beam scanning within M time units and can cover terminal devices within a preset range; M is less than or equal to a first threshold; the first codebook set is included in a predefined beamforming codebook set; in M ​​time units, the beam performing beam scanning in the i-th time unit is different from the beam performing beam scanning in the i+j-th time unit; i is a positive integer less than or equal to M, i+j is a positive integer less than or equal to M, and j is a positive integer; beam scanning is performed according to the beam corresponding to the second codebook set.

[0005] In the present application, the network device can obtain a second codebook set based on a beam selection model, and then perform beam scanning based on the beam corresponding to the second codebook set to complete coverage of all terminal devices in the coverage area within a certain time range. At the same time, it can avoid repeated scanning of the beam, achieve efficient and intelligent execution of beam scanning, and help improve the flexibility and efficiency of beam scanning.

[0006] In combination with the first aspect, in a possible implementation, the method further includes: performing dimensionality reduction processing on the initial codebook set according to the correlation between the codebooks in the initial codebook set to obtain a first codebook set; the initial codebook set is a predefined beamforming codebook set, and the number of codebooks in the second codebook set is less than the number of codebooks in the first codebook set. It can be seen that by performing dimensionality reduction processing on the initial codebook set, the search space of the beam can be reduced, which is conducive to improving the efficiency of beam scanning.

[0007] In combination with the first aspect, in a possible implementation, the initial codebook set includes N codebooks, each of the N codebooks includes S codewords, N is a positive integer greater than 1, and S is a positive integer; the initial codebook set is subjected to dimensionality reduction processing according to the correlation between the codebooks in the initial codebook set to obtain the first codebook set, including: according to the difference between the S codewords in the initial codebook set and the correlation between the first codebook and the second codebook, the initial codebook set is subjected to dimensionality reduction processing to obtain the first codebook set; the first codebook and the second codebook are codebooks in the N codebooks, and the first codebook is different from the second codebook. It can be seen that the initial codebook set can be subjected to dimensionality reduction processing more effectively by combining the difference between the codewords and the correlation between the codebooks.

[0008] In combination with the first aspect, in a possible implementation, the initial codebook set is subjected to dimensionality reduction processing according to the difference between the S codewords in the initial codebook set and the correlation between the first codebook and the second codebook to obtain the first codebook set, including: when the difference between the first codeword and the second codeword in the first codebook is greater than the second threshold value and the correlation between the first codebook and the second codebook is less than the third threshold value, the first codebook is used as a codebook in the first codebook set; the first codeword and the second codeword are the two codewords with the smallest difference among the S codewords in the first codebook. It can be seen that by using a codebook with a large difference between codewords and a small correlation between codebooks as a codebook in the first codebook set, it is conducive to determining a more reasonable first codebook set.

[0009] In combination with the first aspect, in a possible implementation, the method further includes: obtaining a second threshold and a third threshold from the threshold configuration information. It can be seen that the second threshold and the third threshold can be pre-configured, which is conducive to improving the flexibility of the dimensionality reduction process.

[0010] In combination with the first aspect, in a possible implementation, the above-mentioned determining the second codebook set from the first codebook set based on the beam selection model includes: determining the second codebook set from the first codebook set based on codebook state information, system feedback information and the beam selection model; the codebook state information is used to indicate the number of times the beam corresponding to the first codebook set has performed beam scanning, and the system feedback information is used to indicate the coverage performance of the beam corresponding to the first codebook set performing beam scanning. It can be seen that combining the codebook state information and the system feedback information can make the beam scanning efficiency of the beam corresponding to the determined first codebook set higher.

[0011] In combination with the first aspect, in a possible implementation, the codebook state information includes first codebook state information corresponding to the first time unit and second codebook state information corresponding to the second time unit, and the beam selection model includes a first neural network and a second neural network; the above-mentioned determination of the second codebook set from the first codebook set based on the codebook state information, the system feedback information and the beam selection model includes: determining a reference codebook set from the first codebook set based on the first codebook state information, the first network parameter and the first neural network; obtaining system feedback information based on the reference codebook set; adjusting the first network parameter based on the system feedback information and the second neural network to obtain the second network parameter; determining the second codebook set from the first codebook set based on the second codebook state information, the second network parameter and the first neural network. It can be seen that the second codebook set can be dynamically learned by using the codebook state information and the system feedback information through the first neural network and the second neural network, which is conducive to improving the intelligence of beam scanning.

[0012] In combination with the first aspect, in a possible implementation, adjusting the first network parameters based on the system feedback information and the second neural network to obtain the second network parameters includes: calculating the reward data using the reward function based on the system feedback information; adjusting the first network parameters based on the reward data and the second neural network to obtain the second network parameters. The reward function can enable beams with high beam scanning efficiency to obtain larger reward data, which is conducive to the beam selection model selecting beams with higher scanning efficiency.

[0013] In combination with the first aspect, in a possible implementation manner, the system feedback information includes the signal strength of the beam corresponding to the reference codebook set, or the system feedback information includes the signal-to-noise ratio of the beam corresponding to the reference codebook set, or the reference signal received power of the beam corresponding to the reference codebook set, or the quality of service of the beam corresponding to the reference codebook set, or the access delay information of the beam corresponding to the reference codebook set.

[0014] In combination with the first aspect, in a possible implementation, the obtaining of system feedback information based on the reference codebook set includes: transmitting a beam corresponding to the reference codebook set to a terminal device; and obtaining system feedback information from the terminal device. It can be seen that obtaining the system feedback information is conducive to the beam selection model to flexibly adjust the strategy of selecting the beam based on the system feedback information.

[0015] In combination with the first aspect, in a possible implementation manner, the beam selection model may be a reinforcement learning model.

[0016] In a second aspect, an embodiment of the present application provides another beam scanning method, which can be executed by a network device, or by a device matched with the network device, such as a processor, a chip, or a chip system. The method may include: based on a beam selection model, determining a first beam set from multiple beams supported by the network device; the beam corresponding to the first beam set performs beam scanning within M time units and can cover terminal devices within a preset range; M is less than or equal to a first threshold; in the M time units, the beam performing beam scanning in the i-th time unit is different from the beam performing beam scanning in the i+j-th time unit; i is a positive integer less than or equal to M, i+j is a positive integer less than or equal to M, and j is a positive integer; beam scanning is performed according to the beam corresponding to the first beam set.

[0017] In the present application, the network device can also obtain the beam direction for performing beam scanning based on the beam selection model. This method can obtain a larger beam search space, which is beneficial to improving the coverage of the network device and improving the efficiency of beam scanning.

[0018] In conjunction with the second aspect, in a possible implementation, the beam selection model includes a first selection model and a second selection model; based on the beam selection model, determining a first beam set from multiple beams supported by the network device includes: based on the first selection model, determining a reference beam set from multiple beams supported by the network device; there are first beams and second beams with a correlation greater than a second threshold in the beams corresponding to the reference beam set; based on the second selection model, determining the first beam set from the reference beam set. It can be seen that combining the first selection model and the second selection model is conducive to determining a more effective beam direction.

[0019] In conjunction with the second aspect, in a possible implementation, the first selection model may be a reinforcement learning model, and the second selection model may be a long short-term memory model.

[0020] In a third aspect, an embodiment of the present application provides a communication device, which may be a network device, or a device in a network device, or a device that can be used in combination with a network device. Among them, the communication device may also be a chip system. The communication device may execute the method described in the first aspect or the second aspect. The functions of the communication device may be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more units or modules corresponding to the above functions. The unit or module may be software and / or hardware. The operations and beneficial effects performed by the communication device may refer to the methods and beneficial effects described in the first aspect or the second aspect above.

[0021] In a fourth aspect, an embodiment of the present application provides a communication device, the communication device comprising a processor, the processor being used to execute the method as described in the first aspect or the method as described in the second aspect.

[0022] In a fifth aspect, an embodiment of the present application provides a communication device, which includes a processor, the processor is coupled to a memory, and the memory is used to store programs or instructions. When the program or instructions are executed by the processor, the communication device executes the method described in the first aspect or the second aspect.

[0023] In a possible implementation, the communication device further includes a memory. Optionally, the processor and the memory are integrated together. Optionally, the memory and the processor are independently configured.

[0024] In a sixth aspect, an embodiment of the present application provides a communication device, which includes a processor and an interface circuit, wherein the interface circuit is used to receive signals from other communication devices outside the communication device and transmit them to the processor or send signals from the processor to other communication devices outside the communication device, and the processor is used to implement the method described in the first aspect or the second aspect through a logic circuit or executing code instructions.

[0025] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are executed by a communication device, the method described in the first aspect or the second aspect is implemented.

[0026] In an eighth aspect, an embodiment of the present application provides a computer program product comprising instructions, and when a communication device reads and executes the instructions, the communication device executes a method as described in any one of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of a single-beam exhaustive scanning provided in an embodiment of the present application;

[0028] Figure 2A A schematic diagram of the architecture of a communication system applied to an embodiment of the present application;

[0029] Figure 2B A schematic diagram of the architecture of a beam scanning system applied to an embodiment of the present application;

[0030] Figure 3 A schematic diagram of a flow chart of a beam scanning method provided in an embodiment of the present application;

[0031] Figure 4 A schematic flow chart of another beam scanning method provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of a training process of a beam selection model provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram comparing the scanning efficiency of a beam scanning method and an exhaustive beam scanning method provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of a flow chart of another beam scanning method provided in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of a training process for selecting a beam direction by combining a reinforcement learning model with an LSTM model provided in an embodiment of the present application;

[0036] Fig. 9 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0037] Fig.10 A schematic diagram of the structure of another communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship.

[0039] It should be understood that in this application, "at least one" means one or more; "plurality" means two or more. In addition, "equal to" in this application can be used in conjunction with "greater than" or "less than". When "equal to" is used in conjunction with "greater than", the technical solution of "greater than" is adopted; when "equal to" is used in conjunction with "less than", the technical solution of "less than" is adopted.

[0040] In the present application, "sending information to... (e.g., a terminal device)" can be understood as the destination of the information being the terminal device. This can include sending information to the terminal device directly or indirectly. "Receiving information from... (e.g., a terminal device)" or "receiving information from... (e.g., a terminal device)" can be understood as the source of the information being the terminal device, which can include receiving information from the terminal device directly or indirectly. The information may be processed as necessary between the source and destination of the information, such as format changes, but the destination can understand the valid information from the source. Similar expressions in the present application can be understood similarly and will not be repeated here.

[0041] Before describing the embodiments of the present application, a brief explanation of the relevant concepts involved in the embodiments of the application is given.

[0042] 1. Beam

[0043] A beam is a communication resource, and a beam can be a wide beam, or a narrow beam, or other types of beams. The technology for forming a beam can be a beamforming technology or other technical means. The beamforming technology can specifically be a digital beamforming technology, an analog beamforming technology, or a hybrid digital / analog beamforming technology. Optionally, a beam can also be referred to as a precoding vector, and a beam can be directly replaced by a precoding vector, or a precoding vector can be directly replaced by a beam. Optionally, the same communication device (such as a terminal device or a network device) can have different precoding vectors, and different devices can also have different precoding vectors, that is, corresponding to different beams, and different beams can correspond to different directions. It can be understood that the use of different beams by the device indicates that the device uses different precoding vectors. Optionally, the uplink precoding vector and the downlink precoding vector are further distinguished, or the precoding vector used to send information and the precoding vector used to receive information are distinguished.

[0044] Optionally, the beam can be divided into a transmit beam and a receive beam of a network device, and a transmit beam and a receive beam of a terminal device. The beam of a network device may include a transmit beam and a receive beam of a network device. The beam of a terminal device may include a transmit beam and a receive beam of a terminal device. The transmit beam of a network device is used to describe the beamforming information of the network device's transmit side, and the receive beam of a network device is used to describe the beamforming information of the network device's receive side. The transmit beam of a terminal device is used to describe the beamforming information of the terminal device's transmit side, and the receive beam of a terminal device is used to describe the beamforming information of the terminal device's receive side. Among them, the receive beam can be equivalent to a spatial transmission filter, a spatial domain transmission filter, a spatial domain reception filter, and a spatial reception filter; the transmit beam can be equivalent to a spatial domain filter, a spatial domain transmission filter, a spatial domain transmission filter, and a spatial transmission filter. The receive beam on the terminal device side and the transmit beam on the network device side can be a downlink spatial filter, and the transmit beam on the terminal device side and the receive beam on the network device side can be an uplink spatial filter.

[0045] 2. Beam forming

[0046] Beamforming technology can also be called beamforming technology, or it can also be called precoding technology. The network equipment can determine the beamforming matrix that matches the channel state based on known or estimated channel information to process the signal to be transmitted, so that the signal to be transmitted after beamforming is adapted to the channel, so that the signal receiving device obtains better signal reception quality, for example, a higher signal to interference plus noise ratio (SINR).

[0047] 3. Beam sweeping

[0048] Beam scanning refers to scanning the beam within a certain spatial area within a certain period of time. In NR, the transmission loss of high-frequency carriers is large, and beamforming can be used to increase the transmission distance of wireless signals. Since the angle covered by each beam is limited, NR uses beam scanning to achieve the purpose of covering the service range of the cell. Beam scanning refers to sending physical channels or reference signals using beams in different directions at different times. A cell usually needs to send multiple SSBs to complete a beam scan. The following are the currently available beam scanning methods:

[0049] 1. Single beam exhaustive scanning

[0050] Single beam exhaustive scanning means scanning a beam in one direction in each time slot and performing beam scanning in the full beam range. Specifically, the network device can scan a beam in one direction in each time slot, and the terminal device can calculate the received signal strength of each beam to obtain the best communication beam pair.

[0051] For example, see Figure 1 , Figure 1 A schematic diagram of a single-beam exhaustive scan provided in an embodiment of the present application. Figure 1 As shown, the network device 101 can scan a beam in one direction in each time slot, and a beam in one direction can correspond to one SSB, and different SSBs can be distinguished by SSB index. For example, beam 102 can correspond to SSB index 0, and beam 103 can correspond to SSB index 7. Figure 1 The network device can complete the transmission of all SSBs in an SSB burst set within a half frame, that is, the network device can perform beam scanning within a half frame. Among them, the terminal device 104 can know that the received signal strength of beam 105 is the largest by calculating the received signal strength of each beam, and the terminal device 106 can know that the received signal strength of beam 103 is the largest by calculating the received signal strength of each beam.

[0052] 2. Multi-beam exhaustive scanning

[0053] Multi-beam exhaustive scanning refers to scanning beams in multiple directions in each time slot, and performing beam scanning within the full beam range. Specifically, network devices can be configured with multi-antenna multi-link and adopt hybrid beamforming. Network devices can simultaneously scan beams in multiple directions in each time slot, and terminal devices can calculate the received signal strength of each beam to obtain the best communication beam pair.

[0054] Based on the above description, it can be seen that the network device successfully covers all terminal devices in the area. The current beam scanning solutions all adopt an exhaustive scanning method, resulting in low beam scanning efficiency and high energy consumption. Based on this, the embodiment of the present application provides a beam scanning method, which can perform beam scanning more intelligently and is conducive to improving beam scanning efficiency.

[0055] The embodiments of the present application can be applied to various communication systems, such as: 5G or NR system, long term evolution (LTE) system, LTE frequency division duplex (FDD) system, LTE time division duplex (TDD) system, etc. The embodiments of the present application can also be applied to future communication systems, such as the sixth generation mobile communication system. The embodiments of the present application can also be applied to device to device (D2D) communication, vehicle to everything (V2X) communication, machine to machine (M2M) communication, machine type communication (MTC), and Internet of things (IoT) communication system or other communication system. Optionally, the embodiments of the present application can also be applied to long-distance communication scenarios, such as satellite communication scenarios where the distance between the terminal device and the network device is constantly changing, or other long-distance communication scenarios, and the embodiments of the present application are not limited to this.

[0056] See also Figure 2A , Figure 2A FIG. 1 is a schematic diagram of the architecture of a communication system applied to an embodiment of the present application. Figure 2A As shown, the communication system 200 may include a network device 201 and a terminal device 202 , and optionally, may also include a terminal device 203 . Figure 2A The device form and device quantity shown are for example only and do not constitute a limitation on the embodiments of the present application. In the embodiments of the present application, the terminal device 202 and the terminal device 203 are within the coverage of the network device 201, and the terminal device 202 and the terminal device 203 can calculate the received signal strength of each beam when the network device 201 performs beam scanning.

[0057] The network device (such as network device 201) in the embodiment of the present application can be a next generation NodeB (gNB) or a next generation evolved NodeB (ng-eNB). Among them, gNB can provide the user plane function and control plane function of NR, and ng-eNB can provide the user plane function and control plane function of evolved universal terrestrial radio access (EUTRA). It should be noted that gNB and ng-eNB are only a name used to indicate a base station that supports a 5G network system and have no restrictive meaning. The network device can also be a base station (base transceiverstation, BTS) in a GSM system or a CDMA system, a base station (nodeB, NB) in a WCDMA system, or an evolved base station (evolutional node B, eNB or eNodeB) in an LTE system. Alternatively, the network device may also be a relay station, an access point, a vehicle-mounted device, a wearable device, a network-side device in a network after 5G, or a network device in a future evolved PLMN network, a road site unit (RSU), etc.

[0058] The terminal devices in the embodiments of the present application (such as terminal devices 202 and 203) may also be referred to as user equipment (UE), mobile station (MS), mobile terminal (MT), etc., or devices for providing voice or data connectivity to users, or IoT devices. For example, the terminal devices include handheld devices with wireless connection functions, vehicle-mounted devices, etc. At present, terminal devices can be: mobile phones, tablet computers, laptops, PDAs, mobile internet devices (MID), wearable devices (such as smart watches, smart bracelets, pedometers, etc.), vehicle-mounted devices (such as cars, bicycles, electric vehicles, airplanes, ships, trains, high-speed trains, etc.), satellite terminals, virtual reality (VR) devices, augmented reality (AR) devices, smart point of sale (POS) machines, customer-premises equipment (CPE), wireless terminals in industrial control, smart home devices (such as refrigerators, TVs, air conditioners, electric meters, etc.), smart robots, robotic arms, workshop equipment, wireless terminals in unmanned driving, wireless terminals in telemedicine, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, flight equipment (such as smart robots, hot air balloons, drones, airplanes), etc. The terminal device can also be other devices with terminal functions, for example, the terminal device can also be a device that serves as a terminal function in D2D communication.

[0059] See also Figure 2B , Figure 2B FIG. 1 is a schematic diagram of the architecture of a beam scanning system used in an embodiment of the present application. Figure 2B As shown, the beam scanning system 210 includes a network device 211, a terminal device 212, a terminal device 213 and a terminal device 214. In addition, the beam scanning system 210 is also deployed with a beam selection model 215.

[0060] In an embodiment of the present application, the beam selection model 215 can be applied to the network device 211, and the beam selection model 215 can be preliminarily trained based on the first codebook set. The network device 211 can perform beam scanning based on the result of the preliminary training, and obtain system feedback information of the terminal device (such as the terminal device 212, the terminal device 213, and the terminal device 214). Then, the training parameters of the beam selection model 215 can be updated according to the system feedback information to continuously optimize the performance of the beam selection model. Here, the optimization target of the beam selection model 215 can be the time required for the network device 211 to perform beam scanning based on the selected beam to cover all terminal devices within a preset range. After completing the training of the beam selection model 215, the beam selection model 215 determines the second codebook set from the first codebook set. Based on the beam corresponding to the second codebook set, the network device 211 performs beam scanning within M time units to cover all terminal devices within the preset range. Wherein, M can be a positive integer less than or equal to the first threshold. Optionally, the beam selection model 215 can also be used to determine a first beam set from multiple beams supported by the network device 211. The network device 211 performs beam scanning within M time units based on the beams corresponding to the first beam set, which can also cover all terminal devices within a preset range. Compared with the exhaustive scanning method, this can perform beam scanning more intelligently, thereby helping to improve beam scanning efficiency.

[0061] The beam scanning method provided in the embodiments of the present application is described in detail below.

[0062] See also Figure 3 , Figure 3 A schematic flow chart of a beam scanning method provided in an embodiment of the present application. The method may include but is not limited to the following steps:

[0063] Optionally, in S301, the network device performs dimensionality reduction processing on the initial codebook set according to the correlation between codebooks in the initial codebook set to obtain a first codebook set.

[0064] Optionally, in an embodiment of the present application, the network device may first perform dimensionality reduction processing on the initial codebook set according to the correlation between the codebooks in the initial codebook set to obtain a first codebook set. The initial codebook set may include multiple codebooks, each codeword in the codebook may be used to precode the signal, and the precoded signal may be propagated in space according to a certain spatial directivity to form a beamforming effect, so one codeword in the codebook may correspond to one beam.

[0065] In some embodiments, the codebooks in the initial codebook set may be discrete Fourier transform (DFT) codebooks. Each codebook may be represented in the form of a matrix, and the codebook may also be described as a precoding matrix, or may also be described as a predefined beamforming codebook. For example, assuming that the number of transmitting antennas of the network device is N t , then the network device can support sending N t beams, a DFT codebook can be expressed as:

[0066]

[0067] in, and

[0068] In some embodiments, the network device may support multi-beam scanning, that is, it may scan multiple beams in one time slot, which is beneficial to improve the efficiency of beam scanning. RF beams, then there can be N s Scanning methods, here, N s It can be expressed as:

[0069]

[0070] Among them, N t is the total number of beams, N RF The number of beams scanned for each time slot.

[0071] Optional, N s One of the scanning modes can also be described as a beam scanning mode. That is, a beam scanning mode can be used to indicate the beam scanned by the network device in a time slot. Each beam scanning mode can correspond to a codebook, N s N corresponding to the beam scanning mode s The codebooks can form a codebook set, namely the initial codebook set, which can be expressed as

[0072]

[0073] in, B j represents the index of the codeword in the codebook corresponding to the jth beam scanning mode, which can be understood as, B j represents the index of the beam corresponding to the j-th beam scanning mode, satisfying |B j | c =N RF ,|| c Represents a counting operation.

[0074] In some embodiments, since the initial codebook set includes codebooks corresponding to all beam scanning modes of multi-beam scanning, the initial codebook set W all The dimension of is large, which makes the beam search space huge when the network device performs multi-beam scanning. Based on this, the initial codebook set can be reduced in dimension to obtain the first codebook set.

[0075] In some embodiments, the initial codebook set may be processed in a dimension reduction manner by performing dimension reduction processing on the initial codebook set based on the correlation between codebooks in the initial codebook set. Here, the correlation between codebooks in the initial codebook set, i.e., N s In order to minimize the correlation between codebooks in the initial codebook set, two criteria can be used when reducing the dimensionality of the initial codebook set: ① For the same beam scanning mode, the beam directions scanned in a time slot are greatly different to avoid beam interference; ② For different beam scanning modes, the similarity is low to avoid repeated beam scanning.

[0076] Specifically, for the above criterion ①, in the same beam scanning mode, the difference in beam directions of different beams can be measured by the difference between codewords in the codebook corresponding to the beam scanning mode. If, in the codebook corresponding to a beam scanning mode, the two codewords with the smallest difference satisfy the difference greater than the second threshold, then it can be considered that criterion ① is met. Optionally, it can also be measured by the similarity between codewords in the codebook corresponding to the beam scanning mode. In other words, if in the codebook corresponding to a beam scanning mode, the two codewords with the largest similarity satisfy the similarity less than the fourth threshold, then it can also be considered that criterion ① is met.

[0077] For the above criterion ②, in different beam scanning modes, the similarity of different beam scanning modes can be measured by the similarity between the codebooks corresponding to the beam scanning modes. If the intersection of the codebooks corresponding to any two beam scanning modes is less than the third threshold, then it can be considered that criterion ② is satisfied.

[0078] Therefore, the process of reducing the dimension of the initial codebook set can be summarized as the following algorithm (1):

[0079]

[0080] Where Wall represents the initial codebook set, W B represents the first codebook set obtained by dimensionality reduction, represents the number of codebooks in the first codebook set, represents the difference between the two codewords with the smallest difference in the initial codebook set, ∈ represents the second threshold, represents the intersection of any two codebooks in the first codebook set, and n represents the third threshold.

[0081] In some embodiments, the second threshold (i.e., ∈) and the third threshold (i.e., ∈) used when the initial codebook set is subjected to dimensionality reduction processing can be obtained from the threshold configuration information. That is, the second threshold and the third threshold can be pre-configured parameters. Among them, the smaller the second threshold and the third threshold, the stronger the constraints of the above two criteria can be, and the smaller the dimension of the first codebook set obtained by dimensionality reduction, and the smaller the corresponding beam search space. In different communication systems, the requirements for beam scanning may be different. For example, from the perspective of resource saving and reducing energy consumption, a smaller second threshold and a third threshold can be set; from the perspective of improving beam scanning performance, a larger second threshold and a third threshold can be set. Specifically, they can be set according to the needs, so that the first codebook set that meets the needs can be determined in actual applications. This application does not limit this. By flexibly configuring the second threshold and the third threshold, it is beneficial to increase the flexibility of beam scanning. Optionally, the threshold configuration information can be stored in the form of a table. For example, Table 1 shows threshold configuration information in a tabular form.

[0082] Table 1

[0083] Threshold configuration information Second threshold (ε) The third threshold (η) 1 0.001 0.01 2 0.01 0.1 … … …

[0084] As shown in Table 1, the threshold configuration information may include a variety of different threshold configurations to facilitate flexible selection by the network device. For example, by using the second threshold of 0.001 and the third threshold of 0.01, the dimension of the first codebook set obtained by dimensionality reduction is smaller, and the beam search space when the network device subsequently performs beam scanning is smaller, which is beneficial to saving energy consumption; by using the second threshold of 0.01 and the third threshold of 0.1, the dimension of the first codebook set obtained by dimensionality reduction is larger, and the beam search space when the network device subsequently performs beam scanning is larger, which is beneficial to the network device to explore and obtain a beam with higher scanning efficiency.

[0085] S302: The network device determines a second codebook set from the first codebook set based on a beam selection model.

[0086] In an embodiment of the present application, the network device may determine a second codebook set from a first codebook set based on a beam selection model. The network device may perform beam scanning based on the beam corresponding to the second codebook set to cover all terminal devices within a preset range within M time units, where M is less than or equal to a first threshold. Among them, in the M time units, the beam performing beam scanning in the i-th time unit may be different from the beam performing beam scanning in the i+j-th time unit, i is a positive integer less than or equal to M, i+j is a positive integer less than M, j is a positive integer, and a time unit may be, for example, a time slot. In other words, the network device may perform beam scanning based on the beam corresponding to the second codebook set to avoid repeated scanning and complete coverage of all terminal devices within the preset range within the preset time range. Here, the preset range may be the coverage range of the network device.

[0087] Among them, the beam selection model can be used to intelligently select beams so that the network device can efficiently perform beam scanning. It can be understood that if the network device uses the beam selected by the beam selection model to perform beam scanning, and can complete the coverage of all terminal devices within the preset range in the shortest possible time, then it can be considered that the performance of the beam selection model is good and the network device performs beam scanning efficiently.

[0088] In some embodiments, the beam selection model can be modeled as the following optimization problem (I):

[0089]

[0090] stC1:T∈{1,2,...,T max},

[0091]

[0092]

[0093]

[0094] in, is the codebook corresponding to nT beam scanning modes. The network device can perform beam scanning based on the beam scanning mode corresponding to one codebook in one time slot. T is the time required to perform beam scanning. Indicates the nth t The codebook corresponding to the beam scanning pattern of time slots, Indicates the nth t The index of the beam in the beam scanning pattern of time slots, N s Indicates that the network device supports a total of N s Beam scanning mode.

[0095] In some embodiments, since the first codebook set can be obtained by dimensionality reduction processing of the initial codebook set, it means that the beam selection model can select beams based on the beams corresponding to the first codebook set. In this way, the above optimization problem (I) can be transformed into the following optimization problem (II):

[0096]

[0097] stC1:T∈{1,2,...,T max},

[0098]

[0099]

[0100]

[0101] Among them, N B represents the number of codebooks in the first codebook set, that is, the number of beam scanning modes after dimensionality reduction processing.

[0102] For the four constraints (i.e., C1, C2, C3, and C4) included in the above optimization problem, C1 means that the time for beam scanning is no more than T in units of time slots. max time slots; C2 indicates that a codeword is selected from a predefined codebook. The predefined codebook in optimization problem (1) may be from the initial codebook set, and the predefined codebook in optimization problem (2) may be from the first codebook set, that is, from the initial codebook set after dimensionality reduction processing; C3 indicates that different codebooks are selected in different time slots to avoid repeated scanning; C4 indicates that beam scanning is performed within T time slots to cover all terminal devices within the preset range. Among them, C4 It can be expressed as:

[0103]

[0104] Among them, K a Indicates the number of all terminal devices within the preset range, u k,t It can indicate whether the kth terminal device is successfully covered in the tth time slot, u k,t It can be expressed as:

[0105]

[0106] in, represents the received signal strength of the kth terminal device in the tth time slot, for example, it can be the average RSRP of the received signal of the kth terminal device in the tth time slot. Here, if is greater than γ, then u k,t=I, which means that if the received signal strength of the k-th terminal device in the t-th time slot is greater than the threshold γ, then it is considered that the k-th terminal is successfully covered in the t-th time slot.

[0107] In addition, the kth terminal device has a beam scanning pattern W for the jth j , the terminal device can decode the i-th beam, and the mathematical model of the terminal device receiving the signal can be expressed as:

[0108]

[0109] Among them, y k,j (f) represents the signal received by the kth terminal device, w(φ i ) is the codeword corresponding to the i-th beam, which can also be described as the beamforming vector of the i-th beam. x(f) can represent the data service of the k-th terminal device, z(f) is the noise, and h k is the channel representation of the network device for the kth terminal device. Here, the channel from the kth terminal device to the jth subcarrier of the network device can be represented as:

[0110]

[0111] in, α k,l represents the path attenuation, L k represents the multipath number, τ k,l represents the delay, and Δf represents the subcarrier spacing.

[0112] Based on formula (6) and formula (7), the average RSRP of the signal received by the kth terminal device can be expressed as:

[0113]

[0114] in, N c Indicates the number of subcarriers.

[0115] In some embodiments, the beam selection model can be a reinforcement learning model, that is, the solution of the above optimization problem can be completed based on the reinforcement learning model. Among them, the reinforcement learning model is a mathematical framework for autonomous strategy learning through experience. It is an agent that learns in a "trial and error" manner, and the reward obtained by interacting with the environment guides the behavior, with the goal of making the agent obtain the maximum reward value. In the reinforcement learning model, after the agent performs an action, the environment will be converted to a new state, and the environment will give a reward signal (which can be a positive reward or a negative reward) for the new state. Subsequently, the agent can perform new actions according to a certain strategy based on the new state and the reward fed back by the environment. Through reinforcement learning, the agent can know what state it is in and what action it should take to obtain the maximum reward. In an embodiment of the present application, based on the reinforcement learning model, the network device can know how to perform beam scanning to serve all terminal devices within the coverage area in the shortest time. Optionally, the beam selection model can also be other machine learning models, which are not limited in this application.

[0116] S303: The network device performs beam scanning according to the beam corresponding to the second codebook set.

[0117] In an embodiment of the present application, after the network device determines the second codebook set from the first codebook set based on the beam selection model, it can perform beam scanning based on the beam corresponding to the second codebook set. The second codebook set includes multiple codebooks, one codebook can correspond to a beam scanning mode, and the network device can use a beam scanning mode corresponding to a codebook in a time slot to simultaneously scan multiple beams, thereby quickly covering all terminal devices within a preset range.

[0118] exist Figure 3 In the illustrated embodiment, the network device can obtain a second codebook set based on a beam selection model, and then perform beam scanning based on the beam corresponding to the second codebook set to complete coverage of all terminal devices in the coverage area within a certain time range, thereby achieving efficient and intelligent execution of beam scanning, which is conducive to improving the flexibility and efficiency of beam scanning.

[0119] In some embodiments, the beam selection model based on reinforcement learning may include a first neural network and a second neural network. The process of determining the second codebook set based on the beam selection model can be referred to in Figure 4 The embodiment shown.

[0120] See also Figure 4 , Figure 4 A flowchart of another beam scanning method provided in an embodiment of the present application. The method may include but is not limited to the following steps:

[0121] S401: A network device determines a reference codebook set from a first codebook set based on first codebook state information, a first network parameter, and a first neural network.

[0122] In an embodiment of the present application, the network device may first determine a reference codebook set from the first codebook set based on the first codebook state information, the first network parameter, and the first neural network. The first codebook set may be obtained after the dimensionality reduction processing of the initial codebook set, and the first codebook state information may be the codebook state information corresponding to the first time slot, which may indicate the number of times the beam corresponding to the first codebook set has been scanned in the first time slot. The first network parameter may be a parameter for updating the first neural network. The first neural network may select a codebook from the first codebook set based on the first codebook state information and the first network parameter. The reference codebook set may be understood as a codebook set selected by the first neural network based on the codebook state information corresponding to the first time slot and the first network parameter.

[0123] In some embodiments, the codebook state information may be expressed as:

[0124]

[0125] Among them, s t Includes the tth time slot N t The number of times each beam has been scanned, e.g. It can be expressed that the index of the tth time slot is N t The number of times the beam has been scanned. It can be understood that s t Contains information about the interactive environment.

[0126] Exemplarily, assuming that none of the beams corresponding to the first codebook set in the first time slot have been scanned, the first codebook state information can be expressed as [0, 0, ..., 0]; assuming that only the beam with index 1 in the beams corresponding to the first codebook set in the first time slot has been scanned once, the first codebook state information can be expressed as [1, 0, ..., 0].

[0127] Here, by introducing the codebook state information, the first neural network can be guided to perform training in combination with whether each beam has been scanned.

[0128] In some embodiments, the first neural network may be an actor neural network in a reinforcement learning model. The action in the actor neural network may be a beam scanning pattern selected by the network device in the tth time slot, which may be expressed as:

[0129]

[0130] Among them, N B It can represent the number of beam scanning modes in the first codebook set, It can represent the beam scanning mode selected by the network device in the tth time slot.

[0131] S402: The network device transmits a beam corresponding to a reference codebook set to the terminal device. Correspondingly, the terminal device receives the beam from the network device.

[0132] In an embodiment of the present application, after obtaining a reference codebook set based on the codebook state information corresponding to the first time slot and the first network parameter, the network device can transmit a beam corresponding to the reference codebook set to the terminal device.

[0133] S403, the terminal device sends system feedback information to the network device. Correspondingly, the network device receives the system feedback information from the terminal device.

[0134] In an embodiment of the present application, the terminal device may calculate the signal strength of the beam corresponding to the reference codebook set transmitted by the network device, and send system feedback information to the network device. Accordingly, the network device may receive system feedback information from the terminal device. The system feedback information may be used to indicate the coverage performance of the beam corresponding to the reference codebook set performing beam scanning.

[0135] Optionally, the system feedback information may be the signal strength of the beam corresponding to the reference codebook set. The greater the signal strength, the better the coverage performance of the beam corresponding to the reference codebook set. Optionally, the system feedback information may also be the signal-to-interference-plus-noise ratio (SINR) of the beam corresponding to the reference codebook set. The greater the SINR value, the better the coverage performance of the beam corresponding to the reference codebook set. Optionally, the system feedback information may also be the reference signal received power (RSRP) of the beam corresponding to the reference codebook set. The greater the RSRP value, the better the coverage performance of the beam corresponding to the reference codebook set. Optionally, the system feedback information may also be the quality of service (QoS) of the beam corresponding to the reference codebook set. The greater the QoS value, the better the coverage performance of the beam corresponding to the reference codebook set. Optionally, the system feedback information may also be the access delay information of the beam corresponding to the reference codebook set. The smaller the access delay information, the better the coverage performance of the beam corresponding to the reference codebook set.

[0136] S404, the network device calculates reward data using a reward function based on the system feedback information.

[0137] In an embodiment of the present application, after the network device obtains the system feedback information, a reward function can be used to calculate the reward data. Among them, the reward function can be used to guide the adjustment of the first network parameter of the first neural network. In the case where the system feedback information reflects that the beam coverage performance corresponding to the reference codebook set is better, the value of the reward data calculated using the reward function is larger. In other words, a beam with good coverage performance can obtain a larger reward value. Optionally, the reward function may include an attenuation factor, which can make the codebook set corresponding to the beam that can scan the terminal device faster obtain more rewards.

[0138] In some embodiments, when the system feedback information is the signal strength of the beam corresponding to the reference codebook set, the reward function can be expressed as:

[0139]

[0140] Among them, r t It can represent the reward data corresponding to the tth time slot, μ t It can represent the attenuation factor corresponding to the tth time slot, s t,i =0 can indicate that the number of times the i-th beam has performed beam scanning in the t-th time slot is 0, that is, it has not been scanned yet. In addition, It can be expressed as:

[0141]

[0142] in, It can represent the maximum value of the signal strength calculated for k terminal devices in the i-th beam.

[0143] Since the goal is to enable the network device to cover all terminal devices (i.e., k terminal devices) within a preset range through beam scanning, In the case of , it means that the i-th beam can cover at least one of the k terminal devices, then the i-th beam can be considered a valuable beam and can contribute to the reward function. In other words, for the i-th beam that has not performed beam scanning in the t-th time slot, if the beam can cover at least one terminal device, the corresponding reward data can be calculated by formula (11).

[0144] In some embodiments, when the system feedback information is the SINR of the beam corresponding to the reference codebook set, the reward function can be expressed as:

[0145]

[0146] Among them, for the i-th beam that has not performed beam scanning in the t-th time slot, and when the SINR corresponding to the beam is greater than γ, the corresponding reward data can be calculated by formula (13).

[0147] In some embodiments, when the system feedback information is the RSRP of the beam corresponding to the reference codebook set, the reward function can be expressed as:

[0148]

[0149] Among them, for the i-th beam that has not performed beam scanning in the t-th time slot, and when the RSRP corresponding to the beam is greater than γ, the corresponding reward data can be calculated by formula (14).

[0150] In some embodiments, when the system feedback information is the QoS of the beam corresponding to the reference codebook set, the reward function can be expressed as:

[0151]

[0152] Among them, for the i-th beam that has not performed beam scanning in the t-th time slot, and when the QoS corresponding to the beam is greater than γ, the corresponding reward data can be calculated by formula (15).

[0153] In some embodiments, when the system feedback information is the access delay information of the beam corresponding to the reference codebook set, the reward function can be expressed as:

[0154]

[0155] Here, since the larger the access delay information is, the worse the coverage performance of the beam is, when the access delay information is greater than γ, the calculated reward data is negative. It is worth noting that in the case where the system feedback information is the access delay information of the beam corresponding to the reference codebook set, the network device can count the average access delay information of all terminal devices within the coverage area. In this case, the terminal device may not feed back signal strength or service quality in each time slot, but directly feed back access delay information. For example, the access delay information fed back by the terminal device may be that the access takes 5 time slots.

[0156] S405, the network device adjusts the first network parameters based on the reward data and the second neural network to obtain the second network parameters.

[0157] In an embodiment of the present application, after the network device calculates the reward data, it can adjust the first network parameters based on the reward data and the second neural network to obtain the second network parameters. The goal of the adjustment can be to maximize the reward data. Optionally, the second neural network can include a third network parameter, and the third network parameter can be a parameter for updating the second neural network. Optionally, the second neural network can be a critic neural network in a reinforcement learning model.

[0158] S406: The network device determines a second codebook set from the first codebook set based on the second codebook state information, the second network parameter, and the first neural network.

[0159] In an embodiment of the present application, after the network device obtains the second network parameter, it can determine the second codebook set from the first codebook set according to the second codebook state information, the second network parameter and the first neural network. Among them, the second codebook state information can be the codebook state information corresponding to the second time slot, which can indicate the number of times the beam corresponding to the first codebook set has performed beam scanning in the second time slot. Similarly, the first neural network can select the codebook set again based on the codebook state information corresponding to the second time slot and the second network parameter. Then the network device can obtain the system feedback information based on the codebook set selected in the second time slot, and then calculate the reward data based on the system feedback information using the reward function, and then adjust the second network parameters based on the reward data and the second neural network to obtain new network parameters, and then update the codebook set based on the third codebook state information, the new network parameters and the first neural network. In this way, the loop is iterated until the reward data converges, and the trained first neural network and the second neural network (i.e., the beam selection model) can be obtained. Based on the trained beam selection model, the network device can determine the second codebook set from the first codebook set, and the second codebook set here can be understood as the codebook set corresponding to the beam that can efficiently perform beam scanning that is finally determined.

[0160] S407: The network device performs beam scanning according to the beam corresponding to the second codebook set.

[0161] In an embodiment of the present application, the network device performs beam scanning based on the beam corresponding to the second codebook set, so as to complete coverage of all terminal devices within a preset range within a preset time range.

[0162] In some embodiments, the training process of the beam selection model described in steps S401 to S407 can be found in Figure 5 .like Figure 5 As shown, the network device can be based on the codebook state information s t and parameter π θ Through the actor neural network (first neural network), get action at Among them, the parameter π θ It may include the selection probability of each codebook in the first codebook set, a t It can be expressed that in the t time slot, the network device in the t time slot is based on the parameter π θ The selected codebook, that is, the selected beam scanning mode, may be, for example, the codebook corresponding to the value with the largest probability. t The codebook status information s can be updated later t The network device can then perform beam scanning based on the selected beam scanning mode and obtain system feedback information from the terminal device. Based on the system feedback information, a reward function r can be used. t Calculate reward data. In addition, the critic neural network (second neural network) can obtain a t The corresponding value v t , the critic neural network can also record each acquired t , a t , r t , v t >, then based on this, the network training is performed to update the parameters π of the actor neural network θ Among them, the action with a larger reward data value can correspond to a greater selection probability. After multiple training iterations in this way, the second codebook set corresponding to the beam that can efficiently perform beam scanning can be determined.

[0163] For example, see Figure 6 , Figure 6 A schematic diagram comparing the scanning efficiency of a beam scanning method and an exhaustive beam scanning method provided in an embodiment of the present application. Figure 6 As shown, compared with the current single-beam exhaustive scanning method and multi-beam exhaustive scanning method, the beam scanning method in the embodiment of the present application can cover all terminal devices within a preset range in a shorter time, that is, the coverage rate reaches 100% in a shorter time.

[0164] exist Figure 4 In the illustrated embodiment, through the first neural network and the second neural network, and in combination with the codebook state information and system feedback information of the first codebook set, a beam that can efficiently perform beam scanning can be intelligently selected from the first codebook set, thereby helping to improve the efficiency of beam scanning.

[0165] In the embodiment of the present application, in addition to determining the second codebook set from the first codebook set based on the beam selection model, the first beam set can also be determined from multiple beams supported by the network device based on the beam selection model. That is, the beam selection model can be used to select the beam direction.

[0166] See also​ Figure 7 , Figure 7 A flowchart of another beam scanning method provided in an embodiment of the present application. The method may include but is not limited to the following steps:

[0167] S701: A network device determines a first beam set from multiple beams supported by the network device based on a beam selection model.

[0168] In an embodiment of the present application, the network device may determine a first beam set from multiple beams supported by it based on a beam selection model. The multiple beams supported by the network device may be understood as multiple beams used by the network device to perform beam scanning, and the emission directions of the multiple beams are different. The multiple beams may also be described as multiple beam directions. The beam corresponding to the first beam set may cover all terminal devices within a preset range by performing beam scanning within M time units, where M is less than or equal to a first threshold. Among the M time units, the beam performing beam scanning within the i-th time unit may be different from the beam performing beam scanning within the i+j-th time unit, where i is a positive integer less than or equal to M, i+j is a positive integer less than M, j is a positive integer, and a time unit may be, for example, a time slot. That is, the network device performs beam scanning based on the beam corresponding to the first beam set, which may avoid repeated scanning and complete coverage of all terminal devices within the preset range within the preset time range.

[0169] In some embodiments, the beam selection model may include a first selection model and a second selection model, wherein the first selection model may be used to select a beam with high scanning efficiency from multiple beams, and the second selection model may be used to avoid interference between beams and avoid repeated scanning of beams. Figure 3 and Figure 4 In the embodiment shown, the beam is selected based on the codebook set. Figure 3 and Figure 4 In the illustrated embodiment, the network device can perform dimensionality reduction processing on the initial codebook set based on algorithm (1), thereby obtaining a first codebook set with large inter-beam phase difference and capable of avoiding repeated beam scanning. Figure 7 In the illustrated embodiment, the beam set selected by the network device from multiple beams based on the first selection model may contain beams with similar directions. Therefore, the second selection model can be used to make the selected beams meet the criterion that the beams should have large differences in directions and avoid duplication.

[0170] In some embodiments, the network device may determine a reference beam set from multiple beams supported by the network device based on the first selection model. Since there may be beams with similar directions in the reference beam set, that is, there may be a first beam and a second beam with a correlation greater than a second threshold in the reference beam set, this situation will affect the efficiency of performing beam scanning. In view of this, the network device may determine the first beam set from the reference beam set with the help of the second selection model.

[0171] In some embodiments, the first selection model may be a reinforcement learning model, wherein the reinforcement learning model may be based on an actor-critic framework, including an actor neural network and a critic neural network. The actor neural network may be used to determine the selected beam direction based on the beam state information, and the critic neural network may be used to evaluate the selected beam direction so as to adjust the network parameters used to train the actor neural network so that the actor neural network selects a beam direction with better coverage performance in the next state. Among them, the beam state information may be used to indicate the number of times that multiple beam directions supported by the network device have been scanned. The second selection model may be a long short-term memory model (LSTM), which may be used to assist the actor neural network in further determining a more reasonable beam direction.

[0172] For example, the training process of combining the reinforcement learning model with the LSTM model to select the beam direction can be found in Figure 8 .like Figure 8 As shown, the network device can be based on the beam state information s t The initial decision information is obtained through the actor neural network. The initial decision information includes the selection probability of each beam direction in multiple beam directions. Based on the initial decision information, the action can be determined Among them, action It can represent the beam direction selected by the network device in the tth time slot. Optionally, one sampling can determine one beam direction, and multiple sampling can determine a reference beam set consisting of multiple beam directions. Then the network device can constrain each beam direction in the reference beam set through the LSTM model, so that the selected beam direction avoids inter-beam interference and avoids repeated scanning as much as possible. Optionally, the LSTM model can be connected to fully connected layers (FC). Next, the beam state information s can be updated based on the selected beam direction. t , and performs beam scanning based on the selected beam direction to obtain system feedback information from the terminal device. Based on the system feedback information, the critic neural network adjusts the decision information of the next state. After multiple training iterations, the actor neural network can finally output the beam direction that meets the requirements.

[0173] S702: The network device performs beam scanning according to the beam corresponding to the first beam set.

[0174] In an embodiment of the present application, the network device may perform beam scanning according to the determined beam corresponding to the first beam set, wherein the network device may send multiple beams in a time slot according to the beam corresponding to the first beam set.

[0175] exist Figure 7 In the illustrated embodiment, the network device can obtain the beam direction for performing beam scanning based on the beam selection model. This method can obtain a larger beam search space, which is beneficial to improving the coverage of the network device and improving the efficiency of beam scanning.

[0176] The method provided by the embodiment of the present application is described above, and the communication device involved in the embodiment of the present application is described below.

[0177] The present application provides a communication device that can be used to implement the functions of the above-mentioned network device or terminal device. The communication device can be a network device or a terminal device. The communication device includes a unit corresponding to the method / operation / step / action performed by the network device or the terminal device in the above-mentioned method embodiment, and the unit can be a hardware circuit, or software, or a combination of a hardware circuit and software.

[0178] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of a communication device provided in an embodiment of the present application. The communication device 900 may include an interface unit 901 and a processing unit 902. Specifically, the processing unit 902 is used to process signaling and / or data, the signaling and / or data may be data received by the interface unit 901, and the processed signaling and / or data may also be sent by the interface unit 901;

[0179] In one implementation, when the communication device 900 is a network device, wherein:

[0180] Processing unit 902 is used to determine a second codebook set from the first codebook set based on a beam selection model, and the beam corresponding to the second codebook set performs beam scanning within M time units and can cover terminal devices within a preset range; M is less than or equal to a first threshold; the first codebook set is included in a predefined beamforming codebook set; in the M time units, the beam performing beam scanning in the i-th time unit is different from the beam performing beam scanning in the i+j-th time unit; i is a positive integer less than or equal to M, i+j is a positive integer less than or equal to M, and j is a positive integer; beam scanning is performed according to the beam corresponding to the second codebook set.

[0181] In this embodiment, for the specific implementation of the above-mentioned interface unit 901 and processing unit 902, please refer to Figure 3-Figure 7 The specific implementation steps of the network equipment will not be repeated here.

[0182] In one implementation, when the communication device 900 is a terminal device, wherein:

[0183] The interface unit 901 is configured to receive a beam corresponding to a reference codebook set from a network device, and send system feedback information to the network device.

[0184] In this embodiment, for the specific implementation of the above-mentioned interface unit 901 and processing unit 902, please refer to Figure 3-Figure 7 The specific implementation steps of the terminal device will not be repeated here.

[0185] See also Fig.10 , Fig.10 is a schematic diagram of the structure of another communication device provided in an embodiment of the present application. Fig.10 As shown, the communication device 1000 may be a communication device or a device used in a communication device, the communication device may be a terminal device or a network device, and the device used in the communication device may be a chip system or a chip in the communication device. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0186] The communication device 1000 includes at least one processor 1010, which is used to implement the processing function of the device (such as a terminal device or a network device) in the method provided in the embodiment of the present application. The communication device 1000 may also include a communication interface 1020, which is used to implement the transceiver operation of the device (such as a terminal device or a network device) in the method provided in the embodiment of the present application. In the embodiment of the present application, the communication interface can be a transceiver, a circuit, a bus, a module or other types of communication interfaces, which are used to communicate with other devices through a transmission medium. For example, the communication interface 1020 is used for the device in the communication device 1000 to communicate with other devices. The processor 1010 uses the communication interface 1020 to send and receive data, and is used to implement the method described in the above method embodiment.

[0187] The communication device 1000 may also include at least one memory 1030 for storing program instructions and / or data. The memory 1030 is coupled to the processor 1010. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 1010 may operate in conjunction with the memory 1030. The processor 1010 may execute program instructions stored in the memory 1030. At least one of the at least one memory may be included in the processor.

[0188] The specific connection medium between the communication interface 1020, the processor 1010 and the memory 1030 is not limited in the embodiment of the present application. Fig.10 The memory 1030, the processor 1010 and the communication interface 1020 are connected via a bus. Fig.10 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0189] When the communication device 1000 is specifically a device for a device (such as a terminal device or a network device), for example, when the communication device 1000 is specifically a chip or a chip system, the communication interface 1020 may output or receive a baseband signal. When the communication device 1000 is specifically a device (such as a terminal or a network device), the communication interface 1020 may output or receive a radio frequency signal. In an embodiment of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0190] It should be noted that the above-mentioned communication interface 1020 can be used to execute the function of the above-mentioned interface unit 901, and the above-mentioned processor 1010 can be used to execute the function of the above-mentioned processing unit 902, which will not be repeated here.

[0191] When the above-mentioned communication device is a chip applied to a terminal device, the chip implements the functions of the terminal device in the above-mentioned method embodiment, and the chip receives information from other devices; or the chip sends information to other devices.

[0192] When the communication device is a chip applied to a network device, the chip implements the function of the network device in the above method embodiment. The chip receives information from other devices; or the chip sends information to other devices.

[0193] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0194] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in an access network device or a terminal. Of course, the processor and the storage medium can also be present in a terminal or an access network device as discrete components.

[0195] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it may also be an optical medium, such as a DVD; it may also be a semiconductor medium, such as a solid state drive (SSD).

[0196] In the various embodiments of the present application, unless otherwise specified or provided for in any logical conflict, the terms and / or descriptions between the different embodiments are consistent and may be referenced to each other, and the technical features in the different embodiments may be combined to form new embodiments according to their inherent logical relationships.

[0197] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

[0198] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed, the method executed by the terminal device or the network device in the above method embodiment is implemented.

[0199] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed, the method executed by the terminal device or the network device in the above method embodiment is implemented.

[0200] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0201] The descriptions of the various embodiments provided in this application can refer to each other, and the descriptions of the various embodiments have their own emphasis. For parts that are not described in detail in a certain embodiment, refer to the relevant descriptions of other embodiments. For the convenience and simplicity of description, for example, the functions of the various devices and equipment provided in the embodiments of this application and the steps of execution can refer to the relevant descriptions of the method embodiments of this application, and the various method embodiments and the various device embodiments can also refer to, combine or quote each other.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A beam scanning method, characterized in that: The method comprises: Based on the beam selection model, a second codebook set is determined from the first codebook set, wherein a beam corresponding to the second codebook set performs beam scanning within M time units and can cover terminal devices within a preset range; M is less than or equal to a first threshold; the first codebook set is included in a predefined beamforming codebook set; in the M time units, a beam performing beam scanning in an i-th time unit is different from a beam performing beam scanning in an i+j-th time unit; i is a positive integer less than or equal to M, i+j is a positive integer less than or equal to M, and j is a positive integer; Perform beam scanning according to the beam corresponding to the second codebook set.

2. The method according to claim 1, characterized in that The method further comprises: According to the correlation between codebooks in the initial codebook set, the initial codebook set is subjected to dimensionality reduction processing to obtain a first codebook set; the initial codebook set is a predefined beamforming codebook set, and the number of codebooks in the second codebook set is less than the number of codebooks in the first codebook set.

3. The method according to claim 2, characterized in that The initial codebook set includes N codebooks, each of the N codebooks includes S codewords, N is a positive integer greater than 1, and S is a positive integer; The step of performing dimensionality reduction processing on the initial codebook set according to the correlation between codebooks in the initial codebook set to obtain a first codebook set includes: According to the difference between S codewords in the initial codebook set and the correlation between the first codebook and the second codebook, the initial codebook set is subjected to dimensionality reduction processing to obtain a first codebook set; the first codebook and the second codebook are codebooks in the N codebooks, and the first codebook is different from the second codebook.

4. The method according to claim 2, characterized in that The step of performing dimensionality reduction processing on the initial codebook set according to the difference between the S codewords in the initial codebook set and the correlation between the first codebook and the second codebook to obtain the first codebook set includes: When a difference between a first codeword and a second codeword in a first codebook is greater than a second threshold, and a correlation between the first codebook and the second codebook is less than a third threshold, the first codebook is used as a codebook in the first codebook set; and the first codeword and the second codeword are two codewords with the smallest difference among the S codewords in the first codebook.

5. The method according to claim 3, characterized in that The method further comprises: The second threshold and the third threshold are obtained from the threshold configuration information.

6. The method according to claim 1, characterized in that The determining the second codebook set from the first codebook set based on the beam selection model includes: Based on codebook state information, system feedback information and a beam selection model, a second codebook set is determined from the first codebook set; the codebook state information is used to indicate the number of times the beam corresponding to the first codebook set has performed beam scanning, and the system feedback information is used to indicate the coverage performance of the beam corresponding to the first codebook set performing beam scanning.

7. The method according to claim 6, characterized in that The codebook state information includes first codebook state information corresponding to a first time unit and second codebook state information corresponding to a second time unit, and the beam selection model includes a first neural network and a second neural network; The determining the second codebook set from the first codebook set based on the codebook state information, the system feedback information and the beam selection model includes: Determine a reference codebook set from a first codebook set based on the first codebook state information, a first network parameter, and the first neural network; Acquire system feedback information based on the reference codebook set; Based on the system feedback information and the second neural network, adjusting the first network parameters to obtain second network parameters; A second codebook set is determined from the first codebook set based on the second codebook state information, the second network parameter, and the first neural network.

8. The method according to claim 7, characterized in that: The adjusting the first network parameters based on the system feedback information and the second neural network to obtain the second network parameters includes: Based on the system feedback information, a reward function is used to calculate reward data; Based on the reward data and the second neural network, the first network parameters are adjusted to obtain second network parameters.

9. The method according to any one of claims 6 to 8, characterized in that: The system feedback information includes a signal strength of a beam corresponding to the reference codebook set, or the system feedback information includes a signal-to-noise ratio of a beam corresponding to the reference codebook set, or a reference signal received power of a beam corresponding to the reference codebook set, or a quality of service of a beam corresponding to the reference codebook set, or access delay information of a beam corresponding to the reference codebook set.

10. The method according to claim 7, characterized in that The acquiring system feedback information based on the reference codebook set includes: Transmitting a beam corresponding to a reference codebook set to a terminal device; Obtaining system feedback information from the terminal device.

11. The method according to any one of claims 1 to 10, characterized in that: The beam selection model is a reinforcement learning model.

12. A beam scanning method, characterized in that: The method comprises: Based on the beam selection model, a first beam set is determined from multiple beams supported by the network device; the beam corresponding to the first beam set performs beam scanning within M time units and can cover terminal devices within a preset range; M is less than or equal to a first threshold; in the M time units, the beam performing beam scanning in the i-th time unit is different from the beam performing beam scanning in the i+j-th time unit; i is a positive integer less than or equal to M, i+j is a positive integer less than or equal to M, and j is a positive integer; Beam scanning is performed according to the beam corresponding to the first beam set.

13. The method according to claim 11, characterized in that The beam selection model includes a first selection model and a second selection model; The determining, based on the beam selection model, a first beam set from a plurality of beams supported by the network device includes: Based on the first selection model, a reference beam set is determined from a plurality of beams supported by the network device; among the beams corresponding to the reference beam set, there are a first beam and a second beam whose correlation is greater than a second threshold; Based on the second selection model, a first set of beams is determined from the reference sets of beams.

14. The method according to claim 13, characterized in that The first selection model is a reinforcement learning model, and the second selection model is a long short-term memory model.

15. A communication device, characterized in that: Comprising a module for executing the method as claimed in any one of claims 1 to 11, or a module for executing the method as claimed in any one of claims 12 to 14.

16. A communication device, characterized in that: The method comprises a processor, wherein the processor is configured to implement the method according to any one of claims 1 to 11 or the method according to any one of claims 12 to 14 through a logic circuit and / or through executing a computer program or instruction.

17. The communication device according to claim 16, characterized in that: Also includes: The memory is used to store the computer program or instructions.

18. A communication device, characterized in that: The invention comprises a processor and an interface circuit, wherein the interface circuit is used to receive signals from other communication devices outside the communication device and transmit them to the processor or send signals from the processor to other communication devices outside the communication device, and the processor is used to implement the method as described in any one of claims 1 to 11 or the method as described in any one of claims 12 to 14 through a logic circuit or executing code instructions.

19. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or an instruction. When the computer program or the instruction is executed by the communication device, the method according to any one of claims 1 to 11 or the method according to any one of claims 12 to 14 is implemented.

Images