Performance monitoring method and apparatus, and communication system

Abstract

Provided in the embodiments of the present application are a performance monitoring method and apparatus, and a communication system. The performance monitoring apparatus is applied to an artificial-intelligence function or model of a terminal device. The apparatus comprises a first processor, which is configured to: on the basis of the measurement of a reference signal within a first time window, use an artificial-intelligence function or model to obtain predicted beam-related information within a second time window, the second time window being subsequent to the first time window, the first time window comprising one or more time instances, and the second time window comprising one or more time instances; and on the basis of the measurement of a reference signal within the second time window, and the predicted beam-related information within the second time window, monitor the performance of the artificial-intelligence function or model.

Description

Method and apparatus for monitoring performance, communication system TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of communication technology. BACKGROUND

[0002] In New Radio Release 14 (NR Rel-14), Artificial Intelligence or Machine Learning (AI / ML) over the air interface is studied. AI / ML can be used for the following use cases: Channel State Information (CSI) feedback enhancement, beam management, positioning enhancement. CSI feedback enhancement can include CSI prediction, CSI compression; beam management can include spatial domain beam prediction (i.e., BM case-1), temporal beam prediction (i.e., BM case-2); positioning enhancement can include direct positioning, AI / ML assisted positioning.

[0003] Among them, the beam management includes: beam prediction in the spatial domain; temporal beam prediction. The network device can configure a set (for example, set B) of beams (i.e., reference signals) for measurement for the terminal device, and the measurement for the set B is used as the input of the AI / ML function / model. The network device can also configure a set (for example, set A) of beams (i.e., reference signals) for prediction for the terminal device, for example, set A can be used for inference.

[0004] In some sub-use cases, a bilateral model can be used, i.e., the AI / ML function or model is at the terminal device side and at the network device side. In other sub-use cases, a unilateral model can be used, i.e., the AI / ML function or model is at the terminal device side or at the network device side. For beam management, the AI / ML model can be at the terminal device side and / or at the network device side.

[0005] In CSI prediction, the AI / ML function or model can be configured at the terminal device side, the terminal device can measure the reference signal on one or more time instances of the observation window, and predict the CSI on one or more time instances in the future prediction window.

[0006] It should be noted that the above introduction to the technical background is only for the convenience of clearly and completely describing the technical scheme of the present application, and for the understanding of those skilled in the art. The above technical scheme cannot be considered as known to those skilled in the art only because it is described in the background section of the present application.

[0007] SUMMARY

[0008] In a scenario where beam management is performed with an AI / ML function or model, the AI / ML function or model can be subjected to performance monitoring to check the performance of the AI / ML function or model, thereby facilitating control of the AI / ML function or model, such as at least one of activation, deactivation, selection, switching, and fallback of the AI / ML function or model.

[0009] The inventors have found that, for time beam prediction (i.e., BM case-2), when the AI / ML function or model is subjected to performance monitoring, the operations related to reference signals need to be further clarified.

[0010] To address at least one of the above issues, embodiments of the present disclosure provide a method and apparatus for monitoring performance, and a communication system, in which a terminal device can use reference signals at one or more time instances within a prediction window to perform performance monitoring of an artificial intelligence function or model, thereby enabling the terminal device and a network device to have a unified understanding of operations related to reference signals in the performance monitoring process, and facilitating accurate performance monitoring.

[0011] According to an aspect of embodiments of the present disclosure, there is provided an apparatus for monitoring performance, applied to an artificial intelligence function or model of a terminal device, wherein the apparatus comprises a first processor configured to:

[0012] obtain, using the artificial intelligence function or model, predicted beam-related information within a second time window (prediction window) according to measurements of reference signals within a first time window (observation window), the second time window being subsequent to the first time window, the first time window comprising one or more time instances, and the second time window comprising one or more time instances; and

[0013] monitor performance of the artificial intelligence function or model according to measurements of reference signals within the second time window and the predicted beam-related information within the second time window.

[0014] According to another aspect of embodiments of the present disclosure, there is provided an apparatus for monitoring performance, applied to an artificial intelligence function or model of a network device, wherein the apparatus comprises a second processor configured to:

[0015] obtaining, using an artificial intelligence function or model, predicted beam-related information for a second time window based on measurement results of reference signals by the terminal device for a first time window, the second time window being subsequent to the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and

[0016] monitoring performance of the artificial intelligence function or model based on measurement results of reference signals by the terminal device for the second time window and the predicted beam-related information for the second time window.

[0017] One of the beneficial effects of the embodiments of the present application is that the terminal device can use reference signals for one or more time instances in the prediction window to monitor performance of the artificial intelligence function or model, whereby the terminal device and the network device can have a unified understanding of operations related to the reference signals in the performance monitoring process, facilitating accurate performance monitoring.

[0018] Reference will now be made in detail to certain embodiments of the application, examples of which are illustrated in the accompanying drawings. While the application will be described in conjunction with the attached drawings, it will be understood that the application is not limited to the embodiments selected for brief description and illustrated in the drawings. The application encompasses any and all embodiments within the scope of the claims.

[0019] Features described and / or illustrated with respect to one implementation can be used in one or more other implementations in the same or similar manner, in combination with or in place of features in other implementations, and / or in place of or in combination with like features.

[0020] It should be emphasized that the term "comprises / comprising" when used in this text is taken to mean that the features, integers, steps or components referred to are present, but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof. BRIEF DESCRIPTION OF DRAWINGS

[0021] Elements and features depicted in one drawing or implementation of an embodiment of the application can be combined with elements and features depicted in one or more other drawings or implementations. Also, in the drawings, like reference numerals designate corresponding parts throughout the several views, and can be used to designate corresponding parts in more than one implementation.

[0022] Fig. 1 is a schematic illustration of a communication system according to an embodiment of the application;

[0023] Fig. 2 is a schematic illustration of a method of monitoring performance according to an embodiment of the application;

[0024] FIG. 3 is a schematic diagram of reference signals in a first time window and a second time window;

[0025] FIG. 4 is another schematic diagram of reference signals in a first time window and a second time window;

[0026] FIG. 5 is a schematic diagram of a transmission occasion of performance monitoring results;

[0027] FIG. 6 is another schematic diagram of a transmission occasion of performance monitoring results;

[0028] FIG. 7 is a schematic diagram of a method for monitoring performance according to an embodiment of the present application;

[0029] FIG. 8 is a schematic diagram of measurement results of reference signals in a first time window and a second time window;

[0030] FIG. 9 is another schematic diagram of measurement results of reference signals in a first time window and a second time window;

[0031] FIG. 10 is yet another schematic diagram of measurement results of reference signals in a first time window and a second time window;

[0032] FIG. 11 is a schematic diagram of an apparatus for monitoring performance according to an embodiment of the present application;

[0033] FIG. 12 is a schematic diagram of an apparatus for monitoring performance according to an embodiment of the present application;

[0034] FIG. 13 is a schematic diagram of a terminal device according to an embodiment of the present application;

[0035] FIG. 14 is a schematic diagram of a network device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] The foregoing and other features of the present application will become apparent to those skilled in the art upon consideration of the following description of specific embodiments of the present application, taken in conjunction with the accompanying drawings. In the description of embodiments of the present application, specific terminology is employed for the sake of clarity. However, the application is not intended to be limited to the specific terminology so selected. A detailed description of specific embodiments of the application is provided herein.

[0037] In the embodiments of the present application, the terms "first", "second", and the like are used to distinguish different elements from each other, but do not indicate spatial arrangement or time sequence of the elements, and the elements should not be limited by the terms. The term "and / or" includes any one and all combinations of the associated listed terms. The terms "comprise", "include", "have", and the like, refer to the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies.

[0038] In the embodiments of the present application, the singular form "a", "an" and "the" include the plural form, should be broadly understood as "one" or "a kind of", and not limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least partially according to", and the term "based on" should be understood as "at least partially based on", unless the context clearly indicates otherwise.

[0039] In the embodiments of the present application, the term "communication network" or "wireless communication network" can refer to a network that conforms to any communication standard, such as Long Term Evolution (LTE), LTE-Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed Packet Access (HSPA), etc.

[0040] In addition, the communication between devices in the communication system can be carried out according to any stage communication protocol, which can include but is not limited to the following communication protocols: 1G (generation), 2G, 2.5G, 2.75G, 3G, 4G, 4.5G and 5G, New Radio (NR), future 6G, etc., and / or other currently known or to be developed in the future. Communication protocol.

[0041] In the embodiments of the present application, the term "network device" refers to, for example, a device that accesses a communication network and provides services for a terminal device in a communication system. The network device can include but is not limited to the following devices: base station (BS), access point (AP), transmission reception point (TRP), broadcast transmitter, mobile management entity (MME), gateway, server, radio network controller (RNC), base station controller (BSC), etc.

[0042] The base station can include, but is not limited to, a NodeB (or NB), an evolved NodeB (or eNB), and a 5G base station (or gNB), an IAB donor, and the like, and can further include a remote radio head (RRH), a remote radio unit (RRU), a relay, or a low-power node (for example, a femto, a pico, and the like). The term "base station" can include some or all functions of them, and each base station can provide communication coverage for a specific geographic area. The term "cell" can refer to a base station and / or its coverage area, depending on the context in which the term is used.

[0043] In the embodiments of the present application, the term "user equipment" (UE) or "terminal equipment" (TE) refers to a device that accesses a communication network through a network device and receives network services, for example. The terminal equipment can be fixed or mobile, and can also be referred to as a mobile station (MS), a terminal, a subscriber station (SS), an access terminal (AT), a station, a mobile terminal (MT), and the like.

[0044] The terminal equipment can include, but is not limited to, a cellular phone, a personal digital assistant (PDA), a wireless modem, a wireless communication device, a handheld device, a machine type communication device, a laptop computer, a cordless phone, a smartphone, a smartwatch, a digital camera, and the like.

[0045] For another example, in an Internet of Things (IoT) scenario or the like, the terminal equipment can also be a machine or device that performs monitoring or measurement, and can include, but is not limited to, a machine type communication (MTC) terminal, a vehicle-mounted communication terminal, a device to device (D2D) terminal, a machine to machine (M2M) terminal, and the like.

[0046] In addition, the term "network side" or "network device side" refers to the side of the network, which can be a certain base station, or can include one or more network devices as above. The term "user side" or "terminal side" or "terminal device side" refers to the side of the user or terminal, which can be a certain UE, or can include one or more terminal devices as above. In this paper, "device" can refer to network device or terminal device without special indication.

[0047] In the following description, the terms "uplink control signal" and "uplink control information (UCI, Uplink Control Information)" or "physical uplink control channel (PUCCH, Physical Uplink Control Channel)" can be interchangeable without causing confusion, and the terms "uplink data signal" and "uplink data information" or "physical uplink shared channel (PUSCH, Physical Uplink Shared Channel)" can be interchangeable;

[0048] The terms "downlink control signal" and "downlink control information (DCI, Downlink Control Information)" or "physical downlink control channel (PDCCH, Physical Downlink Control Channel)" can be interchangeable, and the terms "downlink data signal" and "downlink data information" or "physical downlink shared channel (PDSCH, Physical Downlink Shared Channel)" can be interchangeable.

[0049] In addition, the uplink signal can include uplink data signal and / or uplink control signal and / or PRACH and / or SRS, etc., and can also be referred to as uplink transmission (UL transmission) or uplink information or uplink channel. Transmitting / receiving the uplink transmission on the uplink resource can be understood as transmitting / receiving the uplink transmission using the uplink resource. The downlink signal can include downlink data signal and / or downlink control signal and / or synchronization signal (SS, such as PSS / SSS) and / or broadcast channel (PBCH) and / or SSB (SS / PBCH block, including PSS, SSS and PBCH and its DMRS) and / or CSI-RS, etc., and can also be referred to as downlink transmission (DL transmission) or downlink information or downlink channel. Transmitting / receiving the downlink transmission on the downlink resource can be understood as transmitting / receiving the downlink transmission using the downlink resource.

[0050] In the embodiments of the present application, the high-layer signaling may be, for example, radio resource control (RRC) signaling; the RRC signaling may include, for example, an RRC message, such as a broadcast / common RRC message / signaling (for example, a master information block (MIB), system information), a dedicated RRC message / signaling; or an RRC information element (RRC IE); or an information field included in the RRC message or RRC information element (or an information field included in the information field). The high-layer signaling may also be, for example, medium access control (MAC) signaling; or a MAC control element (MAC CE). However, the present application is not limited thereto.

[0051] In the embodiments of the present application, multiple means at least two, or two or more.

[0052] In the embodiments of the present application, predefined means defined by a protocol or determined according to a rule defined by a protocol, without additional configuration. Configuration / indication means direct or indirect configuration / indication by a network device through high-layer signaling and / or physical layer signaling. The configuration / indication may be performed by introducing a high-layer parameter in the high-layer signaling, where the high-layer parameter means a field and / or an information element / unit / element (IE) in the high-layer signaling. The physical layer signaling may be, for example, control information (DCI) carried by a physical downlink control channel or control information carried by a sequence, but is not limited thereto.

[0053] For ease of description, the following describes a base station as an example of an access network device. In the following description, “if”, “in the case of” and “when” may be used interchangeably without causing confusion.

[0054] The following describes scenarios of the embodiments of the present application by way of examples, but the present application is not limited thereto.

[0055] FIG. 1 is a schematic diagram of a communication system according to an embodiment of the present application, which schematically illustrates a case taking a terminal device and a network device as examples. As shown in FIG. 1, the communication system 100 may include a network device 101 and terminal devices 102 and 103. For simplicity, FIG. 1 illustrates only two terminal devices and one network device as examples, but the embodiments of the present application are not limited thereto.

[0056] In the embodiments of the present application, the network device 101 and the terminal devices 102, 103 can perform existing services or future implementable service transmission. For example, these services can include, but are not limited to, enhanced mobile broadband (eMBB), massive machine type communication (mMTC), and ultra-reliable and low-latency communication (URLLC), etc.

[0057] It is worth noting that FIG. 1 shows that both terminal devices 102, 103 are within the coverage of the network device 101, but the present application is not limited thereto. Both terminal devices 102, 103 can be outside the coverage of the network device 101, or one terminal device 102 is within the coverage of the network device 101 and the other terminal device 103 is outside the coverage of the network device 101.

[0058] In the embodiments of the present application, one or more AI / ML functions or models can be configured and run in the network device and / or the terminal device. The AI / ML functions or models can be used for various signal processing functions of wireless communication, such as channel state information (CSI) prediction, CSI compression, beam prediction, positioning management, etc.; the present application is not limited thereto.

[0059] In the embodiments of the present application, the following terms have the same meaning, and they can be replaced with each other: AI / ML, artificial intelligence, artificial intelligence or machine learning.

[0060] In the embodiments of the present application, the AI / ML functionality / model can also be referred to as AI / ML function or model, artificial intelligence or machine learning function or model, artificial intelligence function or model, etc., which have the same meaning and can be replaced with each other in the present application.

[0061] Embodiments of the first aspect

[0062] The embodiments of the present application provide a method for monitoring performance, which is described from the terminal device side.

[0063] FIG. 2 is a schematic diagram of a method for monitoring performance according to an embodiment of the present application. As shown in FIG. 2, the method for monitoring performance includes the following steps.

[0064] 201. obtain, using the artificial intelligence functionality or model, predicted beam-related information for a second time window based on measurements of reference signals in a first time window, the second time window being subsequent to the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and

[0065] 202. monitor performance of the artificial intelligence functionality or model based on measurements of reference signals in the second time window and the predicted beam-related information for the second time window.

[0066] In some embodiments, the artificial intelligence functionality or model refers to an AI / ML feature / feature group enabled by a configuration, wherein the configuration is supported based on a condition indicated by a terminal device capability.

[0067] For example, the AL / ML functionality or model can be one or more functionalities or models, or can be one or more logical models, or can be one or more sub- functionalities, or can be one or more features, or can be one or more feature groups.

[0068] Embodiments of the present application are described, for example, in the context of time beam prediction in beam management (i.e., BM case-2) scenarios.

[0069] In operation 201, the artificial intelligence functionality or model is set at the terminal device side.

[0070] As shown in FIG. 2, the method of monitoring performance further comprises:

[0071] 203. the terminal device receives a corresponding reference signal at each time instance or a partial time instance of the second time window and measures the received reference signal.

[0072] In operation 203, the measurement result obtained by measuring the reference signal is, for example, beam-related information.

[0073] In at least some embodiments, operation 203 can be located before operation 202.

[0074] In some embodiments of operation 202, the terminal device can compare the beam-related information obtained by measuring the reference signal in the second time window with the predicted beam-related information for the second time window obtained in operation 201 to monitor the performance of the artificial intelligence functionality or model.

[0075] In this application, the first time window can be referred to as an observation window or a measurement window, for example, and the second time window can be referred to as a prediction window, for example. The lengths of the first time window and the second time window can be pre-configured or configured by a network device to a terminal device.

[0076] FIG. 3 is a schematic diagram of reference signals in the first time window and the second time window, and FIG. 4 is another schematic diagram of reference signals in the first time window and the second time window. As shown in FIG. 3 and FIG. 4, the reference signals in the first time window and the second time window can be channel state information reference signals (CSI-RS) and / or synchronization signal and PBCH blocks (SSB), for example.

[0077] As shown in FIG. 3 and FIG. 4, the second time window is after the first time window. The first time window can include one or more time instances, which can be represented as T1, T2, T3 and T4, for example. The second time window can include one or more time instances, which can be represented as T5, T6, T7 and T8, for example.

[0078] In this application, a time instance is a time point or a time point represented by other time units, for example. The other time units are time slots, frames or sub-frames, for example.

[0079] As shown in FIG. 3 and FIG. 4, the terminal device can receive (e.g., from a network device) the reference signals on the one or more time instances of the first time window. The reference signals received by the terminal device in the first time window can be referred to as first reference signals, which are used for inference or prediction.

[0080] In some examples of operation 201, the terminal device performs measurement on the first reference signals to obtain measurement results (the measurement results are beam-related information obtained by measurement, for example), infers the measurement results using an artificial intelligence function or model, and obtains predicted channel state information (CSI) in the second time window. The predicted channel state information in the second time window can correspond to one or more time instances in the second time window, for example, and for each time instance in the second time window, the terminal device obtains predicted beam-related information for the time instance.

[0081] In some embodiments, the terminal device is configured to perform performance monitoring on the artificial intelligence function or model used in operation 201.

[0082] In operation 203, the terminal device receives a corresponding reference signal at each time instance or part of time instances of the second time window, wherein the reference signal received by the terminal device in the second time window can be referred to as a second reference signal, and the second reference signal is used for performance monitoring of the artificial intelligence function or model.

[0083] In the example shown in FIG. 3, the terminal device receives a corresponding reference signal (i.e., a second reference signal) at each time instance of the second time window, for example, a reference signal is received at T5, T6, T7 and T8. In the example shown in FIG. 4, the terminal device receives a corresponding reference signal (i.e., a second reference signal) at part of the time instances of the second time window, for example, a reference signal is received at T5 and T6. In addition, the terminal device can receive a corresponding reference signal (i.e., a second reference signal) at 1 time instance of the second time window.

[0084] FIGS. 3 and 4 show the case where 1 time instance corresponds to 1 reference signal, but the present application is not limited thereto. For example, 1 time instance can correspond to 2 or more reference signals at the time instance of receiving the reference signal.

[0085] In addition, in the present application, the reference signal received at 1 time instance can also be referred to as the reference signal of the time instance.

[0086] In operation 202, the terminal device can measure the second reference signal, compare the measurement result (e.g., the measured beam-related information) with the predicted beam-related information in the second time window obtained in operation 201, obtain a performance monitoring result, and thus monitor the performance of the artificial intelligence function or model.

[0087] As shown in FIG. 2, the performance monitoring method further comprises:

[0088] 204. The terminal device sends the performance monitoring result to the network device.

[0089] In operation 204, the performance monitoring result obtained in operation 202 can be sent to the network device. For example, the performance monitoring result obtained in operation 202 can be a performance metric of the artificial intelligence function or model, so in operation 204, the terminal device sends the performance metric to the network device.

[0090] For example, for the time beam prediction (i.e., BM case-2) scenario in beam management, the artificial intelligence function or model is configured at the terminal device side, the network device sends reference signals to the terminal device at at least one time instance of the second time window (e.g., a prediction window) to monitor the performance of the artificial intelligence function or model, and the terminal device measures the reference signals at each time instance in the prediction window and obtains performance monitoring results (e.g., performance indicators) corresponding to the reference signals at each time instance. Optionally, the terminal device can send the performance monitoring results to the network device.

[0091] FIG. 5 is a schematic diagram of a sending occasion of the performance monitoring results, and FIG. 6 is another schematic diagram of a sending occasion of the performance monitoring results.

[0092] As shown in FIG. 5, in some examples of operation 204, the terminal device sends the performance indicators respectively after measuring the reference signals at each time instance in the second time window (i.e., after obtaining the measured beam-related information), where the sent performance indicators can be performance indicators obtained based on the latest measurement results. For example, after measuring the reference signal at time instance T5, the terminal device obtains a performance indicator corresponding to the reference signal at time instance T5 based on the measurement result and sends the performance indicator to the network device; after measuring the reference signal at time instance T6, the terminal device obtains a performance indicator corresponding to the reference signal at time instance T6 based on the measurement result and sends the performance indicator to the network device; and for time instances T7 and T8, the operations are similar to those for time instances T5 and T6.

[0093] As shown in FIG. 6, in other examples of operation 204, the terminal device sends the performance indicators after measuring the reference signals at two or more time instances in the second time window (i.e., after obtaining the measured channel state information). Wherein the terminal device can calculate performance indicators corresponding to the reference signals at the two or more time instances in the second time window respectively based on the measurements of the reference signals at the two or more time instances, send an average of the two or more performance indicators to the network device, or send the two or more performance indicators together to the network device. For example, the terminal device obtains performance indicators (e.g., 4 performance indicators) corresponding to the reference signals at time instances T5, T6, T7 and T8 respectively based on the measurement results of the reference signals at the time instances, and then sends an average of the 4 performance indicators to the network device or sends the 4 performance indicators together (e.g., through one report) to the network device.

[0094] As shown in FIG. 2, the performance monitoring method further includes:

[0095] 205. sending the beam-related information obtained by measuring the reference signals in the second time window to the network device.

[0096] As described before, the terminal device can measure the reference signals in the second time window to obtain beam-related information (i.e., measured beam-related information), which includes at least one of beam information (e.g., ID of the reference signal) and measured layer 1 reference signal received power (L1-RSRP).

[0097] For example, for the time beam prediction (i.e., BM case-2) scenario in beam management, the artificial intelligence function or model is configured at the terminal device side, the network device sends the reference signals to the terminal device at at least one time instance of the second time window (e.g., prediction window) to monitor the performance of the artificial intelligence function or model, and the terminal device measures the reference signals at each time instance of the prediction window and obtains the performance monitoring result (e.g., performance indicator) corresponding to the reference signals at each time instance. Optionally, the terminal device can send the beam-related information obtained by measuring the reference signals at each time instance of the prediction window to the network device.

[0098] In operation 205, the terminal device sends the beam-related information obtained by measuring the reference signals in the second time window to the network device. For example, the terminal device obtains the measured beam-related information according to the reference signals at time instance T5, and sends the measured beam-related information to the network device.

[0099] In some examples, the measured beam-related information can be sent after being obtained for the reference signals at one time instance of the second time window, or the beam-related information of each time instance can be obtained by measuring the reference signals at multiple time instances of one second time window respectively, and then the beam-related information of each of the multiple time instances of the one second time window is sent together.

[0100] In operation 205, the beam-related information obtained by measuring the reference signals in the second time window can be sent together with the performance indicator or separately. For example, the terminal device obtains the measured beam-related information according to the reference signals at time instance T5, and further obtains the performance indicator corresponding to the reference signals at time instance T5. The terminal device can send the measured beam-related information and the performance indicator together or separately.

[0101] In this application, the terminal device can operate 205 under a predetermined condition. For example, in the case where the performance of the artificial intelligence function or model (for example, the performance of the artificial intelligence function or model can be characterized by a performance index, and the greater the value of the performance index, the better the performance) is lower than a threshold value, operation 205 is performed, that is, the terminal device sends the network device the beam-related information obtained by measuring the reference signal in the second time window. Wherein, the threshold value can be predefined, or configured by the network device to the terminal device.

[0102] In this application, for the spatial domain beam prediction (i.e., BM case-1) and / or time beam prediction (i.e., BM case-2) scenario of beam management, the operation of monitoring the performance of the artificial intelligence function or model can be triggered by the terminal device or triggered by the network device. For example, the reference signal for monitoring the performance of the artificial intelligence function or model can be sent to the terminal device as needed.

[0103] In some examples, when the terminal device finds that the quality of the beam-related information predicted by the artificial intelligence function or model is deteriorating, the terminal device can send a request to the network device to request monitoring the performance of the activated artificial intelligence function or model, wherein the request can be sent through at least one of a physical uplink control channel (PUCCH), a physical uplink shared channel (PUSCH), and a physical random access channel (PRACH).

[0104] In other examples, the terminal device or the network device can request to monitor the performance of the unactivated artificial intelligence function or model, so as to decide whether to anable / activate the unactivated artificial intelligence function or model according to the performance monitoring result.

[0105] It is worth noting that the above FIG. 2 only schematically illustrates the embodiments of the present application, but the present application is not limited thereto. For example, the execution order between the various operations can be appropriately adjusted, and in addition, some operations can be added or some operations can be reduced. Those skilled in the art can make appropriate modifications based on the above description, and not only limited to the description of the above FIG. 2.

[0106] The above various embodiments only exemplarily illustrate the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications can be made on the basis of the above various embodiments. For example, the above various embodiments can be used alone, or one or more of the above various embodiments can be combined.

[0107] From the above embodiments, the terminal device can use the reference signals at one or more time instances within the prediction window to perform performance monitoring on the artificial intelligence function or model, thereby the terminal device and the network device can have a unified understanding of the operations related to the reference signals in the performance monitoring process, and facilitate accurate performance monitoring.

[0108] Embodiments of the second aspect

[0109] Embodiments of the present application provide a method for monitoring performance, which is described from the network device side. In embodiments of the second aspect, the artificial intelligence function or model is arranged at the network device side.

[0110] FIG. 7 is a schematic diagram of a method for monitoring performance. As shown in FIG. 7, the method comprises:

[0111] 701. using an artificial intelligence function or model, obtaining predicted beam-related information in a second time window according to measurement results of reference signals in a first time window by a terminal device, the second time window being after the first time window, the first time window comprising one or more time instances, and the second time window comprising one or more time instances; and

[0112] 702. monitoring performance of the artificial intelligence function or model according to measurement results of reference signals in the second time window by the terminal device and the predicted beam-related information in the second time window.

[0113] In operation 701, the description of the first time window, the second time window, the reference signals in the first time window, and the reference signals in the second time window can refer to the related description in the embodiments of the first aspect.

[0114] For example, in the scenario of time beam prediction (i.e., BM case-2), the network device sends reference signals (e.g., CSI-RS and / or SSB) to the terminal device at multiple time instances of the first time window (e.g., an observation window or a measurement window), the terminal device measures the reference signals in the first time window, and sends the measurement results (e.g., measured beam-related information) to the network device, and the network device uses an artificial intelligence model to predict the beam-related information in the second time window according to the measurement results, i.e., obtains the predicted beam-related information in the second time window.

[0115] In some examples of operation 702, the network device compares the beam-related information obtained by the terminal device by measuring the reference signals in the second time window with the predicted beam-related information in the second time window to monitor the performance of the artificial intelligence function or model.

[0116] As shown in FIG. 7, the method further comprises:

[0117] 703. The network device transmits corresponding reference signals to the terminal device at each time instance or part of time instances of the second time window; and

[0118] 704. The network device receives beam-related information obtained by the terminal device measuring the reference signals in the second time window.

[0119] For example, in operation 703, the network device transmits corresponding reference signals to the terminal device at each time instance or part of time instances of the second time window; then, the terminal device measures the reference signals in the second time window, obtains measurement results, for example, beam-related information (i.e., measured beam-related information), and transmits the obtained measurement results (i.e., beam-related information) to the network device (operation 704). Thus, in operation 702, the network device can monitor the performance of the artificial intelligence function or model using the beam-related information obtained by the terminal device measuring the reference signals in the second time window.

[0120] In operation 703, the network device can transmit reference signals in the second time window at the time instances as shown in FIG. 3 and FIG. 4.

[0121] In operation 704, the network device can receive beam-related information of the reference signals in the second time window in the following three ways:

[0122] Way 1: The network device receives beam-related information of each reference signal in the second time window respectively, for example, the terminal device transmits measurement results of the reference signals at each time instance of the second time window to the network device after measuring the reference signals at the time instance.

[0123] Way 2: The network device receives beam-related information of each reference signal in the second time window together, for example, the terminal device transmits measurement results of the reference signals at all time instances of the second time window to the network device together (e.g., through one report) after measuring the reference signals at all time instances, i.e., the measurement results of the reference signals are transmitted once for one second time window.

[0124] Way 3: The network device receives beam-related information of the reference signals in the second time window together or respectively and beam-related information obtained by measuring the reference signals in the first time window, for example, the terminal device transmits beam-related information of the reference signals in the second time window and beam-related information obtained by measuring the reference signals in the first time window to the network device together (e.g., through one report) or respectively (e.g., through different reports).

[0125] In operation 704, the beam-related information of each reference signal in the second time window includes at least one of beam information and a measured layer 1 reference signal received power (L1-RSRP).

[0126] In some embodiments, in the case where the beam-related information of each reference signal in the second time window includes a layer 1 reference signal received power (L1-RSRP), the network device can receive the beam-related information of each reference signal in the second time window reported in a differential reference signal received power (RSRP) manner. In this way, the overhead of transmission can be saved.

[0127] For example, the network device receives a first layer 1 reference signal received power (L1-RSRP) in the second time window and a difference between each other layer 1 reference signal received power (L1-RSRP) and the first layer 1 reference signal received power (L1-RSRP), where the first layer 1 reference signal received power (L1-RSRP) in the second time window is the largest one of a plurality of layer 1 reference signal received powers (L1-RSRP) in the second time window.

[0128] For another example, the network device receives a second layer 1 reference signal received power (L1-RSRP) of each time instance in the second time window and a difference between each other layer 1 reference signal received power (L1-RSRP) and the second layer 1 reference signal received power (L1-RSRP) of the time instance, where the second layer 1 reference signal received power (L1-RSRP) of each time instance in the second time window is the largest one of a plurality of layer 1 reference signal received powers (L1-RSRP) of each time instance in the second time window.

[0129] FIG. 8 is a schematic diagram of measurement results of reference signals in a first time window and a second time window. As shown in FIG. 8, for each time instance of the first time window and each time instance of the second time window, a measurement result of a reference signal received at the time instance is obtained, which is, for example, a layer 1 reference signal received power (L1-RSRP).

[0130] As shown in FIG. 8, in the measurement results of reference signals in the second time window, the largest layer 1 reference signal received power (L1-RSRP) is taken as a first layer 1 reference signal received power (L1-RSRP). When the terminal device transmits the measurement results of reference signals in the second time window, the first layer 1 reference signal received power (L1-RSRP) is transmitted, as well as a difference between each other layer 1 reference signal received power (L1-RSRP) and the first layer 1 reference signal received power (L1-RSRP).

[0131] FIG. 9 is another schematic diagram of measurement results of reference signals in the first time window and the second time window. As shown in FIG. 9, for each time instance of the first time window and each time instance of the second time window, a measurement result of a reference signal received at the time instance is obtained, which is, for example, a layer 1 reference signal received power (L1-RSRP).

[0132] As shown in FIG. 9, in the second time window, a maximum layer 1 reference signal received power (L1-RSRP) is selected from a plurality of measurement results (e.g., a plurality of L1-RSRPs) of a reference signal, as a second layer 1 reference signal received power (L1-RSRP) of the reference signal. When the terminal device transmits the measurement results of the reference signals in the second time window, for each time instance, the terminal device transmits the second layer 1 reference signal received power (L1-RSRP) of the time instance, and a difference between other layer 1 reference signal received powers (L1-RSRPs) of the time instance and the second layer 1 reference signal received power (L1-RSRP).

[0133] In some embodiments of the present application, in the case that the beam-related information of each reference signal in the first time window includes a layer 1 reference signal received power (L1-RSRP), the network device receives the beam-related information of each reference signal in the first time window reported in a differential reference signal received power (RSRP) manner. That is, the terminal device reports the beam-related information of each reference signal in the first time window in a differential reference signal received power (RSRP) manner.

[0134] For example, in the case that the network device receives the beam-related information of each reference signal in the first time window and the beam-related information of each reference signal in the second time window, the network device receives: a third layer 1 reference signal received power (L1-RSRP), and a difference between each layer 1 reference signal received power (L1-RSRP) in the first time window and each layer 1 reference signal received power (L1-RSRP) in the second time window and the third layer 1 reference signal received power (L1-RSRP). The third layer 1 reference signal received power (L1-RSRP) is a maximum layer 1 reference signal received power (L1-RSRP) from a plurality of layer 1 reference signal received powers (L1-RSRPs) in the first time window and a plurality of layer 1 reference signal received powers (L1-RSRPs) in the second time window.

[0135] FIG. 10 is still another schematic diagram of measurement results of reference signals in the first time window and the second time window. As shown in FIG. 9, for each time instance of the first time window and each time instance of the second time window, a measurement result of a reference signal received at the time instance is obtained, which is, for example, a layer 1 reference signal received power (L1-RSRP).

[0136] As shown in FIG. 10, the maximum layer 1 reference signal received power (L1-RSRP) among the measurement results of the reference signals in the first time window and the measurement results of the reference signals in the second time window is taken as a third layer 1 reference signal received power (L1-RSRP). The terminal device transmits the third layer 1 reference signal received power (L1-RSRP) and the difference between the other layer 1 reference signal received powers (L1-RSRP) in the first time window and the second time window and the third layer 1 reference signal received power (L1-RSRP) when transmitting the measurement results of the reference signals in the first time window and the measurement results of the reference signals in the second time window.

[0137] In the present application, the operation of monitoring the performance of the artificial intelligence function or model can be triggered by the terminal device or triggered by the network device for the spatial domain beam prediction (i.e., BM case-1) and / or the time beam prediction (i.e., BM case-2) scenario of beam management. For example, the reference signal for monitoring the performance of the artificial intelligence function or model can be transmitted to the terminal device as needed.

[0138] In some examples, the network device transmits a request to the terminal device to request monitoring the performance of the activated or unactivated artificial intelligence function or model. For example, the network device can request monitoring the performance of the unactivated artificial intelligence function or model, so as to decide whether to anable / activate the unactivated artificial intelligence function or model according to the performance monitoring result.

[0139] It is worth noting that the above FIG. 7 only schematically illustrates the embodiments of the present application, but the present application is not limited thereto. For example, the execution order between the operations can be appropriately adjusted, and in addition, some operations can be added or some operations can be reduced. Those skilled in the art can make appropriate modifications according to the above description, and the present application is not limited to the description of the above FIG. 7.

[0140] The above embodiments only exemplarily illustrate the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications can be made on the basis of the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0141] Embodiments of the third aspect

[0142] The embodiments of the present application provide a device for monitoring performance. The device can be a terminal device, or can be one or more components or assemblies configured in the terminal device. The same content as the embodiments of the first and second aspects will not be described herein.

[0143] FIG. 11 is a schematic diagram of an apparatus for monitoring performance of an embodiment of the application, as shown in FIG. 11, the apparatus 1100 for monitoring performance of an embodiment of the application comprises a first processor 1101 configured to:

[0144] obtaining, using an artificial intelligence function or model, predicted beam-related information for a second time window based on measurements of reference signals in a first time window, the second time window being after the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and

[0145] monitoring performance of the artificial intelligence function or model based on measurements of reference signals in the second time window and the predicted beam-related information for the second time window.

[0146] In some embodiments, the apparatus 1100 further comprises:

[0147] a first receiver 1102 configured to receive and measure reference signals at each time instance or a part of time instances of the second time window.

[0148] the first processor is configured to compare the beam-related information obtained from the measurements of the reference signals in the second time window with the predicted beam-related information for the second time window to monitor performance of the artificial intelligence function or model.

[0149] In some embodiments, the apparatus 1100 further comprises:

[0150] a first transmitter 1102 configured to transmit a performance metric of the artificial intelligence function or model to a network device.

[0151] In some embodiments, the first processor is configured to cause the first transmitter to transmit the performance metric respectively after measuring the reference signals at each time instance of the second time window; or

[0152] the first processor is configured to cause the first transmitter to transmit the performance metric after measuring the reference signals at more than two time instances of the second time window.

[0153] In some embodiments, the first processor is configured to calculate the performance metric corresponding to each of the reference signals respectively based on the measurements of the reference signals at more than two time instances of the second time window, and the first transmitter is configured to transmit an average of the performance metrics of the more than two performance metrics to the network device or transmit the more than two performance metrics together to the network device.

[0154] In some embodiments, the first transmitter transmits beam-related information measured on reference signals in the second time window to the network device.

[0155] In some embodiments, the beam-related information measured on reference signals in the second time window includes at least one of beam information and measured layer 1 reference signal received power (L1-RSRP).

[0156] In some embodiments, the first transmitter transmits channel state information measured on reference signals in the second time window together with or separately from the performance indicator.

[0157] In some embodiments, the operation of monitoring the performance of the artificial intelligence function or model is triggered by the terminal device or the network device.

[0158] In some embodiments, the first transmitter of the apparatus transmits a request to the network device to request monitoring of the performance of the artificial intelligence function or model, and the request is transmitted through at least one of a physical uplink control channel (PUCCH), a physical uplink shared channel (PUSCH), and a physical random access channel (PRACH).

[0159] The above embodiments are only exemplary, and the present application is not limited thereto. Any suitable modification can be made to the above embodiments. For example, the above embodiments can be used alone or in combination.

[0160] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The performance monitoring apparatus 1100 can also include other components or modules, and the specific content of these components or modules can be referred to related technologies.

[0161] In addition, for the sake of simplicity, only the connection relationship or signal path between the components or modules is exemplarily shown in FIG. 11, but it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above components or modules can be implemented by hardware facilities such as processors, memories, transmitters, receivers, etc.; the present application is not limited thereto.

[0162] Embodiments of the fourth aspect

[0163] The embodiments of the present application provide a performance monitoring apparatus. The apparatus can be a network device, or a component or component configured in the network device. The same content as the embodiments of the first to third aspects will not be described again.

[0164] FIG. 12 is a schematic diagram of an apparatus for monitoring performance of an artificial intelligence function or model according to an embodiment of the present application. As shown in FIG. 12, the apparatus 1200 for monitoring performance of an artificial intelligence function or model comprises a second processor 1201 configured to:

[0165] obtain, using an artificial intelligence function or model, predicted beam-related information for a second time window based on measurement results of reference signals in a first time window by a terminal device, the second time window being after the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and

[0166] monitor performance of the artificial intelligence function or model based on measurement results of reference signals in the second time window by the terminal device and the predicted beam-related information for the second time window.

[0167] In some embodiments, the second processor compares the beam-related information measured by the terminal device for the reference signals in the second time window with the predicted beam-related information for the second time window to monitor performance of the artificial intelligence function or model.

[0168] In some embodiments, the apparatus 1200 further comprises:

[0169] a second transmitter 1202 configured to transmit corresponding reference signals to the terminal device at each time instance or a partial time instance of the second time window; and

[0170] a second receiver 1203 configured to receive beam-related information measured by the terminal device for the reference signals in the second time window.

[0171] In some embodiments, the second receiver receives the beam-related information for each reference signal in the second time window separately; or

[0172] the second receiver receives the beam-related information for each reference signal in the second time window together; or

[0173] the second receiver receives the beam-related information for the reference signals in the second time window together or separately and the beam-related information measured for the reference signals in the first time window.

[0174] In some embodiments, the beam-related information for each reference signal in the second time window comprises at least one of beam information and measured layer 1 reference signal received power (L1-RSRP).

[0175] In some embodiments, in the case that the beam-related information of each reference signal within the second time window comprises a layer 1 reference signal received power (L1-RSRP), the second receiver receives the beam-related information of each reference signal within the second time window in a differential reference signal received power (RSRP) manner.

[0176] In some embodiments, the second receiver receives a first layer 1 reference signal received power (L1-RSRP) within the second time window and a difference between each other layer 1 reference signal received power (L1-RSRP) and the first layer 1 reference signal received power (L1-RSRP), the first layer 1 reference signal received power (L1-RSRP) within the second time window being a largest one of a plurality of layer 1 reference signal received powers (L1-RSRP) within the second time window; or

[0177] the second receiver receives a second layer 1 reference signal received power (L1-RSRP) of each time instance within the second time window and a difference between an other layer 1 reference signal received power (L1-RSRP) of the time instance and the second layer 1 reference signal received power (L1-RSRP) of the time instance, the second layer 1 reference signal received power (L1-RSRP) of each time instance within the second time window being a largest one of a plurality of layer 1 reference signal received powers (L1-RSRP) of each time instance within the second time window.

[0178] In some embodiments, in the case that the beam-related information of each reference signal within the first time window comprises a layer 1 reference signal received power (L1-RSRP),

[0179] the second receiver receives the beam-related information of each reference signal within the first time window in a differential reference signal received power (RSRP) manner.

[0180] In some embodiments, the second receiver receives:

[0181] the third layer 1 reference signal received power (L1-RSRP); and

[0182] a difference between each layer 1 reference signal received power (L1-RSRP) within the first time window and each layer 1 reference signal received power (L1-RSRP) within the second time window and the third layer 1 reference signal received power (L1-RSRP).

[0183] The third layer 1 reference signal received power (L1-RSRP) is the largest one of the multiple layer 1 reference signal received powers (L1-RSRP) in the first time window and the multiple layer 1 reference signal received powers (L1-RSRP) in the second time window.

[0184] In some embodiments,

[0185] The operation of monitoring the performance of the artificial intelligence function or model is triggered by the terminal device or the network device,

[0186] The transmitter of the apparatus sends a request to the terminal device to request monitoring of the performance of the artificial intelligence function or model.

[0187] The above various embodiments are only exemplarily described for the embodiments of the present application, but the present application is not limited thereto, and can be appropriately modified on the basis of the above various embodiments. For example, the above various embodiments can be used alone, or one or more of the above various embodiments can be combined.

[0188] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The apparatus 1200 for monitoring performance can also include other components or modules, and the specific content of these components or modules can be referred to the related art.

[0189] In addition, for the sake of simplicity, only the connection relationship or signal path between the various components or modules is exemplarily shown in FIG. 12, but it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The various components or modules described above can be implemented by hardware facilities such as processors, memories, transmitters, receivers, etc.; the present application is not limited thereto.

[0190] Embodiments of the fifth aspect

[0191] The embodiments of the present application also provide a communication system, which can be referred to FIG. 1, and the same content as the embodiments of the first to fourth aspects will not be described.

[0192] In some embodiments, the communication system 100 can at least include:

[0193] A network device sends a reference signal to a terminal device in a first time window, receives predicted channel state information (CSI) in a second time window obtained by the terminal device using an artificial intelligence model or model, and receives a performance metric of the artificial intelligence function or model sent by the terminal device, wherein the second time window is after the first time window, the first time window includes one or more time instances, and the second time window includes one or more time instances.

[0194] A terminal device obtains predicted channel state information (CSI) in a second time window using an artificial intelligence function or model according to measurement of a reference signal in a first time window, and monitors the performance of the artificial intelligence function or model according to measurement of a reference signal in the second time window and the predicted channel state information in the second time window.

[0195] In the embodiments of the present application, scenarios including network devices and / or terminal devices are taken as examples.

[0196] In the above scenarios, the network device can include at least one of a core network device, a third-party application device, an operation administration and maintenance (OAM), and an access network device.

[0197] The core network device can refer to a device in a core network (CN) that provides service support for a terminal device. As some examples, the core network device can be at least one of a mobility and management entity (MME), an access and mobility management function (AMF) entity, a session management function (SMF) entity, a user plane function (UPF) entity, a Location Management Function (LMF) entity, and the like, which are not all listed here. Among them, the AMF entity can be responsible for access management and mobility management of the terminal, the SMF entity can be responsible for session management, such as session establishment of a user, the UPF entity can be a functional entity of a user plane, mainly responsible for connecting external networks. The LMF entity can manage the overall coordination and scheduling of resources required for the location of the terminal device registered or accessing the core network device. It should be noted that in the embodiments of the present application, the entity can also be referred to as a network element or a functional entity, such as the AMF entity can also be referred to as an AMF network element or an AMF functional entity, and the like.

[0198] The third-party application device can be an over the top server (OTT) or other third-party device.

[0199] The OAM can be a network device for performing operation (Operation), administration (Administration), and maintenance (Maintenance) management work on the network according to the actual needs of the operator network operation.

[0200] The access network device is an access device through which a terminal device accesses a communication system wirelessly. The access network device can be a base station (BS), a node, an evolved NodeB (eNodeB), a transmission reception point (TRP), a base station (gNB) in a 5th generation (5G) mobile communication system, a base station in a 6th generation (6G) mobile communication system, a base station of a future mobile communication system, or an access node in a WiFi system, etc. The access network device can also be a module or unit that completes part of the functions of a base station, for example, at least one of the following modules or units: a central unit (CU), a distributed unit (DU), a CU control plane (CU-CP), a CU user plane (CU-UP), an integrated access backhaul (IAB), or other modules or units. The embodiments of the present application do not limit the specific technology and / or specific device form of the access network device. The access network device can be deployed on land, including indoors / outdoors, can be handheld or vehicle-mounted; it can also be deployed on water, on an airplane, on a balloon or on a satellite; the access network device can be deployed at a fixed location or on a mobile carrier, and the embodiments of the present application do not limit this.

[0201] In the above scenarios, the terminal device can be a device with wireless transceiver function, which can send signals to the access network device and / or receive signals from the access network device. The terminal device can also be referred to as a terminal, a mobile station, a mobile terminal, etc. The terminal device can be a mobile phone, a tablet, or other devices with wireless intelligent transceiver function. The terminal device can be widely used in various scenarios, such as device-to-device (D2D), vehicle-to-everything (V2X) communication, machine-type communication (MTC), Internet of things (IoT), virtual reality, augmented reality, industrial control, autonomous driving, remote medical treatment, or various smart scenarios.

[0202] In the above scenarios, the access network device and the terminal device, the terminal device and the terminal device can communicate through the licensed spectrum, or through the unlicensed spectrum, or through both the licensed spectrum and the unlicensed spectrum. The embodiments of the present application do not limit the spectrum resources used for wireless communication.

[0203] The embodiments of the present application further provide a terminal device, but the present application is not limited thereto, and can also be other devices.

[0204] FIG. 13 is a schematic diagram of a terminal device according to an embodiment of the present application. As shown in FIG. 13, the terminal device 1300 can include a processor 1310 and a memory 1320. The memory 1320 stores data and programs and is coupled to the processor 1313. It is worth noting that this figure is exemplary; other types of structures can also be used to supplement or replace this structure to achieve telecommunication functions or other functions.

[0205] For example, the processor 1310 can be configured to execute programs to implement the method for monitoring performance according to the embodiments of the first aspect.

[0206] As shown in FIG. 13, the terminal device 1300 can further include a communication module 1330, an input unit 1340, a display 1350, and a power supply 1360. The functions of the above components are similar to those of the prior art, and will not be described here. It is worth noting that the terminal device 1300 does not necessarily include all the components shown in FIG. 13, and the above components are not essential. In addition, the terminal device 1300 can also include components not shown in FIG. 13, which can be referred to the prior art.

[0207] The embodiments of the present application further provide a network device, which can be a base station for example, but the present application is not limited thereto, and can also be other network devices.

[0208] FIG. 14 is a schematic diagram of a network device according to an embodiment of the present application. As shown in FIG. 14, the network device 1400 can include a processor 1410 (such as a central processing unit CPU) and a memory 1420. The memory 1420 is coupled to the processor 1410. The memory 1420 can store various data, and further store programs for information processing 1430, and execute the programs 1430 under the control of the processor 1410.

[0209] For example, the processor 1410 can be configured to execute programs to implement the method for monitoring performance according to the embodiments of the second aspect.

[0210] In addition, as shown in FIG. 14, the network device 1400 can further include a transceiver 1440 and an antenna 1450, etc. The functions of the above components are similar to those in the prior art, and thus are not described here. It is worth noting that the network device 1400 does not necessarily include all the components shown in FIG. 14; in addition, the network device 1400 can include components not shown in FIG. 14, which can be referred to the prior art.

[0211] The embodiments of the present application further provide a computer program, which, when executed in a terminal device, causes the terminal device to perform the method for monitoring performance according to the embodiments of the first aspect.

[0212] The embodiments of the present application further provide a storage medium storing a computer program, which causes a terminal device to perform the method for monitoring performance according to the embodiments of the first aspect.

[0213] The embodiments of the present application further provide a computer program, which, when executed in a network device, causes the network device to perform the method for monitoring performance according to the embodiments of the second aspect.

[0214] The embodiments of the present application further provide a storage medium storing a computer program, which causes a network device to perform the method for monitoring performance according to the embodiments of the second aspect.

[0215] The apparatuses and methods described above can be implemented by hardware, or by a combination of hardware and software. The present application relates to a computer readable program, which, when executed by a logic component, can cause the logic component to implement the apparatuses or constituent components described above, or to implement the various methods or steps described above. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0216] The methods / apparatuses described in connection with the embodiments of the present application can be directly embodied as hardware, software modules executed by a processor, or a combination of the two. For example, one or more of the functional blocks shown in the functional block diagrams, and / or a combination of one or more of the functional blocks, can correspond to individual software modules of a computer program flow, or to individual hardware modules. These software modules can correspond to individual steps shown in the diagrams, respectively. These hardware modules can be implemented by, for example, fixing the software modules with a field programmable gate array (FPGA).

[0217] The software modules can reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a mobile disk, a CD-ROM, or any other form of storage medium known in the art. One storage medium can be coupled to the processor, such that the processor can read information from, and write information to, the storage medium; or the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The software modules can be stored in a memory of the mobile terminal, or in a memory card that can be inserted into the mobile terminal. For example, if the device (e.g., mobile terminal) uses a MEGA-SIM card or a flash memory device of large capacity, the software modules can be stored in the MEGA-SIM card or the flash memory device of large capacity.

[0218] One or more of the functional blocks described in the figures and / or one or more combinations of the functional blocks can be implemented as a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any appropriate combination thereof, for performing the functions described in this application. One or more of the functional blocks described in the figures and / or one or more combinations of the functional blocks can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0219] The application has been described above with the attachment to the specific embodiments; however, it should be clear to those skilled in the art that the descriptions are exemplary and are not a limitation on the scope of protection of the application. Those skilled in the art can make various modifications and changes to the application according to the spirit and principles of the application, and these modifications and changes are also within the scope of the application.

[0220] With regard to the embodiments including the above embodiments, the following notes are also disclosed:

[0221] 1. A method of monitoring performance, applied to an artificial intelligence function or model of a terminal device, wherein the method comprises:

[0222] using the artificial intelligence function or model, obtaining predicted beam-related information for a second time window (prediction window) based on measurements of reference signals in a first time window (observation window), the second time window being subsequent to the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and

[0223] monitoring performance of the artificial intelligence function or model based on the measurements of the reference signals within the second time window and the predicted beam-related information within the second time window.

[0224] 2. A method of monitoring performance of an artificial intelligence function or model applied to a network device, wherein the method comprises:

[0225] using an artificial intelligence function or model to obtain predicted beam-related information within a second time window based on measurements of reference signals within a first time window by a terminal device, the second time window being subsequent to the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and

[0226] monitoring performance of the artificial intelligence function or model based on the measurements of the reference signals within the second time window and the predicted beam-related information within the second time window.

[0227] 3. A terminal device comprising a memory and a processor, the memory storing a computer program, the processor being configured to execute the computer program to implement the method of clause 1.

[0228] 4. A network device comprising a memory and a processor, the memory storing a computer program, the processor being configured to execute the computer program to implement the method of clause 2.

[0229] 5. A computer program product comprising at least a computer program which, when executed by a processor, causes a terminal device to perform the method of clause 1.

[0230] 6. A computer program product comprising at least a computer program which, when executed by a processor, causes a network device to perform the method of clause 2.

Claims

1. An apparatus for monitoring performance, applied to an artificial intelligence function or model of a terminal device, wherein, The apparatus comprises a first processor configured to: obtain, using an artificial intelligence function or model, predicted beam-related information in a second time window according to measurements of reference signals in a first time window, the second time window being after the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and monitor performance of the artificial intelligence function or model according to measurements of reference signals in the second time window and the predicted beam-related information in the second time window.

2. The apparatus of claim 1, wherein, The apparatus further comprises: a first receiver that receives and measures reference signals at each time instance or partial time instance of the second time window, the first processor compares beam-related information measured from the reference signals in the second time window with the predicted beam-related information in the second time window to monitor the performance of the artificial intelligence function or model.

3. The apparatus of claim 2, wherein, The apparatus further comprises: a first transmitter that transmits a performance indicator of the artificial intelligence function or model to a network device.

4. The apparatus of claim 3, wherein the first transmitter transmits the performance indicator respectively after the first processor measures the reference signals at each time instance of the second time window; or the first transmitter transmits the performance indicator after the first processor measures the reference signals at more than two time instances of the second time window.

5. The apparatus of claim 4, wherein the first processor calculates a performance indicator corresponding to each reference signal respectively according to measurements of the reference signals at more than two time instances of the second time window, and the first transmitter transmits an average of the more than two performance indicators to the network device or transmits the more than two performance indicators together to the network device.

6. The apparatus of claim 3, wherein the first transmitter transmits beam-related information measured from the reference signals in the second time window to the network device.

7. The apparatus of claim 6, wherein the beam-related information measured from the reference signals in the second time window comprises at least one of beam information and measured layer 1 reference signal received power (L1-RSRP).

8. The apparatus of claim 6, wherein the first transmitter transmits channel state information measured from the reference signals in the second time window together with or separately from the performance indicator.

9. The apparatus of claim 1, wherein the operation of monitoring the performance of the artificial intelligence function or model is triggered by the terminal device or the network device.

10. The apparatus of claim 9, wherein the first transmitter of the apparatus transmits a request to the network device to request monitoring of the performance of the artificial intelligence function or model, the request being transmitted through at least one of a physical uplink control channel, a physical uplink shared channel and a physical random access channel.

11. An apparatus for monitoring performance, applied to an artificial intelligence function or model of a network device, wherein, The apparatus comprises a second processor configured to: obtaining, using an artificial intelligence function or model, predicted beam-related information in a second time window, according to measurement results of reference signals in the first time window by the terminal device, the second time window being after the first time window, the first time window comprising one or more time instances, the second time window comprising one or more time instances; and monitoring performance of the artificial intelligence function or model, according to measurement results of reference signals in the second time window by the terminal device, and the predicted beam-related information in the second time window.

12. The apparatus of claim 11, wherein, the second processor compares the beam-related information measured by the terminal device on the reference signals in the second time window with the predicted beam-related information in the second time window, to monitor the performance of the artificial intelligence function or model.

13. The apparatus of claim 11, wherein, the apparatus further comprises: a second transmitter that transmits corresponding reference signals to the terminal device at each time instance or partial time instance of the second time window; and a second receiver that receives the beam-related information measured by the terminal device on the reference signals in the second time window.

14. The apparatus of claim 13, wherein, the second receiver receives the beam-related information of each reference signal in the second time window respectively; or the second receiver receives the beam-related information of each reference signal in the second time window together; or the second receiver receives the beam-related information of the reference signals in the second time window together or respectively, and the beam-related information measured on the reference signals in the first time window.

15. The apparatus of claim 4, wherein, the beam-related information of each reference signal in the second time window comprises at least one of beam information and measured layer 1 reference signal received power.

16. The apparatus of claim 15, wherein, in the case that the beam-related information of each reference signal in the second time window comprises layer 1 reference signal received power, the second receiver receives the beam-related information of each reference signal in the second time window reported in a differential reference signal received power manner.

17. The apparatus of claim 16, wherein, the second receiver receives a first layer 1 reference signal received power in the second time window and a difference between each other layer 1 reference signal received power and the first layer 1 reference signal received power, the first layer 1 reference signal received power in the second time window being a largest one of a plurality of layer 1 reference signal received powers in the second time window; or the second receiver receives a first layer 1 reference signal received power in the second time window and a difference between each other layer 1 reference signal received power and the first layer 1 reference power, the first layer 1 reference signal received power in the second time window being a largest one of a plurality of layer 1 reference signal received powers in the second time window. ​ The second receiver receives the second layer 1 reference signal received power of each time instance within the second time window and the difference between the other layer 1 reference signal received power of each time instance and the second layer 1 reference signal received power of the time instance, the second layer 1 reference signal received power of each time instance within the second time window being the largest one of the multiple layer 1 reference signal received powers of each time instance within the second time window.

18. The apparatus of claim 16, wherein, In the case that the beam-related information of each reference signal within the first time window comprises a layer 1 reference signal received power, The second receiver receives the beam-related information of each reference signal within the first time window reported in a differential reference signal received power manner.

19. The apparatus of claim 18, wherein, The second receiver receives: The third layer 1 reference signal received power; and The difference between each layer 1 reference signal received power within the first time window and each layer 1 reference signal received power within the second time window and the third layer 1 reference signal received power, wherein, The third layer 1 reference signal received power is the largest one of the multiple layer 1 reference signal received powers within the first time window and the multiple layer 1 reference signal received powers within the second time window.

20. The apparatus of claim 11, wherein, The operation of monitoring the performance of the artificial intelligence function or model is triggered by the terminal device or the network device, The second transmitter of the apparatus sends a request to the terminal device to request monitoring the performance of the artificial intelligence function or model. ​

Images