Performance monitoring method and apparatus

Abstract

The embodiments of the present application provide a performance monitoring method and apparatus. The method comprises: a terminal device receives a first configuration sent by a network device; according to the first configuration, the terminal device monitors the performance of an AI / ML model which is enabled or has a function activated, the function comprising a feature or a feature group enabled by a configuration indicated by the capability of the terminal device; and, according to the monitoring, the terminal device determines whether the function fails.

Description

Performance monitoring method and device Technical Field

[0001] The embodiments of the present application relate to the field of communication technologies. Background Art

[0002] NR Release 18 studies AI / ML over the air interface. AI / ML can be used for the following use cases: CSI feedback enhancement, beam management, and positioning enhancement. CSI feedback enhancement can include CSI prediction and CSI compression; beam management can include spatial beam prediction and temporal beam prediction; and positioning enhancement can include direct positioning and AI / ML-assisted positioning.

[0003] In some sub-use cases, a bilateral model can be used, with the AI / ML model located on both the terminal device side and the network device side. For example, CSI compression can be a representative use case for a bilateral model. In other sub-use cases, a unilateral model can be used, with the AI / ML model located either on the terminal device side or the network device side.

[0004] It should be noted that the above introduction to the technical background is merely intended to provide a clear and complete description of the technical solutions of this application and facilitate understanding by those skilled in the art. Simply because these solutions are described in the background technology section of this application, it should not be assumed that the above technical solutions are well known to those skilled in the art.

[0005] Summary of the Invention

[0006] The inventors discovered that the performance of AI / ML models located on the terminal device side should be monitored by the terminal device. However, there is currently no clear and accurate solution for how to monitor this.

[0007] To address at least one of the above problems, embodiments of the present application provide a performance monitoring method and apparatus.

[0008] According to one aspect of an embodiment of the present application, a performance monitoring method is provided, including:

[0009] The terminal device receives the first configuration sent by the network device;

[0010] monitoring the performance of an AI / ML model of an enabled or activated function according to the first configuration, the function comprising a feature or feature group enabled by the configuration indicated by the terminal device capability; and

[0011] A determination is made based on the monitoring whether the function has failed.

[0012] According to another aspect of an embodiment of the present application, a performance monitoring device is provided, including:

[0013] a receiving unit configured to receive a first configuration sent by a network device;

[0014] a monitoring unit configured to monitor, according to the first configuration, a performance of an AI / ML model of an enabled or activated function, the function comprising a feature or feature group enabled by the configuration indicated by the terminal device capability; and

[0015] A processing unit determines whether the function fails based on the monitoring.

[0016] According to another aspect of an embodiment of the present application, a performance monitoring method is provided, including:

[0017] The network device sends a first configuration to the terminal device;

[0018] The terminal device monitors the performance of the AI / ML model of the enabled or activated function according to the first configuration, where the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determines whether the function fails based on the monitoring.

[0019] According to another aspect of an embodiment of the present application, a performance monitoring device is provided, including:

[0020] a sending unit configured to send a first configuration to a terminal device;

[0021] The terminal device monitors the performance of the AI / ML model of the enabled or activated function according to the first configuration, where the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determines whether the function fails based on the monitoring.

[0022] According to another aspect of an embodiment of the present application, a communication system is provided, including:

[0023] A network device that sends a first configuration to a terminal device;

[0024] A terminal device that monitors the performance of an AI / ML model of an enabled or activated function according to the first configuration, wherein the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determines whether the function fails based on the monitoring.

[0025] One of the beneficial effects of the embodiments of the present application is that it can accurately monitor the AI / ML model located on the terminal device side, thereby improving the accuracy and reliability of AI / ML.

[0026] With reference to the following description and accompanying drawings, specific embodiments of the present application are disclosed in detail, indicating the manner in which the principles of the present application can be employed. It should be understood that the embodiments of the present application are not limited in scope. Within the spirit and scope of the appended claims, the embodiments of the present application include many variations, modifications and equivalents.

[0027] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0028] It should be emphasized that the term "include / comprising" when used herein refers to the presence of features, integers, steps or components, but does not exclude the presence or addition of one or more other features, integers, steps or components. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The elements and features described in one figure or one embodiment of the present application can be combined with the elements and features shown in one or more other figures or embodiments. In addition, in the accompanying drawings, similar reference numerals represent corresponding parts in several figures and can be used to indicate corresponding parts used in more than one embodiment.

[0030] FIG1 is a schematic diagram of a communication system according to an embodiment of the present application;

[0031] FIG2 is a schematic diagram of a performance monitoring method according to an embodiment of the present application;

[0032] FIG3 is a schematic diagram of performance monitoring of a function according to an embodiment of the present application;

[0033] FIG4 is an example diagram of performance monitoring for multiple functions according to an embodiment of the present application;

[0034] FIG5 is a schematic diagram of a performance monitoring method according to an embodiment of the present application;

[0035] FIG6 is a schematic diagram of a performance monitoring device according to an embodiment of the present application;

[0036] FIG7 is a schematic diagram of a performance monitoring device according to an embodiment of the present application;

[0037] FIG8 is a schematic diagram of a network device according to an embodiment of the present application;

[0038] FIG9 is a schematic diagram of a terminal device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] The above and other features of the present application will become apparent through the following description with reference to the accompanying drawings. In the description and the accompanying drawings, specific embodiments of the present application are disclosed in detail, which illustrate some embodiments in which the principles of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the present application includes all modifications, variations and equivalents that fall within the scope of the appended claims.

[0040] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements from the name, but do not indicate the spatial arrangement or temporal order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0041] In the embodiments of this application, the singular forms "a," "the," etc. include plural forms and should be broadly understood to mean "a" or "a type" rather than being limited to "one." Furthermore, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. Furthermore, the term "according to" should be understood to mean "at least in part based on...", and the term "based on" should be understood to mean "at least in part based on...", unless the context clearly indicates otherwise.

[0042] In the embodiments of the present application, the term "communication network" or "wireless communication network" may refer to a network that complies with any of the following communication standards, such as Long Term Evolution (LTE), enhanced Long Term Evolution (LTE-A, LTE-Advanced), Wideband Code Division Multiple Access (WCDMA), High-Speed ​​Packet Access (HSPA), etc.

[0043] Furthermore, communication between devices in the communication system may be carried out according to communication protocols of any stage, for example, including but not limited to the following communication protocols: 1G (generation), 2G, 2.5G, 2.75G, 3G, 4G, 4.5G and 5G, New Radio (NR), future 6G, etc., and / or other communication protocols currently known or to be developed in the future.

[0044] In the embodiments of the present application, the term "network device" refers to, for example, a device in a communication system that connects a terminal device to the communication network and provides services to the terminal device. Network devices may include, but are not limited to, the following devices: base station (BS), access point (AP), transmission reception point (TRP), broadcast transmitter, mobile management entity (MME), gateway, server, radio network controller (RNC), base station controller (BSC), etc.

[0045] Among them, base stations may include but are not limited to: NodeB (NodeB or NB), evolved NodeB (eNodeB or eNB) and 5G base station (gNB), IAB host, etc., and may also include remote radio head (RRH, Remote Radio Head), remote radio unit (RRU, Remote Radio Unit), relay (relay) or low-power node (such as femeto, pico, etc.). The term "base station" can include some or all of their functions. Each base station can provide communication coverage for a specific geographical area. The term "cell" can refer to a base station and / or its coverage area, depending on the context in which the term is used.

[0046] In the embodiments of the present application, the term "user equipment" (UE) or "terminal equipment" (TE) refers to, for example, a device that accesses a communication network through a network device and receives network services. A terminal device can be fixed or mobile and may also be referred to as a mobile station (MS), a terminal, a subscriber station (SS), an access terminal (AT), a station, and so on.

[0047] Among them, terminal devices may include but are not limited to the following devices: cellular phones, personal digital assistants (PDAs), wireless modems, wireless communication devices, handheld devices, machine-type communication devices, laptop computers, cordless phones, smart phones, smart watches, digital cameras, etc.

[0048] For another example, in scenarios such as the Internet of Things (IoT), the terminal device can also be a machine or device for monitoring or measurement, including but not limited to: machine type communication (MTC) terminal, vehicle-mounted communication terminal, device-to-device (D2D) terminal, machine-to-machine (M2M) terminal, and so on.

[0049] In addition, the term "network side" or "network device side" refers to one side of the network, which can be a base station or one or more network devices as described above. The term "user side" or "terminal side" or "terminal device side" refers to the user or terminal side, which can be a UE or one or more terminal devices as described above. Unless otherwise specified herein, "device" can refer to either network equipment or terminal equipment.

[0050] The following describes the scenarios of the embodiments of the present application through examples, but the present application is not limited thereto.

[0051] FIG1 is a schematic diagram of a communication system according to an embodiment of the present application, schematically illustrating a situation using a terminal device and a network device as an example. As shown in FIG1 , a communication system 100 may include a network device 101 and terminal devices 102 and 103. For simplicity, FIG1 illustrates only two terminal devices and one network device as an example, but the embodiments of the present application are not limited thereto.

[0052] In the embodiment of the present application, existing services or future services can be transmitted between the network device 101 and the terminal devices 102 and 103. For example, these services may include but are not limited to: enhanced mobile broadband (eMBB), massive machine type communication (mMTC), and ultra-reliable and low-latency communication (URLLC), etc.

[0053] It is worth noting that FIG1 shows that both terminal devices 102 and 103 are within the coverage range of network device 101, but the present application is not limited thereto. Both terminal devices 102 and 103 may not be within the coverage range of network device 101, or one terminal device 102 may be within the coverage range of network device 101 while the other terminal device 103 is outside the coverage range of network device 101.

[0054] In the embodiments of the present application, the high-layer signaling may be, for example, radio resource control (RRC) signaling; for example, an RRC message, including, for example, an MIB, system information, or a dedicated RRC message; or an RRC information element (RRC IE). The high-layer signaling may also be, for example, MAC (Medium Access Control) signaling; or a MAC control element (MAC CE). However, the present application is not limited thereto.

[0055] In embodiments of the present application, one or more AI / ML models may be configured and run in a network device and / or a terminal device. The AI / ML models may be used for various signal processing functions in wireless communications, such as CSI prediction, CSI compression, beamforming, positioning management, and the like; however, the present application is not limited thereto.

[0056] Embodiments of the first aspect

[0057] The embodiment of the present application provides a performance monitoring method. FIG2 is a schematic diagram of the performance monitoring method of the embodiment of the present application. As shown in FIG2 , the method includes:

[0058] 201. A terminal device receives a first configuration sent by a network device.

[0059] 202, the terminal device monitors the performance of an AI / ML model of an enabled or activated function according to a first configuration, wherein the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and

[0060] 203. The terminal device determines whether the function fails based on monitoring.

[0061] It is worth noting that FIG2 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG2 above.

[0062] In some embodiments, functionality refers to an AI / ML feature / feature group enabled by a configuration, where the configuration is supported based on conditions indicated by UE capabilities.

[0063] For example, the AL / ML function may be one or more functions, or one or more logical models, or one or more sub-functions, or one or more features, or one or more feature groups.

[0064] For another example, the function can be to use AI / ML for spatial beam prediction, or to use AI / ML for time beam prediction, or to use AI / ML for CSI prediction, or to use AI / ML for direct positioning, or to use AI / ML for assisted positioning, and so on.

[0065] In some embodiments, the AI / ML model is located on a terminal device, and the terminal device monitors the performance of the AI / ML model and determines (determines, detects) whether the AI / ML performance corresponding to one or more functions of the AI / ML model is functioning properly.

[0066] In some embodiments, the first configuration includes at least one of the following: a performance metric, a parameter for controlling the monitoring, and a reference signal configuration. This application is not limited thereto, and the first configuration may include other information / parameters / conditions / resource configurations used for performance monitoring. Furthermore, the first configuration may include any one of the aforementioned information, or any combination of two or more.

[0067] For example, the AI / ML function is located on the UE side. After the AI / ML function is enabled or activated, the UE monitors the AI / ML operation according to a first configuration from the network side. The first configuration may include performance metrics, parameters for controlling monitoring (e.g., parameters for failure event triggering, parameters for activation / deactivation triggering, such as counters, timers, thresholds, conditions), CSI-RS resource configuration, etc. The performance metrics may include AI / ML output performance, data input / output distribution, measurement statistics compared with input statistics, etc.

[0068] In some embodiments, the terminal device receives a second configuration sent by the network device; and the terminal device sends AI / ML performance information corresponding to the function to the network device according to the second configuration.

[0069] For example, when the UE is able to detect and determine that the AI / ML function cannot work normally, the UE sends the AI / ML performance information according to the second configuration from the network side.

[0070] In some embodiments, the second configuration includes at least one of the following: a reporting configuration, uplink resources for transmitting the AI / ML performance information, and a method for reporting the AI / ML performance information; the method includes periodic reporting, semi-continuous reporting, or aperiodic reporting; the AI / ML performance information includes at least one of the following: a function failure indication, input data drift, and output data drift; the present application is not limited thereto. Furthermore, the second configuration or performance information may include any one of the above information, or any combination of two or more.

[0071] In some embodiments, the function is one or more, and the terminal device performs AI / ML performance monitoring for each function separately.

[0072] FIG3 is a schematic diagram of a function performance monitoring process according to an embodiment of the present application. As shown in FIG3 , the process includes:

[0073] 301. The terminal device receives a reference signal for AI / ML performance monitoring sent by a network device.

[0074] 302. The terminal device determines whether the function fails according to the reference signal.

[0075] For example, the UE monitors the AI / ML performance of a certain function and detects whether the performance is degraded. For example, the UE can detect (determine) whether a functionality failure occurs.

[0076] 303. When the function fails, the terminal device sends indication information to the network device; and 304. The terminal device receives response information fed back by the network device based on the indication information, where the response information includes at least one of the following: functionality activation information, functionality deactivation information, functionality fallback information, functionality switching information, or reconfiguration information. The present application is not limited thereto. In addition, the response information may include any one of the above information, or any combination of two or more.

[0077] It is worth noting that FIG3 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG3 above.

[0078] In some embodiments, the terminal device determines whether a function has failed based on a performance metric; wherein, when the performance metric is lower than a threshold, the failure instances are counted; and when multiple consecutive failure instances are detected within a time window, the function is determined to have failed.

[0079] For example, the UE can detect whether a failure has occurred based on a performance metric (e.g., SGCS). If the performance metric is below a certain threshold, the failure instance is counted. If multiple consecutive failure instances are detected within a time window, the function can be considered (determined, declared) to have failed.

[0080] In some embodiments, the reference signal is a periodic signal for AI / ML performance monitoring of a function, for example, a periodic CSI-RS.

[0081] In some embodiments, the reference signal for AI / ML performance monitoring of a function is configured by radio resource control (RRC), or the reference signal for AI / ML performance monitoring of a function is determined by the terminal device.

[0082] For example, a base station may configure a periodic reference signal for performance monitoring of a function. For example, whether a reference signal is used for monitoring, measurement, or prediction may be explicitly identified through RRC configuration. For another example, it is also possible not to explicitly configure whether a reference signal is used for monitoring, measurement, or prediction, and the UE may independently determine whether a reference signal is used for monitoring, measurement, or prediction.

[0083] For another example, if no reference signal for performance monitoring is configured, the reference signal used by the UE for measurement and / or inference may be used for performance monitoring of one or more functions.

[0084] For another example, the reference signal may be semi-persistent or aperiodic.

[0085] In some embodiments, the indication information includes a failure indication of a function.

[0086] In some embodiments, a failure indication of a function is sent via at least one of: dedicated random access channel resources, uplink control channel (PUCCH) resources, MAC CE, two-step random access, or radio resource control (RRC) message.

[0087] For example, the function failure indication may be sent via a dedicated PRACH. For one function, a dedicated PRACH resource may be configured for the UE.

[0088] For another example, the function failure indication may be sent via PUCCH (eg, similar to PUCCH SR) resources. For a function, resources similar to PUCCH SR may be configured for the UE.

[0089] For another example, a function failure indication may be sent via a MAC CE. This MAC CE may be newly defined. A new MAC CE may be introduced for a function, or an existing MAC CE may be reused to send the function failure indication.

[0090] For another example, the function failure indication may be sent via two-step RACH. For example, the UE may send a preamble and a PUSCH to the base station, the function failure information being included in the PUSCH, and the UE receives an RA response sent by the base station.

[0091] For another example, the function failure indication may be sent via an RRC message. A separate RRC message may be defined for each function.

[0092] In the embodiments of the present application, different functions may use the same or different methods to indicate function failure. For example, a dedicated RACH resource may be used to send a function failure indication for function 1, a MAC CE may be used to send a function failure indication for function 2, an RRC message may be used to send a function failure indication for function 3, and so on.

[0093] In some embodiments, the response information is directed to a function and is sent via at least one of the following: downlink control information (DCI), MAC CE, or RRC message.

[0094] For example, after receiving a function failure indication from the UE, the base station may send a response for activation / deactivation / fallback of a function or a response for function switching / reconfiguration to the UE. The response may be sent via a DCI / MAC CE / RRC message.

[0095] In some embodiments, the reference signal is a periodic signal for AI / ML performance monitoring of multiple functions.

[0096] For example, for model / performance monitoring, one or more steps in the monitoring process shown in Figure 3 can be used for multiple functions. For example, the reference signal used for performance monitoring and / or functional failure indication can be used for AI / ML for spatial beam prediction and AI / ML for temporal beam prediction.

[0097] In some embodiments, the reference signal for AI / ML performance monitoring of multiple functions is configured by radio resource control (RRC), or the reference signal for AI / ML performance monitoring of multiple functions is determined by the terminal device.

[0098] For example, a base station may configure a periodic reference signal for performance monitoring of multiple functions. For example, whether a reference signal is used for monitoring, measurement, or prediction may be explicitly identified through RRC configuration. For another example, it is also possible not to explicitly configure whether a reference signal is used for monitoring, measurement, or prediction, and the UE may independently determine whether a reference signal is used for monitoring, measurement, or prediction.

[0099] For another example, if no reference signal for performance monitoring is configured, the reference signal used by the UE for measurement and / or inference may be used for performance monitoring of one or more functions.

[0100] For another example, the reference signal may be semi-persistent or aperiodic.

[0101] In some embodiments, the indication information includes an indication of failure of one or more functions.

[0102] In some embodiments, the failure indication of one or more functions is sent via at least one of the following: uplink control channel resources, MAC CE, two-step random access, and radio resource control (RRC) message.

[0103] For example, a function failure indication can be sent via a resource similar to a PUCCH SR. A resource similar to a PUCCH SR can be configured for several functions. For example, a new MAC CE can be introduced that at least contains information about which functions failed.

[0104] FIG4 is an example diagram of performance monitoring for multiple functions according to an embodiment of the present application. As shown in FIG4 , the process includes:

[0105] 401. The terminal device receives a reference signal for AI / ML performance monitoring sent by the network device.

[0106] 402. The terminal device determines whether the function fails according to the reference signal.

[0107] For example, the UE monitors the AI / ML performance of multiple functions separately and detects whether the performance is degraded. For example, the UE can detect whether a functionality failure occurs.

[0108] 403: When at least one function fails, the terminal device sends a request message to the network device;

[0109] For example, if at least one function fails, the UE sends resources like a PUCCH SR.

[0110] 404. The terminal device receives an uplink grant sent by the network device.

[0111] For example, after receiving a resource such as a PUCCH SR, the base station sends a DCI containing an uplink grant for PUSCH transmission.

[0112] 405, the terminal device sends a MAC CE indicating at least one function failure to the network device;

[0113] For example, the UE sends a MAC-CE to the gNB to indicate which function has failed.

[0114] 406. The terminal device receives the response information fed back by the network device.

[0115] It is worth noting that FIG4 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG4 above.

[0116] For another example, a function failure indication may be sent via a MAC CE. This MAC CE may be newly defined. A new MAC CE may be introduced for multiple functions, and this MAC CE may indicate at least which function failed. Alternatively, an existing MAC CE may be reused to send the function failure indication.

[0117] For another example, the function failure indication may be sent via two-step RACH. For example, the UE may send a preamble and a PUSCH to the base station, the function failure information being included in the PUSCH, and the UE receives an RA response sent by the base station.

[0118] For another example, the function failure indication may be sent via an RRC message. For multiple functions, a separate RRC message may be defined, and the RRC message may indicate at least which function has failed.

[0119] In the embodiments of the present application, different functions may use the same or different methods to indicate function failure. For example, for some functions, a resource such as a PUCCH SR may be used to send a function failure indication, for other functions, a MAC CE may be used to send a function failure indication, for still other functions, an RRC message may be used to send a function failure indication, and so on.

[0120] In some embodiments, the response information is for multiple functions and is sent via at least one of the following: downlink control information (DCI), MAC CE, or RRC message.

[0121] For example, after receiving a function failure indication from the UE, the base station may send a response to the UE for activation / deactivation / fallback of one or more functions or a response for function switching / reconfiguration. The response may be sent via a DCI / MAC CE / RRC message.

[0122] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0123] It can be seen from the above embodiments that the embodiments of the present application can accurately monitor the AI / ML model located on the terminal device side, thereby improving the accuracy and reliability of AI / ML.

[0124] Embodiments of the second aspect

[0125] The embodiment of the present application provides a performance monitoring method, which is described from the perspective of a network device. The embodiment of the second aspect can be combined with the embodiment of the first aspect, or implemented separately, and the same contents as the embodiment of the first aspect will not be repeated.

[0126] FIG5 is a schematic diagram of a performance monitoring method according to an embodiment of the present application. As shown in FIG5 , the method includes:

[0127] 501, the network device sends a first configuration to the terminal device;

[0128] The terminal device monitors the performance of the AI / ML model of the enabled or activated function according to the first configuration, where the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determines whether the function fails based on the monitoring.

[0129] In some embodiments, as shown in FIG5 , the method may further include:

[0130] 502. The network device sends a second configuration to the terminal device; and receives AI / ML performance information corresponding to the function sent by the terminal device according to the second configuration.

[0131] It is worth noting that FIG5 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG5 above.

[0132] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0133] It can be seen from the above embodiments that the embodiments of the present application can accurately monitor the AI / ML model located on the terminal device side, thereby improving the accuracy and reliability of AI / ML.

[0134] Embodiments of the third aspect

[0135] The embodiment of the present application provides a performance monitoring device, which may be, for example, a terminal device, or one or more components or assemblies configured in the terminal device, and the same contents as those in the first to third aspects of the embodiment will not be repeated.

[0136] FIG6 is a schematic diagram of a performance monitoring device according to an embodiment of the present application. As shown in FIG6 , the performance monitoring device 600 includes:

[0137] A receiving unit 601 receives a first configuration sent by a network device;

[0138] a monitoring unit 602 configured to monitor, according to the first configuration, the performance of an AI / ML model of an enabled or activated function, wherein the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and

[0139] The processing unit 603 determines whether the function fails according to the monitoring.

[0140] In some embodiments, the AI / ML model is located on the terminal device side.

[0141] In some embodiments, the first configuration includes at least one of the following: a performance metric, a parameter for controlling the monitoring, and a reference signal configuration.

[0142] In some embodiments, the receiving unit 601 further receives a second configuration sent by the network device.

[0143] In some embodiments, as shown in FIG6 , the performance monitoring device 600 may further include:

[0144] A sending unit 604 sends the AI / ML performance information corresponding to the function to the network device according to the second configuration.

[0145] In some embodiments, the second configuration includes at least one of the following: reporting configuration, uplink resources for sending the AI / ML performance information, and a method for reporting the AI / ML performance information; the method includes periodic reporting, semi-continuous reporting, or non-periodic reporting; the AI / ML performance information includes at least one of the following: function failure indication, input data drift, and output data drift.

[0146] In some embodiments, the function is one or more, and the terminal device performs AI / ML performance monitoring for each function separately.

[0147] In some embodiments, the receiving unit 601 receives a reference signal for AI / ML performance monitoring sent by the network device; and the processing unit 603 determines whether the function fails based on the reference signal.

[0148] In some embodiments, when the function fails, the sending unit 604 sends an indication message to the network device; and the receiving unit 601 also receives response information fed back by the network device based on the indication message, and the response information includes at least one of the following: function activation information, function deactivation information, function fallback information, function switching information or reconfiguration information.

[0149] In some embodiments, the processing unit 603 determines whether the function has failed based on a performance metric; wherein, when the performance metric is lower than a threshold, the failure instances are counted; and when multiple consecutive failure instances are detected within a time window, the function is determined to have failed.

[0150] In some embodiments, the reference signal is a periodic signal for AI / ML performance monitoring of a function.

[0151] In some embodiments, the reference signal for performing AI / ML performance monitoring on a function is configured by radio resource control (RRC), or the reference signal for performing AI / ML performance monitoring on a function is determined by the terminal device.

[0152] In some embodiments, the indication information includes a failure indication of a function.

[0153] In some embodiments, the failure indication of the one function is sent via at least one of the following: dedicated random access channel resources, uplink control channel resources, MAC CE, two-step random access, and radio resource control (RRC) message.

[0154] In some embodiments, the response information is for the one function and is sent via at least one of the following: downlink control information (DCI), MAC CE, or RRC message.

[0155] In some embodiments, the reference signal is a periodic signal for AI / ML performance monitoring of multiple functions.

[0156] In some embodiments, the reference signal for performing AI / ML performance monitoring on multiple functions is configured by radio resource control (RRC), or the reference signal for performing AI / ML performance monitoring on multiple functions is determined by the terminal device.

[0157] In some embodiments, the indication information includes a failure indication of one or more functions.

[0158] In some embodiments, the failure indication of the one or more functions is sent via at least one of the following: uplink control channel resources, MAC CE, two-step random access, and radio resource control (RRC) message.

[0159] In some embodiments, the response information is for the one or more functions and is sent via at least one of the following: downlink control information (DCI), MAC CE, or RRC message.

[0160] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0161] It is worth noting that the above description only describes the components or modules related to the present application, but the present application is not limited thereto. The performance monitoring device 600 may also include other components or modules. For details of these components or modules, reference may be made to related technologies.

[0162] In addition, for the sake of simplicity, FIG6 only illustrates the connection relationship or signal direction between various components or modules. However, it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors, memories, transmitters, and receivers; the implementation of this application is not limited to this.

[0163] It can be seen from the above embodiments that the embodiments of the present application can accurately monitor the AI / ML model located on the terminal device side, thereby improving the accuracy and reliability of AI / ML.

[0164] Embodiments of the fourth aspect

[0165] The embodiment of the present application provides a performance monitoring device, which may be, for example, a network device, or one or more components or assemblies configured on the network device, and the same contents as those in the first to third aspects of the embodiment will not be repeated.

[0166] FIG7 is a schematic diagram of a performance monitoring device according to an embodiment of the present application. As shown in FIG7 , the performance monitoring device 700 includes:

[0167] A sending unit 701, which sends a first configuration to a terminal device;

[0168] The terminal device monitors the performance of the AI / ML model of the enabled or activated function according to the first configuration, where the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determines whether the function fails based on the monitoring.

[0169] In some embodiments, as shown in FIG7 , the performance monitoring device 700 may further include:

[0170] The receiving unit 702 receives indication information sent by the terminal device when the function fails.

[0171] In some embodiments, the sending unit 702 further sends response information according to the indication information, and the response information includes at least one of the following: function activation information, function deactivation information, function fallback information, function switching information or reconfiguration information.

[0172] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0173] It is worth noting that the above description only describes the components or modules related to the present application, but the present application is not limited thereto. The performance monitoring device 700 may also include other components or modules. For details of these components or modules, reference may be made to related technologies.

[0174] In addition, for the sake of simplicity, FIG7 only illustrates the connection relationship or signal direction between various components or modules. However, it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors, memories, transmitters, and receivers; the implementation of this application is not limited to this.

[0175] It can be seen from the above embodiments that the embodiments of the present application can accurately monitor the AI / ML model located on the terminal device side, thereby improving the accuracy and reliability of AI / ML.

[0176] Embodiments of the fifth aspect

[0177] An embodiment of the present application also provides a communication system, and reference may be made to FIG1 . The contents that are the same as those in the first to fourth aspects of the embodiments will not be repeated.

[0178] In some embodiments, the communication system 100 may include at least:

[0179] A network device that sends a first configuration to a terminal device;

[0180] A terminal device that monitors the performance of an AI / ML model of an enabled or activated function according to the first configuration, wherein the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determines whether the function fails based on the monitoring.

[0181] An embodiment of the present application further provides a network device, which may be, for example, a base station, but the present application is not limited thereto and may also be other network devices.

[0182] Figure 8 is a schematic diagram illustrating the structure of a network device according to an embodiment of the present application. As shown in Figure 8 , network device 800 may include a processor 810 (e.g., a central processing unit (CPU)) and a memory 820 ; the memory 820 is coupled to the processor 810 . The memory 820 may store various data and may also store an information processing program 830 , which is executed under the control of the processor 810 .

[0183] For example, the processor 810 may be configured to execute a program to implement the performance monitoring method as described in the embodiment of the second aspect. For example, the processor 810 may be configured to perform the following control: sending a first configuration to a terminal device; wherein the terminal device monitors the performance of an AI / ML model of an enabled or activated function according to the first configuration, wherein the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determining whether the function fails based on the monitoring.

[0184] In addition, as shown in FIG8 , network device 800 may further include: a transceiver 840 and an antenna 850, etc.; wherein, the functions of the above components are similar to those in the prior art and are not described in detail here. It is worth noting that network device 800 does not necessarily include all the components shown in FIG8 ; in addition, network device 800 may also include components not shown in FIG8 , and reference may be made to the prior art for details.

[0185] The embodiment of the present application also provides a terminal device, but the present application is not limited thereto and may also be other devices.

[0186] Figure 9 is a schematic diagram of a terminal device according to an embodiment of the present application. As shown in Figure 9 , terminal device 900 may include a processor 910 and a memory 920. Memory 920 stores data and programs and is coupled to processor 910. It should be noted that this diagram is exemplary; other types of structures may be used to supplement or replace this structure to implement telecommunication or other functions.

[0187] For example, the processor 910 may be configured to execute a program to implement the performance monitoring method as described in the embodiment of the first aspect. For example, the processor 910 may be configured to perform the following control: receiving a first configuration sent by a network device; monitoring the performance of an AI / ML model of an enabled or activated function according to the first configuration, wherein the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determining whether the function fails based on the monitoring.

[0188] As shown in Figure 9 , the terminal device 900 may further include: a communication module 930, an input unit 940, a display 950, and a power supply 960. The functions of these components are similar to those in the prior art and are not described in detail here. It is worth noting that the terminal device 900 does not necessarily include all of the components shown in Figure 9 , and these components are not essential. Furthermore, the terminal device 900 may also include components not shown in Figure 9 , for which reference may be made to the prior art.

[0189] An embodiment of the present application also provides a computer program, wherein when the program is executed in a terminal device, the program causes the terminal device to execute the performance monitoring method described in the embodiment of the first aspect.

[0190] An embodiment of the present application also provides a storage medium storing a computer program, wherein the computer program enables a terminal device to execute the performance monitoring method described in the embodiment of the first aspect.

[0191] An embodiment of the present application also provides a computer program, wherein when the program is executed in a network device, the program causes the network device to execute the performance monitoring method described in the embodiment of the second aspect.

[0192] An embodiment of the present application also provides a storage medium storing a computer program, wherein the computer program enables a network device to execute the performance monitoring method described in the embodiment of the second aspect.

[0193] The above devices and methods of the present application can be implemented by hardware or by a combination of hardware and software. The present application relates to such a computer-readable program that, when executed by a logic component, enables the logic component to implement the devices or components described above, or enables the logic component to implement the various methods or steps described above. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0194] The method / device described in conjunction with the embodiments of the present application can be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams shown in the figure and / or one or more combinations of functional block diagrams can correspond to various software modules of the computer program flow or to various hardware modules. These software modules can respectively correspond to the various steps shown in the figure. These hardware modules can be implemented by solidifying these software modules, for example, using a field programmable gate array (FPGA).

[0195] The software module may be located in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in the memory of the mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0196] One or more of the functional blocks and / or one or more combinations of functional blocks described in the accompanying drawings may be implemented as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any appropriate combination thereof for performing the functions described in this application. One or more of the functional blocks and / or one or more combinations of functional blocks described in the accompanying drawings may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0197] The present application has been described above in conjunction with specific embodiments. However, those skilled in the art should understand that these descriptions are merely illustrative and are not intended to limit the scope of protection of the present application. Those skilled in the art may make various modifications and variations to the present application based on the spirit and principles of the present application, and such modifications and variations are also within the scope of the present application.

[0198] Regarding the implementation methods including the above embodiments, the following additional notes are also disclosed:

[0199] 1. A performance monitoring method, comprising:

[0200] The terminal device receives the first configuration sent by the network device;

[0201] monitoring the performance of an AI / ML model of an enabled or activated function according to the first configuration, the function comprising a feature or feature group enabled by the configuration indicated by the terminal device capability; and

[0202] A determination is made based on the monitoring whether the function has failed.

[0203] 2. A performance monitoring method, comprising:

[0204] The network device sends a first configuration to the terminal device;

[0205] The terminal device monitors the performance of the AI / ML model of the enabled or activated function according to the first configuration, where the function includes a feature or feature group enabled by the configuration indicated by the terminal device capability; and determines whether the function fails based on the monitoring.

[0206] 3. A terminal device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the performance monitoring method as described in Note 1.

[0207] 4. A network device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the performance monitoring method as described in Note 2.

Claims

1. A performance monitoring device, comprising: A receiving unit, configured to receive a first configuration sent by a network device; a monitoring unit, which monitors the performance of the AI / ML model of an enabled or activated function according to the first configuration, wherein the function includes a feature or a feature group enabled by the configuration indicated by the terminal device capability; as well as A processing unit determines whether the function fails based on the monitoring.

2. The device according to claim 1, wherein: The AI / ML model is located on the terminal device side, and the first configuration includes at least one of the following: performance metrics, parameters for controlling the monitoring, and reference signal configuration.

3. The device according to claim 1, wherein: The receiving unit further receives a second configuration sent by the network device; and the apparatus further includes: A sending unit, which sends the AI / ML performance information corresponding to the function to the network device according to the second configuration.

4. The device according to claim 3, wherein: The second configuration includes at least one of the following: reporting configuration, uplink resources for sending the AI / ML performance information, and a method for reporting the AI / ML performance information; the method includes periodic reporting, semi-continuous reporting, or non-periodic reporting; the AI / ML performance information includes at least one of the following: function failure indication, input data drift, and output data drift.

5. The device according to claim 1, wherein: The function is one or more, and the terminal device performs AI / ML performance monitoring for each function separately.

6. The device according to claim 1, wherein: The receiving unit receives a reference signal for AI / ML performance monitoring sent by the network device; The processing unit determines whether the function fails according to the reference signal.

7. The device according to claim 6, wherein: The device also includes: a sending unit, which sends indication information to the network device when the function fails; and The receiving unit also receives response information fed back by the network device according to the indication information, and the response information includes at least one of the following: function activation information, function deactivation information, function rollback information, function switching information or reconfiguration information.

8. The device according to claim 1, wherein: The processing unit determines whether the function has failed based on the performance metric; Wherein, when the performance metric is lower than a threshold, the failure instances are counted; when multiple consecutive failure instances are detected within a time window, it is determined that the function has failed.

9. The device according to claim 6, wherein: The reference signal is a periodic signal for AI / ML performance monitoring of a function.

10. The device according to claim 9, wherein: The reference signal for performing AI / ML performance monitoring on a function is configured by radio resource control, or the reference signal for performing AI / ML performance monitoring on a function is determined by the terminal device.

11. The device according to claim 7, wherein: The indication information includes a function failure indication.

12. The device according to claim 11, wherein The failure indication of the one function is sent through at least one of the following: dedicated random access channel resources, uplink control channel resources, MAC CE, two-step random access, and radio resource control message.

13. The device according to claim 11, wherein: The response information is for the one function and is sent through at least one of the following: downlink control information, MAC CE, and radio resource control message.

14. The device according to claim 6, wherein: The reference signal is a periodic signal for AI / ML performance monitoring for multiple functions.

15. The device according to claim 14, wherein: The reference signal for performing AI / ML performance monitoring on multiple functions is configured by wireless resource control, or the reference signal for performing AI / ML performance monitoring on multiple functions is determined by the terminal device.

16. The device according to claim 7, wherein: The indication information includes failure indications of one or more functions.

17. The device according to claim 16, wherein: The failure indication of the one or more functions is sent through at least one of the following: uplink control channel resources, MAC CE, two-step random access, and radio resource control message.

18. The device according to claim 16, wherein: The response information is for the one or more functions and is sent through at least one of the following: downlink control information, MAC CE, and radio resource control message.

19. A performance monitoring device, comprising: A sending unit, configured to send a first configuration to a terminal device; The terminal device performs the AI / ML model of the enabled or activated function according to the first configuration. The function includes monitoring the performance of the terminal device, wherein the function includes a feature or a feature group enabled by a configuration indicated by the terminal device capability; and determining whether the function fails based on the monitoring.

20. A communication system comprising: A network device, which sends a first configuration to a terminal device; a terminal device that monitors the performance of an AI / ML model of an enabled or activated function according to the first configuration, wherein the function includes a feature or a feature group enabled by the configuration indicated by the terminal device capability; and determining whether the function fails based on the monitoring.

Images