Performance monitoring method and device, equipment and storage medium

Abstract

The invention provides a performance monitoring method and device, equipment and a storage medium, is applied to terminal equipment, and relates to the technical field of communication, and the method comprises the following steps: sending a performance monitoring request of an AI model to network equipment according to a trigger event; and receiving a response of the performance monitoring request sent by the network equipment, and performing performance monitoring on the AI model according to the response of the performance monitoring request, thereby saving communication resources of model monitoring.

Description

Technical Field

[0001] This application relates to the field of communication technologies, and in particular, to a performance monitoring method, apparatus, device, and storage medium. Background Art

[0002] With the development of artificial intelligence (AI) technology and machine learning (ML), AI and / or ML models (hereinafter referred to as AI models) can be used to improve the performance of communication systems. Among them, the performance monitoring of AI models is particularly important.

[0003] Currently, the base station can periodically send reference signals related to the AI model to the terminal device, and then periodically start the performance monitoring of the AI model. However, in many periods, the AI model does not need to be frequently monitored for performance, which will cause waste of communication resources. Summary of the Invention

[0004] This application relates to a performance monitoring method, apparatus, device, and storage medium, which solves the technical problem of waste of communication resources in the performance monitoring of AI models.

[0005] In a first aspect, this application provides a performance monitoring method, which is applied to a terminal device. The method includes:

[0006] Sending a performance monitoring request for the AI model to the network device according to a trigger event;

[0007] Receiving a response to the performance monitoring request sent by the network device, and performing performance monitoring on the AI model according to the response to the performance monitoring request.

[0008] In one implementation, the performance monitoring request includes an identifier of the AI model and / or an identifier of a function.

[0009] In one implementation, the performance monitoring request includes an identifier indicating the applicable model, an identifier of the applicable function, and an identifier of the applicable scenario corresponding to the performance monitoring request.

[0010] In one implementation, the performance monitoring request includes a start time and / or an end time of performance monitoring.

[0011] In one implementation, the performance monitoring request includes target information, and the target information is related to a performance metric of the AI model and / or a measurement value of the AI model.

[0012] In one implementation, the target information includes at least one of the following:

[0013] The intermediate key performance indicators related to the AI model;

[0014] The final key performance indicators related to the AI model;

[0015] The statistical information of the information related to the channel state information CSI;

[0016] The distribution information of the information related to the CSI;

[0017] The reference signal received power related to the AI model;

[0018] The reference signal received quality related to the AI model;

[0019] Signal-to-interference-plus-noise ratio;

[0020] Speed information;

[0021] Location information.

[0022] In one embodiment, the trigger event includes a first event related to the AI model and / or function and a second event related to the performance of the terminal device.

[0023] In one embodiment, when the AI model is not activated, not selected, or not paired, the first event includes at least one of the following:

[0024] Sending the identifier of the supported AI model and / or the identifier of the function;

[0025] Receiving the identifier of the AI model supported by the cell covered by the current base station and / or the identifier of the function.

[0026] In one embodiment, when the AI model is activated, selected, or paired, the first event is an event determined based on at least one of the following information:

[0027] Measured final key performance indicators;

[0028] The statistical information and / or distribution information of the measured information related to the CSI;

[0029] The statistical information and / or distribution information of the information related to the CSI inferred by the AI model;

[0030] The generalization index of the AI model;

[0031] The update information of the AI model and / or the transfer information of the AI model.

[0032] In one embodiment, the first event is an event determined based on the measured final key performance indicators, and the first event includes at least one of the following:

[0033] The first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter, and the first parameter is proportional to the performance of the AI model;

[0034] The second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter, and the second parameter is inversely proportional to the performance of the AI model;

[0035] The number of times that the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter within the first time period is greater than the first threshold;

[0036] The number of times that the second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter within the first time period is greater than the first threshold;

[0037] The deviation of the parameter in the final key performance indicator from the first target value is greater than the second threshold, and the first target value is the target value corresponding to the parameter in the final key performance indicator.

[0038] In one implementation, the first event is an event determined based on statistical information and / or distribution information of measurement CSI-related information, and the first event includes at least one of the following:

[0039] The third parameter in the statistical information and / or distribution information of the measurement CSI-related information is greater than the threshold corresponding to the third parameter, and the third parameter is inversely proportional to the performance of the AI model;

[0040] The fourth parameter in the statistical information and / or distribution information of the measurement CSI-related information is less than the threshold corresponding to the fourth parameter, and the fourth parameter is proportional to the performance of the AI model;

[0041] The number of times that the third parameter in the statistical information and / or distribution information of the measurement CSI-related information is greater than the threshold corresponding to the third parameter within the second time period is greater than the third threshold;

[0042] The number of times that the fourth parameter in the statistical information and / or distribution information of the measurement CSI-related information is less than the threshold corresponding to the fourth parameter within the second time period is greater than the third threshold;

[0043] The deviation of the parameter in the statistical information and / or distribution information of the measurement CSI-related information from the second target value is greater than the fourth threshold, and the second target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the measurement CSI-related information.

[0044] In one implementation, the first event is an event determined based on statistical information and / or distribution information of CSI-related information inferred by the AI model, and the first event includes at least one of the following:

[0045] The fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter, and the fifth parameter is inversely proportional to the performance of the AI model;

[0046] The sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter, and the sixth parameter is directly proportional to the performance of the AI model;

[0047] The number of times that the fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter within a third time period is greater than a fifth threshold;

[0048] The number of times that the sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter within a third time period is greater than the fifth threshold;

[0049] The deviation between the parameter in the statistical information and / or distribution information of the inferred CSI-related information and a third target value is greater than a sixth threshold, where the third target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the inferred CSI-related information.

[0050] In one implementation, the first event is an event determined based on the generalization metric of the AI model, and the first event includes at least one of the following:

[0051] The duration during which the parameter in the performance metric of the received reference signal is less than a seventh threshold is greater than an eighth threshold;

[0052] The number of times that the parameter in the performance metric of the received reference signal is less than the seventh threshold within a fourth time period is greater than a ninth threshold;

[0053] The quasi co-location information of the received reference signal changes;

[0054] The dataset label of the measured CSI-related information changes;

[0055] The duration during which the parameter in the measured speed information is greater than a tenth threshold is greater than an eleventh threshold;

[0056] The number of times that the parameter in the measured speed information is greater than the tenth threshold within a fifth time period is greater than a twelfth threshold;

[0057] The distance difference between two positions at the measured sixth interval duration is greater than the thirteenth threshold;

[0058] The operating center frequency, subcarriers, and / or bandwidth configuration change.

[0059] In one embodiment, the first event is an event determined based on the update information and / or model transfer information of the AI model, and the first event includes at least one of the following:

[0060] Receiving the model update information and / or model transfer information sent by the network device;

[0061] After receiving the model update information and / or model transfer information sent by the network device, no indication related to performance monitoring is received within the seventh duration.

[0062] In one embodiment, the second event is an event determined based on upper layer signaling, and the second event includes at least one of the following:

[0063] A handover occurs in the cell where it is located;

[0064] The first monitoring quantity is greater than the fourteenth threshold, and the first monitoring quantity is inversely proportional to the performance of the terminal device;

[0065] The second monitoring quantity is less than the fifteenth threshold, and the first monitoring quantity is directly proportional to the performance of the terminal device;

[0066] The first parameter of the final key performance indicator at a future moment predicted based on the final key performance indicator at the current moment is less than the threshold corresponding to the first parameter;

[0067] The second parameter of the final key performance indicator at a future moment predicted based on the final key performance indicator at the current moment is greater than the threshold corresponding to the second parameter.

[0068] In one embodiment, the thresholds, durations, and target values are obtained based on protocol pre - definition and / or network - side configuration and / or a combination of protocol pre - definition and network - side configuration.

[0069] In one embodiment, the CSI - related information includes the CSI information, beam information, and positioning information.

[0070] In one embodiment, the response to the performance monitoring request includes a reply or indication of the performance monitoring request.

[0071] In one embodiment, sending the performance monitoring request of the AI model to the network device includes:

[0072] Sending the performance monitoring request to the network device based on the scheduling request SR of the uplink control information UCI;

[0073] Alternatively,

[0074] send the performance monitoring request to the network device based on the Media Access Control - Control Element (MAC - CE) signaling of the media intervention control cell;

[0075] Alternatively,

[0076] send the performance monitoring request to the network device based on the Physical Uplink Control Channel (PUCCH) resources or Physical Uplink Shared Channel (PUSCH) resources of the reservation period.

[0077] In a second aspect, the present application provides another performance monitoring method, which is applied to a network device. The method includes:

[0078] Receive a performance monitoring request of an AI model sent by a terminal device, where the performance monitoring request is determined by the terminal device based on a triggering event;

[0079] Send a response to the performance monitoring request to the terminal device.

[0080] In an implementation manner, sending the response to the performance monitoring request to the terminal device includes:

[0081] Judge whether the AI model needs to perform performance monitoring according to the information carried in the performance monitoring request, and obtain a judgment result;

[0082] Send a response to the performance monitoring request to the terminal device according to the judgment result.

[0083] In an implementation manner, sending the response to the performance monitoring request to the terminal device includes:

[0084] Receive a Scheduling Request (SR) sent by the terminal device;

[0085] Send a response to the SR to the terminal device, where the response to the SR includes Downlink Control Information (DCI) for uplink transmission resource allocation, and the DCI is scrambled based on the Radio Network Temporary Identity (RNTI) of the terminal device.

[0086] In a third aspect, the present application provides a performance monitoring device, which is applied to a terminal device. The performance monitoring device includes a sending module, a receiving module, and a monitoring module, where:

[0087] The sending module is configured to send a performance monitoring request of an AI model to a network device according to a triggering event;

[0088] The receiving module is configured to receive a response to the performance monitoring request sent by the network device;

[0089] The monitoring module is configured to monitor the performance of the AI model according to the response to the performance monitoring request.

[0090] In one embodiment, the performance monitoring request includes the identifier of the AI model and / or the identifier of the function.

[0091] In one embodiment, the performance monitoring request includes the identifier of the applicable model, the identifier of the applicable function, and the identifier of the applicable scenario corresponding to the performance monitoring request.

[0092] In one embodiment, the performance monitoring request includes the start time and / or the end time of the performance monitoring.

[0093] In one embodiment, the performance monitoring request includes target information, which is related to the performance metrics of the AI model and / or the measurement values of the AI model.

[0094] In one embodiment, the target information includes at least one of the following:

[0095] The intermediate key performance indicators related to the AI model;

[0096] The final key performance indicators related to the AI model;

[0097] The statistical information of the information related to the channel state information CSI;

[0098] The distribution information of the information related to the CSI;

[0099] The reference signal received power related to the AI model;

[0100] The reference signal received quality related to the AI model;

[0101] Signal-to-interference-plus-noise ratio;

[0102] Speed information;

[0103] Location information.

[0104] In one embodiment, the trigger event includes a first event related to the AI model and / or function and a second event related to the performance of the terminal device.

[0105] In one embodiment, when the AI model is not activated, not selected, or not paired, the first event includes at least one of the following:

[0106] Sending the identifier of the supported AI model and / or the identifier of the function;

[0107] Receiving the identifier of the AI model and / or the identifier of the function supported by the cell covered by the current base station.

[0108] In one implementation, when the AI model is activated, selected, or paired, the first event is an event determined based on at least one of the following pieces of information:

[0109] The measured final key performance indicator;

[0110] The statistical information and / or distribution information of the measured CSI-related information;

[0111] The statistical information and / or distribution information of the CSI-related information inferred by the AI model;

[0112] The generalization metric of the AI model;

[0113] The update information of the AI model and / or the transfer information of the AI model.

[0114] In one implementation, the first event is an event determined based on the measured final key performance indicator, and the first event includes at least one of the following:

[0115] A first parameter in the final key performance indicator is less than a threshold corresponding to the first parameter, and the first parameter is proportional to the performance of the AI model;

[0116] A second parameter in the final key performance indicator is greater than a threshold corresponding to the second parameter, and the second parameter is inversely proportional to the performance of the AI model;

[0117] The number of times that the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter within a first time period is greater than a first threshold;

[0118] The number of times that the second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter within the first time period is greater than the first threshold;

[0119] The deviation of a parameter in the final key performance indicator from a first target value is greater than a second threshold, where the first target value is the target value corresponding to the parameter in the final key performance indicator.

[0120] In one implementation, the first event is an event determined based on the statistical information and / or distribution information of the measured CSI-related information, and the first event includes at least one of the following:

[0121] A third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than a threshold corresponding to the third parameter, and the third parameter is inversely proportional to the performance of the AI model;

[0122] The fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is less than the threshold corresponding to the fourth parameter, and the fourth parameter is proportional to the performance of the AI model;

[0123] The number of times that the third parameter in the statistical information and / or distribution information of the measured CSI-related information within the second time period is greater than the threshold corresponding to the third parameter is greater than the third threshold;

[0124] The number of times that the fourth parameter in the statistical information and / or distribution information of the measured CSI-related information within the second time period is less than the threshold corresponding to the fourth parameter is greater than the third threshold;

[0125] The deviation of the parameter in the statistical information and / or distribution information of the measured CSI-related information from the second target value is greater than the fourth threshold, and the second target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the measured CSI-related information.

[0126] In one implementation, the first event is an event determined based on the statistical information and / or distribution information of the CSI-related information inferred by the AI model, and the first event includes at least one of the following:

[0127] The fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter, and the fifth parameter is inversely proportional to the performance of the AI model;

[0128] The sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter, and the sixth parameter is proportional to the performance of the AI model;

[0129] The number of times that the fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information within the third time period is greater than the threshold corresponding to the fifth parameter is greater than the fifth threshold;

[0130] The number of times that the sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information within the third time period is less than the threshold corresponding to the sixth parameter is greater than the fifth threshold;

[0131] The deviation of the parameter in the statistical information and / or distribution information of the inferred CSI-related information from the third target value is greater than the sixth threshold, and the third target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the inferred CSI-related information.

[0132] In one implementation, the first event is an event determined based on the generalization index of the AI model, and the first event includes at least one of the following:

[0133] The duration during which the parameter in the performance metric of the received reference signal is less than the seventh threshold is greater than the eighth threshold.

[0134] The number of times the parameter in the performance metric of the received reference signal is less than the seventh threshold within the fourth duration is greater than the ninth threshold.

[0135] The quasi - co - location information of the received reference signal changes.

[0136] The dataset label of the measured CSI - related information changes.

[0137] The duration during which the parameter in the measured speed information is greater than the tenth threshold is greater than the eleventh threshold.

[0138] The number of times the parameter in the measured speed information is greater than the tenth threshold within the fifth duration is greater than the twelfth threshold.

[0139] The distance difference between two positions separated by the sixth duration is greater than the thirteenth threshold.

[0140] The operating center frequency, sub - carriers, and / or bandwidth configuration change.

[0141] In one embodiment, the first event is an event determined based on the update information and / or model transfer information of the AI model, and the first event includes at least one of the following:

[0142] Receiving the model update information and / or model transfer information sent by the network device;

[0143] After receiving the model update information and / or model transfer information sent by the network device, no performance monitoring - related indication is received within the seventh duration.

[0144] In one embodiment, the second event is an event determined based on upper - layer signaling, and the second event includes at least one of the following:

[0145] The cell where the device is located undergoes a handover;

[0146] The first monitoring quantity is greater than the fourteenth threshold, and the first monitoring quantity is inversely proportional to the performance of the terminal device;

[0147] The second monitoring quantity is less than the fifteenth threshold, and the first monitoring quantity is directly proportional to the performance of the terminal device;

[0148] The first parameter of the final key performance metric at a future time predicted based on the final key performance metric at the current time is less than the threshold corresponding to the first parameter;

[0149] The second parameter of the final key performance indicator at a future moment predicted according to the final key performance indicator at the current moment is greater than the threshold corresponding to the second parameter.

[0150] In one implementation manner, the threshold, duration, and target value are obtained based on protocol pre - definition and / or network - side configuration and / or a combination of protocol pre - definition and network - side configuration.

[0151] In one implementation manner, the CSI - related information includes the CSI information, beam information, and positioning information.

[0152] In one implementation manner, the response to the performance monitoring request includes a reply or indication of the performance monitoring request.

[0153] In one implementation manner, the sending module is specifically configured to include:

[0154] Send the performance monitoring request to the network device based on the scheduling request SR of the uplink control information UCI;

[0155] Or,

[0156] Send the performance monitoring request to the network device based on the media access control cell MAC - CE signaling;

[0157] Or,

[0158] Send the performance monitoring request to the network device based on the reserved - period physical uplink control channel PUCCH resource or physical uplink shared channel PUSCH resource.

[0159] In a fourth aspect, the present application provides another performance monitoring device, which is applied to a network device. The performance monitoring device includes a receiving module and a sending module, where:

[0160] The receiving module is configured to receive a performance monitoring request of an AI model sent by a terminal device, where the performance monitoring request is determined by the terminal device based on a triggering event;

[0161] The sending module is configured to send a response to the performance monitoring request to the terminal device.

[0162] In one implementation manner, the sending module is specifically configured to:

[0163] Judge whether the AI model needs to be monitored according to the information carried in the monitoring request, and obtain a judgment result;

[0164] Send a response to the monitoring request to the terminal device according to the judgment result.

[0165] In one implementation manner, the sending module is specifically configured to:

[0166] Based on the information carried in the performance monitoring request, determine whether the AI model needs to be performance monitored to obtain a determination result;

[0167] According to the determination result, send a response to the performance monitoring request to the terminal device.

[0168] In one implementation manner, the sending module is specifically configured to:

[0169] Receive a scheduling request SR sent by the terminal device;

[0170] Send a response to the scheduling request SR to the terminal device, where the response to the scheduling request SR includes downlink control information DCI for uplink transmission resource allocation, and the downlink control information DCI is scrambled based on the radio network temporary identity RNTI of the terminal device.

[0171] In a fifth aspect, the present application provides a terminal device, including a memory, a transceiver, and a processor:

[0172] The memory is used for storing a computer program;

[0173] The transceiver is used for transceiving data under the control of the processor;

[0174] The processor is used for reading the computer program in the memory and performing the following operations:

[0175] Send a performance monitoring request of the AI model to a network device according to a trigger event;

[0176] Receive the response to the performance monitoring request sent by the network device, and monitor the performance of the AI model according to the response to the performance monitoring request.

[0177] In a sixth aspect, the present application provides a network device, including a memory, a transceiver, and a processor:

[0178] The memory is used for storing a computer program;

[0179] The transceiver is used for transceiving data under the control of the processor;

[0180] The processor is used for reading the computer program in the memory and performing the following operations:

[0181] Receive a performance monitoring request of the AI model sent by a terminal device, where the performance monitoring request is determined by the terminal device based on a trigger event;

[0182] Send a response to the performance monitoring request to the terminal device.

[0183] In a seventh aspect, the present application provides a processor-readable storage medium storing a computer program for causing a processor to execute the method described in the first aspect or the method described in the second aspect.

[0184] The present application provides a method, apparatus, device, and storage medium for performance monitoring. In this method, a terminal device can send a performance monitoring request for an AI model to a network device according to a trigger event, receive a response to the performance monitoring request sent by the network device, and perform performance monitoring on the AI model according to the response to the performance monitoring request. In the above method, since the performance monitoring of the AI model can be enabled by the terminal device based on the trigger event, and the trigger event can be associated with the performance of the AI model, the terminal device can avoid the situation of frequently (periodically or semi-periodically) enabling the performance monitoring of the AI model, reduce the communication resources required for monitoring the AI model, and when events such as the failure of the AI model or the decline in the performance of the AI model occur, the terminal device can timely enable the performance monitoring of the AI model, improve the timeliness of the performance monitoring of the AI model, and thus improve the stability of communication. This method can also be used in combination with methods of enabling model monitoring aperiodically, periodically, or semi-periodically to achieve the purpose of quickly and flexibly performing performance monitoring of the AI model according to the available communication resources in the system.

[0185] It should be understood that the content described in the above invention content section is not intended to limit the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0186] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0187] Figure 1 A schematic diagram of a communication scenario provided by an embodiment of the present application;

[0188] Figure 2 A flowchart of a performance monitoring method provided by an embodiment of the present application;

[0189] Figure 3 A schematic diagram of a method for monitoring the performance of an AI model provided by an embodiment of the present application;

[0190] Figure 4 A schematic diagram of another method for monitoring the performance of an AI model provided by an embodiment of the present application;

[0191] Figure 5 Schematic diagram of another method for monitoring the performance of an AI model provided by an embodiment of the present application;

[0192] Figure 6 Schematic diagram of a method for triggering a first event provided by an embodiment of the present application;

[0193] Figure 7 Schematic diagram of another method for triggering a first event provided by an embodiment of the present application;

[0194] Figure 8 Schematic diagram of a method for sending a monitoring request provided by an embodiment of the present application;

[0195] Figure 9 Schematic diagram of another method for sending a monitoring request provided by an embodiment of the present application;

[0196] Figure 10 Schematic diagram of another method for sending a monitoring request provided by an embodiment of the present application;

[0197] Figure 11 Schematic diagram of another method for sending a monitoring request provided by an embodiment of the present application;

[0198] Figure 12 Schematic diagram of another method for sending a monitoring request provided by an embodiment of the present application;

[0199] Figure 13 Schematic diagram of a method for sending a monitoring request provided by an embodiment of the present application;

[0200] Figure 14 Schematic diagram of a method for sending a response to a monitoring request to a terminal device provided by an embodiment of the present application;

[0201] Figure 15 Schematic diagram of the structure of a performance monitoring device provided by an embodiment of the present application;

[0202] Figure 16 Schematic diagram of the structure of another performance monitoring device provided by an embodiment of the present application;

[0203] Figure 17 Schematic diagram of the structure of a terminal device provided by an embodiment of the present application;

[0204] Figure 18 Schematic diagram of the structure of a network device provided by an embodiment of the present application. Detailed implementation manners

[0205] In the embodiments of the present application, the term "and / or" describes the association relationship between associated objects and indicates that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0206] In the embodiments of the present application, the term "plurality" refers to two or more, and other quantifiers are similar.

[0207] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0208] The embodiments of the present application provide a performance monitoring method, device, equipment and storage medium. The terminal device can start monitoring the AI model according to the trigger event, so that there is no need for the network device to frequently start monitoring the performance of the AI model, reducing the consumption of communication resources and improving the timeliness of monitoring the performance of the AI model.

[0209] Among them, the method and the device are based on the same inventive concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can be referred to each other, and the repeated parts will not be described again.

[0210] The technical solutions provided by the embodiments of the present application can be applied to multiple systems. For example, the applicable systems can be Long Term Evolution (LTE) systems, LTE Frequency Division Duplex (FDD) systems, LTE Time Division Duplex (TDD) systems, Long Term Evolution Advanced (LTE-A) systems, Universal Mobile Telecommunications System (UMTS), Worldwide Interoperability for Microwave Access (WiMAX) systems, 5G New Radio (NR) systems and their evolved communication systems, etc. These multiple systems may include terminal devices and network devices. The system may also include a core network part, such as an Evolved Packet System (EPS), a 5G system (5GS), etc.

[0211] The terminal device involved in the embodiments of the present application can be a device that provides voice and / or data connectivity to users, such as a handheld device with wireless connection capabilities, or other processing devices connected to a wireless modem, etc. In different systems, the name of the terminal device may also be different. For example, in a 5G system, the terminal device can be called a User Equipment (UE). The wireless terminal device can be a USB storage device, other personal computer memory devices, and dongles, and can also communicate with one or more core networks (CN) via a Radio Access Network (RAN). The wireless terminal device can be a mobile terminal device, such as a mobile phone (or "cellular" phone) and a computer with a mobile terminal device. For example, it can be a portable, pocket-sized, handheld, computer-integrated, or vehicle-mounted mobile device that exchanges voice and / or data with the wireless access network. For example, devices such as Personal Communication Service (PCS) phones, cordless phones, Session Initiated Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistant (PDA), personal computers, tablets, Machine-type Communication (MTC) terminal devices, etc. The wireless terminal device can also be called a system, subscriber unit, subscriber station, mobile station, mobile, remote station, access point, remote terminal, access terminal, user terminal, user agent, user device, and wireless access points and routers / modems that meet the limitations of this definition, etc., which are not limited in the embodiments of the present application.

[0212] The network device involved in the embodiments of the present application may be a base station, which may include multiple cells that provide services to terminals. Depending on specific application scenarios, the base station may also be referred to as an access point, or may be a device in the access network that communicates with wireless terminal devices through one or more sectors on the air interface, or other names. The network device can be used to mutually replace the received air frames and Internet Protocol (IP) packets, and act as a router between the wireless terminal device and the rest of the access network, where the rest of the access network may include an IP communication network. The network device can also coordinate the management of the attributes of the air interface. For example, the network device involved in the embodiments of the present application may be an evolved network device (eNB or e-NodeB) in a Long Term Evolution (LTE) system, a 5G base station (gNB) in a 5G network architecture (next generation system), etc., or may also be a Home evolved Node B (HeNB), a relay node, a femto, a pico, a network test device, etc. The embodiments of the present application do not limit this. In some network architectures, the network device may include a centralized unit (CU) node and a distributed unit (DU) node, and the centralized unit and the distributed unit may also be geographically separated.

[0213] Next, in combination with Figure 1 , the communication scenario of the present application will be described.

[0214] Figure 1 It is a schematic diagram of a communication scenario provided by the embodiments of the present application. Please refer to Figure 1 , which includes a terminal device and a network device, and both the terminal device and the network device can be used as AI model users. Among them, when the AI model usage scenario is different from its training scenario, it is necessary to monitor the performance of the AI model, and then determine whether the AI model is applicable in the current scenario. The monitor of the AI model can perform operations related to the life cycle management (LCM) of the AI model based on the performance monitoring results of the AI model. For example, when the current AI model is invalid, the monitor of the AI model can perform operations such as model deactivation, model switching, model rollback, and model update, so as to ensure the stability of communication.

[0215] Currently, network devices can send reference signals related to AI models to terminal devices, thereby enabling the monitoring of the performance of AI models. For example, if the AI model is a model for processing channel state information (CSI), the network device can send CSI-related reference signals to the terminal device aperiodically, periodically, or semi-periodically. The terminal device can measure the true CSI based on the reference signal and process (compress and recover) the true CSI through the AI model, and then monitor the performance of the current AI model based on the true CSI and the CSI output by the AI model. However, during most periods, the performance of the AI model is relatively stable. Therefore, the AI model does not need to be frequently monitored for performance, which will result in waste of communication resources. Moreover, in the event of an emergency (such as the failure of the AI model or the degradation of the performance of the AI model), the network device cannot promptly start monitoring the performance of the AI model, thereby resulting in low communication stability.

[0216] To solve the technical problems in the related art, an embodiment of the present application provides a performance monitoring method. The terminal device can send a performance monitoring request for the AI model to the network device according to a trigger event, where the performance monitoring request may include the identifier of the AI model and / or the identifier of the function. Moreover, the trigger event may include an event related to the AI model and / or the function, and an event related to the performance of the terminal device. The terminal device can receive the response to the performance monitoring request sent by the network device and monitor the performance of the AI model according to the response to the performance monitoring request. In this way, since the trigger event is related to the AI model and / or the function, when the AI model fails or the performance of the AI model degrades, the terminal device can promptly start monitoring the performance of the AI model, thereby ensuring the stability of communication quality. Moreover, since the terminal device can start monitoring the performance of the AI model based on the trigger event, there is no need for the network device to frequently (periodically or semi-periodically) start monitoring the performance of the AI model, thereby saving communication resources.

[0217] Next, the performance monitoring method provided by the present application will be described in detail in combination with specific embodiments.

[0218] Figure 2 It is a flowchart of a performance monitoring method provided by an embodiment of the present application. Please refer to Figure 2 This method flow includes:

[0219] S201. The terminal device sends a performance monitoring request for the AI model to the network device according to a trigger event.

[0220] Among them, the triggering event can be used to trigger the terminal device to send a performance monitoring request for the AI model to the network device. For example, when the terminal device detects that the triggering event occurs, the terminal device can send a performance monitoring request for the AI model to the network device.

[0221] Among them, the triggering event can include a first event related to the AI model and / or function and a second event related to the performance of the terminal device. For example, if the user throughput is less than the preset throughput, this event can be the first event; if the block error rate is greater than the preset block error rate, this event can be the first event. Among them, the first event can be any event related to the AI model and / or function, and this application embodiment does not limit this. For example, if the power consumption of the terminal device rises, this event can be the second event; if the remaining computing power of the terminal device is less than the preset computing power, this event can be the second event. Among them, the second event can be any event related to the performance of the terminal device, and this application embodiment does not limit this.

[0222] Optionally, the AI model can be a model for processing CSI information, the AI model can be a model for processing beam information, and the AI model can also be a model for processing positioning information. This application embodiment does not limit this.

[0223] Among them, the performance monitoring request can be used to request performance monitoring of the AI model. For example, the performance monitoring request can request performance monitoring of the AI model, and the monitoring request can also trigger the activation of the performance monitoring of the AI model. Among them, performance monitoring can include model-based performance monitoring (i.e., performance monitoring) and functionality-based monitoring (the functionality can be the functionality of the AI model or the AI functionality in communication, and this application embodiment does not limit this). The terminal device can perform life cycle management on the AI model based on the results of the performance monitoring.

[0224] Among them, the performance monitoring request can include the identifier of the AI model and / or the identifier of the function. For example, the performance monitoring request can include the identity identification number ID of the AI model, the performance monitoring request can include the ID of the function, and the performance monitoring request can also include the ID of the AI model and the ID of the function. This application embodiment does not limit this.

[0225] For example, if the performance monitoring request sent by the terminal device to the network device includes the identifier of AI model 1, the terminal device requests to monitor the performance of AI model 1; if the performance monitoring request sent by the terminal device to the network device includes the identifier of AI model 2, the terminal device requests to monitor the performance of AI model 2. For example, if the performance monitoring request sent by the terminal device to the network device includes the identifier of function A, the terminal device requests to monitor the performance of function A; if the performance monitoring request sent by the terminal device to the network device includes the identifier of function B, the terminal device requests to monitor the performance of function B. In this way, the terminal device can accurately monitor the performance of the AI model and / or function.

[0226] Optionally, the performance monitoring request may include the identifier of the applicable model, the identifier of the applicable function, and the identifier of the applicable scenario corresponding to the performance monitoring request. For example, the performance monitoring request may include the identifier of the function of the AI model corresponding to the performance monitoring request and the identifier of the applicable scenario corresponding to the performance monitoring request.

[0227] Optionally, the performance monitoring request may include the start time and / or the end time of the performance monitoring. For example, the performance monitoring request may include the start time of the performance monitoring, the performance monitoring request may also include the end time of the performance monitoring, and the performance monitoring request may further include the start time and the end time of the performance monitoring. For example, if the performance monitoring request includes the start time of the performance monitoring, the terminal device may start monitoring the performance of the AI model at this start time; if the performance monitoring request includes the end time of the performance monitoring, the terminal device may stop monitoring the performance of the AI model at this end time; if the performance monitoring request includes the start time and the end time of the performance monitoring, the terminal device may monitor the performance of the AI model during the period between the start time and the end time. In this way, the terminal device can flexibly monitor the performance of the AI model.

[0228] It should be noted that if the start time of the performance monitoring is not included in the performance monitoring request, when the network device receives the performance monitoring request, it can execute the performance monitoring process of the AI model (the performance monitoring process of the AI model starts from the network device sending the reference signal related to the AI model. However, in this application, the start of the performance monitoring process of the AI model is determined by the performance monitoring request sent by the terminal device).

[0229] It should be noted that if the end time of the performance monitoring is not included in the performance monitoring request, the terminal device may stop the performance monitoring after a preset duration of starting the performance monitoring, or the terminal device may also stop the performance monitoring at any time. This application embodiment does not make any limitation in this regard.

[0230] It should be noted that the start time and end time in the embodiments of the present application can be absolute time or offset time, and the embodiments of the present application do not limit this.

[0231] Optionally, the performance monitoring request may include target information. Among them, the target information may be related to the performance metrics and / or measurement values of the AI model. Optionally, the target information may include at least one of the following: intermediate key performance indicators related to the AI model, eventual key performance indicators related to the AI model, statistical information of information related to channel state information CSI, distribution information of information related to CSI, speed information, and location information.

[0232] Among them, the intermediate (Intermediate) key performance indicators (Key Performance Indicators, KPI) related to the AI model can indicate the performance of the AI model. For example, if the AI model is a model for processing CSI, the terminal device can obtain the real CSI, perform compression and restoration processing on the real CSI based on the AI model to obtain the predicted CSI. The terminal device can calculate the intermediate KPI (performance metric calculation) of the AI model based on the real CSI and the predicted CSI, and determine the inference accuracy and accuracy of the AI model according to the intermediate KPI.

[0233] Among them, the eventual key performance indicators related to the AI model can be used to indicate the performance of the AI model. For example, the eventual KPI may include parameters such as throughput, hypothetical throughput, hypothetical block error rate (BLER), and block error rate. The above parameters can accurately reflect the quality of the communication system. The terminal device can determine the performance of the AI model based on the quality of the communication system. For example, if the throughput is low, it indicates that the quality of the current communication system is poor and the performance of the AI model is low.

[0234] Among them, the information related to channel state information CSI may include CSI information, beam information, and positioning information. The information related to CSI may also include any information that can affect CSI. The embodiments of the present application do not limit this.

[0235] Among them, the statistical information of information related to CSI may include information such as average delay, delay spread, Doppler shift, and Doppler spread. The distribution information related to CSI may include LOS (Line-of-Sight) distribution, NLOS (Non-Line-of-Sight) distribution, etc. The embodiments of the present application do not limit this.

[0236] Among them, the reference signal received power (RSRP) related to the AI model can be the power intensity of the reference signal received by the terminal device. For example, the RSRP can measure the signal strength of the terminal device or the cell. For example, if the AI model is a model for processing CSI, the reference signal can be a CSI-related reference signal; if the AI model is a model for processing beam information, the reference signal can be a beam-related reference signal; if the AI model is a model for processing positioning information, the reference signal can be a positioning-related reference signal.

[0237] Among them, the reference signal received quality (RSRQ) related to the AI model can be the quality of the reference signal received by the terminal device. For example, the terminal device can determine the signal quality based on the RSRQ.

[0238] Among them, the signal-to-interference plus noise ratio (SINR) can be the ratio of the intensity of the received useful signal to the intensity of the received interference signal (noise plus interference). For example, based on the SINR, the terminal device can accurately determine the impact of interference other than noise on the signal.

[0239] Among them, the speed information can include the absolute value of the linear velocity, the absolute value of the angular velocity, the linear velocity acceleration, and the angular velocity acceleration of the terminal device. The speed information can include any information related to the speed of the terminal device, and the embodiments of the present application do not limit this.

[0240] Among them, the position information can be information related to the position of the terminal device. For example, the position information can be the current position of the terminal device, or the position difference between two positions measured by the terminal device within a period of time. The embodiments of the present application do not limit this.

[0241] It should be noted that the performance indicators and measurement values of the above target information can be L1 (layer 1) or L3 (layer 3) filtered, and the embodiments of the present application do not limit this.

[0242] In this way, when the target information is included in the performance monitoring request of the AI model, the network device can determine whether to enable the performance monitoring of the AI model according to the target information. Although the judgment process for whether to enable the performance monitoring of the AI model is determined by the network device, the start of this judgment process is determined by the performance monitoring request sent by the terminal device. Therefore, the network device does not need to frequently enable the performance monitoring of the AI model, thereby saving communication resources.

[0243] Optionally, the terminal device may send a performance monitoring request for the AI model to the network device according to the following three feasible implementation manners:

[0244] One feasible implementation manner:

[0245] Send a performance monitoring request to the network device based on the scheduling request (SR) of the uplink control information (UCI). For example, when the terminal device uses UCI reporting, the terminal device may send an SR and the physical uplink shared channel (PUSCH) information required for uplink transmission (such as the number of reported quantities).

[0246] Another feasible implementation manner:

[0247] Send a performance monitoring request to the network device based on the media access control control element (MAC-CE) signaling. For example, the terminal device may send a performance monitoring request using the MAC-CE signaling, and the target information may also be included in the MAC-CE signaling.

[0248] Another feasible implementation manner:

[0249] Send a performance monitoring request to the network device based on the reserved-period physical uplink control channel (PUCCH) resource or PUSCH resource. For example, the terminal device may report the performance monitoring request in the reserved-period PUCCH resource or PUSCH resource based on the L1 resource pre-configuration.

[0250] S202. The network device sends a response to the performance monitoring request to the terminal device.

[0251] Among them, after receiving the performance monitoring request sent by the terminal device, the network device may respond to the performance monitoring request. Optionally, the response to the performance monitoring request may include a reply or indication to the monitoring request. For example, the response to the performance monitoring request may include consent to enable the performance monitoring of the AI model and / or consent to activate the performance monitoring of the AI model, and the response to the performance monitoring request may include refusal to enable the performance monitoring of the AI model and / or refusal to activate the performance monitoring of the AI model.

[0252] Among them, after determining the response to the monitoring request, the network device may send the response to the monitoring request to the terminal device.

[0253] Among them, the network device can send a response to the performance monitoring request to the terminal device according to the following feasible implementation methods: receive the scheduling request (SR) sent by the terminal device, and send a response to the SR sent by the terminal device.

[0254] Among them, the response to the SR includes downlink control information (DCI) for uplink transmission resource allocation, and the DCI is scrambled based on the radio network temporary identifier (RNTI) of the terminal device. For example, when the terminal device sends a performance monitoring request for the AI model to the network device based on the SR of the UCI, the terminal device can send the SR on the PUCCH or the random access channel (RACH). After receiving the SR, the network device can send an uplink grant as a response to the SR. Among them, the uplink grant can include the DCI for uplink transmission resource allocation, and the DCI can be scrambled based on the RNTI of the terminal device (in order to distinguish the DCI response on the network side for the uplink transmission scheduling request of other UEs, the RNTI of this terminal device can be a newly defined dedicated RNTI). If the terminal device receives the same RNTI and can descramble the DCI, the terminal device can determine that the network device has received the performance monitoring request sent by the terminal device.

[0255] Optionally, when the terminal device sends a performance monitoring request for the AI model based on the MAC-CE, the response information of the network device is the DCI format for scheduling PUSCH transmission. This DCI format may include the same Hybrid Automatic Repeat reQuest (HARQ) process number as that of the PUSCH for the MAC-CE sent by the terminal device for the performance monitoring request, and a flipped Network Device Interface (NDI) field. When the terminal device detects this DCI, it determines that the network device has received the monitoring request. For example, the NDI field can be a 1-bit indication field. If the NDI field is inverted (from 0 to 1, or from 1 to 0), it indicates that the data transmission is a new transmission rather than a retransmission. For example, the DCI responded by the base station can be the DCI format for scheduling PUSCH transmission. Among them, the HARQ process ID of the newly scheduled PUSCH is different from the HARQ process ID of the PUSCH for the MAC-CE sent by the terminal device before (the HARQ process ID can be incremented). Therefore, if the terminal device checks that this DCI has the same HARQ process ID as the PUSCH for the MAC-CE sent before, it means that the DIC responded by the base station corresponds to the MAC-CE sent by the terminal device.

[0256] S203. The terminal device performs performance monitoring on the AI model according to the response to the performance monitoring request.

[0257] Among them, the terminal device can receive the response to the performance monitoring request sent by the network device and perform performance monitoring on the AI model according to the response to the performance monitoring request. For example, if the response to the performance monitoring request is to agree to enable performance monitoring, the terminal device can receive the reference signal of this AI model sent by the network device, and then perform performance monitoring on this AI model. If the response to the monitoring request is to refuse to enable performance monitoring, the terminal device can stop enabling performance monitoring, or continue to send a performance monitoring request to the network device. This application embodiment does not make a limitation on this.

[0258] Optionally, after the terminal device sends a performance monitoring request for the AI model to the network device, if the terminal device does not receive a response to the performance monitoring request within a certain period of time, the terminal device may resend the performance monitoring request to the network device. The performance monitoring request may be a newly generated performance monitoring request or an already generated performance monitoring request. This application embodiment does not limit this. For example, after the terminal device sends a performance monitoring request to the network device and does not receive a response to the performance monitoring request within 10 seconds, the terminal device may resend the performance monitoring request to the network device.

[0259] Optionally, after the terminal device sends a performance monitoring request to the network device multiple times, if the terminal device does not receive a response to the performance monitoring request, the terminal device may send performance monitoring of the AI model to the network device according to a trigger event. For example, after the terminal device sends 3 performance monitoring requests to the network device and still does not receive a response to the performance monitoring request, the terminal device may regenerate a performance monitoring request according to the trigger event (more target information, etc. may be included in the monitoring request), and send the performance monitoring request to the network device. In this way, the accuracy and timeliness of performance monitoring of the AI model can be improved.

[0260] Next, in combination with Figures 3 - 5 , the process of the terminal device performing performance monitoring on the AI model will be described. Among them, the AI model is a model for processing CSI.

[0261] Figure 3 This is a schematic diagram of a method for performance monitoring of an AI model provided by an embodiment of this application. In the embodiment shown in Figure 3 , the part of the model for compressing CSI is located in the terminal device, and the part of the model for decompressing CSI is located in the network device. The network device performs performance monitoring on the AI model. Please refer to Figure 3 , which includes a terminal device and a network device. When the terminal device enables performance monitoring of the AI model, the network device may send a reference signal related to channel state information to the terminal device. The terminal device may obtain the measured channel state information according to the reference signal, and the terminal device may compress the measured channel state information according to the compression part of the model.

[0262] Please refer to Figure 3, the terminal device can send the compressed channel state information to the network device. The network device can decompress the compressed channel state information based on the decompression part of the model to obtain the decompressed channel state information. The terminal device can also send the measured channel state information to the network device. The network device can determine the performance of the model according to the measured channel state information and the decompressed channel state information. In this way, the network device can calculate the intermediate KPI according to the measured CSI and the decompressed CSI, and then determine the performance of the AI model according to the intermediate KPI, improving the accuracy of model performance monitoring.

[0263] Figure 4 Another schematic diagram of the method for monitoring the performance of the AI model provided by the embodiments of this application. In Figure 4 the shown embodiment, the part of the model for compressing CSI is located in the terminal device, and the part of the model for decompressing CSI is located in the network device. The terminal device monitors the performance of the AI model. Please refer to Figure 4 , which includes a terminal device and a network device. When the terminal device starts to monitor the performance of the AI model, the network device can send a reference signal related to the channel state information to the terminal device. The terminal device can obtain the measured channel state information according to the reference signal, and the terminal device can compress the measured channel state information according to the compression part of the model.

[0264] Please refer to Figure 4 , the terminal device can send the compressed channel state information to the network device. The network device can decompress the compressed channel state information based on the decompression part of the model to obtain the decompressed channel state information. The network device can send the decompressed channel state information to the terminal device. The terminal device can determine the performance of the model according to the measured channel state information and the decompressed channel state information. In this way, the terminal device can calculate the intermediate KPI according to the measured CSI and the decompressed CSI, and then determine the performance of the AI model according to the intermediate KPI, improving the accuracy of model performance monitoring.

[0265] Figure 5 Another schematic diagram of the method for monitoring the performance of the AI model provided by the embodiments of this application. In Figure 5 the shown embodiment, the part of the model for compressing CSI is located in the terminal device, and the part of the model for decompressing CSI is located in the network device. The terminal device monitors the AI model. The terminal device also includes an alternative model for the part of the model for decompressing CSI. Please refer to Figure 5, including a terminal device and a network device. When the terminal device enables performance monitoring of the AI model, the network device may send a reference signal related to channel state information to the terminal device. The terminal device may obtain the measured channel state information based on the reference signal, and the terminal device may compress the measured channel state information according to the compressed part of the model.

[0266] Please refer to Figure 5 , the terminal device may send the compressed channel state information to the network device, and the network device may decompress the compressed channel state information based on the decompressed part of the model to obtain the decompressed channel state information. The terminal device may decompress the compressed channel state information according to the alternative model, and determine the performance of the model based on the measured channel state information and the decompressed channel state information. In this way, the terminal device may calculate the intermediate KPI based on the measured CSI and the decompressed CSI, and then determine the performance of the AI model according to the intermediate KPI, improving the accuracy of model performance monitoring.

[0267] It should be noted that Figures 3 - 5 In the embodiments shown, the serial numbers 1, 2,..., 7 are only for explaining the steps and do not limit the execution order of the steps.

[0268] An embodiment of the present application provides a performance monitoring method. The terminal device may send a performance monitoring request of the AI model to the network device according to a trigger event. Among them, the trigger event may include a first event related to the AI model and / or function, and a second event related to the performance of the terminal device. The monitoring request may include the identifier of the AI model, the identifier of the function of the AI model, the start time and end time of performance monitoring, and target information related to the performance metrics and measurement values of the AI model. The terminal device may receive the response to the performance monitoring request sent by the network device, and perform performance monitoring on the AI model according to the response to the performance monitoring request. In this way, since the terminal device can initiate the performance monitoring process without the network device frequently (periodically or semi-periodically) initiating performance monitoring of the AI model, communication resources can be saved.

[0269] In Figure 2 Based on the embodiments shown, when the AI model is not activated, not selected, or not paired, the first event may include at least one of the following: sending the identifier and / or function identifier of the supported AI model, receiving the identifier and / or function identifier of the AI model supported by the cell covered by the current base station. Next, in combination with Figures 6 - 7 , the process of triggering the first event in this scenario will be described.

[0270] Figure 6 It is a schematic diagram of a method for triggering the first event provided by an embodiment of the present application. In Figure 6In the illustrated embodiment, the first event is that the terminal device sends the identifier of the supported AI model and / or the identifier of the function. Please refer to Figure 6 , and the method flow includes:

[0271] S601. The terminal device sends the identifier of the AI model supported by the terminal device and / or the identifier of the function to the network device.

[0272] Among them, when the AI model is not activated, not selected, or not paired, the AI model or function in the current scenario has not been enabled. Therefore, the terminal device can send a performance monitoring request for the AI model when activating the AI model, selecting the AI model, or pairing the AI model.

[0273] Among them, the first event can be the event of sending the identifier of the supported AI model and / or the identifier of the function. For example, the terminal device can send the identifier of the AI model currently supported by the terminal device (which can also be the identifier of the function) to the network device, and the network device can determine the activated AI model according to the identifier of the AI model. For example, when the terminal device sends the identifiers of the AI models supported by the terminal device to the network device, including the identifier of AI model 1, the identifier of AI model 2, and the identifier of AI model 3, if the network device supports AI model 1 and AI model 2, the network device can activate AI model 1 and AI model 2.

[0274] It should be noted that when the terminal device accesses a new network, the terminal device can send the identifier of the supported AI model and / or the identifier of the function to the network device. The terminal device can also send the identifier of the supported AI model and / or the identifier of the function to the network device in any feasible scenario. The embodiments of the present application do not limit this.

[0275] S602. The terminal device sends a performance monitoring request to the network device.

[0276] Among them, when the terminal device sends the identifier of the supported AI model and / or the identifier of the function to the network device, the terminal device can determine that a trigger event has occurred, and the terminal device can send a performance monitoring request for the AI model to the network device. For example, when the terminal device sends the identifier of the supported AI model to the network device, the network device can send the identifier of the activated AI model to the AI model, and the terminal device can send a performance monitoring request to the network device. The monitoring request can include the identifier of the activated AI model. In this way, when the AI model is activated, the terminal device can start monitoring the performance of the AI model, thereby improving the stability of communication.

[0277] Optionally, the performance monitoring request may include the start time and / or the end time of performance monitoring. For example, when the AI model is not activated, not selected, or not paired, and the AI model and / or function has not been enabled, after the network device activates, selects, or pairs the AI model, the terminal device may instruct the network model to perform the performance monitoring process of the AI model after a period of time when the AI model is activated, selected, or paired, which can improve the accuracy of the performance monitoring of the AI model.

[0278] It should be noted that in this scenario, the target information may also be carried in the performance monitoring request, and the embodiments of the present application do not limit this.

[0279] The embodiments of the present application provide a method for triggering a first event, sending the identifier of the AI model supported by the terminal device and / or the identifier of the function to the network device, and sending a model monitoring request to the network device. In this way, when the terminal device accesses a new network, the terminal device can timely send the supported AI model and / or function to the network device, thereby activating, selecting, or pairing the AI model and / or function jointly supported by the terminal device and the network device. Moreover, when the AI model is activated, selected, or paired, the terminal device can start the performance monitoring of the AI model to improve the communication stability.

[0280] Figure 7 It is a schematic diagram of another method for triggering a first event provided by the embodiments of the present application. In Figure 7 the illustrated embodiment, the first event is that the terminal device receives the identifier of the AI model supported by the cell covered by the current base station and / or the identifier of the function. Please refer to Figure 7 , and the method flow includes:

[0281] S701. The terminal device receives the identifier of the AI model supported by the cell covered by the current base station and / or the identifier of the function.

[0282] Among them, the terminal device may receive the identifier of the AI model supported by the cell covered by the current base station where it is located and / or the identifier of the function. For example, when the terminal device performs cell handover, the AI model has not been activated, not selected, or not paired. Therefore, the terminal device can determine the activated, selected, or paired AI model or function and start the performance monitoring of the AI model. For example, the network device may send the identifier of the AI model supported by the cell (which may also be the identifier of the function) to the terminal device, and the terminal device may determine the activated, selected, and paired AI model based on the identifier of the AI model supported by the cell and the identifier of the AI model supported by the terminal device.

[0283] For example, the network device sends the identifiers of the AI models supported by the cell to the terminal device, including: the identifier of AI model 1, the identifier of AI model 2, and the identifier of AI model 3. If the terminal device supports AI model 1 and AI model 2, the terminal device can request to activate AI model 1 and AI model 2.

[0284] It should be noted that when the terminal device is located in a new cell or performs cell handover, the terminal device can receive the identifier of the AI model supported by the cell sent by the network device and / or the identifier of the function. The terminal device can also receive the identifier of the AI model supported by the cell sent by the network device and / or the identifier of the function in any feasible scenario. The embodiments of the present application do not limit this.

[0285] S702. The terminal device sends a performance monitoring request to the network device.

[0286] Among them, when the terminal device receives the identifier of the AI model supported by the cell covered by the current base station sent by the network device and / or the identifier of the function, the terminal device can determine that a trigger event has occurred, and the terminal device can send a performance monitoring request for the AI model to the network device. For example, when the terminal device receives the identifier of the AI model supported by the cell sent by the network device, the terminal device can determine the activated AI model, send the identifier of the activated AI model to the network device, and the terminal device can send a performance monitoring request for this AI model to the network device. In this way, when the AI model is activated, the terminal device can start the performance monitoring of this AI model, thereby improving the stability of communication.

[0287] Optionally, the performance monitoring request may include the identifier of the AI model, the identifier of the function, the start time of performance monitoring, the end time of performance monitoring, and the target information. The embodiments of the present application do not limit this.

[0288] The embodiments of the present application provide a method for triggering a first event. The terminal device receives the identifier of the AI model supported by the cell covered by the current base station and / or the identifier of the function, and the terminal device sends a performance monitoring request to the network device. In this way, when the terminal device switches cells, the terminal device can timely activate the AI model or function jointly supported by the terminal device and the network device, and when the AI model or function is activated, the terminal device can start performance monitoring, thereby improving the stability of communication.

[0289] Based on any of the above embodiments, when the AI model is activated, selected, or paired, the terminal device may determine a first event according to at least one piece of information, where the at least one piece of information may include the measured final key performance indicator, the statistical information and / or distribution information of the measured CSI-related information, the statistical information and / or distribution information of the CSI-related information inferred by the AI model, the generalization indicator of the AI model, the update information of the AI model, and / or the transmission information of the AI model. When the terminal device determines to trigger the first event, the terminal device may send a performance monitoring request for the AI model to the network device.

[0290] Next, in combination with Figures 8 - 12 , the process of the terminal device sending a performance monitoring request in this scenario will be described. It should be noted that the thresholds, durations, and target values in the embodiments of the present application are obtained based on protocol predefinitions and / or network-side configurations and / or a combination of protocol predefinitions and network-side configurations.

[0291] Figure 8 FIG. is a schematic diagram of a method for sending a performance monitoring request provided by an embodiment of the present application. Please refer to Figure 8 , in Figure 8 In the shown embodiment, the first event is an event determined based on the measured final key performance indicator. The method flow includes:

[0292] S801. Obtain the measured final key performance indicator.

[0293] Among them, the measured final key performance indicator may be the final KPI measured by the terminal device. For example, the final KPI measured by the terminal device may include parameters such as user throughput and block error rate, and the embodiments of the present application do not limit this.

[0294] It should be noted that the terminal device may measure the final KPI according to any feasible implementation method, and the embodiments of the present application do not limit this.

[0295] S802. Determine to trigger the first event according to the measured final key performance indicator.

[0296] Among them, if the first event is an event determined based on the measured final key performance indicator, the first event includes at least one of the following:

[0297] The first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter;

[0298] The second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter

[0299] The number of times that the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter within the first duration is greater than the first threshold;

[0300] The number of times that the second parameter in the final key performance indicator within the first duration is greater than the threshold corresponding to the second parameter is greater than the first threshold;

[0301] The deviation between the parameter in the final key performance indicator and the first target value is greater than the second threshold.

[0302] Among them, the first parameter can be proportional to the performance of the AI model. For example, the larger the first parameter in the final KPI, the higher the performance of the AI model, and the smaller the first parameter in the final KPI, the lower the performance of the AI model. For example, the first parameter can be the user throughput. If the user throughput is large, it indicates good communication quality, and thus the performance of the AI model is high. If the user throughput is small, it indicates poor communication quality, and thus the performance of the AI model is low.

[0303] Among them, the second parameter is inversely proportional to the performance of the AI model. For example, the larger the second parameter in the final KPI, the lower the performance of the AI model, and the smaller the second parameter in the final KPI, the higher the performance of the AI model. For example, the second parameter can be the block error rate. If the block error rate is large, it indicates poor communication quality, and thus the performance of the AI model is low. If the block error rate is small, it indicates good communication quality, and thus the performance of the AI model is high.

[0304] Among them, the first target value is the target value corresponding to the parameter in the final key performance indicator. For example, if the parameter in the final KPI is throughput, the first target value can be the target value corresponding to the throughput. If the parameter of the final KPI is the block error rate, the first target value can be the target value corresponding to the block error rate.

[0305] It should be noted that the first threshold, the second threshold, the first duration, the first target value, the threshold corresponding to the first parameter, and the threshold corresponding to the second parameter can be predefined by the protocol and / or configured by the network device and / or obtained by mixing protocol predefined and network side configuration, or can be determined based on any other feasible implementation manner. The embodiments of the present application do not limit this.

[0306] Optionally, if the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter, the terminal device can determine to trigger the first event. For example, the first parameter can be throughput, and the threshold corresponding to the throughput can be the preset throughput. If the throughput measured by the terminal device is less than the preset throughput, it indicates that the current communication quality is poor. The terminal device triggers the first event, and then can send a performance monitoring request for the AI model to the network device.

[0307] Optionally, if the second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter, the terminal device may determine to trigger a first event. For example, the second parameter may be the block error rate, and the threshold corresponding to the block error rate may be a preset block error rate. If the block error rate measured by the terminal device is greater than the preset block error rate, it indicates that the current communication quality is poor. The terminal device triggers the first event and can then send a performance monitoring request for the AI model to the network device.

[0308] Optionally, if the number of times the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter within the first time period is greater than the first threshold, the terminal device may determine to trigger a first event. For example, the first parameter may be the throughput, the threshold corresponding to the throughput may be Threshold 1, and the first threshold may be Threshold 2. Within the first time period, if the number of times the throughput measured by the terminal device (the terminal device may measure the throughput based on a preset sampling frequency) is less than Threshold 1 is 10 times, and 10 times is greater than Threshold 2, the terminal device may determine that the current communication quality is poor. The terminal device triggers the first event and can then send a performance monitoring request for the AI model to the network device. For example, the first time period is 10 seconds, the threshold corresponding to the throughput is the preset throughput, and the first threshold is 3 times. If the number of times the throughput measured by the terminal device within 10 seconds is less than the preset throughput is 10 times, the terminal device may determine to trigger the first event and send a performance monitoring request for the AI model to the network device.

[0309] Optionally, if the number of times the second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter within the first time period is greater than the first threshold, the terminal device may determine to trigger a first event. For example, the second parameter may be the block error rate, the threshold corresponding to the block error rate may be Threshold 1, and the second threshold may be Threshold 2. Within the first time period, if the number of times the block error rate measured by the terminal device is less than Threshold 1 is 10 times, and 10 times is greater than Threshold 2, the terminal device may determine that the current communication quality is poor. The terminal device triggers the first event and can then send a performance monitoring request for the AI model to the network device. For example, the first time period is 10 seconds, the threshold corresponding to the block error rate is the preset block error rate, and the first threshold is 3 times. If the number of times the block error rate measured by the terminal device within 10 seconds is less than the preset block error rate is 4 times, the terminal device may determine to trigger the first event and send a performance monitoring request for the AI model to the network device.

[0310] Optionally, if the deviation between the parameter in the final key performance indicator and the first target value is greater than the second threshold, the terminal device may determine to trigger a first event. For example, the parameter in the final key performance indicator may include a first parameter and a second parameter, where each parameter has a corresponding first target value. If the deviation between the parameter in the final KPI and the corresponding first target value of the parameter is greater than the second threshold, it indicates that the current communication quality is poor, and the terminal device may send a performance monitoring request for the AI model to the network device. For example, the parameter of the final KPI is throughput, and the first target value corresponding to the throughput is the preset throughput. If the difference between the throughput measured by the terminal device and the preset throughput is greater than the second threshold, the terminal device may determine to trigger a first event and send a performance monitoring request to the network device.

[0311] S803. Send a performance monitoring request to the network device.

[0312] Among them, when the terminal device determines to trigger the first event according to the final KPI, the terminal device may send a monitoring request for the AI model to the network device.

[0313] The embodiment of the present application provides a method for sending a monitoring request. The terminal device may obtain the measured final key performance indicator, and determine that the terminal device triggers a first event according to the measured final key performance indicator, and send a monitoring request to the network device. In this way, the terminal device can determine the communication quality according to the final KPI, and determine whether to enable the monitoring of the AI model based on the communication quality. In this way, when an unexpected event of a decrease in communication quality occurs, the terminal device can timely perform performance monitoring on the AI model, improve the accuracy of performance monitoring of the AI model, and thus improve the stability of communication.

[0314] Figure 9 This is a schematic diagram of another method for sending a performance monitoring request provided by the embodiment of the present application. Please refer to Figure 9 , in Figure 9 In the embodiment shown, the first event is an event determined based on the statistical information and / or distribution information of the measured CSI-related information. The method flow includes:

[0315] S901. Obtain the statistical information and / or distribution information of the measured CSI-related information.

[0316] The statistical information and / or distribution information of the measured CSI-related information may include information such as the average delay, delay spread, etc. corresponding to the CSI-related information measured by the terminal device. For example, the CSI-related information may be CSI information, and the statistical information and / or distribution information corresponding to the CSI information measured by the terminal device may include the average delay, delay spread, Doppler frequency shift, Doppler spread, LOS distribution, NLOS distribution, channel covariance matrix, and Kullback-Leibler (KL) divergence, etc. related to the CSI information.

[0317] It should be noted that the terminal device can obtain the statistical information and / or distribution information of the measured CSI-related information according to any feasible implementation manner, and the embodiments of the present application do not limit this.

[0318] S902. Determine to trigger a first event according to the statistical information and / or distribution information of the measured CSI-related information.

[0319] Wherein, if the first event is an event determined based on the statistical information and / or distribution information of the measured CSI-related information, the first event includes at least one of the following:

[0320] The third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than the threshold corresponding to the third parameter;

[0321] The fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is less than the threshold corresponding to the fourth parameter;

[0322] The number of times that the third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than the threshold corresponding to the third parameter within a second time period is greater than a third threshold;

[0323] The number of times that the fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than the threshold corresponding to the fourth parameter within a second time period is greater than a third threshold;

[0324] The deviation between the parameter in the statistical information and / or distribution information of the measured CSI-related information and a second target value is greater than a fourth threshold.

[0325] Wherein, the third parameter may be inversely proportional to the performance of the AI model. For example, the larger the third parameter in the statistical information and distribution information, the lower the performance of the AI model, and the smaller the third parameter in the statistical information and distribution information, the higher the performance of the AI model. For example, the third parameter may be the average delay. If the average delay is large, it indicates that the communication quality is poor, and thus the performance of the AI model is low. If the average delay is small, it indicates that the communication quality is good, and thus the performance of the AI model is high.

[0326] Among them, the fourth parameter can be directly proportional to the performance of the AI model. For example, the larger the fourth parameter in the statistical information and distribution information, the higher the performance of the AI model; the smaller the fourth parameter in the statistical information and distribution information, the lower the performance of the AI model. Herein, the fourth parameter can be any parameter in the statistical information and distribution information that is directly proportional to the performance of the AI model, and the embodiments of this application do not limit this.

[0327] The second target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the measured CSI-related information. For example, if the parameter in the statistical information and distribution information is the average delay, the second target value can be the target value corresponding to the average delay; if the parameter in the statistical information and distribution information is the delay spread, the second target value can be the target value corresponding to the delay spread.

[0328] It should be noted that the third threshold, the fourth threshold, the second duration, the second target value, the threshold corresponding to the third parameter, and the threshold corresponding to the fourth parameter can be predefined by the protocol and / or configured by the network device, or can be determined based on any other feasible implementation manner, and the embodiments of this application do not limit this.

[0329] Optionally, if the third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than the threshold corresponding to the third parameter, the terminal device can determine to trigger the first event. For example, the third parameter can be the average delay, and the threshold corresponding to the average delay can be a preset delay. If the average delay calculated by the terminal device is greater than the preset delay, it indicates that the current communication quality is poor, and the terminal device triggers the first event, and then can send a performance monitoring request for the AI model to the network device.

[0330] Optionally, if the fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is less than the threshold corresponding to the fourth parameter, the terminal device can determine to trigger the first event. For example, if the fourth parameter calculated by the terminal device is less than the threshold corresponding to the fourth parameter, it indicates that the current communication quality is poor, and the terminal device triggers the first event, and then can send a performance monitoring request for the AI model to the network device.

[0331] Optionally, if the number of times that the third parameter in the statistical information and / or distribution information of the CSI-related information measured within the second duration is greater than the threshold corresponding to the third parameter is greater than the third threshold, the terminal device may determine to trigger the first event. For example, the third parameter may be the average delay, the threshold corresponding to the average delay may be Threshold 1, and the third threshold may be Threshold 2. Within the second duration, if the number of times that the average delay calculated by the terminal device is greater than Threshold 1 is 10 times, and 10 times is greater than Threshold 2, the terminal device may determine that the current communication quality is poor, the terminal device triggers the first event, and further may send a performance monitoring request for the AI model to the network device. For example, the second duration is 10 seconds, the threshold corresponding to the average delay is the preset delay, and the third threshold is 3 times. If the number of times that the average delay calculated by the terminal device within 10 seconds is greater than the preset delay is 4 times, the terminal device may determine to trigger the first event and send a performance monitoring request for the AI model to the network device.

[0332] Optionally, if the number of times that the fourth parameter in the statistical information and / or distribution information of the CSI-related information measured within the second duration is less than the threshold corresponding to the fourth parameter is greater than the third threshold, the terminal device may determine to trigger the first event. For example, the threshold corresponding to the fourth parameter may be Threshold 1, and the third threshold may be Threshold 2. Within the second duration, if the number of times that the fourth parameter calculated by the terminal device is less than Threshold 1 is 4 times, and 4 times is greater than Threshold 2, the terminal device may determine that the current communication quality is poor, the terminal device triggers the first event, and further may send a performance monitoring request for the AI model to the network device. For example, the second duration is 10 seconds, the threshold corresponding to the fourth parameter is the preset threshold, and the third threshold is 3 times. If the number of times that the fourth parameter in the statistical information and / or distribution information of the CSI-related information measured by the terminal device within 10 seconds is less than the preset threshold is 4 times, the terminal device may determine to trigger the first event and send a performance monitoring request for the AI model to the network device.

[0333] Optionally, if the deviation between the parameters in the statistical information and / or distribution information of the measured CSI-related information and the second target value is greater than the fourth threshold, the terminal device may determine to trigger the first event. For example, the parameters in the statistical information and / or distribution information of the measured CSI-related information may include a third parameter and a fourth parameter, where each parameter has a corresponding second target value. If the deviation between the parameters in the statistical information and / or distribution information of the measured CSI-related information and the second target value corresponding to the parameter is greater than the fourth threshold, it indicates that the current communication quality is poor, and the terminal device may send a performance monitoring request for the AI model to the network device. For example, if the parameter in the statistical information and / or distribution information of the measured CSI-related information is the average delay, and the second target value corresponding to the average delay is the preset delay, if the difference between the average delay calculated by the terminal device and the preset delay is greater than the fourth threshold, the terminal device may determine to trigger the first event and send a performance monitoring request to the network device.

[0334] S903. Send a performance monitoring request to the network device.

[0335] Among them, when the terminal device determines to trigger the first event according to the statistical information and / or distribution information of the measured CSI-related information, the terminal device may send a performance monitoring request for the AI model to the network device.

[0336] The embodiments of the present application provide a method for sending a performance monitoring request, which obtains the statistical information and / or distribution information of the measured CSI-related information, determines that the terminal device triggers the first event according to the statistical information and / or distribution information of the measured CSI-related information, and sends a performance monitoring request to the network device. In this way, the terminal device can determine the communication quality according to the statistical information and / or distribution information of the measured CSI-related information, and determine whether to enable the performance monitoring of the AI model based on the communication quality. In this way, when an unexpected event of communication quality degradation occurs, the terminal device can timely perform performance monitoring on the AI model, improve the accuracy of the performance monitoring of the AI model, and thus improve the stability of communication.

[0337] Figure 10 This is a schematic diagram of another method for sending a performance monitoring request provided by the embodiments of the present application. Please refer to Figure 10 , in Figure 10 In the shown embodiment, the first event is an event determined based on the statistical information and / or distribution information of the CSI-related information inferred by the AI model. The method flow includes:

[0338] S1001. Obtain the statistical information and / or distribution information of the CSI-related information inferred by the AI model.

[0339] Among them, the statistical information and / or distribution information of the CSI-related information for AI model inference may include information such as the average delay and delay spread corresponding to the CSI-related information for AI model inference. For example, the CSI-related information may be beam information, etc. The statistical information and / or distribution information corresponding to the beam information for AI model inference may include the average delay, delay spread, Doppler frequency shift, Doppler spread, LOS distribution, NLOS distribution, channel covariance matrix, and KL (Kullback-Leibler) divergence associated with the beam information, etc.

[0340] It should be noted that the terminal device can obtain the statistical information and / or distribution information of the CSI-related information for AI model inference according to any feasible implementation manner, and the embodiments of the present application do not limit this.

[0341] S1002. Determine to trigger the first event according to the statistical information and / or distribution information of the CSI-related information for AI model inference.

[0342] Among them, if the first event is an event determined based on the statistical information and / or distribution information of the CSI-related information for AI model inference, the first event includes at least one of the following:

[0343] The fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter;

[0344] The sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter;

[0345] The number of times that the fifth parameter in the statistical information and / or distribution information of the CSI-related information inferred within the third time period is greater than the fifth parameter corresponding threshold is greater than the fifth threshold;

[0346] The number of times that the sixth parameter in the statistical information and / or distribution information of the CSI-related information inferred within the third time period is greater than the sixth parameter corresponding threshold is greater than the fifth threshold;

[0347] The deviation between the parameter in the statistical information and / or distribution information of the inferred CSI-related information and the third target value is greater than the sixth threshold.

[0348] Among them, the fifth parameter may be inversely proportional to the performance of the AI model. For example, the larger the fifth parameter in the statistical information and distribution information, the lower the performance of the AI model, and the smaller the fifth parameter in the statistical information and distribution information, the higher the performance of the AI model. For example, the fifth parameter may be the average delay. If the average delay is large, it indicates that the communication quality is poor, and thus the performance of the AI model is low. If the average delay is small, it indicates that the communication quality is good, and thus the performance of the AI model is high.

[0349] Among them, the sixth parameter can be proportional to the performance of the AI model. For example, the larger the sixth parameter in the statistical information and distribution information, the higher the performance of the AI model, and the smaller the sixth parameter in the statistical information and distribution information, the lower the performance of the AI model. Among them, the sixth parameter can be any parameter in the statistical information and distribution information that is proportional to the performance of the AI model, and the embodiments of the present application do not limit this.

[0350] Among them, the third target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the CSI-related information for inference. For example, if the parameter in the statistical information and distribution information is the average delay, the third target value can be the target value corresponding to the average delay. If the parameter in the statistical information and distribution information is the delay spread, the third target value can be the target value corresponding to the delay spread.

[0351] It should be noted that the fifth threshold, the sixth threshold, the third duration, the third target value, the threshold corresponding to the fifth parameter, and the threshold corresponding to the sixth parameter can be predefined by the protocol and / or configured by the network device, or can be determined based on any other feasible implementation manner. The embodiments of the present application do not limit this.

[0352] Optionally, if the fifth parameter in the statistical information and / or distribution information of the CSI-related information for inference is greater than the threshold corresponding to the fifth parameter, the terminal device can determine to trigger the first event. For example, the fifth parameter can be the average delay, and the threshold corresponding to the average delay can be a preset delay. If the average delay calculated by the terminal device is greater than the preset delay, it indicates that the current communication quality is poor, and the terminal device triggers the first event, and then can send a performance monitoring request for the AI model to the network device.

[0353] Optionally, if the sixth parameter in the statistical information and / or distribution information of the CSI-related information for inference is less than the threshold corresponding to the sixth parameter, the terminal device can determine to trigger the first event. For example, if the sixth parameter calculated by the terminal device is less than the threshold corresponding to the sixth parameter, it indicates that the current communication quality is poor, and the terminal device triggers the first event, and then can send a performance monitoring request for the AI model to the network device.

[0354] Optionally, if the number of times the fifth parameter in the statistical information and / or distribution information of the CSI-related information inferred within the third time period is greater than the threshold corresponding to the fifth parameter and is greater than the fifth threshold, the terminal device may determine to trigger the first event. For example, the fifth parameter may be the average delay, the threshold corresponding to the average delay may be Threshold 1, and the fifth threshold may be Threshold 2. Within the second time period, if the number of times the average delay calculated by the terminal device is greater than Threshold 1 is 4 times, and 4 times is greater than Threshold 2, the terminal device may determine that the current communication quality is poor, trigger the first event, and then may send a performance monitoring request for the AI model to the network device. For example, the third time period is 10 seconds, the threshold corresponding to the average delay is the preset delay, and the fifth threshold is 3 times. If the number of times the average delay calculated by the terminal device is greater than the preset delay within 10 seconds is 4 times, the terminal device may determine to trigger the first event and send a performance monitoring request for the AI model to the network device.

[0355] Optionally, if the number of times the sixth parameter in the statistical information and / or distribution information of the CSI-related information inferred within the third time period is less than the threshold corresponding to the sixth parameter and is greater than the fifth threshold, the terminal device may determine to trigger the first event. For example, the threshold corresponding to the sixth parameter may be Threshold 1, and the fifth threshold may be Threshold 2. Within the second time period, if the number of times the sixth parameter calculated by the terminal device is less than Threshold 1 is 4 times, and 4 times is greater than Threshold 2, the terminal device may determine that the current communication quality is poor, trigger the first event, and then may send a performance monitoring request for the AI model to the network device. For example, the third time period is 10 seconds, the threshold corresponding to the sixth parameter is the preset threshold, and the fifth threshold is 3 times. If the number of times the sixth parameter calculated by the terminal device is less than the preset threshold within 10 seconds is 4 times, the terminal device may determine to trigger the first event and send a performance monitoring request for the AI model to the network device.

[0356] Optionally, if the deviation between the parameter in the statistical information and / or distribution information of the CSI-related information inferred and the third target value is greater than the sixth threshold, the terminal device may determine to trigger the first event. For example, the parameter in the statistical information and / or distribution information of the CSI-related information inferred may include the fifth parameter and the sixth parameter. Among them, each parameter has a corresponding third target value. If the deviation between the parameter in the statistical information and / or distribution information of the CSI-related information inferred and the third target value corresponding to the parameter is greater than the sixth threshold, it indicates that the current communication quality is poor, and the terminal device may send a performance monitoring request for the AI model to the network device. For example, the parameter in the statistical information of the CSI-related information is the average delay, and the third target value corresponding to the average delay is the preset delay. If the difference between the average delay calculated by the terminal device and the preset delay is greater than the sixth threshold, the terminal device may determine to trigger the first event and send a performance monitoring request to the network device.

[0357] S1003. Send a performance monitoring request to the network device.

[0358] Among them, when the terminal device determines to currently trigger the first event according to the statistical information and / or distribution information of the inferred CSI-related information, the terminal device may send a performance monitoring request for the AI model to the network device.

[0359] The embodiment of the present application provides a method for sending a performance monitoring request, obtains the statistical information and / or distribution information of the inferred CSI-related information, determines that the terminal device triggers the first event according to the statistical information and / or distribution information of the inferred CSI-related information, and sends a performance monitoring request to the network device. In this way, the terminal device can determine the communication quality according to the statistical information and / or distribution information of the inferred CSI-related information, and determine whether to enable the performance monitoring of the AI model based on the communication quality. In this way, when an unexpected event of communication quality decline occurs, the terminal device can timely perform performance monitoring on the AI model, improve the accuracy of the performance monitoring of the AI model, and further improve the stability of communication.

[0360] Figure 11 It is a schematic diagram of another method for sending a performance monitoring request provided by the embodiment of the present application. Please refer to Figure 11 , in Figure 11 In the shown embodiment, the first event is an event determined based on the generalization index of the AI model. The method flow includes:

[0361] S1101. Obtain the generalization index of the AI model.

[0362] Among them, the generalization index of the AI model can indicate the performance of the AI model. For example, the generalization index of the AI model may include the environment, scenario, speed, location, physical layer carrier resource configuration, etc. of the terminal device (the AI model is located in the terminal device).

[0363] It should be noted that the terminal device may obtain the generalization index of the AI model according to any feasible implementation manner, and the embodiment of the present application does not limit this.

[0364] S1102. Determine to trigger the first event according to the generalization index of the AI model.

[0365] Among them, if the first event is an event determined based on the generalization index of the AI model, the first event includes at least one of the following:

[0366] The duration when the parameter in the performance index of the received reference signal is less than the seventh threshold is greater than the eighth threshold;

[0367] The number of times when the parameter in the performance index of the received reference signal is less than the seventh threshold within the fourth duration is greater than the ninth threshold;

[0368] The quasi co-location information of the received reference signal changes;

[0369] The dataset label of the measured CSI-related information changes;

[0370] The duration during which the parameter in the measured speed information is greater than the tenth threshold is greater than the eleventh threshold;

[0371] The number of times the parameter in the measured speed information is greater than the tenth threshold within the fifth duration is greater than the twelfth threshold;

[0372] The distance difference between two positions separated by the measured sixth duration is greater than the thirteenth threshold.

[0373] Wherein, the reference signal can be a signal related to the CSI processed by the AI model. For example, if the AI model is a model for processing CSI, the reference signal can be a CSI-related reference signal; if the AI model is a model for processing beams, the reference signal can be a beam-related reference signal; if the AI model is a model for processing positioning, the reference signal can be a positioning-related reference signal.

[0374] Wherein, the performance indicators of the reference signal can include SINR, L1 (L1-level filtering)-RSRP, L1-RSRQ, and L3 (L3-level filtering)-RSRP. The performance indicators of the reference signal can also include any other arbitrary indicators, which are not limited in the embodiments of the present application. For example, the terminal device can receive the reference signal sent by the network device.

[0375] Wherein, the quasi co-location (QCL) information can include QCL source and QCL type.

[0376] It should be noted that the fourth duration, the fifth duration, the sixth duration, the seventh threshold, the eighth threshold, the ninth threshold, the tenth threshold, the eleventh threshold, the twelfth threshold, and the thirteenth threshold can be predefined by the protocol and / or configured by the network device, or can be determined based on any other feasible implementation manner, which is not limited in the embodiments of the present application.

[0377] Optionally, if the duration during which the parameter in the performance indicator of the reference signal received by the terminal device is less than the seventh threshold is greater than the eighth threshold, the terminal device can determine to trigger a first event. For example, the performance indicator of the reference signal can be SINR. If the duration during which SINR is less than the seventh threshold is greater than the eighth threshold, it indicates that the current communication quality is poor. The terminal device can determine to trigger the first event and send a performance monitoring request to the network device.

[0378] Optionally, if the number of times the parameter in the performance metric of the reference signal received by the terminal device is less than the seventh threshold within the fourth time period is greater than the ninth threshold, the terminal device may determine to trigger a first event. For example, the performance metric of the reference signal may be SINR. If within the fourth time period, the number of times the SINR determined by the terminal device is less than the seventh threshold is greater than the ninth threshold, it indicates that the current communication quality is poor. The terminal device may determine to trigger the first event and send a performance monitoring request to the network device. For example, the fourth time period is 10 seconds, and the ninth threshold is 3 times. If within 10 seconds, the number of times the SINR determined by the terminal device is less than the seventh threshold is 5 times, the terminal device may determine to trigger the first event and send a performance monitoring request to the network device.

[0379] Optionally, if the quasi co-location information of the reference signal received by the terminal device changes, the terminal device may determine to trigger a first event. For example, if the QCL source of the reference signal received by the terminal device changes, the terminal device may determine to trigger the first event and send a performance monitoring request to the network device. For example, if the QCL type of the reference signal received by the terminal device changes, the terminal device may determine to trigger the first event and send a performance monitoring request to the network device.

[0380] Optionally, if the dataset label of the CSI-related information measured by the terminal device changes, the terminal device may determine to trigger a first event. For example, the dataset label may indicate the environment and scenario of the terminal device. For example, based on the dataset label, the terminal device can determine whether it is indoors or outdoors. For example, if the terminal device moves from indoors to outdoors, the dataset label changes, and the terminal device may determine to trigger the first event and send a performance monitoring request to the network device.

[0381] Optionally, if the duration during which the parameter in the speed information measured by the terminal device is greater than the tenth threshold is greater than the eleventh threshold, the terminal device may determine to trigger a first event. For example, the parameter in the speed information may be the absolute value of the linear velocity. If the duration during which the absolute value of the linear velocity measured by the terminal device is greater than the tenth threshold is greater than the eleventh threshold, it indicates that the terminal device is moving fast. The terminal device may determine to trigger the first event and send a performance monitoring request for the model to the network device. For example, the tenth threshold is 10 m / s, and the eleventh threshold is 60 seconds. If the absolute value of the linear velocity measured by the terminal device is 15 m / s within 100 seconds, the terminal device may determine to trigger the first event and send a performance monitoring request for the model to the network device.

[0382] Optionally, if the number of times that the parameter in the speed information measured by the terminal device is greater than the tenth threshold within the fifth time period is greater than the twelfth threshold, the terminal device may determine to trigger the first event. For example, the parameter in the speed information may be the absolute value of the linear velocity. Within the fifth time period, if the number of times that the absolute value of the linear velocity measured by the terminal device (the terminal device may measure the linear velocity according to the sampling frequency) is greater than the tenth threshold is greater than the twelfth threshold, it indicates that the terminal device is moving fast. The terminal device may determine to trigger the first event and send a performance monitoring request for the model to the network device. For example, the fifth time period is 10 seconds, the tenth threshold is 10 m / s, and the twelfth threshold is 5 times. If the number of times that the absolute value of the linear velocity measured by the terminal device is greater than 10 m / s within 10 seconds is 10 times, the terminal device may determine to trigger the first event and send a performance monitoring request for the model to the network device.

[0383] Optionally, if the distance difference between two positions measured by the terminal device at an interval of the sixth time period is greater than the thirteenth threshold, the terminal device may determine to trigger the first event. For example, the terminal device measures the position at time 1 to obtain position 1, and the terminal device measures the position at time 2 to obtain position 2 (the time period between time 1 and time 2 is the sixth time period). If the distance difference between position 1 and position 2 is greater than the thirteenth threshold, the terminal device may determine to trigger the first event and send a performance monitoring request to the network device. For example, if the thirteenth threshold is 1000 m and the sixth time period is 60 seconds, and the distance between two positions measured by the terminal device at an interval of 60 seconds is 2000 m, the terminal device may determine to trigger the first event and send a performance monitoring request to the network device.

[0384] Optionally, if the operating center frequency, subcarrier, and / or bandwidth configuration of the terminal device changes, the terminal device may determine to trigger the first event. For example, if the operating center frequency of the terminal device changes, the terminal device may determine to trigger the first event. If the subcarrier of the terminal device changes, the terminal device may determine to trigger the first event. If the bandwidth configuration of the terminal device changes, the terminal device may determine to trigger the first event.

[0385] S1103. Send a performance monitoring request to the network device.

[0386] Among them, when the terminal device determines to currently trigger the first event according to the generalization index of the AI model, the terminal device may send a performance monitoring request for the AI model to the network device.

[0387] An embodiment of the present application provides a method for sending a performance monitoring request, obtaining a generalization metric of an AI model, determining, according to the generalization metric of the AI model, that a terminal device triggers a first event, and sending a performance monitoring request to a network device. In this way, the terminal device can determine that the network state of the terminal device has changed according to the generalization metric of the AI model, and then can timely perform performance monitoring on the AI model, improve the accuracy of AI model performance monitoring, save communication resources, and then improve the stability of communication.

[0388] Figure 12 It is a schematic diagram of another method for sending a performance monitoring request provided by an embodiment of the present application. Please refer to Figure 12 , in Figure 12 In the embodiment shown, the first event is an event determined based on the update information and / or model transfer information of the AI model, and the method flow includes:

[0389] S1201. Receive the model transfer information and / or model update information sent by the network device.

[0390] Among them, the update information of the AI model can indicate the update of the AI model, and the transfer information of the AI model can indicate the transfer of the AI model. For example, the terminal device can receive the update information of the AI model and the transfer information of the AI model sent by the network device or the server, and the terminal device can perform operations for life cycle management of the AI model based on the above information.

[0391] It should be noted that the terminal device can obtain the model transfer information or model update information of the AI model according to any feasible implementation manner, and the embodiments of the present application do not limit this.

[0392] S1202. Send a performance monitoring request to the network device.

[0393] Among them, if the first event is an event determined based on the update information or model transfer information of the AI model, the first event includes at least one of the following:

[0394] Receiving the model update information and / or model transfer information sent by the network device;

[0395] After receiving the model update information and / or model transfer information sent by the network device, no performance monitoring-related indication is received within the seventh time period.

[0396] It should be noted that the seventh time period can be predefined by the protocol and / or configured by the network device, or can be determined based on any other feasible implementation manner, and the embodiments of the present application do not limit this.

[0397] It should be noted that the terminal device can also receive the model update information and model transfer information sent by the server, and the embodiments of the present application do not limit this.

[0398] Optionally, if the terminal device receives the model update information and / or model transfer information sent by the network device, the terminal device can determine to trigger the first event. For example, if the terminal device receives the model update information sent by the network device, it means that the terminal device can perform operations for life cycle management of the AI model. Therefore, the terminal device can send a performance monitoring request for the AI model to the network device, and then monitor the performance of the updated AI model. For example, if the terminal device receives the model transfer information sent by the network device, it means that the terminal device can perform operations for life cycle management of the AI model. Therefore, the terminal device can send a performance monitoring request for the AI model to the network device, and then monitor the performance of the transferred AI model.

[0399] Optionally, if the terminal device does not receive an indication related to performance monitoring within the seventh time period after receiving the model update information and / or model transfer information sent by the network device, the terminal device can determine to trigger the first event. For example, after the terminal device receives the model update information and / or model transfer information sent by the network device, the terminal device can monitor the performance of the updated AI model or the transferred AI model. If the terminal device does not receive an indication related to performance monitoring sent by the network device, the terminal device can determine to trigger the first event and send a performance monitoring request to the network device.

[0400] The embodiments of the present application provide a method for sending a performance monitoring request, which receives the model transfer information and / or model update information sent by the network device and sends a performance monitoring request to the network device. In this way, after the AI model is updated or transferred, the terminal device can start monitoring the performance of the AI model, thereby improving the accuracy of performance monitoring and the communication quality.

[0401] Based on any of the above embodiments, the trigger event includes a second event related to the performance of the terminal device, where the second event can be an event determined based on the upper layer signaling of the terminal device. Next, in combination with Figure 13 ., the process of the terminal device sending a performance monitoring request when determining to trigger the second event in this scenario will be described.

[0402] Figure 13 It is a schematic diagram of a method for sending a performance monitoring request provided by the embodiments of the present application. Please refer to Figure 13 ., the method flow includes:

[0403] S1301. Obtain the upper layer signaling.

[0404] Among them, the upper-layer signaling may include any information that interacts with the upper-layer protocol layer in the terminal device. The embodiments of the present application do not limit this, and the terminal device may obtain the upper-layer signaling according to any feasible implementation manner. The embodiments of the present application do not limit this either.

[0405] S1302. Determine to trigger a second event according to the upper-layer signaling.

[0406] Among them, if the second event is an event determined based on the upper-layer signaling, the second event includes at least one of the following:

[0407] A handover occurs in the cell where it is located;

[0408] The first monitored quantity is greater than the fourteenth threshold;

[0409] The second monitored quantity is less than the fifteenth threshold;

[0410] The first parameter of the final key performance indicator at a future moment predicted according to the final key performance indicator at the current moment is less than the threshold corresponding to the first parameter;

[0411] The second parameter of the final key performance indicator at a future moment predicted according to the final key performance indicator at the current moment is greater than the threshold corresponding to the second parameter.

[0412] Among them, the first monitored quantity is inversely proportional to the performance of the terminal device. For example, the first monitored quantity may include the power consumption, used memory, etc. of the terminal device. The embodiments of the present application do not limit this.

[0413] Among them, the second monitored quantity is directly proportional to the performance of the terminal device. For example, the second monitored quantity may include the remaining computing power, remaining memory, etc. of the terminal device. The embodiments of the present application do not limit this.

[0414] It should be noted that the fourteenth threshold and the fifteenth threshold may be predefined by the protocol and / or configured by the network device, or may be determined based on any other feasible implementation manner. The embodiments of the present application do not limit this.

[0415] Optionally, if a handover occurs in the cell where the terminal device is located, the terminal device may determine to trigger a second event. For example, when a handover occurs in the cell where the terminal device is located, the terminal device needs to determine the currently available AI model. Therefore, the terminal device may determine to trigger a second event and send a monitoring request to the network device. For example, the terminal device sending a monitoring request to the network device may be triggered by the cell handover of the terminal device. The cell handover measurement time and the cell handover signaling of the terminal device may both trigger the terminal device to send a monitoring request for the AI model.

[0416] Optionally, if the first monitored quantity is greater than the fourteenth threshold, the terminal device may determine to trigger a second event. For example, the first monitored quantity may be the power consumption of the terminal device. If the power consumption of the terminal device is greater than the fourteenth threshold, it indicates that the performance of the terminal device will affect the communication quality. The terminal device may determine to trigger a second event and send a monitoring request to the network device. For example, the first monitored quantity may be the power consumption, and the fourteenth threshold is the preset power consumption. If the current power consumption obtained by the terminal device is greater than the preset power consumption, it indicates that the current performance of the terminal device is poor, which will further affect the communication quality of the terminal device. Therefore, the terminal device may determine to trigger a second event.

[0417] Optionally, if the second monitored quantity is less than the fifteenth threshold, the terminal device may determine to trigger a second event. For example, the second monitored quantity may be the remaining computing power of the terminal device. If the remaining computing power of the terminal device is less than the fourteenth threshold, it indicates that the performance of the terminal device will affect the communication quality. The terminal device may determine to trigger a second event and send a monitoring request to the network device. For example, the second monitored quantity may be the remaining computing power, and the fifteenth threshold is the preset computing power. If the current remaining computing power obtained by the terminal device is less than the preset computing power, it indicates that the current performance of the terminal device is poor, which will further affect the communication quality of the terminal device. Therefore, the terminal device may determine to trigger a second event.

[0418] Optionally, if the first parameter of the final key performance indicator at the future moment predicted based on the final key performance indicator at the current moment is less than the threshold corresponding to the first parameter, the terminal device may determine to trigger a second event. For example, the first parameter may be the throughput. If the throughput in the final KPI at the future moment predicted by the terminal device based on the current throughput is less than the threshold corresponding to the throughput, the terminal device may determine to trigger a second event.

[0419] Optionally, if the second parameter of the final key performance indicator at the future moment predicted based on the final key performance indicator at the current moment is greater than the threshold corresponding to the second parameter, the terminal device may determine to trigger a second event. For example, the second parameter may be the block error rate. If the block error rate in the final KPI at the future moment predicted by the terminal device based on the current throughput is greater than the threshold corresponding to the block error rate, the terminal device may determine to trigger a second event.

[0420] S1303. Send a performance monitoring request to the network device.

[0421] Among them, when the terminal device determines to trigger a second event according to the upper-layer signaling, the terminal device may send a performance monitoring request of the AI model to the network device.

[0422] An embodiment of the present application provides a method for sending a performance monitoring request. The method includes obtaining upper-layer signaling, determining, according to the upper-layer signaling, that a terminal device triggers a second event, and sending a performance monitoring request to a network device. In this way, the terminal device can determine, according to the upper-layer signaling, a second event related to the performance of the terminal device. Since this second event can affect the communication of the terminal device, when the terminal device triggers the second event, the terminal device can timely send a performance monitoring request of the model to the network device, improving the timeliness and accuracy of performance monitoring.

[0423] Based on any of the above embodiments, below, in combination with Figure 14 , the process of the network device sending a response to the performance monitoring request to the terminal device will be described.

[0424] Figure 14 It is a schematic diagram of a method for sending a response to a performance monitoring request to a terminal device provided by an embodiment of the present application. Please refer to Figure 14 , and the method flow includes:

[0425] S1401. The network device receives a performance monitoring request of the AI model sent by the terminal device.

[0426] Among them, the monitoring request may be determined by the terminal device based on a trigger event.

[0427] It should be noted that the trigger event and the information carried in the monitoring request may refer to the embodiments shown in Figures 2 - 13 , and the embodiments of the present application will not be elaborated herein.

[0428] S1402. The network device sends a response to the performance monitoring request to the terminal device.

[0429] Among them, when the network device sends a response to the performance monitoring request to the terminal device, specifically, it may be: judging whether the AI model needs to perform performance monitoring according to the information carried in the performance monitoring request, obtaining a judgment result, and sending a response to the performance monitoring request to the terminal device according to the judgment result.

[0430] Among them, when the network device receives the performance monitoring request, it can determine whether to perform performance monitoring on the AI model according to the target information in the performance monitoring request. For example, if the information carried in the performance monitoring request indicates a large throughput, the network device may refuse to perform performance monitoring on the AI model; if the information carried in the monitoring request indicates a small throughput, the network device may agree to perform performance monitoring on the AI model.

[0431] Optionally, if the judgment result is to agree to perform performance monitoring on the AI model, the response to the performance monitoring request sent by the network device to the terminal device can be "agree"; if the judgment result is to refuse to perform performance monitoring on the AI model, the response to the performance monitoring request sent by the network device to the terminal device can be "refuse".

[0432] It should be noted that if the target information is not included in the performance monitoring request, the network device may send a response to the performance monitoring request to the terminal device, and the response may be to agree to start performance monitoring or to agree to activate performance monitoring.

[0433] The embodiment of the present application provides a method for sending a response to a performance monitoring request to a terminal device. The network device receives a performance monitoring request of an AI model sent by the terminal device, determines whether the AI model needs to be performance-monitored according to the information carried in the performance monitoring request, obtains a judgment result, and sends a response to the performance monitoring request to the terminal device according to the judgment result. In this way, the network device can accurately determine whether to start performance monitoring on the AI model according to the target information reported by the terminal device, and thus can improve the accuracy of performance monitoring of the AI model.

[0434] Figure 15 It is a schematic structural diagram of a performance monitoring device provided by an embodiment of the present application. Please refer to Figure 15 The performance monitoring device 1500 includes a sending module 1501, a receiving module 1502, and a monitoring module 1503, where:

[0435] The sending module 1501 is configured to send a performance monitoring request of the AI model to the network device according to a trigger event;

[0436] The receiving module 1502 is configured to receive the response to the performance monitoring request sent by the network device;

[0437] The monitoring module 1503 is configured to monitor the performance of the AI model according to the response to the performance monitoring request.

[0438] In one embodiment, the performance monitoring request includes an identifier of the AI model and / or an identifier of a function.

[0439] In one embodiment, the performance monitoring request includes an identifier indicating an applicable model corresponding to the performance monitoring request, an identifier of an applicable function, and an identifier of an applicable scenario.

[0440] In one embodiment, the performance monitoring request includes a start time of performance monitoring and / or an end time of performance monitoring.

[0441] In one embodiment, the performance monitoring request includes target information related to the performance metrics of the AI model and / or the metric values of the AI model.

[0442] In one embodiment, the target information includes at least one of the following:

[0443] The intermediate key performance metrics related to the AI model;

[0444] The final key performance metrics related to the AI model;

[0445] The statistical information of the information related to the channel state information (CSI);

[0446] The distribution information of the information related to the CSI;

[0447] The reference signal received power related to the AI model;

[0448] The reference signal received quality related to the AI model;

[0449] Signal-to-interference-plus-noise ratio;

[0450] Speed information;

[0451] Location information.

[0452] In one embodiment, the trigger event includes a first event related to the AI model and / or function and a second event related to the performance of the terminal device.

[0453] In one embodiment, when the AI model is not activated, not selected, or not paired, the first event includes at least one of the following:

[0454] Sending the identifier of the supported AI model and / or the identifier of the function;

[0455] Receiving the identifier of the AI model supported by the cell covered by the current base station and / or the identifier of the function.

[0456] In one embodiment, when the AI model is activated, selected, or paired, the first event is an event determined based on at least one of the following information:

[0457] The measured final key performance metrics;

[0458] The statistical information and / or distribution information of the measured CSI-related information;

[0459] The statistical information and / or distribution information of the CSI-related information inferred by the AI model;

[0460] The generalization metrics of the AI model;

[0461] The update information of the AI model and / or the transmission information of the AI model.

[0462] In one implementation, the first event is an event determined based on the measured final key performance indicator, and the first event includes at least one of the following:

[0463] A first parameter in the final key performance indicator is less than a threshold corresponding to the first parameter, and the first parameter is proportional to the performance of the AI model;

[0464] A second parameter in the final key performance indicator is greater than a threshold corresponding to the second parameter, and the second parameter is inversely proportional to the performance of the AI model;

[0465] The number of times that the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter within a first time period is greater than a first threshold;

[0466] The number of times that the second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter within the first time period is greater than the first threshold;

[0467] The deviation between a parameter in the final key performance indicator and a first target value is greater than a second threshold, and the first target value is the target value corresponding to the parameter in the final key performance indicator.

[0468] In one implementation, the first event is an event determined based on the statistical information and / or distribution information of the measured CSI-related information, and the first event includes at least one of the following:

[0469] A third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than a threshold corresponding to the third parameter, and the third parameter is inversely proportional to the performance of the AI model;

[0470] A fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is less than a threshold corresponding to the fourth parameter, and the fourth parameter is proportional to the performance of the AI model;

[0471] The number of times that the third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than the threshold corresponding to the third parameter within a second time period is greater than a third threshold;

[0472] The number of times that the fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is less than the threshold corresponding to the fourth parameter within the second time period is greater than the third threshold;

[0473] The deviation between the parameter in the statistical information and / or distribution information of the measured CSI-related information and the second target value is greater than a fourth threshold, where the second target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the measured CSI-related information.

[0474] In one implementation, the first event is an event determined based on the statistical information and / or distribution information of the CSI-related information inferred by the AI model. The first event includes at least one of the following:

[0475] The fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter, and the fifth parameter is inversely proportional to the performance of the AI model;

[0476] The sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter, and the sixth parameter is directly proportional to the performance of the AI model;

[0477] The number of times that the fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter within a third time period is greater than a fifth threshold;

[0478] The number of times that the sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter within a third time period is greater than the fifth threshold;

[0479] The deviation between the parameter in the statistical information and / or distribution information of the inferred CSI-related information and the third target value is greater than a sixth threshold, where the third target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the inferred CSI-related information.

[0480] In one implementation, the first event is an event determined based on the generalization metric of the AI model. The first event includes at least one of the following:

[0481] The duration during which the parameter in the performance metric of the received reference signal is less than a seventh threshold is greater than an eighth threshold;

[0482] The number of times that the parameter in the performance metric of the received reference signal is less than the seventh threshold within a fourth time period is greater than a ninth threshold;

[0483] The quasi co-location information of the received reference signal changes;

[0484] The dataset label of the measured CSI-related information changes;

[0485] The duration for which the parameter in the measured speed information is greater than the tenth threshold is greater than the eleventh threshold;

[0486] The number of times the parameter in the measured speed information is greater than the tenth threshold within the fifth duration is greater than the twelfth threshold;

[0487] The distance difference between two positions separated by the sixth duration of measurement is greater than the thirteenth threshold;

[0488] The operating center frequency, sub - carriers, and / or bandwidth configuration change.

[0489] In one embodiment, the first event is an event determined based on the update information and / or model transfer information of the AI model, and the first event includes at least one of the following:

[0490] Receiving the model update information and / or model transfer information sent by the network device;

[0491] After receiving the model update information and / or model transfer information sent by the network device, no performance monitoring - related indication is received within the seventh duration.

[0492] In one embodiment, the second event is an event determined based on upper - layer signaling, and the second event includes at least one of the following:

[0493] The cell where the device is located undergoes a handover;

[0494] The first monitoring quantity is greater than the fourteenth threshold, and the first monitoring quantity is inversely proportional to the performance of the terminal device;

[0495] The second monitoring quantity is less than the fifteenth threshold, and the first monitoring quantity is directly proportional to the performance of the terminal device;

[0496] The first parameter of the final key performance indicator at a future time predicted based on the final key performance indicator at the current time is less than the threshold corresponding to the first parameter;

[0497] The second parameter of the final key performance indicator at a future time predicted based on the final key performance indicator at the current time is greater than the threshold corresponding to the second parameter.

[0498] In one embodiment, the thresholds, durations, and target values are obtained based on protocol pre - definitions and / or network - side configurations and / or a combination of protocol pre - definitions and network - side configurations.

[0499] In one embodiment, the CSI - related information includes the CSI information, beam information, and positioning information.

[0500] In one embodiment, the response to the performance monitoring request includes a reply or indication of the performance monitoring request.

[0501] In one embodiment, the sending module 1501 is specifically configured to include:

[0502] Send the performance monitoring request to the network device based on the scheduling request SR of the uplink control information UCI;

[0503] Or,

[0504] Send the performance monitoring request to the network device based on the media access control cell MAC-CE signaling;

[0505] Or,

[0506] Send the performance monitoring request to the network device based on the physical uplink control channel PUCCH resource or the physical uplink shared channel PUSCH resource of the reservation period.

[0507] Figure 16 It is a schematic structural diagram of another performance monitoring device provided by the embodiments of the present application. Please refer to Figure 16 , the performance monitoring device 1600 includes a receiving module 1601 and a sending module 1602, where:

[0508] The receiving module 1601 is configured to receive the performance monitoring request of the AI model sent by the terminal device, and the performance monitoring request is determined by the terminal device based on the triggering event;

[0509] The sending module 1602 is configured to send a response to the performance monitoring request to the terminal device.

[0510] In one embodiment, the sending module 1602 is specifically configured to:

[0511] Judge whether the AI model needs to be monitored according to the information carried in the monitoring request, and obtain a judgment result;

[0512] Send a response to the monitoring request to the terminal device according to the judgment result.

[0513] In one embodiment, the sending module 1602 is specifically configured to:

[0514] Judge whether the AI model needs to perform performance monitoring according to the information carried in the performance monitoring request, and obtain a judgment result;

[0515] Send a response to the performance monitoring request to the terminal device according to the judgment result.

[0516] In one embodiment, the sending module is specifically configured to:

[0517] Receive the scheduling request SR sent by the terminal device;

[0518] Send a response to the scheduling request SR to the terminal device, where the response to the scheduling request SR includes downlink control information DCI for uplink transmission resource allocation, and the downlink control information DCI is scrambled based on a radio network temporary identity RNTI of the terminal device.

[0519] It should be noted that the division of units in the embodiments of the present application is illustrative only, and is only a logical function division. In actual implementation, there may be other division methods. In addition, in each embodiment of the present application, each functional unit may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above integrated units may be implemented in the form of hardware or in the form of software functional units.

[0520] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0521] It should be noted here that the above device provided by the present application can implement all the method steps implemented by the above method embodiments, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0522] Figure 17 This is a schematic structural diagram of a terminal device provided by an embodiment of the present application. Please refer to Figure 17 , the terminal device includes a memory 1710, a transceiver 1720, and a processor 1730:

[0523] The memory 1710 is used to store a computer program;

[0524] The transceiver 1720 is used to send and receive data under the control of the processor;

[0525] The processor 1730 is configured to read the computer program in the memory and perform the following operations:

[0526] Send a performance monitoring request of the AI model to a network device according to a trigger event;

[0527] Receive a response to the performance monitoring request sent by the network device, and monitor the performance of the AI model according to the response to the performance monitoring request.

[0528] In one embodiment, the performance monitoring request includes an identifier of the AI model and / or an identifier of a function.

[0529] In one embodiment, the performance monitoring request includes an identifier indicating an applicable model corresponding to the performance monitoring request, an identifier of an applicable function, and an identifier of an applicable scenario.

[0530] In one embodiment, the performance monitoring request includes a start time of performance monitoring and / or an end time of performance monitoring.

[0531] In one embodiment, the performance monitoring request includes target information related to a performance metric of the AI model and / or a metric value of the AI model.

[0532] In one embodiment, the target information includes at least one of the following:

[0533] Intermediate key performance indicators related to the AI model;

[0534] Final key performance indicators related to the AI model;

[0535] Statistical information of information related to channel state information CSI;

[0536] Distribution information of the information related to the CSI;

[0537] Reference signal received power related to the AI model;

[0538] Reference signal received quality related to the AI model;

[0539] Signal-to-interference-plus-noise ratio;

[0540] Speed information;

[0541] Location information.

[0542] In one embodiment, the trigger event includes a first event related to the AI model and / or function and a second event related to the performance of the terminal device.

[0543] In one embodiment, when the AI model is not activated, not selected, or not paired, the first event includes at least one of the following:

[0544] Sending the identifier of the supported AI model and / or the identifier of the function;

[0545] Receiving the identifier of the AI model supported by the cell covered by the current base station and / or the identifier of the function.

[0546] In one embodiment, when the AI model is activated, selected, or paired, the first event is an event determined based on at least one of the following information:

[0547] The measured final key performance indicator;

[0548] The statistical information and / or distribution information of the measured CSI-related information;

[0549] The statistical information and / or distribution information of the CSI-related information inferred by the AI model;

[0550] The generalization index of the AI model;

[0551] The update information of the AI model and / or the transfer information of the AI model.

[0552] In one embodiment, the first event is an event determined based on the measured final key performance indicator, and the first event includes at least one of the following:

[0553] The first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter, and the first parameter is proportional to the performance of the AI model;

[0554] The second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter, and the second parameter is inversely proportional to the performance of the AI model;

[0555] The number of times that the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter within the first time period is greater than the first threshold;

[0556] The number of times that the second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter within the first time period is greater than the first threshold;

[0557] The deviation of the parameter in the final key performance indicator from the first target value is greater than the second threshold, and the first target value is the target value corresponding to the parameter in the final key performance indicator.

[0558] In one embodiment, the first event is an event determined based on statistical information and / or distribution information of measured CSI-related information, and the first event includes at least one of the following:

[0559] The third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than the threshold corresponding to the third parameter, and the third parameter is inversely proportional to the performance of the AI model;

[0560] The fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is less than the threshold corresponding to the fourth parameter, and the fourth parameter is directly proportional to the performance of the AI model;

[0561] The number of times that the third parameter in the statistical information and / or distribution information of the measured CSI-related information is greater than the threshold corresponding to the third parameter within a second time period is greater than a third threshold;

[0562] The number of times that the fourth parameter in the statistical information and / or distribution information of the measured CSI-related information is less than the threshold corresponding to the fourth parameter within a second time period is greater than the third threshold;

[0563] The deviation of the parameter in the statistical information and / or distribution information of the measured CSI-related information from a second target value is greater than a fourth threshold, and the second target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the measured CSI-related information.

[0564] In one embodiment, the first event is an event determined based on statistical information and / or distribution information of CSI-related information inferred by the AI model, and the first event includes at least one of the following:

[0565] The fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter, and the fifth parameter is inversely proportional to the performance of the AI model;

[0566] The sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter, and the sixth parameter is directly proportional to the performance of the AI model;

[0567] The number of times that the fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter within a third time period is greater than a fifth threshold;

[0568] The number of times that the sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter within a third time period is greater than the fifth threshold;

[0569] The deviation between the parameter in the statistical information and / or distribution information of the CSI related to the inference and the third target value is greater than a sixth threshold, where the third target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the CSI related to the inference.

[0570] In one implementation, the first event is an event determined based on a generalization metric of the AI model, and the first event includes at least one of the following:

[0571] The duration during which the parameter in the performance metric of the received reference signal is less than a seventh threshold is greater than an eighth threshold;

[0572] The number of times the parameter in the performance metric of the received reference signal is less than the seventh threshold within a fourth duration is greater than a ninth threshold;

[0573] The quasi - co - location information of the received reference signal changes;

[0574] The dataset label of the measured CSI - related information changes;

[0575] The duration during which the parameter in the measured speed information is greater than a tenth threshold is greater than an eleventh threshold;

[0576] The number of times the parameter in the measured speed information is greater than the tenth threshold within a fifth duration is greater than a twelfth threshold;

[0577] The distance difference between two positions measured at an interval of a sixth duration is greater than a thirteenth threshold;

[0578] The operating center frequency, sub - carriers, and / or bandwidth configuration change.

[0579] In one implementation, the first event is an event determined based on the update information and / or model transfer information of the AI model, and the first event includes at least one of the following:

[0580] Receiving model update information and / or model transfer information sent by the network device;

[0581] After receiving the model update information and / or model transfer information sent by the network device, no performance monitoring - related indication is received within a seventh duration.

[0582] In one implementation, the second event is an event determined based on upper - layer signaling, and the second event includes at least one of the following:

[0583] The cell where the terminal device is located undergoes a handover;

[0584] A first monitoring quantity is greater than a fourteenth threshold, and the first monitoring quantity is inversely proportional to the performance of the terminal device;

[0585] The second monitored quantity is less than the fifteenth threshold, and the first monitored quantity is proportional to the performance of the terminal device;

[0586] A first parameter of the final key performance indicator at a future moment predicted based on the final key performance indicator at the current moment is less than the threshold corresponding to the first parameter;

[0587] A second parameter of the final key performance indicator at a future moment predicted based on the final key performance indicator at the current moment is greater than the threshold corresponding to the second parameter.

[0588] In one implementation, the threshold, duration, and target value are obtained based on protocol pre - definition and / or network - side configuration and / or a combination of protocol pre - definition and network - side configuration.

[0589] In one implementation, the CSI - related information includes the CSI information, beam information, and positioning information.

[0590] In one implementation, the response to the performance monitoring request includes a reply or indication of the performance monitoring request.

[0591] In one implementation, sending the performance monitoring request of the AI model to the network device includes:

[0592] Sending the performance monitoring request to the network device based on a scheduling request SR of uplink control information UCI;

[0593] Or,

[0594] Sending the performance monitoring request to the network device based on a media access control cell MAC - CE signaling;

[0595] Or,

[0596] Sending the performance monitoring request to the network device based on a reserved - period physical uplink control channel PUCCH resource or a physical uplink shared channel PUSCH resource.

[0597] In one implementation, the terminal device 1700 may further include a user interface 1740. For different terminal devices, the user interface 1740 may also be an interface capable of externally or internally connecting to required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, etc.

[0598] Among them, in Figure 17Among them, the bus architecture may include any number of interconnected buses and bridges, specifically various circuits of one or more processors represented by processor 1703 and memory represented by memory 1710 are linked together. The bus architecture can also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, so they will not be further described herein. The bus interface provides an interface. The transceiver 1720 can be multiple components, that is, including a transmitter and a receiver, and provides a unit for communicating with various other devices on a transmission medium, and these transmission mediums include wireless channels, wired channels, optical cables and other transmission mediums. The processor 1730 is responsible for managing the bus architecture and general processing, and the memory 1701 can store the data used by the processor 1730 when executing operations.

[0599] Optionally, the processor 1730 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD), and the processor can also adopt a multi-core architecture.

[0600] The processor 1730 is used to execute any of the methods provided in the embodiments of the present application according to the obtained executable instructions by calling the computer program stored in the memory 1710. The processor 1730 and the memory 1710 can also be physically separated.

[0601] It should be noted here that the above-mentioned physical devices provided in the present application can implement all the method steps implemented by the physical devices in the above method embodiments, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described herein.

[0602] Figure 18 It is a schematic structural diagram of a network device provided in an embodiment of the present application. Please refer to Figure 18 This network device includes a memory 1810, a transceiver 1820, and a processor 1830:

[0603] The memory 1810 is used to store a computer program;

[0604] The transceiver 1820 is used to send and receive data under the control of the processor;

[0605] The processor 1830 is configured to read a computer program in the memory and perform the following operations:

[0606] Receive a performance monitoring request of an AI model sent by a terminal device, where the performance monitoring request is determined by the terminal device based on a triggering event;

[0607] Send a response to the performance monitoring request to the terminal device.

[0608] In one implementation, sending the response to the performance monitoring request to the terminal device includes:

[0609] Judge whether the AI model needs to be performance-monitored according to the information carried in the performance monitoring request, and obtain a judgment result;

[0610] Send a response to the performance monitoring request to the terminal device according to the judgment result.

[0611] In one implementation, sending the response to the performance monitoring request to the terminal device includes:

[0612] Receive a scheduling request SR sent by the terminal device;

[0613] Send a response to the scheduling request SR to the terminal device, where the response to the scheduling request SR includes downlink control information DCI for uplink transmission resource allocation, and the downlink control information DCI is scrambled based on the radio network temporary identity RNTI of the terminal device.

[0614] Among them, in Figure 18 The bus architecture may include any number of interconnected buses and bridges, specifically various circuits of one or more processors represented by the processor 1830 and the memory represented by the memory 1810 are linked together. The bus architecture can also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, and thus will not be further described herein. The bus interface provides an interface. The transceiver 1820 may be multiple elements, that is, including a transmitter and a receiver, and provides a unit for communicating with various other devices on a transmission medium, and these transmission mediums include wireless channels, wired channels, optical fiber cables, and other transmission mediums. The processor 1830 is responsible for managing the bus architecture and general processing, and the memory 1810 can store the data used by the processor 1830 when performing operations.

[0615] Optionally, the processor 1830 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0616] It should be noted here that the above-mentioned physical device provided in this application can implement all the method steps implemented by the physical device in the above method embodiments, and can achieve the same technical effects. Therefore, the same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0617] This application embodiment also provides a processor-readable storage medium storing a computer program for causing the processor to execute the method described in any one of the above method embodiments.

[0618] The processor-readable storage medium may be any available medium or data storage device accessible by a computer, including but not limited to magnetic memories (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical memories (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memories (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NAND FLASH), solid state drives (SSD)).

[0619] This application embodiment also provides a computer program product including a computer program which, when executed by a processor, implements the method described in any one of the above method embodiments.

[0620] Those skilled in the art should understand that the embodiments of this application may be provided as a method, a system, or a computer program product. Therefore, this application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0621] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 and / or means for implementing the functions specified in one or more of the blocks.

[0622] These processor-executable instructions can also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce a manufacture including instruction means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 and / or means for implementing the functions specified in one or more of the blocks.

[0623] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 and / or means for implementing the functions specified in one or more of the blocks.

[0624] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. A performance monitoring method, characterized in that, Applied to a terminal device, the method includes: Sending a performance monitoring request for an AI model to a network device according to a trigger event; Receiving a response to the performance monitoring request sent by the network device, and performing performance monitoring on the AI model according to the response to the performance monitoring request.

2. The method according to claim 1, characterized in that The performance monitoring request includes an identifier of the AI model and / or an identifier of a function.

3. The method according to claim 1 or 2, characterized in that, The performance monitoring request includes an identifier indicating an applicable model corresponding to the performance monitoring request, an identifier of an applicable function, and an identifier of an applicable scenario.

4. The method according to any one of claims 1 to 3, characterized in that, The performance monitoring request includes a start time of performance monitoring and / or an end time of performance monitoring.

5. The method according to any one of claims 1-4, characterized in that, The performance monitoring request includes target information, and the target information is related to a performance metric of the AI model and / or a measurement value of the AI model.

6. The method according to claim 5, wherein The target information includes at least one of the following: Intermediate key performance indicators related to the AI model; Final key performance indicators related to the AI model; Statistical information of information related to channel state information CSI; Distribution information of the information related to the CSI; Reference signal received power related to the AI model; Reference signal received quality related to the AI model; Signal-to-interference-plus-noise ratio; Speed information; Location information.

7. The method according to any one of claims 1-6, characterized in that, The trigger event includes a first event related to the AI model and / or function and a second event related to the performance of the terminal device.

8. The method according to claim 7, characterized in that, When the AI model is not activated, not selected, or not paired, the first event includes at least one of the following: Sending an identifier of a supported AI model and / or an identifier of a function; Receiving an identifier of an AI model supported by a cell covered by the current base station and / or an identifier of a function.

9. The method according to claim 7, wherein When the AI model is activated, selected, or paired, the first event is an event determined based on at least one of the following information: Measured final key performance indicators; Statistical information and / or distribution information of measured information related to CSI; Statistical information and / or distribution information of information related to CSI inferred by the AI model; Generalization metrics of the AI model; Update information of the AI model and / or transfer information of the AI model.

10. The method according to claim 9, wherein The first event is an event determined based on measured final key performance indicators, and the first event includes at least one of the following: A first parameter in the final key performance indicator is less than a threshold corresponding to the first parameter, and the first parameter is proportional to the performance of the AI model; A second parameter in the final key performance indicator is greater than a threshold corresponding to the second parameter, and the second parameter is inversely proportional to the performance of the AI model; The number of times that the first parameter in the final key performance indicator is less than the threshold corresponding to the first parameter within a first time period is greater than a first threshold; The number of times that the second parameter in the final key performance indicator is greater than the threshold corresponding to the second parameter within the first time period is greater than the first threshold; The deviation of a parameter in the final key performance indicator from a first target value is greater than a second threshold, and the first target value is a target value corresponding to the parameter in the final key performance indicator.

11. The method according to claim 9, wherein The first event is an event determined based on statistical information and / or distribution information of measurement CSI-related information, and the first event includes at least one of the following: The third parameter in the statistical information and / or distribution information of the measurement CSI-related information is greater than the threshold corresponding to the third parameter, and the third parameter is inversely proportional to the performance of the AI model; The fourth parameter in the statistical information and / or distribution information of the measurement CSI-related information is less than the threshold corresponding to the fourth parameter, and the fourth parameter is directly proportional to the performance of the AI model; The number of times that the third parameter in the statistical information and / or distribution information of the measurement CSI-related information is greater than the threshold corresponding to the third parameter within the second time period is greater than the third threshold; The number of times that the fourth parameter in the statistical information and / or distribution information of the measurement CSI-related information is less than the threshold corresponding to the fourth parameter within the second time period is greater than the third threshold; The deviation between the parameter in the statistical information and / or distribution information of the measurement CSI-related information and the second target value is greater than the fourth threshold, and the second target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the measurement CSI-related information.

12. The method according to claim 9, wherein The first event is an event determined based on statistical information and / or distribution information of CSI-related information inferred by the AI model, and the first event includes at least one of the following: The fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter, and the fifth parameter is inversely proportional to the performance of the AI model; The sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter, and the sixth parameter is directly proportional to the performance of the AI model; The number of times that the fifth parameter in the statistical information and / or distribution information of the inferred CSI-related information is greater than the threshold corresponding to the fifth parameter within the third time period is greater than the fifth threshold; The number of times that the sixth parameter in the statistical information and / or distribution information of the inferred CSI-related information is less than the threshold corresponding to the sixth parameter within the third time period is greater than the fifth threshold; The deviation between the parameter in the statistical information and / or distribution information of the inferred CSI-related information and the third target value is greater than the sixth threshold, and the third target value is the target value corresponding to the parameter in the statistical information and / or distribution information of the inferred CSI-related information.

13. The method according to claim 9, characterized in that, The first event is an event determined based on the generalization index of the AI model, and the first event includes at least one of the following: The duration during which the parameter in the performance index of the received reference signal is less than the seventh threshold is greater than the eighth threshold; The number of times that the parameter in the performance index of the received reference signal is less than the seventh threshold within the fourth time period is greater than the ninth threshold; The quasi co-location information of the received reference signal changes; The dataset label of the measurement CSI-related information changes; The duration during which the parameter in the measured speed information is greater than the tenth threshold is greater than the eleventh threshold; The number of times that the parameter in the measured speed information is greater than the tenth threshold within the fifth time period is greater than the twelfth threshold; The distance difference between two positions at the sixth time interval of the measurement is greater than the thirteenth threshold; The operating center frequency, subcarrier, and / or bandwidth configuration change.

14. The method according to claim 9, wherein The first event is an event determined based on the update information and / or model transfer information of the AI model. The first event includes at least one of the following: Receiving the model update information and / or model transfer information sent by the network device; After receiving the model update information and / or model transfer information sent by the network device, no performance monitoring-related indication is received within the seventh time period.

15. The method according to any one of claims 7 to 14, characterized in that The second event is an event determined based on upper-layer signaling. The second event includes at least one of the following: The cell where the device is located undergoes a handover; The first monitored quantity is greater than the fourteenth threshold, and the first monitored quantity is inversely proportional to the performance of the terminal device; The second monitored quantity is less than the fifteenth threshold, and the first monitored quantity is directly proportional to the performance of the terminal device; The first parameter of the final key performance indicator at a future time predicted based on the final key performance indicator at the current time is less than the threshold corresponding to the first parameter; The second parameter of the final key performance indicator at a future time predicted based on the final key performance indicator at the current time is greater than the threshold corresponding to the second parameter.

16. The method according to any one of claims 10 to 14, characterized in that The thresholds, time periods, and target values are obtained based on protocol predefinitions and / or network-side configurations and / or a combination of protocol predefinitions and network-side configurations.

17. The method according to any one of claims 1-16, characterized in that, The CSI-related information includes the CSI information, beam information, and positioning information.

18. The method according to any one of claims 1-17, characterized in that, The response to the performance monitoring request includes the reply or indication of the performance monitoring request.

19. The method according to any one of claims 1-18, characterized in that, Sending a performance monitoring request for the AI model to the network device includes: Sending the performance monitoring request to the network device based on the scheduling request SR of the uplink control information UCI; Or, Sending the performance monitoring request to the network device based on the media access control cell MAC-CE signaling; Or, Sending the performance monitoring request to the network device based on the reserved-period physical uplink control channel PUCCH resource or physical uplink shared channel PUSCH resource.

20. A performance monitoring method, characterized in that, Applied to a network device, the method includes: Receiving a performance monitoring request for the AI model sent by a terminal device, where the performance monitoring request is determined by the terminal device based on a triggering event; Sending a response to the performance monitoring request to the terminal device.

21. The method according to claim 20, wherein Sending the response to the performance monitoring request to the terminal device includes: Judging whether the AI model needs to perform performance monitoring according to the information carried in the performance monitoring request to obtain a judgment result; Sending a response to the performance monitoring request to the terminal device according to the judgment result.

22. The method according to claim 20 or 21, characterized in that, Sending the response to the performance monitoring request to the terminal device includes: Receiving the scheduling request SR sent by the terminal device; Sending a response to the scheduling request SR to the terminal device, where the response to the scheduling request SR includes downlink control information DCI for uplink transmission resource allocation, and the downlink control information DCI is scrambled based on the radio network temporary identity RNTI of the terminal device.

23. A performance monitoring device, characterized in that, Applied to a terminal device, the performance monitoring device includes a sending module, a receiving module, and a monitoring module, where: The sending module is configured to send a performance monitoring request for an AI model to a network device according to a triggering event; The receiving module is configured to receive a response to the performance monitoring request sent by the network device; The monitoring module is configured to monitor the performance of the AI model according to the response to the performance monitoring request.

24. A performance monitoring device, characterized in that, Applied to a network device, the performance monitoring device includes a receiving module and a sending module, where: The receiving module is configured to receive a performance monitoring request for an AI model sent by a terminal device, and the performance monitoring request is determined by the terminal device based on a triggering event; The sending module is configured to send a response to the performance monitoring request to the terminal device.

25. A terminal device, characterized in that, Including a memory, a transceiver, and a processor: The memory is used to store a computer program; The transceiver is used to send and receive data under the control of the processor; The processor is configured to read the computer program in the memory and perform the following operations: Send a performance monitoring request for an AI model to a network device according to a triggering event; Receive a response to the performance monitoring request sent by the network device, and monitor the performance of the AI model according to the response to the performance monitoring request.

26. A network device, characterized in that, Including a memory, a transceiver, and a processor: The memory is used to store a computer program; The transceiver is used to send and receive data under the control of the processor; The processor is configured to read the computer program in the memory and perform the following operations: Receive a performance monitoring request for an AI model sent by a terminal device, and the performance monitoring request is determined by the terminal device based on a triggering event; Send a response to the performance monitoring request to the terminal device.

27. A processor-readable storage medium, characterized in that, The processor-readable storage medium stores a computer program, and the computer program is used to cause the processor to execute the method according to any one of claims 1 to 19 or execute the method according to claims 20-22.

Images