Model monitoring method and apparatus, device, and readable storage medium

Abstract

The present application relates to the field of communications, and discloses a model monitoring method and apparatus, a device, and a readable storage medium. The method in the embodiments of the present application comprises: a terminal acquiring first information; and performing model monitoring on a first artificial intelligence (AI) model on the basis of the first information, wherein the first AI model is configured to execute a first prediction function; wherein the first information comprises at least one of the following: one or more prediction accuracy requirements; configuration information corresponding to the first prediction function; and a first parameter, used for determining a prediction accuracy requirement associated with the first prediction function, the one or more prediction accuracy requirements being associated with at least one of the following: a prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model.

Description

Model monitoring method, device, equipment and readable storage medium

[0001] Cross-reference to Related Applications

[0002] The present application claims priority to the Chinese patent application No. 202411272532.0, filed on September 11, 2024, and entitled "Model monitoring method, device, equipment and readable storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application belongs to the field of communication technology, and particularly relates to a model monitoring method, device, equipment and readable storage medium. BACKGROUND

[0004] In the related art, a terminal can manage a model through a model monitoring result, for example, when the model performance cannot meet the accuracy requirement, the model is deactivated, switched or rolled back. However, if the accuracy requirement of the model performance is too high, the model may not work normally, and if the accuracy requirement of the model performance is too low, the prediction performance of the model may be affected. Therefore, how to select a suitable accuracy requirement to monitor the model to balance the normal work of the model and the prediction performance of the model is an urgent problem to be solved. SUMMARY

[0005] The embodiments of the present application provide a model monitoring method, device, equipment and readable storage medium, which can balance the normal work of the model and the prediction performance of the model.

[0006] In a first aspect, a model monitoring method is provided, the method comprising:

[0007] obtaining first information by a terminal;

[0008] monitoring a first artificial intelligence (AI) model according to the first information, wherein the first AI model is used to perform a first prediction function;

[0009] The first information comprises at least one of the following:

[0010] one or more prediction accuracy requirements;

[0011] configuration information corresponding to the first prediction function;

[0012] a first parameter used to determine a prediction accuracy requirement associated with the first prediction function;

[0013] The one or more prediction accuracy requirements are associated with at least one of the following:

[0014] a prediction window, a measurement reduction ratio MRR, a frequency point, a cell, a beam, an AI function, an AI model.

[0015] In a second aspect, a model monitoring method is provided, the method comprising:

[0016] a network-side device sending first information to a terminal, the first information being used for the terminal to perform model monitoring on a first artificial intelligence (AI) model, wherein the first AI model is used to perform a first prediction function;

[0017] The first information comprises at least one of:

[0018] one or more prediction accuracy requirements;

[0019] configuration information corresponding to the first prediction function;

[0020] a first parameter used to determine a prediction accuracy requirement associated with the first prediction function;

[0021] The one or more prediction accuracy requirements are associated with at least one of:

[0022] a prediction window, a measurement reduction ratio MRR, a frequency point, a cell, a beam, an AI function, an AI model.

[0023] In a third aspect, a wireless communication apparatus is provided, comprising:

[0024] a processing module configured to obtain first information;

[0025] perform model monitoring on a first artificial intelligence (AI) model according to the first information, wherein the first AI model is used to perform a first prediction function;

[0026] The first information comprises at least one of:

[0027] one or more prediction accuracy requirements;

[0028] configuration information corresponding to the first prediction function;

[0029] a first parameter used to determine a prediction accuracy requirement associated with the first prediction function;

[0030] The one or more prediction accuracy requirements are associated with at least one of:

[0031] a prediction window, a measurement reduction ratio MRR, a frequency point, a cell, a beam, an AI function, an AI model.

[0032] In a fourth aspect, a wireless communication apparatus is provided, comprising:

[0033] The sending module is configured to send first information to a terminal, the first information being used for model monitoring of a first artificial intelligence (AI) model by the terminal, wherein the first AI model is used to perform a first prediction function.

[0034] The first information comprises at least one of the following:

[0035] One or more prediction accuracy requirements;

[0036] Configuration information corresponding to the first prediction function;

[0037] A first parameter used to determine a prediction accuracy requirement associated with the first prediction function;

[0038] The one or more prediction accuracy requirements are associated with at least one of the following:

[0039] A prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model.

[0040] In a fifth aspect, a wireless communication apparatus is provided, which is configured to perform the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

[0041] In a sixth aspect, a terminal is provided, which comprises a processor and a memory, the memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method according to the first aspect.

[0042] In a seventh aspect, a terminal is provided, which comprises a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the method according to the first aspect.

[0043] In an eighth aspect, a network-side device is provided, which comprises a processor and a memory, the memory storing programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method according to the second aspect.

[0044] In a ninth aspect, a network-side device is provided, which comprises a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the method according to the second aspect.

[0045] In a tenth aspect, a readable storage medium is provided, which stores programs or instructions, and the programs or instructions, when executed by a processor, implement the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

[0046] In an eleventh aspect, a wireless communication system is provided, comprising: a terminal configured to perform the steps of the method according to the first aspect, and a network-side device configured to perform the steps of the method according to the second aspect.

[0047] In a twelfth aspect, a chip is provided, comprising a processor and a communication interface, the communication interface being coupled to the processor, the processor being configured to run a program or an instruction to implement the method according to the first aspect or the method according to the second aspect.

[0048] In a thirteenth aspect, a computer program / program product is provided, stored in a storage medium, and executed by at least one processor to implement the steps of the method according to the first aspect or the method according to the second aspect.

[0049] In the embodiments of the present application, the terminal can select, according to the first information, a prediction accuracy requirement associated with at least one of the prediction window, the MRR, the frequency point, the cell, the AI model and the AI function to perform model monitoring on the first AI model, or determine, according to the configuration information corresponding to the first prediction function and / or the first parameter, a target prediction accuracy requirement used for performing model monitoring on the first AI model, which is beneficial to the terminal to select a suitable prediction accuracy requirement to perform model monitoring on the first AI model, guarantee the normal operation of the AI model, and take into account the prediction performance of the AI model. BRIEF DESCRIPTION OF DRAWINGS

[0050] FIG. 1 is a schematic diagram of a communication system architecture according to an embodiment of the present application.

[0051] FIG. 2 is a schematic diagram of a neuron structure.

[0052] FIG. 3 is a schematic diagram of a typical neural network.

[0053] FIG. 4 is an AI / ML function framework diagram for air interface according to an embodiment of the present application.

[0054] FIG. 5 is a schematic diagram of a model monitoring method according to an embodiment of the present application.

[0055] FIG. 6 is a schematic diagram of another model monitoring method according to an embodiment of the present application.

[0056] FIG. 7 is a schematic diagram of yet another model monitoring method according to an embodiment of the present application.

[0057] FIG. 8 is a schematic diagram of yet another model monitoring method according to an embodiment of the present application.

[0058] FIG. 9 is a schematic diagram of yet another model monitoring method according to an embodiment of the present application.

[0059] FIG. 10 is a schematic block diagram of a wireless communication device according to an embodiment of the present application.

[0060] FIG. 11 is a schematic block diagram of a wireless communication device according to an embodiment of the present application.

[0061] FIG. 12 is a schematic block diagram of a communication device according to an embodiment of the present application.

[0062] FIG. 13 is a schematic diagram of a hardware structure of a terminal according to an embodiment of the present application.

[0063] FIG. 14 is a schematic block diagram of a network-side device according to an embodiment of the present application.

[0064] FIG. 15 is a schematic block diagram of another network-side device according to an embodiment of the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0066] The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second" are generally a category, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, the protection scope of "A or B" at least covers three schemes, namely, scheme one: including A and not including B; scheme two: including B and not including A; scheme three: including A and including B. In addition, the terms "A and / or B", "at least one of A and B", "at least one of A or B" also at least cover the above three schemes, respectively. The character " / " generally represents that the objects before and after are in an "or" relationship.

[0067] The term "indication" in this application can be a direct indication (or explicit indication) or an indirect indication (or implicit indication). The direct indication can be understood as that the sender explicitly informs the receiver of specific information, operations to be performed or requested results, etc. in the sent indication. The indirect indication can be understood as that the receiver determines the corresponding information according to the indication sent by the sender, or judges and determines the operations to be performed or the requested results according to the judgment result.

[0068] It is worth noting that the technology described in the embodiments of the present application is not limited to the Long Term Evolution (LTE) / LTE-Advanced (LTE-A) system, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA) or other systems. The terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the described technology can be used in the above-mentioned systems and radio technologies, and also in other systems and radio technologies. The following description describes a New Radio (NR) system for example purposes, and NR terminology is used in most of the following description, but these technologies can also be applied to systems other than the NR system, such as a 6th Generation (6G) communication system. th

[0069] ​FIG. 1 shows a block diagram of a wireless communication system to which embodiments of the present application can be applied. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal 11 can be a terminal-side device such as a mobile phone, a Tablet Personal Computer, a Laptop Computer, a notebook computer, a Personal Digital Assistant (PDA), a palmtop computer, a netbook, an Ultra-mobile Personal Computer (UMPC), a Mobile Internet Device (MID), an Augmented Reality (AR) device, a Virtual Reality (VR) device, a robot, a wearable device, a flight vehicle, a Vehicle User Equipment (VUE), a shipboard device, a Pedestrian User Equipment (PUE), a smart home (a home device with a wireless communication function such as a refrigerator, a television, a washing machine, or furniture), a game console, a Personal Computer (PC), a kiosk, or a self-service machine. The wearable device includes a smart watch, a smart bracelet, a smart earphone, smart glasses, smart jewelry (a smart bracelet, a smart necklace, a smart ring, a smart necklace, a smart anklet, a smart necklace, etc.), a smart wristband, smart clothes, etc. The vehicle-mounted device can also be referred to as a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip, or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal 11 is not limited in the embodiments of the present application.

[0070] In the embodiments of the present application, the terminal can also be referred to as a User Equipment (UE), a terminal device, an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile, a remote station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent, or a user equipment, etc.

[0071] The network-side device 12 can include an access network device or a core network device, wherein the access network device can also be referred to as a radio access network (RAN) device, a radio access network function or a radio access network unit. The access network device can include a base station, a wireless local area network (WLAN) access point (AP) or a wireless fidelity (WiFi) node, etc. Among them, the base station can be referred to as a node B (NB), an evolved node B (eNB), a next generation node B (gNB), a new radio node B (NR node B), an access point, a relay base station (RBS), a serving base station (SBS), a base transceiver station (BTS), a radio base station, a radio transceiver, a basic service set (BSS), an extended service set (ESS), a home node B (HNB), a home evolved node B, a transmit / receive point (TRP), or some other suitable term in the art, as long as the same technical effect is achieved. The base station is not limited to a specific technical term, and it should be noted that in the embodiments of the present application, only the base station in the NR system is taken as an example for introduction, and the specific type of the base station is not limited.

[0072] The core network device can also be referred to as a core network node, a core network function, or a core network network element, etc., which includes but is not limited to at least one of the following: a mobility management entity (MME), an access and mobility management function (AMF), a session management function (SMF), a user plane function (UPF), a policy control function (PCF), a policy and charging rules function (PCRF), an edge application server discovery function (EASDF), a unified data management (UDM), a unified data repository (UDR), a home subscriber server (HSS), a centralized network configuration (CNC), a network repository function (NRF), a network exposure function (NEF), a local NEF (L-NEF), a binding support function (BSF), an application function (AF), a location management function (LMF), a gateway mobile location center (GMLC), a network data analytics function (NWDAF), etc. It should be noted that only the core network device in the NR system is taken as an example for introduction in the embodiments of the present application, and the specific type of the core network device is not limited. If the name of the core network device mentioned in the embodiments of the present application changes in the subsequent protocol version (for example, 6G), it is also within the protection scope of the present application.

[0073] Optionally, the core network device can be implemented by one or more function modules in one device, or can be implemented by multiple devices together, and the embodiments of the present application do not make specific limitations thereto. It can be understood that the above function modules can be network elements in a hardware device, can be software function modules running on a special hardware, or can be virtualized function modules instantiated on a platform (for example, a cloud platform).

[0074] In order to facilitate understanding of the embodiments of the present application, the related artificial intelligence (AI) technology of the present application is described.

[0075] AI technology has been widely applied in various fields. Integrating AI technology into wireless communication networks to improve technical indicators such as throughput, latency, and user capacity is an important task for future wireless communication networks. AI models have various implementation methods, such as neural networks, decision trees, support vector machines, and Bayesian classifiers. The following takes the neural network as an example for description, but does not limit the specific type of AI module.

[0076] A neural network is an operation model composed of multiple neuron nodes connected to each other, wherein the connection between nodes represents the weighted value from the input signal to the output signal, which is called weight; each node performs weighted summation on different input signals and outputs through a specific activation function (f), and FIG. 2 is a schematic diagram of a neuron structure, wherein a1, a2, …, an represent inputs, w1, w2, …, wn represent weights (multiplicative coefficients), b represents bias (additive coefficient), σ represents activation function, and y represents output, wherein z = a1*w1 + a2*w2 + … + an*wn + b. Common activation functions include Sigmoid, tanh, ReLU (Rectified Linear Unit, linear rectification function, modified linear unit), etc.

[0077] A typical neural network is shown in FIG. 3, which includes an input layer, a hidden layer, and an output layer. Through different connection methods of multiple neurons, weights, and activation functions, different outputs can be generated, and then the mapping relationship from input to output can be fitted. Each upper-level node is connected to all lower-level nodes, and the neural network is a fully connected neural network, which can also be called a deep neural network (DNN).

[0078] Deep learning adopts deep neural networks with multiple hidden layers, greatly improves the ability of network to learn features, and can fit complex nonlinear mapping from input to output, so it is widely used in speech and image processing. In addition to deep neural networks, deep learning also includes convolutional neural network (CNN), recurrent neural network (RNN) and other commonly used basic structures for different tasks.

[0079] The parameters of the neural network are optimized by a gradient optimization algorithm. Gradient optimization algorithm is a class of algorithm for minimizing or maximizing an objective function (or called loss function), and the objective function is often a mathematical combination of model parameters and data. For example, given data X and its corresponding label Y, a neural network model f(.) can be constructed. With the model, the output f(x) can be predicted according to the input x, and the difference between the predicted value and the true value (f(x)-Y) can be calculated, which is the loss function. Our goal is to find the appropriate W, b to minimize the value of the above loss function. The smaller the loss value is, the closer the model is to the true situation.

[0080] The common optimization algorithm at present, for example, includes error back propagation (BP) algorithm. The basic idea of BP algorithm is that the learning process consists of two processes of forward propagation of signals and backward propagation of errors. When forward propagation, the input sample is transmitted from the input layer, processed layer by layer through each hidden layer, and then transmitted to the output layer. If the actual output of the output layer does not match the expected output, the backward propagation of error is entered. Error back propagation is to transmit the output error to the input layer through the hidden layer in a certain form, and allocate the error to all units of each layer, so as to obtain the error signal of each layer unit, which can be used as the basis for correcting the weight of each unit. The process of adjusting the weight of each layer through forward propagation of signals and backward propagation of errors is repeated. The process of continuously adjusting the weight is the learning and training process of the network. This process continues until the error of the network output is reduced to an acceptable level, or until the preset learning times are reached.

[0081] Common optimization algorithms can include, but are not limited to: Gradient Descent, Stochastic Gradient Descent (SGD), mini-batch gradient descent, Momentum, Nesterov (Stochastic Gradient Descent with Momentum), Adagrad (ADAptive GRADient descent), Adadelta, RMSprop (root mean square prop), Adam (Adaptive Moment Estimation), and the like.

[0082] These optimization algorithms, when error backpropagation, are based on the error / loss obtained from the loss function, the derivative / partial derivative of the current neuron, the influence of the learning rate, the previous gradient / derivative / partial derivative, etc., to obtain the gradient, and pass the gradient to the previous layer.

[0083] For the convenience of understanding the embodiments of the present application, the related radio resource management (RRM) of the present application is described.

[0084] In the related art, the measurement configuration is mainly composed of a measurement object (MO), a report configuration (ReportConfig) and a measurement identifier (measId).

[0085] Among them, the measurement object can refer to the frequency point to be measured. The report configuration can include: reporting criteria (such as periodic or event triggered); reference signal type, such as synchronization signal block (SSB) or channel state information reference signal (CSI-RS); whether to report beam measurement results, the maximum number of reportable beams, etc. The measurement identifier associates a measurement object and a report configuration, a measurement object can be associated with multiple report configurations, and a report configuration can be associated with multiple measurement objects.

[0086] In the communication system, the measurement object, the measurement identifier and the device configuration can be associated together in the following way:

[0087] Among them, measObjectId represents the measurement object identifier, and reportConfigId represents the report configuration identifier.

[0088] In the related art, event triggered reporting can be configured in the reporting configuration, for example, the events defined in the communication system can include the events in Table 1 as follows.

[0089] Table 1

[0090] For the A3 event, the meanings of the parameters involved in the entering condition and the leaving condition are as follows:

[0091] Mn: the measurement result of the neighbor cell, without considering any offset;

[0092] Ofn: the offset specific to the measurement object of the neighbor cell;

[0093] Ocn: the offset specific to the cell level of the neighbor cell;

[0094] Mp: the measurement result of the special cell (SpCell), without considering any offset;

[0095] Ofp: the offset specific to the measurement object of the SpCell;

[0096] Ocp: the offset specific to the cell level of the SpCell;

[0097] Hys: the hysteresis parameter of the event;

[0098] Off: the offset parameter of the event.

[0099] If the reporting type is event triggered reporting, in order to avoid frequent reporting or ping-pong switching, the network side device configures a trigger time (timeToTrigger) parameter for each event, and if the Layer 3 (L3) filtered signal quality of one or more candidate cells within the timeToTrigger time meets the entering condition of the event, the measurement reporting is triggered.

[0100] In some scenarios, an AI / ML function framework for the air interface is introduced, as shown in FIG. 4, which can include the following modules:

[0101] Data Collection module: used to provide input data for the Model Training module, the Management module and the Interence module.

[0102] Model Training module: responsible for performing AI / ML model training, verification and testing. It can also be responsible for data preparation, such as data preprocessing, for example, converting data into a specific format, etc.

[0103] Management module: responsible for model selection, activation, deactivation, switching or fallback (e.g., fallback to non-AI based methods), etc.

[0104] Interence module: assist to provide the output after applying AI / ML model or AI / ML function.

[0105] Model Storage module: assist to save the trained or updated model.

[0106] Model Transfer / Delivery module: assist to deliver AI / ML model to inference function node.

[0107] In some scenarios, consider introducing AI-based mobility enhancement, for example, can include the following use cases:

[0108] 1. RRM measurement prediction.

[0109] For example, the future RRM measurement prediction result can be predicted in the time domain.

[0110] For another example, the current or future RRM measurement prediction result of other frequency points can be predicted in the frequency domain.

[0111] For another example, the RRM measurement prediction result of other beams in the cell can be predicted in the spatial domain.

[0112] 2. Measurement event prediction, predict whether the measurement event will be met in the future, for example, can include the following two methods:

[0113] Method one: direct measurement event prediction, such as the output of the AI model is the flag or probability of whether the measurement event is met;

[0114] Method two: indirect measurement event prediction: such as the output of the AI model is the RRM prediction result, and whether the measurement event will be met is judged according to the RRM prediction result.

[0115] 3. Radio Link Failure (RLF) prediction, predict whether RLF will occur in the future, for example, can include the following two methods:

[0116] Method one: direct RLF prediction, such as the output of the AI model is the flag or probability of whether RLF will occur;

[0117] Method two: indirect RLF prediction: if the output of the AI model is the signal to interference plus noise ratio (SINR), it is determined whether RLF will occur according to the SINR.

[0118] In the related art, the network side device or the terminal can manage the model based on the model monitoring result, for example, when the model performance cannot meet the accuracy requirement, the model is deactivated, switched or rolled back. However, if the accuracy requirement of the model performance is too high, the model may not work normally, but if the accuracy requirement of the model performance is low, the prediction performance of the model may be affected, therefore, how to select a suitable accuracy requirement to monitor the model to balance the normal work of the model and the prediction performance of the model is an urgent problem to be solved.

[0119] The wireless communication method provided by the embodiments of the present application will be described in detail in combination with the accompanying drawings, some embodiments and application scenarios.

[0120] FIG. 5 is a schematic diagram of a wireless communication method 200 provided by an embodiment of the present application. As shown in FIG. 5, the method 200 includes at least part of the following content:

[0121] S210, the terminal obtains first information;

[0122] S220, performing model monitoring on a first artificial intelligence (AI) model according to the first information, wherein the first AI model is used to perform a first prediction function.

[0123] It should be noted that in the embodiments of the present application, the AI model can also be referred to as an AI unit, a machine learning (ML) model, an ML unit, an AI structure, an AI function, an AI feature, a machine learning model, a neural network, a neural network function, a neural network function, etc., or the AI model can also refer to a processing unit capable of implementing a specific algorithm, formula, processing flow, capability, etc. related to AI, or the AI model can be a processing method, algorithm, function, module or unit for a specific data set, or the AI model can be a processing method, algorithm, function, module or unit running on a graphics processing unit (GPU), neural processing unit (NPU), tensor processing unit (TPU), application specific integrated circuit (ASIC) and other AI / ML related hardware, which is not limited in the present application.

[0124] It should be noted that in the embodiments of the present application, the identification information of the AI model may, for example, be an AI model identification (model ID), an AI structure identification, an AI algorithm identification, or an identification of a specific data set associated with the AI model, or an identification of a specific scene, environment, channel feature, or device related to AI / ML, or an identification of a function, feature, capability, or module related to AI / ML, and the present application does not make a specific limitation thereon.

[0125] In the embodiments of the present application, the first AI model is deployed on the terminal, the terminal can perform a first prediction function based on the first AI model, and the terminal can perform model monitoring on the first AI model based on the method 200.

[0126] It should be understood that in the embodiments of the present application, a second AI model can be deployed on the network side device, the network side device can perform a second prediction function based on the second AI model, and the network side device can also perform model monitoring on the second AI model in a similar manner to the method 200, and the present application does not make a limitation thereon, and the following will take the terminal performing model monitoring on the first AI model as an example to illustrate the model monitoring method provided in the embodiments of the present application.

[0127] In some embodiments of the present application, the first prediction function includes but is not limited to at least one of the following:

[0128] RRM measurement prediction;

[0129] Measurement event prediction;

[0130] RLF prediction;

[0131] HOF (Handover Failure, HOF) prediction;

[0132] Target cell prediction.

[0133] It should be understood that the prediction functions supported by the above-mentioned first AI model are only examples, and the first AI model can also support other prediction functions, and the present application does not make a limitation thereon.

[0134] In some embodiments, the RRM measurement prediction can include but is not limited to at least one of the following:

[0135] Predicting future RRM measurement prediction results in the time domain;

[0136] Predicting current or future RRM measurement prediction results of other frequency points / cells in the frequency domain;

[0137] Predicting RRM measurement prediction results of other beams in the service cell or the neighbor cell in the spatial domain.

[0138] Optionally, the RRM measurement prediction result in the embodiments of the present application can be a predicted signal quality, for example, a Reference Signal Receiving Power (RSRP), a Reference Signal Receiving Quality (RSRQ), a Signal to Interference plus Noise Ratio (SINR), or the like.

[0139] In some embodiments, the input information of the first AI model can be a RRM measurement result or a RRM measurement prediction result of a cell in a second frequency point, and the other frequency point can be a frequency point other than the second frequency point, for example, a first frequency point, wherein the RRM measurement result of the cell in the second frequency point can be obtained by measurement, and the RRM measurement prediction result of the cell in the second frequency point can be obtained by prediction.

[0140] In some embodiments, the input information of the first AI model can be a RRM measurement result or a RRM measurement prediction result of a second cell, and the other cell can be a cell other than the second cell, for example, a first cell, wherein the RRM measurement result of the second cell can be obtained by measurement, and the RRM measurement prediction result of the second cell can be obtained by prediction.

[0141] In some embodiments, the input information of the first AI model can be a RRM measurement result or a RRM measurement prediction result of a second beam, and the other beam can be a beam other than the second beam in a serving cell or a neighbor cell, for example, a first beam, wherein the RRM measurement result of the second beam can be obtained by measurement, and the RRM measurement prediction result of the second beam can be obtained by prediction.

[0142] In some embodiments, the measurement event prediction can include at least one of the following:

[0143] Direct measurement event prediction, for example, the output of the first AI model can be a flag indicating whether a measurement event is met, or a probability of meeting the measurement event;

[0144] Indirect measurement event prediction, for example, the output of the AI model is a RRM measurement prediction result, which is used to determine whether a measurement event is met.

[0145] In some embodiments, the RLF prediction can include at least one of the following:

[0146] Direct RLF prediction, for example, the output of the first AI model is a flag indicating whether an RLF will occur, or a probability of the RLF occurring;

[0147] Indirect RLF prediction, for example, the output of the first AI model can be SINR, which is used to determine whether RLF will occur.

[0148] In some embodiments, HOF prediction can refer to predicting whether HOF will occur.

[0149] In some embodiments, target cell prediction can refer to predicting the target cell for which the terminal performs cell switching.

[0150] In some embodiments, for RRM measurement prediction, measurement event prediction, RLF prediction, a prediction window corresponding to a future time or time period can be defined, for example, the prediction window can be 400 ms in the future, or 1 s or 2 s in the future, etc.

[0151] In some embodiments, for AI model-based prediction, a measurement reduction rate (MRR) can be defined to represent the proportion of measurement reduction through prediction.

[0152] For example, for RRM measurement in the time domain, the measurement time includes time 1, 2, 3, 4, 5, and 6. Among them, time 1, 3, and 5 use actual measurement values, and time 2, 4, and 6 use predicted values. At this time, the measurement reduction rate in the time domain (MRRT) is 1 / 2.

[0153] For example, for RRM measurement in the spatial domain: the set of all transmission beams (Tx beams) is {Tx beam 1, Tx beam 2, Tx beam 3, Tx beam 4}, where Tx beam 1 and Tx beam 2 are actual measurement beams, and Tx beam 3 and Tx beam 4 are predicted beams. At this time, the measurement reduction rate in the spatial domain (MRRS) is 1 / 2.

[0154] In some embodiments of the present application, the terminal obtains first information, including:

[0155] The terminal obtains the first information through pre-defined information; or

[0156] The terminal obtains the first information from the network side device.

[0157] In other words, the first information can be predefined or configured by the network side device.

[0158] For example, the network-side device can send the first information to the terminal through at least one of the following signaling:

[0159] Radio Resource Control (RRC) signaling;

[0160] Medium Access Control Control Element (MAC CE);

[0161] Downlink Control Information (DCI).

[0162] In some embodiments of the present application, the first information comprises at least one of the following:

[0163] one or more prediction accuracy requirements;

[0164] configuration information corresponding to the first prediction function;

[0165] a first parameter for determining a prediction accuracy requirement associated with the first prediction function.

[0166] In some embodiments, the one or more prediction accuracy requirements are associated with at least one of the following:

[0167] a prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model.

[0168] In embodiments of the present application, prediction accuracy requirements of prediction window granularity, MRR granularity, frequency point granularity, cell granularity, beam granularity, AI function granularity, or AI model granularity can be designed, so that the terminal can select a suitable prediction accuracy requirement for model monitoring of the first AI model based on the above parameters associated with the first AI model, which is conducive to ensuring the normal operation of the first AI model while taking into account the prediction performance of the first AI model.

[0169] In some embodiments, since the configuration information corresponding to the first prediction function can be used to determine whether an event is triggered or the probability of event occurrence, when determining the target prediction accuracy requirement used for model monitoring of the first AI model, the configuration information corresponding to the first prediction function performed by the first AI model is considered, which is conducive to the terminal selecting a suitable prediction accuracy requirement for monitoring the first AI model, ensuring the normal operation of the AI model while taking into account the prediction performance of the first AI model.

[0170] In some embodiments, when the terminal determines the target prediction accuracy requirement based on the configuration information corresponding to the first prediction function, the target prediction accuracy requirement can be determined based on the first parameter and the configuration information corresponding to the first prediction function. Optionally, the first parameter can be a scaling factor of the configuration information, or an adjustment amount, etc. Some determination methods of the target prediction accuracy requirement will be further described in the following embodiments.

[0171] Therefore, in the embodiment of the present application, the terminal can select the prediction accuracy requirement associated with at least one of the prediction window, the MRR, the frequency point, the cell, the AI model and the AI function to perform model monitoring on the first AI model according to the first information, or determine the target prediction accuracy requirement used for model monitoring on the first AI model according to the configuration information corresponding to the first prediction function and the first parameter, which is beneficial to the terminal to select a suitable prediction accuracy requirement to perform model monitoring on the first AI model, ensure the normal operation of the AI model, and take into account the prediction performance of the AI model.

[0172] In some embodiments, the one or more prediction accuracy requirements include but are not limited to at least one of the following:

[0173] Prediction accuracy requirement for RRM measurement prediction;

[0174] Prediction accuracy requirement for measurement event prediction;

[0175] Prediction accuracy requirement for RLF prediction;

[0176] Prediction accuracy requirement for HOF prediction;

[0177] Prediction accuracy requirement for target cell prediction.

[0178] Therefore, in the embodiment of the present application, the terminal can obtain the prediction accuracy requirement corresponding to different prediction functions, so as to select a suitable prediction accuracy requirement to perform prediction according to the prediction function to be performed, which can ensure the normal operation of the AI model, and also take into account the prediction performance of the AI model performing the prediction function.

[0179] For example, the measurement event prediction is used to predict whether a measurement event will be triggered in the short term, and the HOF prediction is used to predict whether a ping-pong effect of handover will occur in the future. The events predicted by the two are different, and accordingly, the prediction accuracy requirements should also be designed differently, which is beneficial to ensure the normal operation of the AI model, and also takes into account the prediction performance of the model.

[0180] In some embodiments, the one or more prediction accuracy requirements can include at least one of the following:

[0181] One or more prediction accuracy requirements associated with the prediction window;

[0182] One or more prediction accuracy requirements associated with the MRR;

[0183] One or more prediction accuracy requirements associated with the frequency point;

[0184] One or more prediction accuracy requirements associated with the cell;

[0185] one or more prediction accuracy requirements associated with a beam;

[0186] one or more prediction accuracy requirements associated with an AI function;

[0187] one or more prediction accuracy requirements associated with an AI model.

[0188] In some embodiments, one prediction accuracy requirement can be associated with one prediction window, or one prediction accuracy requirement can be associated with multiple prediction windows.

[0189] In some embodiments, one prediction accuracy requirement can be associated with one MRR, or one prediction accuracy requirement can be associated with multiple MRRs.

[0190] In some embodiments, one prediction accuracy requirement can be associated with one frequency point or one cell or one beam, or one prediction accuracy requirement can be associated with multiple frequency points or multiple cells or multiple beams.

[0191] In some embodiments, one prediction accuracy requirement can be associated with one AI function or one AI model, or one prediction accuracy requirement can be associated with multiple AI functions or multiple AI models.

[0192] In some embodiments, when the first information includes at least one prediction accuracy requirement associated with a frequency point, a cell or a beam, the at least one prediction accuracy requirement associated with the frequency point, the cell or the beam is configured by the network side device in the first configuration information, and the first configuration information is used to configure a measurement object.

[0193] In some embodiments, the prediction accuracy requirement associated with a frequency point, a cell or a beam can be configured in a measurement object, so that the network side device does not need to configure the prediction accuracy requirement associated with the frequency point, the cell or the beam for the terminal through additional signaling, which is conducive to reducing signaling interaction overhead.

[0194] In some embodiments, the first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and wherein the first prediction accuracy requirement is lower than the second prediction accuracy requirement when the length of the first prediction window is greater than the length of the second prediction window, or the time interval between the first prediction window and the current time is greater than the interval between the second prediction window and the current time (for example, the first prediction window is farther away from the current time, and accordingly, the prediction difficulty is greater).

[0195] It can be understood that the prediction difficulty of different prediction windows or MRRs is different, for example, the longer the prediction window is, the greater the MRR is, and the greater the prediction difficulty is, and accordingly, the prediction accuracy requirement should also be set lower to ensure the normal operation of the AI model.

[0196] In addition, different prediction windows or MRRs can be used for different purposes, for example, a short prediction window can be used to determine whether a measurement event will be triggered in the short term, and a long prediction window can be used to determine whether a ping-pong effect of handover will occur in the future. The purposes of the two are different, and the precision requirements should also be designed differently. In the embodiments of the present application, corresponding prediction precision requirements are configured for different prediction windows or MRRs, which is beneficial to ensure the normal operation of the AI model, and at the same time, the prediction performance of the model can also be taken into account.

[0197] In some embodiments of the present application, the model monitoring of the first AI model according to the first information comprises:

[0198] determining a target prediction precision requirement from the one or more prediction precision requirements according to second information;

[0199] monitoring the first AI model according to the target prediction precision requirement;

[0200] The second information comprises at least one of the following:

[0201] a prediction window associated with the first AI model;

[0202] an MRR associated with the first AI model;

[0203] a frequency point associated with the first AI model;

[0204] a cell associated with the first AI model;

[0205] a beam associated with the first AI model;

[0206] an AI function supported by the first AI model;

[0207] the first AI model.

[0208] In some embodiments, the target prediction precision requirement is associated with at least one of the following:

[0209] a first prediction window, the first prediction window being a prediction window associated with the first AI model;

[0210] a first MRR, the first MRR being an MRR associated with the first AI model;

[0211] a first frequency point, the first frequency point being a frequency point associated with the first AI model;

[0212] a first cell, the first frequency point being a cell associated with the first AI model;

[0213] The first AI model is associated with a first beam.

[0214] The first AI model is associated with a first AI function.

[0215] The first AI model.

[0216] In some embodiments, the first AI model being associated with a first prediction window can be understood as that the output of the first AI model is a prediction result within the first prediction window. For example, the prediction result within the first prediction window can be a RRM measurement prediction result within the first prediction window, such as a predicted signal quality. For another example, the prediction result within the first prediction window can be a flag indicating whether a measurement event / RLF / HOF will occur within the first prediction window, or a probability of the measurement event / RLF / HOF occurring within the first prediction window.

[0217] In some embodiments, the first AI model being associated with a first MRR can be understood as that a measurement reduction ratio achieved by performing the first prediction function through the first AI model is the first MRR.

[0218] In some embodiments, the first AI model being associated with a first frequency point can be understood as that the output of the first AI model is a prediction result of a cell within the first frequency point, such as a RRM measurement prediction result of the cell within the first frequency point (e.g. a signal quality of the cell within the first frequency point), or the first AI model can be used to predict the RRM measurement prediction result of the cell within the first frequency point. Optionally, the first frequency point can be a frequency point to be measured or predicted by the terminal, by measuring or predicting the corresponding frequency point, the terminal can obtain a signal quality of a cell within the corresponding frequency point, which can be used for mobility management or radio link monitoring.

[0219] In some embodiments, the first AI model being associated with a first cell can be understood as that the output of the first AI model is a prediction result of the first cell, such as a RRM measurement prediction result of the first cell, or the first AI model can be used to predict the RRM measurement prediction result of the first cell. Optionally, the first cell can be a cell to be measured or predicted by the terminal, by measuring or predicting the corresponding cell, the terminal can obtain a signal quality of the corresponding cell, which can be used for mobility management or radio link monitoring.

[0220] In some embodiments, the first AI model associating the first beam can be understood as that the output of the first AI model is a prediction result of the first beam, for example, a RRM measurement prediction result of the first beam, or the first AI model can be used to predict a RRM measurement prediction result of the first beam. Optionally, the first beam can be a beam to be measured or predicted by the terminal, and by measuring or predicting the corresponding beam, the terminal can obtain the signal quality of the corresponding beam, which can be used for mobility management or radio link monitoring.

[0221] In some embodiments of the present application, the model monitoring of the first AI model according to the first information comprises:

[0222] determining a target prediction accuracy requirement according to the configuration information corresponding to the first prediction function and the first parameter;

[0223] monitoring the first AI model according to the target prediction accuracy requirement.

[0224] Optionally, in this case, the first prediction function can be an indirect event prediction, for example, an indirect measurement event prediction, or an indirect RLF prediction.

[0225] For the indirect event prediction, the output result of the AI model is an intermediate prediction result, which is used to determine the final prediction result. If the prediction accuracy error of the intermediate prediction result is higher than the threshold corresponding to the event configuration, the accuracy of the event prediction cannot be guaranteed. For example, for A3 event prediction, if the event configuration is that the neighbor cell is 3 dB higher than the serving cell to meet the event entering condition, at this time, if the accuracy error of the RRM measurement prediction result output by the AI model is higher than 3 dB, the accuracy of the measurement event prediction cannot be guaranteed.

[0226] Therefore, in the embodiments of the present application, for the indirect event prediction, the terminal can determine the target prediction accuracy requirement according to the configuration information corresponding to the event and the first parameter, which is beneficial to guarantee that the AI model used for event prediction can normally operate, and is beneficial to guarantee the accuracy of the event prediction.

[0227] Optionally, the configuration information corresponding to the first prediction function can include configuration information corresponding to the measurement event prediction.

[0228] Optionally, the measurement event can be an A3 event, and the configuration information corresponding to the A3 event can include, but is not limited to, at least one of the following:

[0229] Mn: neighbor cell measurement result, without considering any offset;

[0230] Ofn: neighbor cell measurement object specific offset;

[0231] Ocn: offset specific to neighbor cell level;

[0232] Mp: SpCell measurement result, without considering any offset;

[0233] Ofp: offset specific to SpCell measurement object;

[0234] Ocp: offset specific to SpCell cell level;

[0235] Hys: hysteresis parameter of the event;

[0236] Off: offset parameter of the event.

[0237] In some embodiments, the target prediction accuracy requirement is determined according to the configuration information corresponding to the first prediction function and the first parameter, including:

[0238] The product of the second parameter and the first parameter is determined as the target prediction accuracy requirement, wherein the second parameter is one parameter or a combination of multiple parameters in the configuration information corresponding to the first prediction function (or the second parameter is determined by the multiple parameters).

[0239] Optionally, the first parameter can be a value between 0 and 1.

[0240] By taking the product of the second parameter and the first parameter as the target prediction accuracy requirement, the prediction accuracy requirement of the intermediate prediction result of the first AI model is improved, and thus the accuracy of event prediction based on the intermediate prediction result of the first AI model is improved.

[0241] For example, the second parameter is the offset parameter Off of the A3 event, which is 3 dB, and the value of the first parameter can be 0.5, and the target prediction accuracy requirement can be 3*0.5=1.5 dB, so that the prediction accuracy requirement of the intermediate prediction result of the first AI model can be improved from 3 dB to 1.5 dB, which is conducive to improving the accuracy of event prediction according to the offset parameter 3 dB for A3 event prediction.

[0242] For another example, the second parameter is a combination of multiple parameters (for example, Mp, Ofp, Ocp, Off, Hys, Ofn, Ocn) in the configuration information corresponding to the A3 event, or is determined by multiple parameters, for example, the second parameter = Mp+Ofp+Ocp+Off+Hys-Ofn-Ocn, and the terminal can determine the product of the second parameter and the first parameter as the target prediction accuracy requirement.

[0243] In some embodiments of the present application, the method 200 further includes:

[0244] In a case where the model monitoring result of the first AI model is less than or equal to the target prediction accuracy requirement, the terminal performs a first operation, wherein the first operation includes at least one of the following:

[0245] The terminal reports third information to the network side device;

[0246] The terminal deactivates the first AI model;

[0247] The terminal switches the first AI model to a second AI model;

[0248] The terminal reverts to a non-AI model based manner to implement the first prediction function.

[0249] In a case where the model monitoring result of the first AI model does not meet the target prediction accuracy requirement corresponding to the first AI model, the terminal can report third information to the network side device, or perform deactivation, switching or reversion of the model.

[0250] In some embodiments, the third information includes but is not limited to at least one of the following:

[0251] The model identifier of the first AI model;

[0252] The function identifier of the first AI model;

[0253] A first indication indicating that the first AI model does not meet the prediction accuracy requirement;

[0254] The model monitoring result of the first AI model, such as the prediction accuracy achieved by the first AI model in performing the first prediction function.

[0255] Therefore, by reporting the third information to the network side device, the network side device can know that the first AI model used by the terminal side cannot meet the prediction accuracy requirement, so that the network side device can configure a new AI model for the terminal to meet the target prediction accuracy requirement, thereby balancing the normal operation of the AI model and the prediction performance of the AI model.

[0256] In some embodiments, the model monitoring result of the first AI model can refer to the prediction accuracy achieved by the first AI model in performing the first prediction function.

[0257] In some embodiments, the first prediction function is RRM measurement prediction, and the prediction accuracy of the first AI model performing the RRM measurement prediction can be determined according to a RRM measurement prediction result and an actual RRM measurement result, for example, a prediction accuracy requirement for the RRM measurement prediction can be a difference between the actual RRM measurement result and the RRM measurement prediction result, for example, a difference between a RRM measurement result of the second beam and a RRM measurement prediction result of the second beam, for example, a difference between a RRM measurement result of the cell in the second frequency and a RRM measurement prediction result of the cell in the second frequency.

[0258] In some embodiments, the first prediction function is measurement event prediction, and the prediction accuracy of the first AI model performing the measurement event prediction can include, but is not limited to, at least one of the following:

[0259] Accuracy of the measurement event prediction, precision of the measurement event prediction, recall of the measurement event prediction, sensitivity of the measurement event prediction, F1 score of the measurement event prediction, specificity of the measurement event prediction.

[0260] In some embodiments, the first prediction function is RLF prediction, and the prediction accuracy of the first AI model performing the RLF prediction can include, but is not limited to, at least one of the following:

[0261] Accuracy of the RLF prediction, precision of the RLF prediction, recall of the RLF prediction, sensitivity of the RLF prediction, F1 score of the RLF prediction, specificity of the RLF prediction.

[0262] In some embodiments, the first prediction function is HOF prediction, and the prediction accuracy of the first AI model performing the HOF prediction can include, but is not limited to, at least one of the following:

[0263] Accuracy of the HOF prediction, precision of the HOF prediction, recall of the HOF prediction, sensitivity of the HOF prediction, F1 score of the HOF prediction, specificity of the HOF prediction.

[0264] In some embodiments, the first prediction function is target cell prediction, and the prediction accuracy of the first AI model performing the target cell prediction can include, but is not limited to, at least one of the following:

[0265] Accuracy of the target cell prediction, precision of the target cell prediction, recall of the target cell prediction, sensitivity of the target cell prediction, F1 score of the target cell prediction, specificity of the target cell prediction.

[0266] In some embodiments, the second AI model supports the first prediction function, and the prediction accuracy of the second AI model meets the target prediction accuracy requirement.

[0267] The following describes an implementation of the model monitoring method provided by the embodiments of the present application in combination with FIG. 6 to FIG. 9.

[0268] Embodiment 1

[0269] In this embodiment 1, the first information can include a prediction accuracy requirement associated with a prediction window or MRR, and the terminal can select a target prediction accuracy requirement based on the prediction window or MRR associated with the first AI model, and perform model monitoring on the first AI model based on the target accuracy requirement.

[0270] As shown in FIG. 6, the model monitoring method can include the following steps:

[0271] S310, the terminal obtains first information, and the first information includes a prediction accuracy requirement associated with a prediction window or MRR.

[0272] For different prediction windows or MRRs, the prediction difficulty of the AI model is different. For example, for a longer prediction window and a larger MRR, the prediction difficulty of the AI model is greater, and the corresponding prediction accuracy requirement (or prediction performance requirement) should also be lower, otherwise the normal operation of the AI model cannot be guaranteed. In addition, the purposes of different prediction windows or MRRs can be different. For example, a short prediction window can be used to determine whether a measurement event will trigger in the short term, and a long prediction window can be used to determine whether a ping-pong handover will occur in the future. The purposes of the two are different, and accordingly, the prediction accuracy requirements should also be different to ensure the normal operation of the AI model.

[0273] In embodiment 1, the corresponding prediction accuracy requirements can be configured for different prediction windows or MRRs, which is beneficial to guarantee the normal operation of the AI model, and also beneficial to take into account the prediction performance of the AI model.

[0274] In some embodiments, the first information includes at least one of the following:

[0275] at least one prediction window, and a prediction accuracy requirement associated with each prediction window;

[0276] at least one MRR, and a prediction accuracy requirement associated with each MRR.

[0277] In some embodiments, the prediction accuracy requirement can be associated with the prediction window one by one, or one prediction accuracy requirement can be associated with multiple prediction windows.

[0278] In some embodiments, the prediction accuracy requirement can be associated with the MRR one by one, or one prediction accuracy requirement can be associated with multiple MRRs.

[0279] Optionally, the first information can be configured by a network side device, or can be predefined.

[0280] At step 320, the terminal determines a target prediction accuracy requirement in the first information based on a first prediction window or a first MRR. The first prediction window is a prediction window associated with the first AI model, and the first MRR is an MRR associated with the first AI model.

[0281] In some embodiments, the target prediction accuracy requirement is a prediction accuracy requirement associated with the first prediction window, or a prediction accuracy requirement associated with the first MRR.

[0282] At S330, the terminal performs model monitoring on the first AI model according to the target prediction accuracy requirement.

[0283] For example, the first AI model is used to perform a first prediction function, where the first prediction function includes at least one of the following: RRM measurement prediction, measurement event prediction, RLF prediction, HOF prediction, and target cell prediction.

[0284] At S340, in a case where a model monitoring result of the first AI model is less than or less than or equal to the target prediction accuracy requirement, the terminal performs a first operation.

[0285] In some embodiments, the model monitoring result of the first AI model can refer to a prediction accuracy achieved by the first AI model in performing the first prediction function.

[0286] In some embodiments, the first operation includes at least one of the following:

[0287] The terminal reports third information to a network side device;

[0288] The terminal deactivates the first AI model;

[0289] The terminal switches the first AI model to a second AI model;

[0290] The terminal reverts to a non-AI model based manner to implement the first prediction function.

[0291] In some embodiments, the third information includes at least one of the following:

[0292] A model identifier of the first AI model;

[0293] A function identifier of the first AI model;

[0294] A first indication indicating that the first AI model does not meet a prediction accuracy requirement;

[0295] A model monitoring result of the first AI model, such as a prediction accuracy achieved by the first AI model in performing the first prediction function.

[0296] Therefore, in the embodiment of the present application, the terminal reports the third information to the network side device, the network side device can know that the first AI model used by the terminal side cannot meet the prediction accuracy requirement, so that the network side device can configure a new AI model for the terminal to meet the target prediction accuracy requirement, so as to balance the normal operation of the AI model and the prediction performance of the AI model.

[0297] Embodiment 2

[0298] In this embodiment 2, the first information can include the prediction accuracy requirement associated with the frequency point or cell or beam, the terminal can select the target prediction accuracy requirement based on the frequency point or cell or beam associated with the first AI model, and perform model monitoring on the first AI model based on the target accuracy requirement.

[0299] As shown in FIG. 7, the model monitoring method can include the following steps:

[0300] S410, the terminal obtains first information, and the first information includes the prediction accuracy requirement associated with the frequency point or cell or beam.

[0301] For different frequency points or cells or beams (here, different means that the frequency point or cell or beam input to the AI model is different from the frequency point or cell or beam predicted by the AI model), the prediction difficulty of the AI model is different. For example, when the target cell and the serving cell are on the same frequency, the prediction performance of the AI model is better, and when the target cell and the serving cell are on different frequencies, the prediction performance of the AI model is poorer. For another example, when the target cell and the serving cell are co-located, the prediction performance of the AI model is better, and when the target cell and the serving cell are not co-located, the prediction performance of the AI model is poorer. For another example, when predicting a high-frequency cell through a low-frequency cell, the performance of the AI model is better, and when predicting a low-frequency cell through a high-frequency cell, the performance of the AI model is poorer.

[0302] In embodiment 2, the corresponding prediction accuracy requirement can be configured for different frequency points or cells or beams, which is beneficial to ensure the normal operation of the AI model, and also beneficial to balance the prediction performance of the AI model.

[0303] In some embodiments, the first information includes at least one of the following:

[0304] at least one frequency point, and the prediction accuracy requirement associated with each frequency point;

[0305] at least one cell, and the prediction accuracy requirement associated with each cell;

[0306] at least one beam, and the prediction accuracy requirement associated with each beam.

[0307] In some embodiments, the prediction accuracy requirement may be associated one-to-one with a frequency point, cell, or beam, or a prediction accuracy requirement may be associated with multiple frequency points, cells, or beams.

[0308] Optionally, in this embodiment 2, the prediction accuracy requirements for frequency points, cells, or beam associations can be configured in the measurement object.

[0309] Optionally, the first information can be configured by the network-side device, or it can be predefined.

[0310] Step 420: The terminal determines the target prediction accuracy requirement based on the first frequency point, the first cell, or the first beam from the first information. Here, the first frequency point is the frequency point associated with the first AI model, the first cell is the cell associated with the first AI model, and the first beam is the beam associated with the first AI model.

[0311] In some embodiments, the target prediction accuracy requirement is the prediction accuracy requirement associated with a first frequency point, a first cell, or a first beam.

[0312] In some embodiments, associating the first AI model with the first frequency point can be understood as the output of the first AI model being the prediction result of the first frequency point. Optionally, the first frequency point can be the frequency point to be measured or predicted by the terminal.

[0313] In some embodiments, associating the first AI model with the first cell can be understood as the output of the first AI model being a prediction result of the first cell, such as the RRM measurement prediction result of the first cell; or, the first AI model can be used to predict the RRM measurement prediction result of the first cell. Optionally, the first cell can be the cell to be measured or predicted by the terminal.

[0314] In some embodiments, associating the first AI model with the first beam can be understood as the output of the first AI model being a prediction result of the first beam, such as an RRM measurement prediction result of the first beam, or the first AI model being used to predict the RRM measurement prediction result of the first beam. Optionally, the first beam can be the beam to be measured or predicted at the terminal.

[0315] S430: The terminal monitors the first AI model according to the target prediction accuracy requirements.

[0316] For example, a first AI model is used to perform a first prediction function, wherein the first prediction function includes at least one of the following: RRM measurement prediction, measurement event prediction, RLF prediction, HOF prediction, and target cell prediction.

[0317] S440, if the model monitoring result of the first AI model is less than or equal to the target prediction accuracy requirement, the terminal executes the first operation.

[0318] In some embodiments, the model monitoring result of the first AI model can indicate a prediction accuracy reached by the first AI model in performing the first prediction function.

[0319] In some embodiments, the first operation comprises at least one of:

[0320] reporting, by the terminal, third information to the network-side device;

[0321] deactivating, by the terminal, the first AI model;

[0322] switching, by the terminal, the first AI model to a second AI model;

[0323] falling back, by the terminal, to a non-AI model-based manner to implement the first prediction function.

[0324] In some embodiments, the third information comprises at least one of:

[0325] a model identifier of the first AI model;

[0326] a function identifier of the first AI model;

[0327] a first indication indicating that the first AI model does not meet the prediction accuracy requirement;

[0328] a model monitoring result of the first AI model, such as a prediction accuracy reached by the first AI model in performing the first prediction function.

[0329] Therefore, in the embodiments of the present application, the terminal reports the third information to the network-side device, so that the network-side device can learn that the first AI model used by the terminal side cannot meet the prediction accuracy requirement, and thus the network-side device can configure a new AI model for the terminal to meet the target prediction accuracy requirement, thereby balancing the normal operation of the AI model and the prediction performance of the AI model.

[0330] Embodiment 3

[0331] In this embodiment 3, the first information can include a prediction accuracy requirement associated with an AI function or an AI model, and the terminal can select a target prediction accuracy requirement based on the first AI model or an AI function supported by the first AI model, and perform model monitoring on the first AI model based on the target accuracy requirement.

[0332] As shown in FIG. 8, the model monitoring method of the embodiments of the present application can include the following steps:

[0333] S510, the terminal obtains first information, and the first information includes a prediction accuracy requirement associated with an AI function or an AI model.

[0334] Optionally, when a prediction window, MRR, frequency point, cell, or beam associated with an AI model or AI function is fixed, the prediction accuracy requirement can be associated with the AI model or AI function, and then the prediction accuracy requirement is indirectly associated with the prediction window, MRR, frequency point, cell, or beam.

[0335] In some embodiments, the first information includes at least one of the following:

[0336] at least one AI model, and a prediction accuracy requirement associated with each AI model;

[0337] at least one AI function, and a prediction accuracy requirement associated with each AI function.

[0338] In some embodiments, the prediction accuracy requirement can be associated with the AI model one by one, for example, when the AI model performs a fixed prediction function, the AI model can be associated with a prediction accuracy requirement.

[0339] In some embodiments, one AI model is associated with multiple prediction accuracy requirements, for example, when the AI model performs multiple prediction functions, each prediction function can be associated with a prediction accuracy requirement.

[0340] In some embodiments, the prediction accuracy requirement can be associated with the AI function one by one, or one AI function can be associated with multiple prediction accuracy requirements, respectively associated with different prediction windows, MRRs, frequency points, cells, or beams.

[0341] Optionally, when the functions of multiple AI functions or AI models are different, the prediction accuracy requirements associated with the multiple AI functions or AI models are different.

[0342] Optionally, when the functions of multiple AI functions or AI models are the same, but the prediction windows, MRRs, frequency points, cells, or beams corresponding to the multiple AI functions or AI models are different, the prediction accuracy requirements associated with the multiple AI functions or AI models are different, so as to ensure the normal operation of the multiple AI functions or AI models when performing different prediction functions, and take into account the prediction performance of the multiple AI functions or AI models.

[0343] Optionally, the first information can be configured by a network side device, or it can be predefined.

[0344] In step 520, the terminal determines a target prediction accuracy requirement in the first information based on the first AI model or the first AI function. The first AI function is an AI function supported by the first AI model.

[0345] In some embodiments, the target prediction accuracy requirement is the prediction accuracy requirement associated with the first AI model or the first AI function.

[0346] S530, the terminal performs model monitoring on the first AI model according to the target prediction accuracy requirement.

[0347] For example, the first AI model is used to perform a first prediction function, where the first prediction function includes at least one of the following: RRM measurement prediction, measurement event prediction, RLF prediction, HOF prediction, target cell prediction.

[0348] S540, in the case where the model monitoring result of the first AI model is less than or less than or equal to the target prediction accuracy requirement, the terminal performs a first operation.

[0349] In some embodiments, the model monitoring result of the first AI model can refer to the prediction accuracy achieved by the first AI model in performing the first prediction function.

[0350] In some embodiments, the first operation includes at least one of the following:

[0351] The terminal reports third information to the network side device;

[0352] The terminal deactivates the first AI model;

[0353] The terminal switches the first AI model to a second AI model;

[0354] The terminal reverts to a non-AI model based manner to implement the first prediction function.

[0355] In some embodiments, the third information includes at least one of the following:

[0356] Model identifier of the first AI model;

[0357] Function identifier of the first AI model;

[0358] First indication indicating that the first AI model does not meet the prediction accuracy requirement;

[0359] Model monitoring result of the first AI model, such as the prediction accuracy achieved by the first AI model in performing the first prediction function.

[0360] Therefore, in the embodiments of the present application, the terminal reports the third information to the network side device, and the network side device can learn that the first AI model used on the terminal side cannot meet the prediction accuracy requirement, so that the network side device can configure a new AI model for the terminal to meet the target prediction accuracy requirement, thereby balancing the normal operation of the AI model and the prediction performance of the AI model.

[0361] Embodiment 4:

[0362] In the embodiment 4, when the terminal performs indirect measurement event prediction, the prediction accuracy requirement can be determined according to the configuration information corresponding to the measurement event and the first parameter.

[0363] As shown in FIG. 9, the model monitoring method of the embodiment of the application can include the following steps:

[0364] S610, obtaining first information, the first information including configuration information corresponding to the measurement event and a first parameter.

[0365] The first parameter is used to determine the prediction accuracy requirement associated with the measurement event prediction.

[0366] Optionally, the first parameter takes a value between 0 and 1.

[0367] Optionally, the configuration information corresponding to the measurement event includes at least one of the following parameters:

[0368] Mn: neighbor cell measurement result, without considering any offset;

[0369] Ofn: neighbor cell measurement object specific offset;

[0370] Ocn: neighbor cell level specific offset;

[0371] Mp: SpCell measurement result, without considering any offset;

[0372] Ofp: SpCell measurement object specific offset;

[0373] Ocp: SpCell level specific offset;

[0374] Hys: hysteresis parameter of the event;

[0375] Off: offset parameter of the event.

[0376] S620, determining a target prediction accuracy requirement according to the configuration information corresponding to the measurement event and the first parameter.

[0377] For example, the target prediction accuracy requirement is determined according to the first parameter and a second parameter, wherein the second parameter includes at least one of the following parameters:

[0378] Mn: neighbor cell measurement result, without considering any offset;

[0379] Ofn: neighbor cell measurement object specific offset;

[0380] Ocn: neighbor cell level specific offset;

[0381] Mp: SpCell measurement result, without considering any offset;

[0382] Ofp: SpCell measurement object specific offset

[0383] Ocp: SpCell cell level specific offset

[0384] Hys: Hysteresis parameter of the event

[0385] Off: Offset parameter of the event

[0386] For example, the product of the first parameter and the second parameter is taken as the target prediction accuracy requirement.

[0387] For example, for an A3 event, assuming that the event is configured to meet the event entry condition when the neighbor cell is 3 dB higher than the serving cell, at this time, if the accuracy error of the RRM measurement prediction result of the AI model is higher than 3 dB, the accuracy of the measurement event prediction cannot be guaranteed with high probability, and therefore the accuracy requirement of the measurement event prediction is related to the event configuration. In embodiment 4, by configuring the first parameter, the terminal determines the prediction accuracy requirement used when indirectly predicting the measurement event according to the configuration information corresponding to the measurement event and the first parameter, so as to ensure that the AI model used for measurement event prediction can normally run, and to facilitate ensuring the accuracy of the measurement event prediction.

[0388] S640, in the case where the model monitoring result of the first AI model is less than or less than or equal to the target prediction accuracy requirement, the terminal performs a first operation.

[0389] In some embodiments, the model monitoring result of the first AI model can refer to the prediction accuracy reached by the first AI model in performing the first prediction function.

[0390] In some embodiments, the first operation includes at least one of the following:

[0391] The terminal reports third information to the network side device;

[0392] The terminal deactivates the first AI model;

[0393] The terminal switches the first AI model to a second AI model;

[0394] The terminal reverts to a non-AI model based manner to implement the first prediction function.

[0395] In some embodiments, the third information includes at least one of the following:

[0396] The model identifier of the first AI model;

[0397] The function identifier of the first AI model;

[0398] A first indication indicating that the first AI model does not meet the prediction accuracy requirement;

[0399] a model monitoring result of the first AI model, such as a prediction accuracy reached by the first AI model in performing the first prediction function.

[0400] In the embodiments of the present application, the terminal reports the third information to the network side device, and the network side device can learn that the first AI model used by the terminal side cannot meet the prediction accuracy requirement, so that the network side device can configure a new AI model for the terminal to meet the target prediction accuracy requirement, thereby balancing the normal operation of the AI model and the prediction performance of the AI model.

[0401] In summary, in the embodiments of the present application, the terminal can perform model monitoring on the first AI model according to the prediction accuracy requirement associated with at least one of the prediction window, the MRR, the frequency point, the cell, the AI model and the AI function according to the first information, or determine the target prediction accuracy requirement used for performing model monitoring on the first AI model according to the configuration information and / or the first parameter corresponding to the first prediction function, which is beneficial to the terminal to select a suitable prediction accuracy requirement to perform model monitoring on the first AI model, ensure the normal operation of the AI model, and balance the prediction performance of the AI model.

[0402] The model monitoring method provided in the embodiments of the present application can be executed by a wireless communication device. In the embodiments of the present application, the wireless communication device executing the model monitoring method is taken as an example to illustrate the wireless communication device provided in the embodiments of the present application.

[0403] The wireless communication device provided in the embodiments of the present application can be a communication device or a component in a communication device, such as a chip, as an example. The communication device can be a terminal, a network side device or a server, etc. For example, the terminal can include but is not limited to the types of the terminal 11 listed above, the network side device can include but is not limited to the types of the network side device 12 listed above, and the embodiments of the present application are not limited specifically.

[0404] The wireless communication device includes a receiving module, a sending module, and a processing module. The receiving module, the sending module, and the processing module can be implemented by software or by hardware. When implemented by hardware, the processing module can be implemented by a processor, which can include a general-purpose processor, a special-purpose processor, or the like, such as a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), an artificial intelligent (AI) processor, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a network processor (NP), a field programmable gate array (FPGA), or other programmable logic devices, a gate circuit, a transistor, a discrete hardware component, or the like. The receiving module and the sending module can be implemented by a communication interface, which can include one or more of a transceiver, a pin, a circuit, a bus, a radio frequency unit, or the like.

[0405] Optionally, referring to FIG. 10, when the wireless communication device is a terminal or a component in a terminal, the wireless communication device 1500 includes:

[0406] a processing module 1510, configured to obtain first information; and

[0407] perform model monitoring on a first artificial intelligent (AI) model according to the first information, where the first AI model is used to perform a first prediction function;

[0408] The first information includes at least one of the following:

[0409] one or more prediction accuracy requirements;

[0410] configuration information corresponding to the first prediction function;

[0411] a first parameter, used to determine a prediction accuracy requirement associated with the first prediction function;

[0412] The one or more prediction accuracy requirements are associated with at least one of the following:

[0413] a prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, or an AI model.

[0414] In some embodiments, the processing module 1510 is further configured to:

[0415] determine a target prediction accuracy requirement from the one or more prediction accuracy requirements according to the second information;

[0416] perform model monitoring on the first AI model according to the target prediction accuracy requirement;

[0417] wherein the second information comprises at least one of:

[0418] a prediction window associated with the first AI model;

[0419] a MRR associated with the first AI model;

[0420] a frequency point associated with the first AI model;

[0421] a cell associated with the first AI model;

[0422] a beam associated with the first AI model;

[0423] an AI function supported by the first AI model;

[0424] the first AI model.

[0425] In some embodiments, the target prediction accuracy requirement is associated with at least one of:

[0426] a first prediction window, the first prediction window being a prediction window associated with the first AI model;

[0427] a first MRR, the first MRR being a MRR associated with the first AI model;

[0428] a first frequency point, the first frequency point being a frequency point associated with the first AI model;

[0429] a first cell, the first frequency point being a cell associated with the first AI model;

[0430] a first beam, the first beam being a beam associated with the first AI model;

[0431] a first AI function, the first AI function being an AI function supported by the first AI model;

[0432] the first AI model.

[0433] In some embodiments, the processing module 1510 is further configured to:

[0434] determine a target prediction accuracy requirement according to the configuration information corresponding to the first prediction function and the first parameter;

[0435] According to the target prediction accuracy requirement, the first AI model is subjected to model monitoring.

[0436] In some embodiments, the processing module 1510 is further configured to:

[0437] In a case where the model monitoring result of the first AI model is less than or less than or equal to the target prediction accuracy requirement, a first operation is performed, wherein the first operation includes at least one of the following:

[0438] reporting third information to a network side device;

[0439] deactivating the first AI model;

[0440] switching the first AI model to a second AI model;

[0441] falling back to a non-AI model based manner to implement the first prediction function;

[0442] The third information includes at least one of the following:

[0443] a model identifier of the first AI model;

[0444] a function identifier of the first AI model;

[0445] a first indication indicating that the first AI model does not meet a prediction accuracy requirement;

[0446] a model monitoring result of the first AI model.

[0447] In some embodiments, the processing module 1510 is further configured to:

[0448] obtain the first information through predefined information; or

[0449] obtain the first information from a network side device.

[0450] In some embodiments, the first information includes a prediction accuracy requirement associated with at least one of a frequency point, a cell and a beam, and the prediction accuracy requirement associated with at least one of the frequency point, the cell and the beam is configured by the network side device in first configuration information, and the first configuration information is used to configure a measurement object.

[0451] In some embodiments, the first information comprises a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and the first prediction accuracy requirement is lower than the second prediction accuracy requirement in a case that a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time.

[0452] In some embodiments, the first parameter has a value between 0 and 1.

[0453] In some embodiments, the one or more prediction accuracy requirements comprise at least one of:

[0454] a prediction accuracy requirement for a radio resource management (RRM) measurement prediction;

[0455] a prediction accuracy requirement for a measurement event prediction;

[0456] a prediction accuracy requirement for a radio link failure (RLF) prediction;

[0457] a prediction accuracy requirement for a handover failure (HOF) prediction;

[0458] a prediction accuracy requirement for a target cell prediction.

[0459] In some embodiments, the first prediction function comprises at least one of:

[0460] an RRM measurement prediction;

[0461] a measurement event prediction;

[0462] an RLF prediction;

[0463] an HOF prediction;

[0464] a target cell prediction.

[0465] In some embodiments, the configuration information corresponding to the first prediction function comprises configuration information corresponding to a measurement event prediction.

[0466] Therefore, in the embodiments of the present application, the terminal can select, according to the first information, a prediction accuracy requirement associated with at least one of the prediction window, the MRR, the frequency point, the cell, the AI model and the AI function to perform model monitoring on the first AI model, or determine, according to the configuration information corresponding to the first prediction function and / or the first parameter, a target prediction accuracy requirement used to perform model monitoring on the first AI model, which is beneficial to the terminal to select a suitable prediction accuracy requirement to perform model monitoring on the first AI model, to ensure the normal operation of the AI model, and to take into account the prediction performance of the AI model.

[0467] Referring to FIG. 11, when the wireless communication apparatus is a network-side device or a component in the network-side device, the wireless communication apparatus 1600 includes:

[0468] The sending module 1610 is configured to send first information to a terminal, the first information being used for model monitoring of a first artificial intelligence (AI) model by the terminal, wherein the first AI model is used to perform a first prediction function.

[0469] The first information includes at least one of the following:

[0470] One or more prediction accuracy requirements;

[0471] Configuration information corresponding to the first prediction function;

[0472] A first parameter used to determine a prediction accuracy requirement associated with the first prediction function;

[0473] The one or more prediction accuracy requirements are associated with at least one of the following:

[0474] A prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model.

[0475] In some embodiments, the first parameter has a value between 0 and 1.

[0476] In some embodiments, the first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, the first prediction accuracy requirement is associated with a first prediction window, the second prediction accuracy requirement is associated with a second prediction window, and the first prediction accuracy requirement is lower than the second prediction accuracy requirement in a case where a length of the first prediction window is greater than a length of the second prediction window or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time.

[0477] In some embodiments, the first information includes a prediction accuracy requirement associated with at least one of a frequency point, a cell, and a beam, the prediction accuracy requirement associated with the at least one of the frequency point, the cell, and the beam being configured by the network-side device in first configuration information, and the first configuration information being used to configure a measurement object.

[0478] In some embodiments, the one or more prediction accuracy requirements include at least one of the following:

[0479] A prediction accuracy requirement for radio resource management (RRM) measurement prediction;

[0480] A prediction accuracy requirement for measurement event prediction;

[0481] Prediction accuracy requirement for radio link failure, RLF, prediction;

[0482] Prediction accuracy requirement for handover failure, HOF, prediction;

[0483] Prediction accuracy requirement for target cell prediction.

[0484] In some embodiments, the first prediction function comprises at least one of:

[0485] RRM measurement prediction;

[0486] Measurement event prediction;

[0487] RLF prediction;

[0488] HOF prediction;

[0489] Target cell prediction.

[0490] In some embodiments, the apparatus 1600 further comprises:

[0491] a receiving module, configured to receive third information from the terminal;

[0492] wherein the third information comprises at least one of:

[0493] model identification of the first AI model;

[0494] function identification of the first AI model;

[0495] a first indication, indicating that the first AI model does not meet the prediction accuracy requirement;

[0496] model monitoring result of the first AI model.

[0497] In some embodiments, the configuration information corresponding to the first prediction function comprises configuration information corresponding to measurement event prediction.

[0498] Therefore, in the embodiments of the present application, the network side device can provide the terminal with the first information, so that the terminal can determine the target prediction accuracy requirement used for performing model monitoring on the first AI model according to the first information, or according to the configuration information corresponding to the first prediction function and / or the first parameter, which is beneficial to the terminal to select a suitable prediction accuracy requirement to perform model monitoring on the first AI model, to ensure the normal operation of the AI model, and to take into account the prediction performance of the AI model.

[0499] The wireless communication device provided by the embodiments of the present application can implement each process of the method embodiments of FIGS. 5 to 9, and achieve the same technical effects. To avoid repetition, details are not described herein.

[0500] As shown in FIG. 12, the embodiments of the present application further provide a communication device 800, which includes a processor 801 and a memory 802, and the memory 802 stores programs or instructions executable on the processor 801. For example, when the communication device 800 is a terminal, the programs or instructions are executed by the processor 801 to implement each step performed by the terminal in the method embodiments of FIGS. 5 to 9, and achieve the same technical effects. When the communication device 800 is a network side device, the programs or instructions are executed by the processor 801 to implement each step performed by the network side device in the method embodiments of FIGS. 5 to 9, and achieve the same technical effects. To avoid repetition, details are not described herein.

[0501] The embodiments of the present application further provide a terminal, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the steps in the method embodiments of FIGS. 5 to 9. The terminal embodiment corresponds to the terminal side method embodiments described above, and each implementation process and implementation manner of the method embodiments can be applied to the terminal embodiment, and achieve the same technical effects. The terminal can be the wireless communication device 1500 shown in FIG. 10. FIG. 13 is a schematic diagram of a hardware structure of a terminal according to an embodiment of the present application.

[0502] The terminal 900 includes, but is not limited to, at least part of the components such as a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, and a processor 910.

[0503] Those skilled in the art can understand that the terminal 900 can further include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 910 through a power management system, so as to realize functions such as power management, discharge management, and power consumption management through the power management system. The terminal structure shown in FIG. 13 does not constitute a limitation on the terminal, and the terminal can include more or fewer components than those shown, or combine certain components, or have different component arrangements, which are not described herein.

[0504] It should be understood that in the embodiments of the present application, the input unit 904 can include a graphics processor 9041 and a microphone 9042, and the graphics processor 9041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 906 can include a display panel 9061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 can include two parts of a touch detection device and a touch controller. The other input devices 9072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, a joystick, and the like, which will not be described here.

[0505] In the embodiments of the present application, after the radio frequency unit 901 receives the downlink data from the network side device, it can be transmitted to the processor 910 for processing. In addition, the radio frequency unit 901 can send uplink data to the network side device. Generally, the radio frequency unit 901 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0506] The memory 909 can be used to store software programs or instructions and various data. The memory 909 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 909 can include a volatile memory or a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 909 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0507] The processor 910 can include one or more processing units; optionally, the processor 910 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 910.

[0508] The processor 910 is configured to obtain first information.

[0509] According to the first information, a first artificial intelligence (AI) model is monitored, wherein the first AI model is used to perform a first prediction function.

[0510] The first information includes at least one of the following:

[0511] One or more prediction accuracy requirements;

[0512] configuration information corresponding to the first prediction function;

[0513] a first parameter for determining a prediction accuracy requirement associated with the first prediction function;

[0514] The one or more prediction accuracy requirements are associated with at least one of the following:

[0515] a prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model.

[0516] It can be understood that the implementation processes of the implementation manners mentioned in the embodiments can refer to the related descriptions of the method embodiments in FIGS. 5 to 9, and achieve the same or corresponding technical effects. To avoid repetition, the details are not described herein again.

[0517] The embodiments of the present application further provide a network side device, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to realize the steps of the method embodiments shown in FIG. 14. The network side device embodiments correspond to the network side device method embodiments described above, and each implementation process and implementation manner of the method embodiments described above can be applied to the network side device embodiments, and the same technical effects can be achieved.

[0518] The embodiments of the present application further provide a network side device, which can be the wireless communication device 1600 shown in FIG. 11. As shown in FIG. 14, the network side device 1000 includes an antenna 1001, a radio frequency device 1002, a baseband device 1003, a processor 1004, and a memory 1005. The antenna 1001 is connected with the radio frequency device 1002. In the uplink direction, the radio frequency device 1002 receives information through the antenna 1001, and sends the received information to the baseband device 1003 for processing. In the downlink direction, the baseband device 1003 processes the information to be sent, and sends it to the radio frequency device 1002. The radio frequency device 1002 processes the received information and sends it out through the antenna 1001.

[0519] The method performed by the network side device in the above embodiments can be implemented in the baseband device 1003, which includes a baseband processor.

[0520] The baseband device 1003 may, for example, include at least one baseband board, which is provided with a plurality of chips, as shown in FIG. 14. One of the chips is, for example, a baseband processor, which is connected with the memory 1005 through a bus interface to call programs in the memory 1005 and perform the network device operations shown in the above method embodiments.

[0521] The network side device can further include a network interface 1006, for example, a common public radio interface (CPRI).

[0522] The network side device 1000 of the embodiments of the present application further includes instructions or programs stored on the memory 1005 and executable on the processor 1004, the processor 1004 invokes the instructions or programs in the memory 1005 to execute the methods performed by the modules shown in FIG. 11 and achieve the same technical effects, to avoid repetition, and therefore will not be described here.

[0523] The radio frequency device 1002 is configured to send first information to the terminal, the first information being used for the terminal to perform model monitoring on a first artificial intelligence (AI) model, wherein the first AI model is used to perform a first prediction function.

[0524] The first information includes at least one of the following:

[0525] One or more prediction accuracy requirements;

[0526] Configuration information corresponding to the first prediction function;

[0527] A first parameter used to determine a prediction accuracy requirement associated with the first prediction function;

[0528] The one or more prediction accuracy requirements are associated with at least one of the following:

[0529] A prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model.

[0530] The embodiments of the present application also provide a network side device. As shown in FIG. 15, the network side device 1100 includes a processor 1101, a network interface 1102, and a memory 1103. The network side device can be the wireless communication device shown in FIG. 11. The network interface 1102 is, for example, a common public radio interface (CPRI).

[0531] The network side device 1100 of the embodiments of the present application further includes instructions or programs stored on the memory 1103 and executable on the processor 1101, the processor 1101 invokes the instructions or programs in the memory 1103 to execute the methods performed by the modules shown in FIG. 11 and achieve the same technical effects, to avoid repetition, and therefore will not be described here.

[0532] The embodiment of the present application further provides a readable storage medium, and the readable storage medium stores a program or instructions, the program or instructions are executed by a processor to realize each process of the method embodiments of FIG. 5 to FIG. 9, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0533] The processor is the processor in the terminal in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. In some examples, the readable storage medium can be a non-transitory readable storage medium.

[0534] The embodiment of the present application further provides a chip, and the chip includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used to run a program or instructions to realize each process of the method embodiments of FIG. 5 to FIG. 9, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0535] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system chip, a system chip, a chip system or a system on chip, etc.

[0536] The embodiment of the present application further provides a computer program / program product, and the computer program / program product is stored in a storage medium, the computer program / program product is executed by at least one processor to realize each process of the method embodiments of FIG. 5 to FIG. 9, and the same technical effects can be achieved. To avoid repetition, details are not described herein.

[0537] The embodiment of the present application further provides a wireless communication system, and the wireless communication system includes a terminal and a network side device, the terminal can be used to execute steps of the model monitoring method, and the network side device can be used to execute steps of the model monitoring method.

[0538] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or the like does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it is to be understood that the methods and apparatuses of the present application can be carried out by specific hardware, software, or a combination thereof, and that the scope of the application is not limited to the specific order of execution of the steps described in the examples. In addition, features described in relation to certain examples can be combined in other examples.

[0539] From the above description of the embodiments, it is clear that the above-mentioned method can be realized by means of a computer software product and a general hardware platform, of course, it can also be realized by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disc, optical disc, etc.), and includes a plurality of instructions for making the terminal or network side device execute the method described in each embodiment of the present application.

[0540] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, the above-mentioned specific embodiments are only illustrative, not restrictive, and those skilled in the art can make many forms of embodiments under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A model monitoring method, wherein, Comprising: The terminal acquires first information; According to the first information, the model monitoring of the first artificial intelligence AI model is carried out, wherein the first AI model is used to execute the first prediction function; Wherein, the first information includes at least one of the following: One or more prediction accuracy requirements; The configuration information corresponding to the first prediction function; First parameter, used to determine the prediction accuracy requirement associated with the first prediction function; Wherein, the one or more prediction accuracy requirements are associated with at least one of the following: Prediction window, measurement reduction ratio MRR, frequency point, cell, beam, AI function, AI model.

2. The method of claim 1, wherein, According to the first information, the model monitoring of the first artificial intelligence AI model includes: According to the second information, the target prediction accuracy requirement is determined in the one or more prediction accuracy requirements; According to the target prediction accuracy requirement, the model monitoring of the first AI model is carried out; Wherein, the second information includes at least one of the following: The prediction window associated with the first AI model; The MRR associated with the first AI model; The frequency point associated with the first AI model; The cell associated with the first AI model; The beam associated with the first AI model; The AI function supported by the first AI model; The first AI model.

3. The method of claim 2, wherein, The target prediction accuracy requirement is associated with at least one of the following: First prediction window, the first prediction window is the prediction window associated with the first AI model; First MRR, the first MRR is the MRR associated with the first AI model; First frequency point, the first frequency point is the frequency point associated with the first AI model; First cell, the first frequency point is the cell associated with the first AI model; First beam, the first beam is the beam associated with the first AI model; First AI function, the first AI function is the AI function supported by the first AI model; The first AI model.

4. The method of claim 1, wherein, According to the first information, the model monitoring of the first artificial intelligence AI model includes: According to the configuration information corresponding to the first prediction function and the first parameter, the target prediction accuracy requirement is determined; According to the target prediction accuracy requirement, the model monitoring of the first AI model is carried out.

5. The method of any one of claims 2-4, wherein, The method further comprises: In the case that the model monitoring result of the first AI model is less than or less than or equal to the target prediction accuracy requirement, the terminal executes the first operation, wherein the first operation includes at least one of the following: The terminal reports third information to the network side device; The terminal deactivates the first AI model; The terminal switches the first AI model to the second AI model; The terminal reverts to the non-AI model based mode to realize the first prediction function; Wherein, the third information includes at least one of the following: The model identifier of the first AI model; The function identifier of the first AI model; First indication, used to indicate that the first AI model does not meet the prediction accuracy requirement; The model monitoring result of the first AI model.

6. The method of any one of claims 1-5, wherein, The terminal acquires first information, including: The terminal acquires the first information through pre-defined information; Or The terminal obtains the first information from a network side device.

7. The method of claim 6, wherein, The first information comprises at least one of a prediction accuracy requirement associated with a frequency point, a cell, and a beam, and the at least one of the prediction accuracy requirement associated with the frequency point, the cell, and the beam is configured by the network side device in first configuration information, and the first configuration information is used for configuring a measurement object.

8. The method of any one of claims 1-7, wherein, The first information comprises a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case where a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement.

9. The method of any one of claims 1-8, wherein, The first parameter has a value between 0 and 1.

10. The method of any one of claims 1-9, wherein, The one or more prediction accuracy requirements comprise at least one of: a prediction accuracy requirement for RRM measurement prediction; a prediction accuracy requirement for measurement event prediction; a prediction accuracy requirement for RLF prediction; a prediction accuracy requirement for HOF prediction; a prediction accuracy requirement for target cell prediction.

11. The method of any one of claims 1-10, wherein, The first prediction function comprises at least one of: RRM measurement prediction; measurement event prediction; RLF prediction; HOF prediction; target cell prediction.

12. The method of any one of claims 1-11, wherein, The configuration information corresponding to the first prediction function comprises configuration information corresponding to measurement event prediction.

13. A model monitoring method, wherein, The network side device sends first information to a terminal, and the first information is used for model monitoring of a first artificial intelligence (AI) model by the terminal, wherein the first AI model is used for performing a first prediction function. The first information comprises at least one of: one or more prediction accuracy requirements; configuration information corresponding to the first prediction function; a first parameter used for determining a prediction accuracy requirement associated with the first prediction function. The one or more prediction accuracy requirements are associated with at least one of: a prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model. The first parameter has a value between 0 and 1.

14. The method of claim 13, wherein, The first information comprises a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case where a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement.

15. The method of claim 13 or 14, wherein, The first information comprises at least one of a prediction accuracy requirement associated with a frequency point, a cell, and a beam, and the at least one of the prediction accuracy requirement associated with the frequency point, the cell, and the beam is configured by the network side device in first configuration information, and the first configuration information is used for configuring a measurement object.

16. The method of any one of claims 13-15, wherein, The one or more prediction accuracy requirements comprise at least one of:

17. The method of any one of claims 13-16, wherein, ​ Prediction accuracy requirement for radio resource management (RRM) measurement prediction; Prediction accuracy requirement for measurement event prediction; Prediction accuracy requirement for radio link failure (RLF) prediction; Prediction accuracy requirement for handover failure (HOF) prediction; Prediction accuracy requirement for target cell prediction.

18. The method of any one of claims 13-17, wherein, The first prediction function comprises at least one of: RRM measurement prediction; Measurement event prediction; RLF prediction; HOF prediction; Target cell prediction.

19. The method of any one of claims 13-18, wherein, The method further comprises: receiving, by the network-side device, third information from the terminal; wherein the third information comprises at least one of: a model identifier of the first AI model; a function identifier of the first AI model; a first indication indicating that the first AI model does not meet the prediction accuracy requirement; a model monitoring result of the first AI model.

20. The method of any one of claims 13-19, wherein, The configuration information corresponding to the first prediction function comprises configuration information corresponding to measurement event prediction.

21. A wireless communication device, wherein, Comprise: a processing module configured to obtain first information; perform model monitoring on a first artificial intelligence (AI) model according to the first information, wherein the first AI model is used to perform a first prediction function; wherein the first information comprises at least one of: one or more prediction accuracy requirements; configuration information corresponding to the first prediction function; a first parameter used to determine a prediction accuracy requirement associated with the first prediction function; wherein the one or more prediction accuracy requirements are associated with at least one of: a prediction window, a measurement reduction ratio (MRR), a frequency point, a cell, a beam, an AI function, and an AI model.

22. The apparatus of claim 21, wherein, The processing module is further configured to: determine a target prediction accuracy requirement from the one or more prediction accuracy requirements according to second information; perform model monitoring on the first AI model according to the target prediction accuracy requirement; wherein the second information comprises at least one of: a prediction window associated with the first AI model; an MRR associated with the first AI model; a frequency point associated with the first AI model; a cell associated with the first AI model; a beam associated with the first AI model; an AI function supported by the first AI model; and the first AI model.

23. The apparatus of claim 21, wherein, The processing module is further configured to: determine a target prediction accuracy requirement according to the configuration information corresponding to the first prediction function and the first parameter; perform model monitoring on the first AI model according to the target prediction accuracy requirement.

24. The apparatus of any one of claims 21-23, wherein, The processing module is further configured to: perform a first operation in a case where a model monitoring result of the first AI model is less than or equal to a target prediction accuracy requirement, wherein the first operation comprises at least one of: reporting third information to a network-side device; deactivating the first AI model; switching the first AI model to a second AI model; falling back to a non-AI model-based manner to implement the first prediction function; wherein the third information comprises at least one of: a model identifier of the first AI model; a function identifier of the first AI model; a first indication indicating that the first AI model does not meet the prediction accuracy requirement; a model monitoring result of the first AI model.

25. The apparatus of any one of claims 21-24, wherein, The first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case that a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement.

26. A wireless communication device, comprising: The first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case that a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement. The first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case that a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement. The first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case that a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement. The first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case that a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement. The first information includes a first prediction accuracy requirement and a second prediction accuracy requirement, wherein the first prediction accuracy requirement is associated with a first prediction window, and the second prediction accuracy requirement is associated with a second prediction window, and in a case that a length of the first prediction window is greater than a length of the second prediction window, or a time interval between the first prediction window and a current time is greater than an interval between the second prediction window and the current time, the first prediction accuracy requirement is lower than the second prediction accuracy requirement. The apparatus further includes: a receiving module configured to receive third information from the terminal; The third information includes at least one of the following:

27. The apparatus of claim 26, wherein, a model identifier of the first AI model; 28. The apparatus of claim 26 or 27, wherein, a function identifier of the first AI model; 29. The apparatus of any one of claims 26-28, wherein, a first indication indicating that the first AI model does not meet a prediction accuracy requirement; a model monitoring result of the first AI model. The apparatus includes a processor and a memory, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the model monitoring method according to any one of claims 1 to 12. The apparatus includes a processor and a memory, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the model monitoring method according to any one of claims 1 to 12. The readable storage medium stores programs or instructions, and the programs or instructions are executed by the processor to implement the steps of the model monitoring method according to any one of claims 1 to 12, or the steps of the model monitoring method according to any one of claims 13 to 20. ​ ​ 30. A terminal, wherein, ​ 31. A network-side device, wherein, ​ 32. A readable storage medium, wherein, ​

Images