Communications device and method

Abstract

The embodiments of this disclosure relate to communication apparatus and methods. In one aspect, a terminal device can evaluate the performance of a first model within a first time period. If a first number of first performance instances of the first model are evaluated within the first time period, the terminal device can evaluate the performance of candidate models of the first model within a second time period. If a second number of second performance instances of the candidates are evaluated within the second time period, the terminal device can perform model management associated with at least one of the first model or the candidates. In this way, model management can be performed at the terminal device side.

Description

Technical Field

[0001] The embodiments disclosed herein relate generally to the telecommunications field, and more specifically to methods, apparatus and computer storage media for communication used in the management of artificial intelligence (AI) / machine learning (ML) models. Background Technology

[0002] Various monitoring methods have been discussed for monitoring the performance of AI / ML models. However, the model management solutions for different monitoring methods are still imperfect and require further development. Summary of the Invention

[0003] Generally, embodiments of this disclosure provide methods, apparatus, and computer storage media for communication used in model management.

[0004] In a first aspect, a terminal device is provided. The terminal device includes a processor configured to: evaluate the performance of a first model within a first time period; evaluate the performance of candidates for the first model within a second time period based on a first number of performance instances of the first model evaluated within the first time period; and perform model management associated with at least one of the first model or the candidates based on a second number of performance instances of the candidates evaluated within the second time period.

[0005] In a second aspect, a terminal device is provided. The terminal device includes a processor configured to: evaluate the performance of a first model within a first time period; and, based on determining that a first performance instance of the first model was evaluated within the first time period, send a first indication of the first performance instance of the first model to a network device.

[0006] In a third aspect, a network device is provided. The network device includes a processor configured to cause the network device to: receive a first indication of a first performance instance of a first model from an end device; send an indication to the end device to evaluate the performance of a candidate of the first model; receive a second indication of a second performance instance of the candidate from the end device; and perform model management associated with at least one of the first model or the candidate.

[0007] In a fourth aspect, a communication method is provided. The method includes: evaluating the performance of a first model within a first time period at a terminal device; evaluating the performance of candidates for the first model within a second time period based on a first number of first performance instances of the first model evaluated within the first time period; and performing model management associated with at least one of the first model or the candidates based on a second number of second performance instances of the candidates evaluated within the second time period.

[0008] In a fifth aspect, a communication method is provided. The method includes: evaluating the performance of a first model within a first time period at a terminal device; and, based on determining that a first performance instance of the first model was evaluated within the first time period, sending a first indication of the first performance instance of the first model to a network device.

[0009] In a sixth aspect, a communication method is provided. The method includes: receiving, at a network device, a first indication of a first performance instance of a first model from a terminal device; sending to the terminal device an indication of evaluating the performance of a candidate of the first model; receiving, from the terminal device, a second indication of a second performance instance of the candidate; and performing model management associated with at least one of the first model or the candidate.

[0010] In a seventh aspect, a computer-readable medium is provided that stores instructions. When executed on at least one processor, these instructions cause the at least one processor to perform the method according to any one of the fourth to sixth aspects of this disclosure.

[0011] The following description will help to understand other features of this disclosure. Attached Figure Description

[0012] The above and other objects, features and advantages of this disclosure will become more apparent from a more detailed description of some embodiments thereof in the accompanying drawings, wherein: Figure 1 Example communication networks are illustrated, which may implement some embodiments of this disclosure; Figure 2 A signaling diagram illustrating the communication process for model management at the terminal device side according to some embodiments of this disclosure is shown; Figure 3 Signaling diagrams illustrating communication processes for model management at the network device side according to some embodiments of this disclosure are shown. Figure 4 A flowchart illustrating a method of communication implemented at a terminal device according to some embodiments of the present disclosure is shown; Figure 5A flowchart illustrating another method of communication implemented at a terminal device according to some embodiments of the present disclosure is shown; Figure 6 Flowcharts illustrating methods of communication implemented at a network device according to some embodiments of the present disclosure are shown; and Figure 7 This is a simplified block diagram of an apparatus suitable for implementing embodiments of this disclosure.

[0013] Throughout the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0014] The principles of this disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and are intended to assist those skilled in the art in understanding and implementing this disclosure, and are not intended to limit the scope of this disclosure. The disclosure described herein can be implemented in various ways other than those described below.

[0015] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0016] As used herein, the term "terminal device" refers to any device with wireless or wired communication capabilities. Examples of terminal devices include, but are not limited to: User Equipment (UE); Personal Computers; Desktop Computers; Mobile Phones; Cellular Phones; Smartphones; Personal Digital Assistants (PDAs); Portable Computers; Tablets; Wearable Devices; Internet of Things (IoT) Devices; Ultra-Reliable and Low-Latency Communication (URLLC) Devices; Internet of Everything (IoE) Devices; Machine-Type Communication (MTC) Devices; Devices for V2X communication on vehicles, where X refers to pedestrians, vehicles, or infrastructure / networks; Devices for Integrated Access and Backhaul (IAB); Devices for Small Data Transmission (SDT); Mobility Devices; Devices for Multicast and Broadcast Services (MBS); Devices for Location Services; Devices for Dynamic / Flexible Duplex in Commercial Networks; RedCap (Red Cap) Devices; Non-Terrestrial Networks (NTNs). In a non-terrestrial network, spacecraft or aircraft vehicles are included. These networks include satellites and high-altitude platforms (HAPs) encompassing unmanned aircraft systems (UAS); extended reality (XR) devices that include different types of reality (such as augmented reality (AR), mixed reality (MR), and virtual reality (VR); unmanned aerial vehicles (UAVs), often referred to as drones (aircraft without any human pilots); equipment on high-speed trains (HSTs); or image capture devices such as digital cameras and sensors; gaming devices; music storage and playback devices; or internet devices that enable wireless or wired internet access and browsing.The "terminal device" may also have "multicast / broadcast" capabilities to support public safety and mission-critical applications, V2X applications, transparent IPv4 / IPv6 multicast delivery, IPTV, smart TV, radio services, wireless software delivery, group communication, and IoT applications. The "terminal device" may also incorporate one or more subscriber identity modules (SIMs), a situation referred to as multi-SIM. The term "terminal device" is used interchangeably with UE, mobile station, subscriber station, mobile terminal, user terminal, or wireless device.

[0017] The term "network device" refers to a device that provides or hosts a cell or coverage area for terminal devices to communicate. Examples of network devices include, but are not limited to, NodeBs (or NBs), evolved NodeBs (eNodeBs or eNBs), next-generation NodeBs (gNBs), transmission reception points (TRPs), remote radio units (RRUs), radioheads (RHs), remote radio heads (RRHs), IAB nodes, low-power nodes (such as femtonodes and piconodes), reconfigurable intelligent surfaces (RISs), and network-controlled repeaters.

[0018] End devices or network devices may have AI / ML capabilities. End devices or network devices typically include models that have been trained on specific functions based on a large amount of collected data and can be used to predict some information.

[0019] Terminal or network devices can operate within several frequency ranges, such as FR1 (410MHz to 7125MHz), FR2 (24.25GHz to 71GHz), bands above 100GHz, and terahertz (THz). Terminal or network devices can also operate on licensed / unlicensed / shared spectrum. In MR-DC applications, terminal devices can have more than one connection to network devices. Terminal or network devices can operate in full-duplex, flexible-duplex, and cross-segmented-duplex modes.

[0020] Network devices may have network energy-saving and self-organizing network (SON) / minimization of drive test (MDT) capabilities. Terminals may have power-saving capabilities.

[0021] The embodiments disclosed herein can be implemented in test equipment (e.g., signal generator, signal analyzer, spectrum analyzer, network analyzer, test terminal equipment, test network equipment, channel simulator).

[0022] In one embodiment, the terminal device may be connected to a first network device and a second network device. One of the first and second network devices may be a master node, and the other may be a slave node. The first and second network devices may use different radio access technologies (RATs). In one embodiment, the first network device may be a first RAT device, and the second network device may be a second RAT device. In one embodiment, the first RAT device is an eNB, and the second RAT device is a gNB. Information related to different RATs may be sent to the terminal device from at least one of the first or second network devices. In one embodiment, first information may be sent from the first network device to the terminal device, and second information may be sent from the second network device directly or via the first network device to the terminal device. In one embodiment, configuration-related information configured by the second network device for the terminal device may be sent from the second network device via the first network device. Reconfiguration-related information configured by the second network device for the terminal device may be sent from the second network device directly or via the first network device to the terminal device.

[0023] As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term “comprising” and its variations should be understood as open terms meaning “including, but not limited to.” The term “based on” should be understood as “at least partially based on.” The terms “one implementation” and “implementation” should be understood as “at least one implementation.” The term “another implementation” should be understood as “at least one other implementation.” The terms “first,” “second,” etc., may refer to different or the same objects. Other explicit and implicit definitions are given below.

[0024] In some examples, values, processes, or devices are described as “best,” “lowest,” “highest,” “minimum,” “maximum,” etc. It should be understood that such descriptions are intended to indicate that a choice can be made among many alternative functionalities used, and that such a choice is not necessarily better, smaller, higher, or otherwise preferred than other choices.

[0025] The embodiments disclosed herein provide a communication solution for model management. In one aspect, a terminal device can evaluate the performance of a first model within a first time period. If a first number of first performance instances of the first model are evaluated within the first time period, the terminal device can evaluate the performance of candidate models of the first model within a second time period. If a second number of second performance instances of the candidates are evaluated within the second time period, the terminal device can perform model management associated with at least one of the first model or the candidates. In this way, model management can be performed at the terminal device side.

[0026] On the other hand, the terminal device can evaluate the performance of a first model within a first time period. If a first performance instance of the first model is evaluated within the first time period, the terminal device can send a first indication of the first performance instance of the first model to the network device. The network device can send an indication to the terminal device to evaluate the performance of candidate models of the first model. The terminal device can evaluate the performance of candidate models within a second time period. If a second performance instance of a candidate model is evaluated within the second time period, the terminal device can send a second indication of the second performance instance of the candidate model to the network device. The network device can perform model management associated with at least one of the first model or candidate models. In this way, model management can be performed at the network device side.

[0027] For convenience, some of the terms in this disclosure are defined as follows.

[0028] AI / ML Models: Data-driven algorithms that apply AI / ML techniques to generate a set of outputs based on a set of inputs.

[0029] AI / ML Model Delivery: A general term for the delivery of AI / ML models from one entity to another in any way. Entity can refer to network nodes / functions (e.g., gNB, location management function (LMF), etc.), UE, proprietary servers, etc.

[0030] AI / ML model inference: The process of using a trained AI / ML model to produce a set of outputs based on a set of inputs.

[0031] AI / ML Model Testing: A sub-process of training used to evaluate the performance of the final AI / ML model using a different dataset than that used for model training and validation. Unlike AI / ML model validation, testing does not assume subsequent model tuning.

[0032] AI / ML model training: The process of training an AI / ML model in a data-driven manner (e.g., by learning input / output relationships) and obtaining a trained AI / ML model for inference.

[0033] AI / ML model transmission: AI / ML models are delivered over the air interface in a manner opaque to the signaling of the Third Generation Partnership Project (3GPP). The delivered content consists of parameters of a model structure known to the receiving end, or a new model with parameters. The delivery may contain a complete model or a partial model.

[0034] AI / ML Model Validation: A sub-process of training used to evaluate the quality of AI / ML models using a different dataset than the dataset used for model training, in order to help select model parameters that demonstrate generalization ability outside of the dataset used for model training.

[0035] Data collection: The process by which network nodes, management entities, or UEs collect data for the purpose of AI / ML model training, data analysis, and inference.

[0036] Federated learning / federated training: A machine learning technique that trains AI / ML models on multiple decentralized edge nodes (e.g., UE, gNB), with each node performing local model training using local data samples. This technique requires multiple interactions between models but does not exchange local data samples.

[0037] Functionality Identification: The process / method for identifying AI / ML functionality for mutual understanding between the network (NW) and the UE. It should be noted that information regarding AI / ML functionality can be shared during functionality identification. The location where AI / ML functionality resides depends on the specific use case and sub-use case.

[0038] Model activation: Enables AI / ML models for specific AI / ML-enabled features.

[0039] Model deactivation: Deactivate the AI / ML model for specific AI / ML enabled features.

[0040] Model download: The model is transferred from the network to the UE.

[0041] Model identification: The process / method for identifying AI / ML models for mutual understanding between NW and UE. The process / method for model identification may or may not be applicable. Information about the AI / ML model may be shared during model identification.

[0042] Model monitoring: The process of monitoring the inference performance of AI / ML models.

[0043] Model parameter update: The process of updating the model parameters.

[0044] Model selection: The process of choosing one AI / ML model from multiple models to activate the same AI / ML-enabled feature. Model selection can be performed simultaneously with model activation, or they can be performed at different times.

[0045] Model switching: The process of deactivating the currently active AI / ML model and activating different AI / ML models for specific AI / ML-enabled features.

[0046] Model update: The process of updating the model parameters and / or model structure.

[0047] Model upload: The model is transferred from the UE to the network.

[0048] Network-side (AI / ML) model: An AI / ML model in which inference is performed entirely at the network.

[0049] Offline field data: Data collected from the field and used for offline training of AI / ML models.

[0050] Offline training: The AI / ML training process in which a model is trained based on a collected dataset and then used or delivered later for inference. Note: This definition is for guidance only. There may be some cases that, while not perfectly fitting this definition, can still be classified as offline training according to generally accepted conventions.

[0051] Online field data: Data collected from the field and used for online training of AI / ML models.

[0052] Online training: The AI / ML training process in which the model used for inference is trained (usually continuously) in (near) real-time as new training samples arrive. The concepts of (near) real-time and non-real-time depend on the context and are relative to the inference timescale. This definition is for guidance only. There may be cases that, while not perfectly fitting this definition, can still be classified as online training according to generally accepted conventions. Fine-tuning / retraining can be accomplished via online or offline training.

[0053] Reinforcement learning (RL): The process of training an AI / ML model in an environment that interacts with the model, based on inputs (also called states) and feedback signals (also called rewards) caused by the model's outputs (also called actions).

[0054] Semi-supervised learning: The process of training a model using a mixture of labeled and unlabeled data.

[0055] Supervised learning: The process of training a model based on the input and its corresponding labels.

[0056] Two-sided (AI / ML) model: A paired AI / ML model on which joint inference is performed, where joint inference includes AI / ML inference, which is jointly performed across the UE and the network. That is, the first part of the inference is performed by the UE first, and then the remaining part is performed by the gNB, and vice versa.

[0057] UE-side (AI / ML) model: An AI / ML model where inference is performed entirely on the UE.

[0058] Unsupervised learning: The process of training a model without using labeled data.

[0059] Proprietary format model: From a 3GPP perspective, this refers to a vendor / device-specific proprietary format ML model. Such models are not mutually recognizable between different vendors and hide model design information from other vendors when shared. Note: An example is a device-specific binary executable format.

[0060] Open format models: From a 3GPP perspective, these are ML models with a specified format that can be mutually recognized and interoperable across different vendors. Such models are mutually recognizable across different vendors and do not hide model design information from other vendors when shared.

[0061] In the context of this disclosure, the terms “model,” “functionality,” and “model / functionality” are used interchangeably. The terms “feature” and “feature group” are used interchangeably. The terms “model” and “model group” are used interchangeably. The terms “functionality,” “functionality group,” and “functionality set” are used interchangeably. The terms “ID,” “index,” “indicator,” and “identifier” are used interchangeably. The term “model monitoring” is used interchangeably with “monitoring,” “performance monitoring,” “performance monitoring of model / functionality,” or “performance monitoring of AI / ML-enabled features.”

[0062] In the context of this disclosure, the term "NW" may refer to "Operations, Administration and Maintenance (OAM)," "Server," or "Advanced Mobile Location (AML) / LMF." The term "in-distribution" may refer to a fitted data distribution, a drift-free data distribution, or a data distribution with tolerable drift. The term "out-of-distribution" may refer to an unfitted data distribution or a data distribution with detectable drift.

[0063] In the context of this disclosure, the terms “management decision,” “management request,” “management instruction,” or “management decision report” may include details regarding model / functionality selection, (de)activation, switching, or rollback. The term “condition” may refer to a supported configuration indicated by a UE capability report relating to model training, model inference, performance monitoring, verification processes, or rollback of an AI / ML model / functionality or a set of models / functionalities. The term “additional conditions” may refer to, for example, application conditions, scenarios, datasets, cell identity (ID), timestamps, signal-to-noise ratio (SNR), beamform, etc. The term “UE internal conditions” may refer to, for example, memory, battery, computing resources, overheating, and other hardware limitations.

[0064] In the context of this disclosure, for the UE portion of the UE-side model and / or dual-side model, AI / ML model identification and model ID-based lifecycle management (LCM) are discussed. Model ID-based LCM operates based on the identified model, which may be associated with specific configurations / conditions related to UE capabilities that enable AI / ML features / feature groups, as well as additional conditions (e.g., scenario, site, and dataset) determined / identified between the UE side and the NW side.

[0065] In the context of this disclosure, the term "model monitoring / performance monitoring metric" may include performance metrics or data required for performance metric calculations, such as: Inference accuracy, including metrics related to key performance indicators (KPIs); System performance, including metrics related to system performance KPIs; Data distribution Input-based: For example, monitoring the effectiveness of AI / ML inputs, such as out-of-distribution detection, input data drift detection, signal-to-noise ratio (SNR), latency spread, etc. Output-based: for example, drift detection of output data; or Applicable conditions.

[0066] As used herein, a model may be equivalent to at least one of the following: AI / ML model, ML model, AI model, data-driven, data processing model, algorithm, functionality, program, process, entity, function, feature, feature group, model ID, ID, functionality ID, configuration ID, scene ID, site ID, or dataset ID. Therefore, the above terms are used interchangeably.

[0067] In some implementations, the model may be represented or associated with channels, resources, resource sets, reference signal (RS) resources, RS resource sets, RS ports, RS port sets, RS port IDs, or RS port ID sets.

[0068] In some implementations, the model may include a set of weight values ​​that can be learned during training, for example for a specific architecture or configuration, where the set of weight values ​​may also be referred to as a parameter set.

[0069] In some implementations, the model can be used to predict target cells, or to predict the measurement results of a set of beams for a future set of candidate cells based on historical measurements of a set of beams from at least one set of candidate cells (e.g., L1-reference signal received power (RSRP), L1-signal to interference plus noise ratio (SINR)).

[0070] In some implementations, the input to the ML model (i.e., the AI ​​input) may refer to the input to the model and indicate the data input into the model, which may be equivalent to data.

[0071] In some implementations, the output of the ML model (i.e., the AI ​​output) can refer to the output of the model and indicate the result produced by the model, which is equivalent to the label / data.

[0072] In some implementations, the benchmark truth label (or benchmark truth tag) for data used to monitor or train an ML model (i.e., AI output) may refer to authoritative, accepted data or true answers or results of the AI / ML model. In some implementations, "benchmark truth," "benchmark truth tag," "benchmark truth tag for data," "input label," "input data," and "data" are used interchangeably. In some implementations, the benchmark truth can be interpreted as actual / factual (i.e., actual / factually measured) data / values / results / sets / parameters that can be used as a reference compared to predictions or inferences.

[0073] The principles and specific implementations of this disclosure will now be described in detail with reference to the accompanying drawings.

[0074] Examples of communication networks Figure 1 An example communication network 100 in which embodiments of this disclosure can be implemented is illustrated. For example... Figure 1 As shown, the communication network 100 includes a terminal device 110 and a network device 120 served by the terminal device 110.

[0075] like Figure 1 As shown, terminal device 110 may have multiple beams, and network device 120 may have multiple beams. A channel (or sub-channel) may be formed between one beam of terminal device 110 and one beam of network device 120. Terminal device 110 may send information to or receive information from network device 120 via one or more sub-channels.

[0076] It should be understood that Figure 1 The number of devices and beams shown is for illustrative purposes only and does not imply any limitation on this disclosure. The communication network 100 may include any suitable number of network devices and / or terminal devices and / or beams appropriate for implementing specific embodiments of this disclosure.

[0077] The communications in communication network 100 may conform to any suitable standard, including but not limited to Global System for Mobile Communications (GSM), Long Term Evolution (LTE), LTE Evolution, LTE-A (LTE-Advanced), New Radio (NR), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), GSM EDGE Radio Access Network (GERAN), Machine Type Communication (MTC), etc. The embodiments of this disclosure may be implemented according to any generation of communication protocols currently known or to be developed in the future. Examples of communication protocols include, but are not limited to, first-generation (1G) communication protocols, second-generation (2G) communication protocols, 2.5G communication protocols, 2.75G communication protocols, third-generation (3G) communication protocols, fourth-generation (4G) communication protocols, 4.5G communication protocols, fifth-generation (5G) communication protocols, 5.5G communication protocols, 5G-Advanced Networks, or sixth-generation (6G) networks.

[0078] Communication from terminal device 110 toward network device 120 is called uplink (UL) communication, while communication from network device 120 toward terminal device 110 in the opposite direction is called downlink (DL) communication. Wireless communication channels may include a physical uplink control channel (PUCCH), a physical uplink shared channel (PUSCH), a physical random-access channel (PRACH), a physical downlink control channel (PDCCH), a physical downlink shared channel (PDSCH), and a physical broadcast channel (PBCH).

[0079] Currently, performance monitoring metrics / methods and signaling procedures have been discussed. However, it remains unclear when or how a UE can declare an AI / ML model underperforming. This may vary depending on the performance metric / method. Furthermore, it is unclear when or how the NW / UE can decide to perform model / functionality selection, (de)activation, switching, or rollback. It may be necessary to verify that the model or method's performance is indeed superior to the current AI / ML model.

[0080] Therefore, embodiments of this disclosure provide a communication solution to offer methods for examining the performance of the current application's model and other AI / ML models or non-AI operations, thereby determining model management. References will follow below. Figures 2 to 3 These solutions are described in detail.

[0081] Example of side-model management implementation In this implementation plan, UE-side monitoring and UE-side management are considered.

[0082] Figure 2 A signaling diagram illustrating a communication process 200 for model management at the terminal device side according to some embodiments of this disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Describe process 200. Process 200 may involve, for example, Figure 1 The terminal device 110 and network device 120 are illustrated. It should be understood that... Figure 2The steps and their order are for illustrative purposes only and are not intended to be restrictive. For example, the order of steps can be changed. Some steps can be omitted, or any other suitable additional steps can be added. Assume the first model is the model currently being applied.

[0083] like Figure 2 As shown, terminal device 110 can send capability information of terminal device 110 to network device 120. In some embodiments, the capability information may be carried in UE assistance information (UAI).

[0084] In some embodiments, the capability information of terminal device 110 may include information about the model management methods supported by terminal device 110. In some embodiments, the capability information of terminal device 110 may include information about one or more timers or counters that the terminal device can maintain for the supported model management methods. In some embodiments, the capability information of terminal device 110 may include information about the storage of input or output data that terminal device 110 can use to monitor.

[0085] It should be understood that the capability information of terminal device 110 may include any other suitable capability information or any combination of the aforementioned capability information.

[0086] Continue to refer to Figure 2 Network device 120 may, for example, send configuration 220 for model management to terminal device 110 via radio resource control (RRC) signaling. In some embodiments, network device 120 may generate the configuration based on capability information of terminal device 110. It should be understood that configuration may also be generated without capability information of terminal device 110.

[0087] In some implementations, the configuration may include the maximum value of one or more of the counters. In some implementations, the configuration may include a timer for monitoring the AI / ML model. In some implementations, the configuration may include a threshold (also referred to herein as a first threshold) for a counter to begin evaluating a candidate or candidate method. In some implementations, the configuration may include a threshold (also referred herein as a second threshold) for a counter used for managing decisions.

[0088] In some implementations, the configuration may include information about one or more candidate AI / ML models, such as one or more model IDs. In some implementations, the configuration may include information about one or more candidate non-AI / ML methods, such as a default resource / resource configuration / reporting configuration, or configuration / resources during initial access (e.g., synchronization signal and physical broadcast channel block (SSB) or physical random access channel (PRACH)), or configuration prior to applying AI / ML. In some implementations, the default resource / resource configuration / reporting configuration associated with the AI / ML model may be explicitly provided, for example, by the NW configuration. In some implementations, the default resource / resource configuration / reporting configuration associated with the AI / ML model may be implicitly provided, such as the resource / resource configuration / reporting configuration with the lowest ID.

[0089] In some implementations, the configuration may include time-related configurations regarding the duration of the evaluation, such as a first time period for evaluating a model of the current application or at least a candidate second time period for evaluating a model of the current application. In some implementations, the configuration may include DL / UL resource / reporting configurations for the terminal device 110 to perform monitoring. In some implementations, the configuration may include performance monitoring methods / metrics and corresponding performance monitoring instance conditions. In some implementations, the configuration may include UL resources for reporting management decisions or transmitting management requests. In some implementations, the configuration may include a set of parameters for UL resources, including transmit power, priority, retransmission, or fallback.

[0090] It should be understood that the configuration used for model management may include any other suitable information or any combination of the above information.

[0091] Continue to refer to Figure 2 In some implementations, terminal device 110 can detect 230 physical layer problems. In this case, terminal device 110 can decide to evaluate the performance of the first model.

[0092] In some implementations, the terminal device may initialize a counter (e.g., N310) for RLF detection and a counter (e.g., N311) for RLF recovery. If a physical layer problem is detected (e.g., poor performance of the first model), the PHY layer of the terminal device 110 may send an asynchrony indication to the higher layers of the terminal device 110. Based on the asynchrony indication, the terminal device 110 may determine that a physical layer problem has been detected and may begin evaluating the performance of the first model.

[0093] In some implementations, when the timer used for RLF detection (e.g., timer T310) stops, the higher layer can reset N310 upon receiving an asynchrony indication from the PHY layer. When N310 reaches its maximum value, the higher layer can start timer T310. In some implementations, if timer T310 is started, the terminal device 110 can determine that a physical layer problem has been detected and can begin evaluating the performance of the first model.

[0094] It should be understood that any other suitable timer may also be considered. For example, beamFailureRecoveryTimer, timeAlignmentTimer, LTM supervisory timer, InactivityTimer, FailureDetectionTimer, RetransmissionTimer, configuredGrantTimer, ProhibitTimer, DRX-related timers, NES-related timers, reducedCap-related timers, cell / secondary cell activation / deactivation timers, or power-saving related timers. It should also be understood that any other suitable counter may also be considered. For example, BFI_COUNTER and beamFailureInstanceMaxCount, LBT_COUNTER, or ResourceSelection_COUNTER.

[0095] In some implementations, terminal device 110 may determine that a physical layer problem has been detected if a beam failure instance indication is received. In some implementations, terminal device 110 may determine that a physical layer problem has been detected if a beam failure is declared. It should be understood that physical layer problems in any other suitable process may also be considered. For example, physical layer problems in cell handover, transfer, or Layer 1 or Layer 2 triggered mobility (LTM) processes.

[0096] When a physical layer problem is detected, terminal device 110 can begin evaluating the performance of the first model. In other words, if the first model provides good performance, then the first model is not the cause of the physical layer problem. In this case, the RLF detection and recovery process can be performed. If the first model does not provide good performance, then the first model is one of the causes of the physical layer problem. In this case, terminal device 110 can begin evaluating the performance of the first model. In this way, continuous model monitoring can be avoided, and model monitoring can be performed only when there is a system performance problem.

[0097] It should be understood that continuous or periodic model monitoring is also feasible. This disclosure does not impose any limitations in this regard.

[0098] Continue to refer to Figure 2Terminal device 110 can evaluate the performance of the first model over a first time period of 240. In other words, terminal device 110 can perform model monitoring on the first model by calculating and evaluating monitoring metrics for each first time period. In some embodiments, model monitoring may be based on inference accuracy. In some embodiments, model monitoring may be based on system performance. In some embodiments, model monitoring may be based on data distribution. In some embodiments, model monitoring may be based on applicable conditions. It should be understood that the performance evaluation of the first model can be performed in any suitable manner, whether existing or developed in the future.

[0099] In some implementations, terminal device 110 may request or recommend resource / report configurations required for performance monitoring, and network device 120 may configure resources / reports based on requests from terminal device 110 or based on AI / ML model requirements. In some implementations, terminal device 110 may perform one or more measurements and comparisons within a first time period (i.e., an evaluation duration) to evaluate the performance of the AI / ML model. In some implementations, terminal device 110 may not need to monitor each model inference output.

[0100] In some implementations, the first time period can be periodic or semi-permanent. This means that the length of each evaluation duration, or the length of each of a plurality of evaluation durations, is the same. In this case, the length of the first time period may be referred to as periodic (and for convenience, also referred to herein as first periodicity).

[0101] In some implementations, the length of the first time period may be sufficient for the terminal device 110 to perform monitoring. In some implementations, the length of the first time period may be sufficient to obtain at least one evaluation result or judgment of the AI / ML model performance. In some implementations, the length of the first time period may be sufficient to satisfy the conditions of an "event" or "instance" related to AI / ML model performance monitoring. For example, an instance may be that the AI / ML model performed well or poorly during the previous evaluation duration. For example, an event may be defined based on one or more performance monitoring metrics. In some implementations, the length of the first time period may be sufficient to generate an indication to a higher level of the terminal device 110. In some implementations, the length of the first time period may be sufficient to cover the time required for one or more of the following: requesting resources, performing measurements, calculating performance monitoring metrics, etc.

[0102] To ensure sufficient length, the first periodicity of the first time period can be determined based on one or more factors. In some implementations, the first periodicity of the first time period may include at least one of the following: a maximum value of the first periodicity; an exact value of the first periodicity; or a minimum value of the first periodicity. The maximum and / or minimum values ​​can be used to control the frequency of monitoring AI / ML model performance.

[0103] In some implementations, the first periodicity may include a set of fixed values. In some implementations, the first periodicity may include a set of configured values. In some implementations, the first periodicity may include a set of values ​​reported or suggested by the terminal device 110.

[0104] In some implementations, the first periodicity may be determined based on a set of requirements of the first model, such as requirements in the AI / ML model description or requirements in AI / ML features or feature groups.

[0105] In some implementations, the first periodicity may be determined based on a method for evaluating the first model. In some implementations, the first periodicity may be determined based on intermediate KPIs. For example, the first periodicity may be determined based on the periodicity of obtaining the baseline truth value of the first model. In another example, the first periodicity may be determined based on the periodicity or scaling periodicity of a reference signal or report used to monitor the first model.

[0106] In some implementations, the first periodicity can be determined based on system KPIs or final KPIs. For example, the first periodicity can be determined based on the periodicity of the KPIs of the first model or the system KPIs.

[0107] In some implementations, the first periodicity may be determined based on the distribution of input or output data. For example, the first periodicity may be determined based on the time required to obtain input or output data (e.g., M input samples or N output samples) for monitoring the first model. M and N should be large enough to obtain statistical values. It should be understood that the precise values ​​of M or N may be fixed, configured, suggested by the terminal device 110, or based on the applied AI / ML model (i.e., the first model).

[0108] In some implementations, the first periodicity may be determined based on the applicable conditions of the first model. For example, the first periodicity may be determined based on the periodicity of checking the applicable conditions of the first model.

[0109] In some implementations, the first periodicity may be determined based on the duration of discontinuous reception (DRX) configured for terminal device 110.

[0110] It should be understood that the first periodicity can be determined based on any combination of the above factors or any other suitable factor.

[0111] In some implementations, the first time period can be aperiodic. This means that the length of each evaluation duration can vary. For example, AI / ML performance monitoring can be based on aperiodic requests or aperiodic triggers. In some implementations, the first time period can be the interval between two monitoring instances or events. In some implementations, the first time period can be the interval between two indications.

[0112] like Figure 2 As shown, to monitor the first model, a higher layer of terminal device 110 can initialize a counter (also referred to herein as the first counter) 241 to 0. The first counter will count the number of "events" or "instances" related to the monitoring of the first model. In some embodiments, different counters can be initialized and maintained for different AI / ML models. The first model can be the currently applied AI / ML model, or any AI / ML model configured to be monitored or requested to be monitored. In some embodiments, the counter can be initialized when one or more signaling signals are received from the NW. In some embodiments, the counter can be initialized when the first model is activated.

[0113] like Figure 2 As shown, the physical (PHY) layer of terminal device 110 can evaluate the performance of the first model within a first time period 242. If a first performance instance of the first model is evaluated within the first time period, terminal device 110 can send an indication of the first performance instance of the first model (also referred to herein as a first indication) to a higher layer 243.

[0114] In some implementations, a first performance instance may indicate that the performance of a first model is poor. In other words, the first indication may be generated only when the performance of the first model is poor. In some alternative implementations, a first performance instance may indicate that the performance of the first model is good. In other words, the first indication may be generated only when the performance of the first model is good. In some implementations, a first indication (e.g., 1 bit) may be generated to indicate whether the performance of the first model is good. In some implementations, a first indication (e.g., multiple bits) may be generated to indicate whether the performance of each of a plurality of models is good. For illustration, the following description uses the example of a first performance instance indicating that the performance of the first model is poor.

[0115] Continue to refer to Figure 2 Upon determining that a first number of first performance instances of the first model have been evaluated within a first duration, terminal device 110 may evaluate the performance of candidate models for a second duration of 250. In some embodiments, terminal device 110 may begin evaluating candidates if a first counter reaches a first threshold (i.e., a first number).

[0116] In some implementations, the higher layers of terminal device 110 may start or restart timer 251 (also referred to herein as the first timer) upon first receiving a first indication (e.g., an initial indication) from the physical layer. In some implementations, the first timer may be multiple evaluation durations, for example, four evaluation durations configured as timer values. Alternatively, the value of the first timer may be any suitable time unit, such as ms, time slots, etc. In some implementations, different timers may be initialized and maintained for different AI / ML models.

[0117] When the first timer is running, the higher layer can increment the first counter by 2521 when a first indication is received from the PHY layer. The higher layer can restart the first timer. In this case, performance is considered questionable if no indication is received. In some alternative implementations, the first timer can be designed differently; for example, the first timer can be used to provide a time window to count how many first indications are received. In this case, there is no need to restart the first timer.

[0118] In some implementations, if no first indication is received from the PHY layer during the first timer's operation (or if a good performance indication is received), the higher layers may keep the first counter unchanged, i.e., maintain the first counter. This option assumes that a successful model inference does not necessarily mean that the first model is performing well.

[0119] In some alternative implementations, if no first indication is received from the PHY layer during the first timer run (or if a good performance indication is received), the higher layer may reset the first counter (e.g., set the first counter to 0). That is, previous "bad" results will not be accumulated. This means that only the first consecutive performance instance is considered a poor AI / ML model performance.

[0120] In some implementations, if the first timer expires, the first counter can be reset (e.g., set to 0). This means that the first model is considered good because there are not enough “bad” instances to declare the first model as poorly performing during the first duration.

[0121] In some implementations, if a first counter is greater than or equal to a threshold (i.e., a first quantity) during the first timer's operation, a higher layer of the terminal device 110 may make a management decision. Alternatively, if the PHY layer generates an indication only when the first model is performing well, the higher layer may start a timer upon receiving an initial indication and increment the counter by 1 upon receiving the indication. When the timer expires, if the counter is less than or equal to the threshold, the higher layer may make a management decision.

[0122] like Figure 2As shown, in some implementations of the management decision, if a first indication of a first number of first performance instances of a first model is received from the PHY layer, the higher layer of the terminal device 110 may instruct the PHY layer 253 to evaluate the performance of a candidate for the first model. In some implementations, the candidate may be another model (also referred to herein as a second model). In some implementations, the candidate may be a non-AI operation or method.

[0123] To monitor candidates, higher layers of terminal device 110 can initialize a counter (also referred to herein as a second counter) to 0. The second counter is used to count the number of “events” or “instances” associated with the monitored candidate. In some implementations, different counters may be initialized and maintained if more than one AI / ML model is to be evaluated.

[0124] like Figure 2 As shown, the PHY layer of terminal device 110 can evaluate the performance of candidates within a second time period of 254. In some embodiments, if a candidate is inactive, the PHY layer of terminal device 110 can activate the candidate. In some embodiments, if multiple candidate AI / ML models are available, at least one AI / ML model can be activated as a candidate.

[0125] In some implementations, the PHY layer of terminal device 110 may evaluate the performance of candidates at least over a second time period. In some implementations, terminal device 110 may compare the performance of candidates with the performance of a first model. If the performance of a candidate is higher than that of the first model, terminal device 110 may determine that a second performance instance of the candidate has been evaluated.

[0126] In some implementations, the second time period can be periodic or semi-permanent. This means that the length of each evaluation duration in the evaluation duration, or the length of each evaluation duration in multiple evaluation durations, is the same. In this case, the length of the second time period may be referred to as the second periodicity.

[0127] In some implementations, the second periodicity may be sufficient for the terminal device 110 to perform monitoring and comparison between the first model and the candidate. In some implementations, the second periodicity may be determined based on at least one of the following: the periodicity required to evaluate the first model (i.e., the first periodicity of the first time period); or the periodicity required to evaluate the candidate.

[0128] In some alternative implementations, the second time period can be non-periodic.

[0129] like Figure 2As shown, if a candidate second performance instance is evaluated during the second time period, the PHY layer of terminal device 110 may send an indication of the candidate second performance instance (also referred to herein as a second indication) to the higher layers of terminal device 110.

[0130] In some implementations, a second performance instance may indicate that a candidate's performance is good. In other words, the second indication may be generated only when the candidate's performance is good. In some alternative implementations, a second performance instance may indicate that a candidate's performance is poor. In other words, the second indication may be generated only when the candidate's performance is poor. In some implementations, the PHY layer may generate a second indication (e.g., 1 bit) to indicate whether the candidate's performance is good. In some implementations, the PHY layer may generate a second indication (e.g., multiple bits) to indicate whether the performance of each of a plurality of models is good. For illustration, the following description uses the example of a second performance instance indicating that a candidate's performance is good.

[0131] Continue to refer to Figure 2 Upon determining that a second number of candidate performance instances have been evaluated during a second duration, terminal device 110 may perform model management 260 associated with at least one of the first model or candidates. In some embodiments, terminal device 110 may perform model management if a second counter reaches a second threshold (i.e., a second number).

[0132] In some implementations, a timer (also referred to herein as a second timer) may be started upon the first receipt of a second instruction from the PHY layer. In some implementations, the second timer may be multiple evaluation durations (i.e., second time periods). In some implementations, different timers may be initialized and maintained for different AI / ML models.

[0133] In some implementations, when a second instruction is received from the PHY layer during the operation of the second timer, the higher layer can increment the second counter by 1 and restart the second timer.

[0134] In some implementations, if no second indication is received from the PHY layer during the second time period, the higher layer may maintain the second counter. In some alternative implementations, if no second indication is received from the PHY layer during the second time period, the higher layer may set the second counter to 0.

[0135] In some implementations, upon receiving a second indication of the second performance instance (i.e., the candidate's performance is good), the higher layer can increment the second counter by 1 and start or restart the second timer.

[0136] In some implementations, upon receiving an indication that a candidate is performing poorly, the higher layer may set the second counter to 0. Alternatively, the higher layer may take no action on the second counter, for example, keeping the second counter unchanged or maintaining the second counter.

[0137] In some implementations, if the second timer expires, the higher layer can reset the second counter (e.g., set the second counter to 0).

[0138] In some implementations, if the second counter is greater than or equal to a second threshold during the operation of the second timer, the higher layer may make a management decision. In some implementations, if a second indication of a second number of candidate second performance instances is received from the PHY layer, the higher layer may perform model management.

[0139] like Figure 2 As shown, in some implementations, terminal device 110 can perform 261 management decisions. For example, a higher layer of terminal device 110 can perform a model switch from a first model to a second model. In another example, a higher layer of terminal device 110 can deactivate the first model. In another example, a higher layer of terminal device 110 can perform a rollback from the first model to a non-AI operation. In another example, a higher layer of terminal device 110 can perform a rollback from the first model to the default model. In another example, a higher layer can deactivate all AI / ML models.

[0140] like Figure 2 As shown, in some implementations, terminal device 110 may send 262 management decision information, such as model switching, rollback, or deactivation information, to network device 120.

[0141] Continue to refer to Figure 2 In some implementations, terminal device 110 may send a request 263 for model management to network device 120. Network device 120 may send a decision 264 for model management (e.g., model switching, rollback, or deactivation) to terminal device 110. Terminal device 110 may execute the decision 265 for model management.

[0142] This concludes the description of the model management solution. It should be understood that the first and second performance instances can be evaluated using any suitable model monitoring method, whether existing or developed in the future. For illustrative purposes, some specific implementations of the solution will be described in conjunction with Implementation Scheme 1 and Implementation Scheme 2.

[0143] Implementation Plan 1 In this implementation, the candidate is an AI / ML model (i.e., the second model). The first and second performance instances are evaluated based on the data distribution of the input and / or output data of the AI / ML model.

[0144] For the first model to be monitored, the terminal device 110 may store the input data or output data of the first model. In some embodiments, if the stored data (also referred to herein as first data) of at least one of the inputs or outputs of the first model is sufficient to determine the data distribution (also referred to herein as first data distribution), the terminal device 110 may determine the first data distribution based on the stored first data.

[0145] In some implementations, when the amount of stored first data is sufficient, the terminal device 110 can check the first data distribution of the stored first data. For example, a counter can be used to count the amount of first data, and the counter increments by 1 when new first data is stored. When the amount of stored first data is higher than or equal to a quantity threshold, the terminal device 110 can check the first data distribution.

[0146] In some implementations, when the data collection time for the stored first data is sufficient, for example, when the data collection time for the stored first data is greater than or equal to a time threshold, the terminal device 110 may check the first data distribution of the stored first data. For example, a timer may be used to control the duration of data storage. The timer may be started when the first data is stored. When the timer stops, the terminal device 110 may check the first data distribution. Alternatively, the terminal device 110 may check the first data distribution periodically.

[0147] In some implementations, when the storage space for the first data is full, for example, when the buffer, variable, or log size for the first data is full, the terminal device 110 may check the distribution of the first data.

[0148] In some implementations, terminal device 110 may release all stored first data (e.g., refresh the buffer of stored first data) after checking the first data distribution. In some implementations, terminal device 110 may release all stored data (e.g., refresh the buffer) after transmitting the first data distribution. In some implementations, terminal device 110 may release all stored data (e.g., refresh the buffer) after incrementing a first counter by 1. In this example, the first counter is used to count data distribution issues, that is, to count the first data distribution indicating a first performance instance.

[0149] In some implementations, if a first data distribution indicates a first performance instance, then terminal device 110 may determine that a first performance instance of the first model has been evaluated. In this case, the PHY layer of terminal device 110 may send a first indication to a higher layer of terminal device 110, and the higher layer may increment a first counter upon receiving the first indication.

[0150] When determining a first number of first performance instances of the first model that have been evaluated within a first duration, terminal device 110 may evaluate the candidate performance of the first model within a second duration. In some embodiments, terminal device 110 may determine the second data distribution based on the stored second data if stored data (also referred to herein as second data) of at least one of the inputs or outputs of the second model is sufficient to determine a data distribution (also referred to herein as second data distribution).

[0151] In some implementations, when the amount of stored second data is sufficient, the terminal device 110 can check the distribution of the stored second data. For example, a counter can be used to count the amount of second data, incrementing by 1 when new second data is stored. When the amount of stored second data is higher than or equal to a quantity threshold, the terminal device 110 can check the distribution of the second data.

[0152] In some implementations, when the data collection time for the stored second data is sufficient, for example, when the data collection time for the stored second data is greater than or equal to a time threshold, the terminal device 110 may check the second data distribution of the stored second data. For example, a timer may be used to control the duration of data storage. The timer may be started when the second data is stored. When the timer stops, the terminal device 110 may check the second data distribution. Alternatively, the terminal device 110 may check the second data distribution periodically.

[0153] In some implementations, when the storage space for the stored second data is full, for example, when the buffer, variable, or log size for the stored second data is full, the terminal device 110 may check the distribution of the second data.

[0154] In some embodiments, terminal device 110 may release all stored second data (e.g., refresh the buffer of stored second data) after checking the second data distribution. In some embodiments, terminal device 110 may release all stored second data (e.g., refresh the buffer) after transmitting the second data distribution. In some embodiments, terminal device 110 may release all stored second data (e.g., refresh the buffer) after incrementing a second counter by 1. In this example, the second counter is used to count within the distribution, that is, to count the second data distribution indicating the second performance instance.

[0155] In some implementations, if the second data distribution indicates a second performance instance, the terminal device 110 may determine that a second performance instance of the second model has been evaluated. In this case, the PHY layer of the terminal device 110 may send a second indication to a higher layer of the terminal device 110, and the higher layer may increment a second counter upon receiving the second indication.

[0156] When determining a second number of second performance instances of candidates that have been evaluated during a second duration, terminal device 110 may perform model management associated with at least one of the first model or candidates.

[0157] This approach provides a method to examine the distribution of input / output data used for model monitoring.

[0158] Implementation Plan 2 In this implementation, the candidate can be an AI / ML model (i.e., the second model) or a non-AI operation. The first and second performance instances are evaluated based on system performance. In this implementation, the first indication is an asynchronous indication, and the second indication is a synchronous indication.

[0159] Terminal device 110 may initialize a counter (e.g., N310) for radio link failure (RLF) detection and a counter (e.g., N311) for RLF recovery. If a physical layer problem is detected (e.g., poor performance of the first model), the PHY layer may send an asynchrony indication to the higher layers as a first indication.

[0160] In some implementations, the asynchrony indication may not be periodic. In some implementations, when the timer used for RLF detection (e.g., T310) stops, the higher layer may reset N310 upon receiving an "asynchrony" signal from the PHY layer. In some implementations, the higher layer may start timer T310 when N310 reaches its maximum value.

[0161] In some implementations, if timer T310 is started, the higher layers can instruct the PHY layer to evaluate the performance of the candidates.

[0162] In some implementations, for the first model to be monitored, the higher layer can initialize a counter to 0. In some implementations, the higher layer can start a timer (e.g., set for a third time period) upon first receiving an asynchrony indication. If a first number of asynchrony indications are received from the physical layer during the timer's operation (e.g., the counter reaches a first number), the higher layer can instruct the physical layer to evaluate the candidate's performance.

[0163] In some implementations, if the candidate provides good performance, the PHY layer may send a synchronization indication to the higher layers. In some implementations, the synchronization indication may not be periodic. In some implementations, the higher layers may stop the third timer upon first receiving the synchronization indication.

[0164] In some implementations, when timer T310 is running, the higher layer can reset N311 upon receiving a "dissynchronization" signal from the PHY layer. In some implementations, the higher layer can stop timer T310 when N311 reaches its maximum value. In some implementations, when timer T310 is stopped, the higher layer can perform model management.

[0165] In some implementations, if timer T310 starts, terminal device 110 may deactivate the first model. In some implementations, if timer T310 expires, terminal device 110 may deactivate all AI / ML models.

[0166] In this way, asynchronous and synchronous indicators can be used to monitor the system performance of AI / ML models.

[0167] This concludes the description of model management on the UE side. It should be understood that the operations described in procedure 200 and in embodiments 1 and 2 can be performed individually or in any suitable combination.

[0168] For illustration, the example process can be described as follows.

[0169] The responsible entity shall: 1> If the AI / ML model is configured with performance monitoring: 2> If performance monitoring instance indications for the AI / ML model have already been received from the lower layer: 3> Start or restart the monitoring timer for the AI / ML model; 3> Increase the monitoring counter of the AI / ML model by 1; 3> If the monitoring counter of the AI / ML model is >= MaxCount: 4> Trigger the action.

[0170] For illustration, another example procedure can be described as follows.

[0171] The responsible entity shall: 1> If the performance monitoring process determines that at least one action has been triggered and not canceled: 2> If UL-SCH resources are available for new transmissions and if UL-SCH resources can accommodate reports: 3> Instructions for generating reports on reuse and assembly processes; 2> Otherwise, if the UL-SCH resource is unavailable: 3> Trigger a request for a performance monitoring report.

[0172] In the above example, "responsible entity" can represent a high-level entity, and "low-level entity" can represent the PHY layer. "Monitoring timer" can represent a timer, "monitoring counter" can represent a counter, "performance monitoring instance indication" can represent an indication, and "MaxCount" can represent a threshold. "Action" can represent management decisions, management decision reports, etc.

[0173] Example of side-model management implementation In this implementation plan, UE-side monitoring and NW-side management are considered.

[0174] Figure 3 A signaling diagram illustrating a communication process 300 for model management at the network device side according to some embodiments of this disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Describe process 300. Process 300 may involve, for example, Figure 1 The terminal device 110 and network device 120 are illustrated. It should be understood that... Figure 3 The steps and their order are for illustrative purposes only and are not intended to be restrictive. For example, the order of steps can be changed. Some steps can be omitted, or any other suitable additional steps can be added. Assume the first model is the model currently being applied.

[0175] like Figure 3 As shown, terminal device 110 can send 310 capability information of terminal device 110 to network device 120. The specific implementation of step 310 is the same as that of step 210, so for the sake of brevity, it will not be repeated here.

[0176] refer to Figure 3 Network device 120 can send configuration 320 for model management to terminal device 110. The specific implementation of step 320 is the same as that of step 220, so it will not be repeated here for the sake of brevity.

[0177] Continue to refer to Figure 3 Network device 120 may send an instruction 330 to evaluate the performance of the first model. In some implementations, network device 120 may send an instruction to evaluate the performance of the first model if a physical layer problem is detected. It should be understood that the sending of the instruction to evaluate the performance of the first model may be triggered by any other suitable means.

[0178] refer to Figure 3 Terminal device 110 can evaluate the performance of the first model within the first time period 340. The specific implementation of step 340 is the same as that described in conjunction with step 240 and implementation schemes 1 and 2, so for the sake of brevity, it will not be repeated here.

[0179] like Figure 3As shown, if a first performance instance of the first model is evaluated within a first time period, the terminal device 110 may send a first indication of the first performance instance of the first model to the network device 120. Further details of the first indication are related to... Figure 2 The details described in Implementation Scheme 1 and Implementation Scheme 2 are the same.

[0180] like Figure 3 As shown, network device 120 can send an instruction to terminal device 110 to evaluate the performance of a candidate for the first model in a 360° angle. In some implementations, the candidate may be a second model or a non-AI operation.

[0181] In some implementations, network device 120 may send an indication to evaluate the performance of a candidate based on the number of received first indications reaching a first threshold. It should be understood that the transmission of the indication to evaluate the performance of a candidate can be triggered by any other suitable means.

[0182] In some implementations, if no first instruction is received from terminal device 110, network device 120 may reset or maintain the first counter for the first model.

[0183] refer to Figure 3 Terminal device 110 can evaluate the performance of candidates during the second time period 370. The specific implementation of step 370 is the same as that described in conjunction with step 250 and implementation schemes 1 and 2, so it will not be repeated here for the sake of brevity.

[0184] like Figure 3 As shown, if a candidate second performance instance is evaluated within the second time period, the terminal device 110 may send a second indication of the candidate second performance instance to the network device 120. Further details of the second indication are related to... Figure 2 The details described in Implementation Scheme 1 and Implementation Scheme 2 are the same.

[0185] Continue to refer to Figure 3 Network device 120 may perform model management 390 associated with at least one of the first model or candidates based on a second instruction. In some embodiments, network device 120 may perform model management based on a second threshold being reached in the number of received second instructions.

[0186] In some implementations, if no second instruction is received from terminal device 110, network device 120 may reset the second counter used for candidates or maintain the second counter.

[0187] This concludes the description of model management on the NW side. It should be understood that the operations described in process 300 can be performed individually or in any suitable combination with the operations described in process 200 and embodiments 1 and 2.

[0188] Example implementation of the method Corresponding to the above process, embodiments of this disclosure provide methods for communication implemented at terminal devices and network devices. These methods will be referenced below. Figures 4 to 6 Describe it.

[0189] Figure 4 A flowchart illustrating a method 400 for communication implemented at a terminal device according to some embodiments of the present disclosure is provided. For example, method 400 may be implemented as follows: Figure 1 The terminal device 110 shown is executed. For discussion purposes, reference will be made below. Figure 1 Method 400 is described below. It should be understood that method 400 may include additional boxes not shown and / or some boxes shown in the figures may be omitted, and the scope of this disclosure is not limited in this respect.

[0190] At box 410, terminal device 110 evaluates the performance of the first model within the first time period.

[0191] In some implementations, terminal device 110 may evaluate the performance of a first model within a first time period through its PHY layer. If a first performance instance of the first model is evaluated within the first time period, the PHY layer may send a first indication of the first performance instance of the first model to a higher layer of terminal device 110.

[0192] In some implementations, if a physical layer problem is detected, the terminal device 110 can evaluate the performance of the first model within a first time period.

[0193] In some implementations, terminal device 110 may determine that a physical layer problem has been detected based on at least one of the following: receiving an asynchrony indication; receiving a beam failure instance indication; starting a timer for radio link failure detection; or declaring a beam failure.

[0194] In some implementations, terminal device 110 may determine the first data distribution based on the stored data if the stored first data of at least one of the inputs or outputs of the first model is sufficient to determine the first data distribution. If the first data distribution indicates a first performance instance, terminal device 110 may determine that a first performance instance of the first model has been evaluated.

[0195] In some implementations, terminal device 110 may determine that the stored first data is sufficient to determine the distribution of the first data based on at least one of the following: the quantity of the stored first data is greater than or equal to a quantity threshold; the data collection time for the stored first data is greater than or equal to a time threshold; or the storage space for the stored first data is full.

[0196] In some implementations, if the first data distribution is determined, the terminal device 110 may release the stored first data.

[0197] In some implementations, the first time period may have a first periodicity, and the first periodicity may be determined based on at least one of the following: the maximum value of the first periodicity; the precise value of the first periodicity; the minimum value of the first periodicity; a set of requirements for the first model; a method for evaluating the first model; or configuring the DRX duration for the terminal device 110.

[0198] In some implementations, the method for evaluating the first model may include at least one of the following: periodicity of obtaining the baseline truth value of the first model; periodicity of scaling of reference signals or reports for monitoring the first model; periodicity of obtaining key performance indicators of the first model; time required to obtain input or output data for monitoring the first model; or periodicity of checking the applicability of the first model.

[0199] At box 420, if a first number of first performance instances of the first model have been evaluated within a first duration, then terminal device 110 evaluates the performance of candidate models of the first model within a second time period. In some implementations, the candidate may be a second model or a non-AI operation.

[0200] In some implementations, if a first indication of the first number of first performance instances of a first model is received from the PHY layer, the higher layer may instruct the PHY layer to evaluate the performance of the candidates.

[0201] In some implementations, the first indication may be an asynchrony indication. In some implementations, if a timer for RLF detection is started, the higher layer may instruct the physical layer to evaluate the candidate's performance. In some implementations, if a first number of asynchrony indications are received from the PHY layer within a third time period, the higher layer may instruct the physical layer to evaluate the candidate's performance.

[0202] In some implementations, terminal device 110 may deactivate the first model if the timer for RLF detection is started. In some implementations, terminal device 110 may stop setting the timer for the third time period if a synchronization indication is received for the first time.

[0203] In some implementations, if no first instruction for evaluating the first model is received from the PHY layer within a first time period, the terminal device 110 may reset the first counter for the first model or maintain the first counter.

[0204] In some implementations, if a candidate is not activated, the terminal device 110 may activate the candidate.

[0205] In some implementations, the terminal device 110 may evaluate candidate performance instances at least within a second time period via the PHY layer. If a candidate second performance instance is evaluated within the second time period, the PHY layer may send a second indication of the candidate second performance instance to a higher layer.

[0206] In some implementations, terminal device 110 may compare the performance of a candidate with the performance of a first model. If the performance of the candidate is higher than that of the first model, terminal device 110 may determine that a second performance instance of the candidate has been evaluated.

[0207] In some implementations where the candidate is a second model, if the stored second data of at least one of the inputs or outputs of the second model is sufficient to determine the second data distribution, then the terminal device 110 may determine the second data distribution based on the stored second data. If the second data distribution indicates a second performance instance, then the terminal device 110 may determine that a second performance instance of the second model has been evaluated.

[0208] In some implementations, terminal device 110 may determine that the stored second data is sufficient to determine the distribution of the second data based on at least one of the following: the quantity of the stored second data is greater than or equal to a quantity threshold; the data collection time for the stored second data is greater than or equal to a time threshold; or the storage space for the stored second data is full.

[0209] In some implementations, if a second data distribution is determined, the terminal device 110 may release the stored second data.

[0210] In some implementations, the second time period may have a second periodicity, and the second periodicity may be determined based on at least one of the following: the first periodicity of the first time period; or the periodicity required for evaluating the candidate.

[0211] At box 430, if a second number of second performance instances of candidates are evaluated during the second duration, the terminal device 110 performs model management associated with at least one of the first model or candidates.

[0212] In some implementations, if a second indication of a second number of second performance instances of a second model is received from the PHY layer, then higher-level executable model management is performed.

[0213] In some implementations, the second indication may be a synchronization indication. In some implementations, if the timer used for RLF detection stops, the terminal device 110 may perform model management.

[0214] In some implementations, if the timer used for RLF detection expires, the terminal device 110 may deactivate a set of models including at least the first model.

[0215] In some implementations, if no second instruction for evaluating candidates is received from the PHY layer during the second time period, the terminal device 110 may reset the second counter for candidates or maintain the second counter.

[0216] In some implementations, terminal device 110 may send model management information to network device 120. In some implementations, terminal device 110 may send a request for model management to network device 120. In some implementations, if a model management decision is received from network device 120, terminal device 110 may execute the model management decision.

[0217] Using method 400, model management can be performed on the terminal device side. It should be understood that the operation of method 400 corresponds to the combination of... Figure 2 The operations described in Implementation Schemes 1 and 2 are also described, and other details are omitted here for the sake of brevity.

[0218] Figure 5 A flowchart illustrating another method 500 for communication implemented at a terminal device according to some embodiments of the present disclosure is provided. For example, method 500 may be implemented as follows: Figure 1 The terminal device 110 shown is executed. For discussion purposes, reference will be made below. Figure 1 Method 500 is described below. It should be understood that method 500 may include additional boxes not shown and / or some boxes shown in the figures may be omitted, and the scope of this disclosure is not limited in this respect.

[0219] At box 510, terminal device 110 evaluates the performance of the first model within the first time period.

[0220] In some implementations, if a physical layer problem is detected, the terminal device 110 can evaluate the performance of the first model within a first time period.

[0221] In some implementations, terminal device 110 may determine that a physical layer problem has been detected based on at least one of the following: receiving an asynchrony indication; receiving a beam failure instance indication; starting a timer for radio link failure detection; or declaring a beam failure.

[0222] In some implementations, terminal device 110 may determine the first data distribution based on the stored data if the stored first data of at least one of the inputs or outputs of the first model is sufficient to determine the first data distribution. If the first data distribution indicates a first performance instance, terminal device 110 may determine that a first performance instance of the first model has been evaluated.

[0223] In some implementations, terminal device 110 may determine that the stored first data is sufficient to determine the distribution of the first data based on at least one of the following: the quantity of the stored first data is greater than or equal to a quantity threshold; the data collection time for the stored first data is greater than or equal to a time threshold; or the storage space for the stored first data is full.

[0224] In some implementations, if the first data distribution is determined, the terminal device 110 may release the stored first data.

[0225] In some implementations, the first time period may have a first periodicity, and the first periodicity may be determined based on at least one of the following: the maximum value of the first periodicity; the precise value of the first periodicity; the minimum value of the first periodicity; a set of requirements for the first model; a method for evaluating the first model; or configuring the DRX duration for the terminal device 110.

[0226] In some implementations, the method for evaluating the first model may include at least one of the following: periodicity of obtaining the baseline truth value of the first model; periodicity of scaling of reference signals or reports for monitoring the first model; periodicity of obtaining key performance indicators of the first model; time required to obtain input or output data for monitoring the first model; or periodicity of checking the applicability of the first model.

[0227] At box 520, if a first performance instance of the first model is evaluated within a first time period, the terminal device 110 sends a first indication of the first performance instance of the first model to the network device 120.

[0228] In some implementations, terminal device 110 may receive instructions from network device 120 to evaluate the performance of a candidate for the first model. In some implementations, the candidate may be a second model or a non-AI operation.

[0229] In some implementations, terminal device 110 may evaluate candidate performance at least within a second time period. If a candidate second performance instance is evaluated within the second time period, terminal device 110 may send a second indication of the candidate second performance instance to network device 120.

[0230] In some implementations, if a candidate is not activated, the terminal device 110 may activate the candidate.

[0231] In some implementations, terminal device 110 may compare the performance of a candidate with the performance of a first model. If the performance of the candidate is higher than that of the first model, terminal device 110 may determine that a second performance instance of the candidate has been evaluated.

[0232] In some implementations where the candidate is a second model, if the stored second data of at least one of the inputs or outputs of the second model is sufficient to determine the second data distribution, then the terminal device 110 may determine the second data distribution based on the stored second data. If the second data distribution indicates a second performance instance, then the terminal device 110 may determine that a second performance instance of the second model has been evaluated.

[0233] In some implementations, terminal device 110 may determine that the stored second data is sufficient to determine the distribution of the second data based on at least one of the following: the quantity of the stored second data is greater than or equal to a quantity threshold; the data collection time for the stored second data is greater than or equal to a time threshold; or the storage space for the stored second data is full.

[0234] In some implementations, if a second data distribution is determined, the terminal device 110 may release the stored second data.

[0235] In some implementations, the second time period may have a second periodicity, and the second periodicity may be determined based on at least one of the following: the first periodicity of the first time period; or the periodicity required for evaluating the candidate.

[0236] In some implementations, the first instruction may be an asynchronous instruction, and the second instruction may be a synchronous instruction.

[0237] Method 500 enables model monitoring at the terminal device side and facilitates model management at the network device side.

[0238] Figure 6 A flowchart illustrating a method 600 for communication implemented at a network device according to some embodiments of the present disclosure is provided. For example, method 600 may be implemented as follows: Figure 1 The network device shown is executed at location 120. For discussion purposes, references will be made below. Figure 1 Method 600 is described below. It should be understood that method 600 may include additional boxes not shown and / or some boxes shown in the figures may be omitted, and the scope of this disclosure is not limited in this respect.

[0239] At box 610, network device 120 receives a first indication of a first performance instance of a first model from terminal device 110.

[0240] In some implementations, if a physical layer problem is detected, network device 120 may send an instruction to terminal device 110 to evaluate the performance of the first model.

[0241] In some implementations, if no first instruction is received from terminal device 110, network device 120 may reset or maintain the first counter for the first model.

[0242] At box 620, network device 120 sends an instruction to terminal device 110 to evaluate the performance of a candidate for the first model. In some implementations, the candidate may be a second model or a non-AI operation.

[0243] At block 630, network device 120 receives a second indication of a candidate second performance instance from terminal device 110. In some embodiments, if no second indication is received, network device 120 may reset or maintain the second counter used for the candidate.

[0244] At box 640, network device 120 performs model management associated with at least one of the first model or candidates.

[0245] Method 600 allows for model management at the network device side. It should be understood that the operations of methods 500 and 600 correspond to the combination of... Figure 3 The operation described herein, and for the sake of brevity, other details are omitted here.

[0246] Example Implementation of the Device Figure 7 This is a simplified block diagram of device 700 suitable for implementing embodiments of the present disclosure. Device 700 can be considered as follows: Figure 1 Another example of the implementation of the terminal device 70 or network device 120 is shown. Therefore, device 700 may be implemented at or be implemented as at least a part of the terminal device or the network device at the terminal device 110 or the network device 120.

[0247] As shown in the figure, device 700 includes a processor 710, a memory 720 coupled to the processor 710, a suitable transceiver 740 coupled to the processor 710, and a communication interface coupled to the transceiver 740. The memory 710 stores at least a portion of a program 730. Depending on the requirements, the transceiver 740 can be used for bidirectional or unidirectional communication. The transceiver 740 may include at least one of a transmitter 742 or a receiver 744. The transmitter 742 and receiver 744 may be functional modules or physical entities. The transceiver 740 has at least one antenna to facilitate communication; however, in practice, the access node mentioned in this application may have several antennas. The communication interface can represent any interface necessary for communication with other network elements, such as the X2 / Xn interface for bidirectional communication between eNBs / gNBs, the S1 / NG interface for communication between the mobility management entity (MME) / access and mobility management function (AMF) / SGW / UPF and eNBs / gNBs, the Un interface for communication between eNBs / gNBs and relay nodes (RNs), or the Uu interface for communication between eNBs / gNBs and terminal equipment.

[0248] Assume that program 730 includes program instructions that, when executed by the associated processor 710, enable device 700 to operate according to embodiments of this disclosure, as referenced herein. Figures 1 to 6 The embodiments discussed herein may be implemented by computer software executable by processor 710 of device 700, or by hardware, or by a combination of software and hardware. Processor 710 may be configured to implement various embodiments of this disclosure. Furthermore, a combination of processor 710 and memory 720 may form a processing unit 750 suitable for implementing various embodiments of this disclosure.

[0249] The memory 720 can be of any type suitable for a local technology network and can be implemented using any suitable data storage technology, such as, as non-limiting examples, non-transitory computer-readable storage media, semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. Although only one memory 720 is shown in device 700, several physically different memory modules may exist in device 700. The processor 710 can be of any type suitable for a local technology network and may include one or more of the following: as non-limiting examples, general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSPs), and processors based on multi-core processor architectures. Device 700 may have multiple processors, such as application-specific integrated circuit (ASIC) chips, which are time-dependent on a clock that synchronizes the main processor.

[0250] In some implementations, the terminal device includes circuitry configured to: evaluate the performance of a first model within a first time period; evaluate the performance of candidates for the first model within a second time period based on a first number of first performance instances of the first model evaluated within the first time period; and perform model management associated with at least one of the first model or the candidates based on a second number of second performance instances of the candidates evaluated within the second time period.

[0251] In some implementations, the terminal device includes circuitry configured to: evaluate the performance of a first model within a first time period; and, based on determining that a first performance instance of the first model was evaluated within the first time period, send a first indication of the first performance instance of the first model to a network device.

[0252] In some implementations, the network device includes circuitry configured to: receive a first indication of a first performance instance of a first model from a terminal device; send an indication to the terminal device to evaluate the performance of a candidate of the first model; receive a second indication of a second performance instance of the candidate from the terminal device; and perform model management associated with at least one of the first model or the candidate.

[0253] As used herein, the term "circuit" can refer to hardware circuitry and / or a combination of hardware circuitry and software. For example, a circuit can be a combination of analog and / or digital hardware circuitry with software / firmware. As another example, a circuit can be any part of a hardware processor with software, including digital signal processors, software, and memory, which work together to enable a device (such as a terminal device or network device) to perform various functions. In yet another example, a circuit can be hardware circuitry and / or a processor (such as a microprocessor or a portion thereof) that requires software / firmware to operate, but which may be absent when operation is not required. As used herein, the term "circuit" also encompasses a specific implementation of hardware circuitry or a processor alone, or a portion thereof, and its accompanying software and / or firmware.

[0254] In summary, the implementation schemes disclosed herein can provide the following solutions.

[0255] In one solution, the terminal device includes a processor configured to: evaluate the performance of a first model within a first time period; evaluate the performance of candidates for the first model within a second time period based on a first number of first performance instances of the first model evaluated within the first time period; and perform model management associated with at least one of the first model or the candidates based on a second number of second performance instances of the candidates evaluated within the second time period.

[0256] In some implementations, the terminal device is configured to evaluate the performance of the first model by: the physical layer of the terminal device evaluating the performance of the first model within the first time period; and, based on determining that a first performance instance of the first model has been evaluated within the first time period, sending a first indication of the first performance instance of the first model to a higher layer of the terminal device.

[0257] In some implementations, the terminal device is configured to evaluate the performance of the candidate by: instructing the physical layer to evaluate the performance of the candidate based on a first indication that a first number of first performance instances of the first model have been received from the physical layer.

[0258] In some implementations, the first indication is an asynchronous indication, and the terminal device is instructed to instruct the physical layer to evaluate the performance of the candidate by: instructing the physical layer to evaluate the performance of the candidate based on the determination of a timer for radio link failure detection; or instructing the physical layer to evaluate the performance of the candidate based on the determination that a first number of asynchronous indications have been received from the physical layer within a third time period.

[0259] In some implementations, the terminal device is further configured to: deactivate the first model based on the start of the timer determined for radio link failure detection; or stop setting the timer for the third time period based on the determination that a synchronization indication has been received for the first time.

[0260] In some implementations, the terminal device is further configured to: reset or maintain a first counter for the first model if it is determined that no first instruction for evaluating the first model is received from the physical layer during the first time period.

[0261] In some implementations, the terminal device is configured to evaluate the performance of the candidate by: the physical layer of the terminal device evaluating the performance of the candidate at least during the second time period; and, based on determining that the second performance instance of the candidate has been evaluated during the second time period, sending a second indication of the second performance instance of the candidate to a higher layer of the terminal device.

[0262] In some implementations, the terminal device is configured to perform model management by a higher layer performing the model management in accordance with a second instruction that determines the second number of the second performance instances of the second model received from the physical layer.

[0263] In some implementations, the second instruction is a synchronization instruction, and the terminal device is instructed to perform the model management by stopping the timer determined for radio link failure detection.

[0264] In some implementations, the terminal device is further configured to: deactivate a set of models, including at least the first model, based on the timer determined to have expired for radio link failure detection.

[0265] In some implementations, the terminal device is further configured to: reset or maintain a second counter for the candidate if it is determined that no second instruction for evaluating the candidate is received from the physical layer during the second time period.

[0266] In some implementations, the terminal device is configured to evaluate the performance of the first model by evaluating the performance of the first model within a first time period based on the determination that a physical layer problem has been detected.

[0267] In some implementations, the terminal device is further configured to determine that the physical layer problem has been detected based on at least one of the following: receiving an asynchrony indication; receiving a beam failure instance indication; starting a timer for radio link failure detection; or declaring a beam failure.

[0268] In some implementations, the terminal device is configured to perform the model management by at least one of the following: sending model management information to the network device; sending a request for model management to the network device; or performing the decision to manage the model management based on a decision that determines that the model management has been received from the network device.

[0269] In another solution, the terminal device includes a processor configured to: evaluate the performance of a first model within a first time period; and, based on determining that a first performance instance of the first model was evaluated within the first time period, send a first indication of the first performance instance of the first model to a network device.

[0270] In some implementations, the terminal device is further configured to: receive from the network device an instruction to evaluate the performance of a candidate of the first model; evaluate the performance of the candidate at least during the second time period; and, based on determining a second performance instance of the candidate evaluated during the second time period, send a second instruction to the network device regarding the second performance instance of the candidate.

[0271] In some implementations, the terminal device is configured to evaluate the performance of the candidate by activating the candidate if it is determined to be inactive.

[0272] In some implementations, the terminal device is configured to evaluate the performance of the candidate at least by comparing the performance of the candidate with the performance of the first model; and by determining, based on the determination that the performance of the candidate is higher than the performance of the first model, the second performance instance of the candidate that has been evaluated.

[0273] In some implementations, the terminal device is configured to evaluate the performance of the first model by: determining the first data distribution based on the stored first data, which determines that at least one of the inputs or outputs of the first model is sufficient to determine the first data distribution; and determining the first performance instance of the first model that has been evaluated based on the determination that the first data distribution indicates the first performance instance.

[0274] In some implementations, the terminal device is further configured to determine that the stored first data is sufficient to determine the distribution of the first data based on at least one of the following: the quantity of the stored first data is greater than or equal to a quantity threshold; the data collection time for the stored first data is greater than or equal to a time threshold; or the storage space for the stored first data is full.

[0275] In some implementations, the terminal device is further configured to: release the stored first data based on the determined first data distribution.

[0276] In some implementations, the candidate is a second model, and the terminal device is configured to evaluate the performance of the candidate by: determining the second data distribution based on the stored second data, which determines that at least one of the inputs or outputs of the second model is sufficient to determine the second data distribution; and determining the second performance instance of the second model that has been evaluated based on the determination of the second data distribution indicating the second performance instance.

[0277] In some implementations, the terminal device is further configured to determine that the stored second data is sufficient to determine the distribution of the second data based on at least one of the following: the quantity of the stored second data is greater than or equal to a quantity threshold; the data collection time for the stored second data is greater than or equal to a time threshold; or the storage space for the stored second data is full.

[0278] In some implementations, the terminal device is further configured to: release the stored second data based on the determined second data distribution.

[0279] In some implementations, the first instruction is an asynchronous instruction, and the second instruction is a synchronous instruction.

[0280] In some implementations, the first time period has a first periodicity, and the first periodicity is determined based on at least one of the following: the maximum value of the first periodicity; the precise value of the first periodicity; the minimum value of the first periodicity; a set of requirements of the first model; a method for evaluating the first model; or configuring the DRX duration for the terminal device.

[0281] In some implementations, the method for evaluating the first model includes at least one of the following: periodicity of obtaining a baseline truth value of the first model; periodicity of scaling of a reference signal or report for monitoring the first model; periodicity of obtaining key performance indicators of the first model; time required to obtain input or output data for monitoring the first model; or periodicity of checking the applicability of the first model.

[0282] In some implementations, the second time period has a second periodicity, and the second periodicity is determined based on at least one of the following: the first periodicity of the first time period; or the periodicity required to evaluate the candidate.

[0283] In some implementations, the candidate is a second model or a non-AI operation.

[0284] In another solution, the network device includes a processor configured to: receive a first indication of a first performance instance of a first model from an end device; send an indication to the end device to evaluate the performance of a candidate of the first model; receive a second indication of a second performance instance of the candidate from the end device; and perform model management associated with at least one of the first model or the candidate.

[0285] In some implementations, the network device is further configured to perform at least one of the following: reset a first counter for the first model or maintain the first counter if a first instruction is not received from the terminal device; reset a second counter for a candidate or maintain the second counter if a second instruction is not received; or send an instruction to the terminal device to evaluate the performance of the first model if a physical layer problem is detected.

[0286] In some implementations, the candidate is a second model or a non-AI operation.

[0287] Generally, various embodiments of this disclosure can be implemented in hardware or special-purpose circuitry, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software executable by a controller, microprocessor, or other computing device. Although various aspects of embodiments of this disclosure are illustrated and described using block diagrams, flowcharts, or other illustrations, it should be understood that, as non-limiting examples, the blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, special-purpose circuitry or logic, general-purpose hardware or controllers or other computing devices, or any combination thereof.

[0288] This disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions (such as those included in program modules) that execute on a target real or virtual processor in a device to perform the functions described above. Figures 1 to 6 The described process or method. Generally, a program module includes routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. The functionality of a program module can be combined in various implementation schemes or split among program modules as needed. The machine-executable instructions used for a program module can be executed on a local or distributed device. In a distributed device, a program module can reside on both local and remote storage media.

[0289] Program code used to perform the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that, when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a stand-alone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0290] The aforementioned program code may be embodied on a machine-readable medium, which may be any tangible medium containing or storing a program used by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media will include electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0291] Furthermore, although the operations are described in a specific order, this should not be construed as requiring such operations to be performed in the specific order shown or in sequential order, or to perform all the illustrated operations to achieve the desired result. In some environments, multitasking and parallel processing can be advantageous. While several specific implementation details are included in the foregoing discussion, these details should not be construed as limiting the scope of this disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features described in the context of individual embodiments may also be implemented in a single embodiment in combination. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0292] Although this disclosure has been described using language specific to structural features and / or methodological actions, it should be understood that the disclosure as defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as examples of implementing the claims.

Claims

1. A terminal device, the terminal device comprising: Processor, the processor being configured to cause the terminal device to: Evaluate the performance of the first model within the first time period; Based on a first number of first performance instances of the first model that were evaluated within a first duration, the performance of the candidate first model is evaluated within a second time period. as well as Based on a second number of second performance instances of the candidates that were evaluated within a second duration, model management associated with the first model or at least one of the candidates is performed.

2. The terminal device according to claim 1, wherein the terminal device is configured to evaluate the performance of the first model in the following manner: The physical layer of the terminal device evaluates the performance of the first model during the first time period; and Based on the determination that a first performance instance of the first model was evaluated within the first time period, a first indication of the first performance instance of the first model is sent to the higher-level layer of the terminal device.

3. The terminal device of claim 2, wherein the terminal device is configured to evaluate the performance of the candidate in the following manner: Based on a first indication that a first number of first performance instances of the first model have been received from the physical layer, the higher layer instructs the physical layer to evaluate the performance of the candidate.

4. The terminal device of claim 3, wherein the first indication is an asynchronous indication, and wherein the terminal device is configured to instruct the physical layer to evaluate the performance of the candidate in such a way as: Based on a timer determined for radio link failure detection, the higher layer instructs the physical layer to evaluate the performance of the candidate; or Based on the determination that a first number of asynchronous indications were received from the physical layer within a third time period, the higher layer instructs the physical layer to evaluate the performance of the candidate.

5. The terminal device according to claim 4, wherein the terminal device is further configured such that: The first model is deactivated based on the timer determined for radio link failure detection; or Based on the confirmation that a synchronization instruction has been received for the first time, the timer for the third time period is stopped.

6. The terminal device of claim 1, wherein the terminal device is configured to evaluate the performance of the candidate in the following manner: The physical layer of the terminal device at least evaluates the performance of the candidates during the second time period; and Based on the determination that the candidate second performance instance was evaluated within the second time period, a second indication of the candidate second performance instance is sent to the higher layer of the terminal device.

7. The terminal device of claim 6, wherein the terminal device is configured to perform the model management in the following manner: The higher layer performs the model management based on a second instruction that determines the second number of the second performance instances of the second model received from the physical layer.

8. The terminal device of claim 6, wherein the second indication is a synchronization indication, and wherein the terminal device is configured to perform the model management in the following manner: The model management is executed based on the timer determined to be used for radio link failure detection.

9. The terminal device according to claim 8, wherein the terminal device is further configured such that: Based on the expiration of the timer used for radio link failure detection, a set of models including at least the first model is deactivated.

10. The terminal device of claim 6, wherein the terminal device is configured to evaluate the performance of the candidate in the following manner: Based on the determination that the candidate is inactive, the candidate is activated.

11. The terminal device of claim 6, wherein the terminal device is configured to evaluate the performance of the candidate at least by: The performance of the candidate model is compared with the performance of the first model; and Based on the determination that the performance of the candidate is higher than that of the first model, a second performance instance of the candidate is determined and evaluated.

12. The terminal device of claim 1, wherein the terminal device is configured to evaluate the performance of the first model in the following manner: Based on the determination that a physical layer problem was detected, the performance of the first model during the first time period is evaluated.

13. The terminal device according to claim 12, wherein the terminal device is further configured such that: The physical layer problem is determined to be detected based on at least one of the following: Asynchrony indication received; A beam failure instance indication has been received. A timer for radio link failure detection is started; or, Beam failure has been declared.

14. The terminal device of claim 1, wherein the terminal device is configured to perform the model management by at least one of the following: Send the model management information to the network device; Send a request for model management to the network device; or, Based on the decision received from the network device regarding model management, the decision of the model management is executed.

15. A terminal device, the terminal device comprising: Processor, the processor being configured to cause the terminal device to: Evaluate the performance of the first model within the first time period; as well as Based on the determination that a first performance instance of the first model was evaluated within the first time period, a first indication of the first performance instance of the first model is sent to the network device.

16. The terminal device according to claim 1 or 15, wherein the terminal device is configured to evaluate the performance of the first model in the following manner: The first data distribution is determined based on the stored first data, which determines that at least one of the inputs or outputs of the first model is sufficient to determine the first data distribution; and Based on the determination of the first data distribution indicating the first performance instance, the first performance instance of the first model is determined.

17. The terminal device according to claim 16, wherein the terminal device is further configured such that: The stored first data is determined to be sufficient to determine the first data distribution based on at least one of the following: The quantity of the first data stored is greater than or equal to the quantity threshold; The data collection time for the first data used for storage is higher than or equal to a time threshold; or The storage space for the first data to be stored is full.

18. The terminal device according to claim 1 or 15, wherein the first time period has a first periodicity, and the first periodicity is determined based on at least one of the following: The maximum value of the first periodicity; The precise value of the first periodicity; The minimum value of the first periodicity; A set of requirements for the first model; A method for evaluating the first model; or Configure the duration of discontinuous reception (DRX) for the terminal device.

19. The terminal device of claim 18, wherein the method for evaluating the first model comprises at least one of the following: Obtain the periodicity of the baseline truth value of the first model; Used to monitor the scaling periodicity of the reference signal or report of the first model; Obtain the periodicity of the key performance indicators of the first model; The time required to obtain the input or output data used to monitor the first model; or Check the periodicity of the applicable conditions for the first model.

20. A network device, the network device comprising: Processor, the processor being configured to cause the network device to: Receive a first indication of the first performance instance of the first model from the terminal device; Send an instruction to the terminal device to evaluate the performance of the candidates for the first model; Receive a second indication of the candidate second performance instance from the terminal device; as well as Perform model management associated with at least one of the first model or the candidates.

Images