Trust management for machine learning

Abstract

Example embodiments of the present disclosure relate to trust management for machine learning. In one method, a user equipment receives, from a network element, a message for requesting capabilities related to trust management for at least one machine learning enabled feature associated with an inter-domain service. The user equipment sends, to the network element, a response informing the capabilities. The user equipment receives, from the network element, a configuration message for a trust management report. The configuration message includes information about at least one request element associated with the at least one inter-domain service. The user equipment receives, from the network element, a request for the trust management report. The user equipment generates the trust management report based on the configuration message. The user equipment then sends, to the network element, the trust management report.

Description

Technical Field

[0001] The various examples and / or embodiments disclosed herein generally relate to the field of telecommunications. Background Technology

[0002] Artificial intelligence (AI) and machine learning (ML) will be integrated into all domains and layers of the 6G system, from devices to the radio access network (RAN), core network, and orchestration / management domain. To achieve maximum consistency and efficiency, this must be done systematically to ensure the interoperability, predictability, and availability of AI resources at all levels of the system, while maximizing the flexibility to introduce new AI-driven technologies. The availability of key AI enablers will allow all stakeholders to build and deploy high-quality, innovative AI models and solutions: access to data for training and inference, access to computing infrastructure for inference and training, and support for end-to-end lifecycle management for AI and ML.

[0003] Assessing trust in 6G networks involves evaluating, for example, the network's ability to deliver reliable connections, protect data, ensure privacy, control algorithm performance, and the reliability of inter-domain services. Trust assessment enhances network reliability and resilience. Some key objectives of reliability assessment and / or trust management are ensuring uninterrupted connectivity even in challenging scenarios, efficient management of large volumes of traffic, and continuous service availability. Summary of the Invention

[0004] In a first aspect of this disclosure, a user equipment is provided. The user equipment includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the user equipment to: receive from a network element a message requesting a capability, the capability being associated with trust management of at least one machine learning-enabled feature, the machine learning-enabled feature being associated with an inter-domain service; send a response to the network element notifying of the capability; receive from the network element a configuration message for a trust management report, the configuration message including information about one or more performance indicators and / or the characteristics of one or more performance indicators to be used as input metrics for the at least one machine learning-enabled feature, and information about at least one request element associated with the at least one inter-domain service, the characteristics of the one or more performance indicators being adjusted according to the capability and the at least one machine learning-enabled feature; receive from the network element a request for a trust management report; generate a trust management report based on the configuration message; and send the trust management report to the network element.

[0005] In a second aspect of this disclosure, a network element is provided. The network element includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the network element to: send a message to a user equipment (UE) requesting a capability, the capability being associated with trust management of at least one machine learning-enabled feature, the machine learning-enabled feature being associated with an inter-domain service; receive a response from the UE notifying the capability; determine, based on the capability and the at least one machine learning-enabled feature, one or more performance indicators and / or characteristics of one or more performance indicators to be used as input metrics for the at least one machine learning-enabled feature associated with trust management, and at least one request element associated with the at least one inter-domain service; send a configuration message for a trust management report to the UE, the configuration message including information about the characteristics of the one or more performance indicators and / or one or more performance indicators, and information about the at least one request element associated with the at least one inter-domain service; and, in response to detecting a performance degradation associated with a radio network service and related to the at least one machine learning-enabled feature, send a request for a trust management report to the UE and receive a trust management report from the UE.

[0006] In a third aspect of this disclosure, a method is provided. The method includes: receiving from a network element a message requesting a capability, the capability being associated with trust management of at least one machine learning-enabled feature, the machine learning-enabled feature being associated with an inter-domain service; sending a response notifying the network element of the capability; receiving from the network element a configuration message for a trust management report, the configuration message including information about one or more performance indicators and / or the characteristics of one or more performance indicators to be used as input metrics for the at least one machine learning-enabled feature, and information about at least one request element associated with the at least one inter-domain service, the characteristics of the one or more performance indicators being adjusted based on the capability and the at least one machine learning-enabled feature; receiving from the network element a request for the trust management report; generating a trust management report based on the configuration message; and sending the trust management report to the network element.

[0007] In a fourth aspect of this disclosure, a method is provided. The method includes: sending a message from a network element to a user equipment (UE) requesting a capability, the capability being associated with trust management of at least one machine learning-enabled feature, the machine learning-enabled feature being associated with an inter-domain service; receiving a response from the UE notifying the capability; determining, based on the capability and the at least one machine learning-enabled feature, one or more performance indicators and / or characteristics of one or more performance indicators to be used as input metrics for the at least one machine learning-enabled feature associated with trust management, and at least one request element associated with the at least one inter-domain service; sending a configuration message for a trust management report to the UE, the configuration message including information about the characteristics of the one or more performance indicators and / or one or more performance indicators, and information about the at least one request element associated with the at least one inter-domain service; and in response to detecting a performance degradation associated with a radio network service and related to the at least one machine learning-enabled feature, sending a request for a trust management report to the UE, and receiving a trust management report from the UE.

[0008] In a fifth aspect of this disclosure, an apparatus is provided. The apparatus includes: means for receiving from a network element a message requesting a capability, the capability being associated with trust management of at least one machine learning-enabled feature, the machine learning-enabled feature being associated with an inter-domain service; means for sending a response notifying the network element of the capability; means for receiving from the network element a configuration message for a trust management report, the configuration message including information about one or more performance indicators and / or the characteristics of one or more performance indicators to be used as input metrics for the at least one machine learning-enabled feature, and information about at least one request element associated with the at least one inter-domain service, the characteristics of the one or more performance indicators being adjusted according to the capability and the at least one machine learning-enabled feature; means for receiving the request for the trust management report from the network element; means for generating a trust management report based on the configuration message; and means for sending the trust management report to the network element.

[0009] In a sixth aspect of this disclosure, an apparatus is provided. The apparatus includes: means for sending a message from a network element to a user equipment (UE) requesting a capability, the capability being associated with trust management of at least one machine learning-enabled feature, the machine learning-enabled feature being associated with an inter-domain service; means for receiving a response from the UE notifying the capability; means for determining, based on the capability and the at least one machine learning-enabled feature, one or more performance indicators and / or characteristics of one or more performance indicators to be used as input metrics for the at least one machine learning-enabled feature associated with trust management, and at least one request element associated with the at least one inter-domain service; means for sending a configuration message for a trust management report to the UE, the configuration message including information about the characteristics of the one or more performance indicators and / or one or more performance indicators, and information about the at least one request element associated with the at least one inter-domain service; and means for sending a request for a trust management report to the UE in response to detecting a performance degradation associated with a radio network service and related to the at least one machine learning-enabled feature, and means for receiving the trust management report from the UE.

[0010] In a seventh aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to perform at least the method according to a third aspect.

[0011] In an eighth aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to perform at least the method according to the fourth aspect. Attached Figure Description

[0012] Some exemplary embodiments will now be described with reference to the accompanying drawings, in which: Figure 1 An example communication environment in which example embodiments of the present disclosure may be implemented is shown; Figure 2 Example signaling flows for trust management for ML according to some embodiments of this disclosure are shown; Figure 3 An overview of a trust management process for ML according to some embodiments of this disclosure is shown; Figure 4 A diagram illustrating examples of general KPI trends according to some embodiments of the present disclosure is shown; Figure 5 Example signaling flows for trust management for ML according to some embodiments of this disclosure are shown; Figure 6 Example signaling flows for trust management for ML according to some embodiments of this disclosure are shown; Figure 7A Example signaling flows for trust management for ML according to some embodiments of this disclosure are shown; Figure 7B Example signaling flows for trust management for ML according to some embodiments of this disclosure are shown; Figure 7C Example signaling flows for trust management for ML according to some embodiments of this disclosure are shown; Figure 8 A simplified block diagram of a device suitable for implementing example embodiments of the present disclosure is shown; and Figure 9 A block diagram of an example computer-readable medium according to some example embodiments of the present disclosure is shown.

[0013] In all the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0014] The principles of this disclosure will now be described with reference to some exemplary embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand this disclosure, without implying any limitation on the scope of this disclosure. The embodiments described herein can be implemented in various ways other than those described below.

[0015] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0016] References to “an embodiment,” “embodiment,” “example embodiment,” etc., in this disclosure indicate that the described examples and / or embodiments may include a particular feature, structure, or characteristic, but not every example and / or embodiment must include that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same example and / or embodiment. It should be understood that although the terms “first,” “second,” etc., may be used before nouns(s) herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another, and they do not restrict the order of the nouns(s). For example, a first element may be referred to as a second element without departing from the scope of the example embodiment, and similarly, a second element may be referred to as a first element. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0017] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements is connected by “and” or “or”, means at least any one of the elements, or at least any two or more of the elements, or at least all of the elements.

[0018] As used herein, unless explicitly stated otherwise, the “responding to A” action does not indicate that the action is performed immediately after “A” occurs, but may include one or more intermediate steps.

[0019] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the terms “comprising,” “including,” “having,” “possessing,” “containing,” and / or “covering,” as used herein, specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0020] As used in this application, the term "circuit" may refer to one or more or all of the following: (a) Hardware circuit implementation only (e.g., implemented with purely analog and / or digital circuits) and (b) A combination of hardware circuitry and software, such as (if applicable): (i) A combination of (multiple) analog and / or digital hardware circuits and software / firmware, and (ii) Any part of a hardware processor(s) having software (including (multiple) digital signal processors, software, and (multiple) memories, which work together to enable a device (such as a mobile phone or server) to perform various functions) and (c) The operation requires software (e.g., firmware) for the operation of (multiple) hardware circuits and / or (multiple) processors, such as (multiple) microprocessors or parts thereof, but the software may be absent when the operation does not require the software.

[0021] This definition of "circuit" applies to all uses of the term in this application, including in any claim. As another example, as used in this application, the term "circuit" also covers only hardware circuitry or processors (or processors in general) or a portion thereof and its accompanying software and / or firmware implementations. For example, where applicable to a particular claim element, the term "circuit" also covers baseband integrated circuits or processor integrated circuits for mobile devices or similar integrated circuits in servers, cellular network devices, or other computing or network devices.

[0022] As used herein, the term "communication network" refers to a network that conforms to any suitable communication standard, such as New Radio (NR), Long Term Evolution (LTE), LTE-A Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed ​​Packet Access (HSPA), Narrowband Internet of Things (NB-IoT), etc. Furthermore, communication between terminal devices and network devices in a communication network can be performed according to any suitable generation of communication protocol, including but not limited to first-generation (1G), second-generation (2G), 2.5G, 2.75G, third-generation (3G), fourth-generation (4G), 4.5G, fifth-generation (5G), Advanced 5G, 5.5G, sixth-generation (6G) communication protocols and / or any other currently known or future-developed protocols. Embodiments of this disclosure can be applied to various communication systems. Given the rapid development of communications, there will certainly be future types of communication technologies and systems that embody the nature of this disclosure. The scope of this disclosure should not be considered limited to the aforementioned systems.

[0023] As used herein, the term "network element" or "network device" refers to a device or node in a communications network through which terminal devices access the network and receive services. Depending on the terminology and technology applied, a network element can refer to a base station (BS) or access point (AP), such as a Node B (NodeB or NB), an evolved Node B (eNodeB or eNB), an NR NB (also known as a gNB), a Remote Radio Unit (RRU), a Radio Head (RH), a Remote Radio Head (RRH), a repeater, an Integrated Access and Backhaul (IAB) node, a low-power node (such as a femtosecond or picosecond), a Non-Terrestrial Network (NTN) or Non-Land Network Equipment (such as satellite network equipment, Low Earth Orbit (LEO) satellites, and Geostationary Earth Orbit (GEO) satellites), spacecraft network equipment, etc. In some example embodiments, the Radio Access Network (RAN) split architecture includes a centralized unit (CU) and a distributed unit (DU). This architecture can be applied to IAB donor nodes. An IAB node includes a mobile terminal (IAB-MT) portion that behaves similarly to a UE facing its parent node, and a DU portion that behaves similarly to a base station facing the next-hop IAB node.

[0024] The term "terminal device" refers to any terminal device capable of wireless communication. As an example and not a limitation, a terminal device may also be referred to as a communication device, user equipment (UE), subscriber station (SS), portable subscriber station, mobile station (MS), or access terminal (AT). Terminal devices can include, but are not limited to, mobile phones, cellular phones, smartphones, Voice over IP (VoIP) phones, wireless local loop phones, tablet computers, wearable terminal devices, personal digital assistants (PDAs), portable computers, desktop computers, image capture terminal devices (such as digital cameras), gaming terminal devices, music storage and recycle bins, in-vehicle wireless terminal devices, wireless endpoints, mobile stations, laptop embedded devices (LEEs), laptop-mounted devices (LMEs), USB dongles, smart devices, wireless customer premises equipment (CPEs), Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. The terminal equipment may also correspond to the mobile termination (MT) portion of an IAB node (e.g., a relay node). In the following description, the terms "terminal equipment," "communication equipment," "terminal," "user equipment," "user device," and "UE" are used interchangeably.

[0025] The core network (CN) functionality described herein can be implemented as a CN entity or CN node, comprising a combination of hardware processing circuitry and software and / or firmware including machine-readable instructions, or software including machine-readable instructions executable by at least one processor of the hardware processing circuitry. The hardware processing circuitry includes at least one processor and at least one memory storing machine-readable instructions executable by the at least one processor of the hardware processing circuitry. The processor includes any or some combinations of the following: accelerator, microprocessor, core of a multi-core microprocessor, microcontroller, programmable integrated circuit, programmable gate array, digital signal processor, central processing unit, graphics processing unit, tensor processing unit. The memory includes any or some combination of volatile or non-volatile memory (e.g., flash memory, cache, random access memory (RAM), and / or read-only memory (ROM)). The memory stores machine-readable instructions for the software and / or firmware executed by at least one processor of the hardware processing circuitry. The machine-readable instructions are executable by at least one processor of the hardware processing circuitry, causing the hardware processing circuitry to perform the actions or operations of the methods described herein. For example, the session management function described in this paper can be implemented as a session management entity, and the session management policy control function described in this paper can be implemented as a session management policy control entity.

[0026] As used herein, the terms “resource,” “transmission resource,” “resource block,” “physical resource block” (PRB), “uplink resource,” or “downlink resource” can refer to any resource used to perform communication (e.g., communication between a terminal device and a network device), such as resources in the time domain, resources in the frequency domain, resources in the spatial domain, resources in the code domain, or any other combination of time-domain, frequency-domain, spatial-domain, and / or code-domain resources capable of communication. In the following, unless explicitly stated otherwise, resources in the frequency and time domains will be used as examples of transmission resources used to describe some exemplary embodiments of this disclosure. Note that the exemplary embodiments of this disclosure are equally applicable to other resources in other domains.

[0027] As used in this paper, the term "model" refers to the association between an input and an output learned from training data, thus enabling the generation of a corresponding output for a given input after training. Model generation can be based on machine learning (ML) techniques, which can also be referred to as artificial intelligence (AI) techniques. Typically, an ML model can be built that receives input information and makes predictions based on that information. As used in this paper, "model" is equivalent to an AI / ML model or a data-driven / data processing algorithm / process.

[0028] As used in this paper, the term "interpretable ML" refers to the ability of an ML model to enable humans to understand the decisions or predictions it makes. As used in this paper, the term "fair ML" refers to the process of correcting and eliminating biases in an ML model. As used in this paper, the term "robust ML" refers to the process of automatically handling various forms of errors or corruption in an ML model, as well as changes in the underlying data distribution.

[0029] Figure 1 An example communication environment 100 in which exemplary embodiments of the present disclosure may be implemented is illustrated. In communication environment 100, UE 110 and network element 120 communicate with each other. Network element 120 serves UE 110. The transmission direction from network element 120 to UE 110 is referred to as downlink (DL), and the transmission direction from UE 110 to network element 120 is referred to as uplink (UL). In DL, network element 120 is a transmitting (TX) device (or transmitter), and UE 110 is a receiving (RX) device (or receiver). In UL, UE 110 is a TX device (or transmitter), and network element 120 is an RX device (or receiver).

[0030] exist Figure 1 In the example, network element 120 has a specific coverage area or service area, which is shown as cell 101. UE 110 is located in cell 101 covered by the second device 120. In communication environment 100, network element 120 can send data and control information to UE 110, and UE 110 can also send data and control information to network element 120.

[0031] In some examples, the communication environment 100 may additionally include a data server 130. The data server 130 may include any entity from a CN, or Operation, Administration and Maintenance (OAM), or Multi-Access Edge Computing (MEC), which can provide the network element 120 with the necessary functionality to collect, store, and provide services for additional interpretable data related to the UE 110. The data server 130 can communicate with the UE 110 and / or the network element 120.

[0032] It should be understood that Figure 1 The number of devices, equipment, and their connections shown are for illustrative purposes only and do not imply any limitation. Communication environment 100 may include any suitable number of devices. Although not shown, it should be understood that one or more additional devices may be located in cell 101, and one or more additional cells may be deployed in communication environment 100. For example, in Figure 1 The diagram depicts a terminal device; however, it should be understood that the communication environment 100 may include additional terminal devices besides the one shown.

[0033] Communication in communication environment 100 may be implemented according to any suitable communication protocol, including but not limited to cellular communication protocols, wireless local network communication protocols such as IEEE 802.11, and / or any other currently known or future-developed protocols. Furthermore, communication may utilize any suitable wireless communication technology, including but not limited to: Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Frequency Division Duplex (FDD), Time Division Duplex (TDD), Multiple Input Multiple Output (MIMO), Orthogonal Frequency Division Multiple Access (OFDM), Discrete Fourier Transform Extended OFDM (DFT-s-OFDM), and / or any other currently known or future-developed technologies.

[0034] In addition to basic connectivity, one or more AI / ML-based functions may be employed at network element 120, UE 110, or both to optimize system performance. For example, at least one ML-enabled feature associated with radio network services may be employed at UE 110 and / or network element 120.

[0035] With the rapid development of AI and ML technologies, their application in communication systems has become increasingly common. However, the use of AI / ML systems introduces varying degrees of risk. While the EU Artificial Intelligence Act (EU AI Act) does not strictly apply to 3GPP communication systems, it still sets corresponding obligations for providers and users based on the risk level of AI. Although many AI systems pose extremely low risks, they still need to be assessed, and transparency requirements are an integral part of the new rules.

[0036] The credibility of an AI / ML system encompasses various dimensions, such as data integrity, model performance, and the reliability of the inference process. Ensuring that AI / ML functions operate within acceptable credibility parameters is essential to maintaining system integrity, operational reliability, and user confidence.

[0037] The study emphasizes the need for a “native trust architecture” for 6G systems, which differs from the concept of “5G trust” and is applicable to the diverse requirements of an ecosystem with multiple stakeholders. It highlights the importance of managing trust during the ML training, testing, and inference phases. This can include preprocessing techniques to ensure data trustworthiness and the establishment of trustworthiness indicators. The proposed working definition of trustworthiness in 6G systems is summarized below. Trustworthiness (in 6G) is a measurable and coherent level of trust and / or confidence in the values ​​provided by 6G system stakeholders. Trustworthiness can be quantified as a multidimensional dynamic parameter, encompassing aspects of reliability, security, privacy, resilience, and reputation. The complex values ​​assigned to this parameter can be based on (but are not limited to) policies, behavioral evidence, implementation-specific algorithms, etc. Within the context of AI / ML use cases studied in 3GPP NR, which typically cover Open Systems Interconnection (OSI) Layer 1 (L1) to Layer 3 (L3) applications, consider the testability and verification processes for UE-side ML models or UE-side functions enabled by ML models. The necessary assemblies for verifying and testing ML-enabled solutions are still under development. Several aspects are use case-specific, particularly regarding the use of performance monitoring processes.

[0038] Establishing credibility in AI / ML solutions involves post-deployment validation of AI / ML enabling features. This work assumes that, when operating in the field, two factors can affect performance: a) updating, fine-tuning, adding, or removing features (if applicable) on the model, resulting in functional changes that can impact performance; and b) data drift or mismatch between the conditions encountered by the user in the field and the training data, which can also affect performance. Both options can be available to address potential variations in performance. These options are not mutually exclusive.

[0039] The first option is to validate the AI ​​model or functional changes before deploying or activating them in already deployed UEs. This validation process considers the UE hardware on which the model is deployed and activated, and further research is needed to determine whether validation can be performed in the field (i.e., on individual devices) or under laboratory conditions (i.e., depending on the device type or model). The feasibility of performing validation on-device and reasoning about other active functions, as well as the possibility of capturing model inputs and other necessary test data (such as ground-based data) during conformance testing for subsequent validation of new or updated models, are also under investigation.

[0040] The second option involves using performance monitoring and lifecycle management (LCM) processes. While performance monitoring can be defined or addressed through other standardization efforts, additional requirements and tests can be considered to ensure the consistency and accuracy of monitoring metrics or other relevant data transmitted from the UE. Additionally, where applicable, monitoring can be used to manage fallback mechanisms, feature changes, model updates, model switching, and model migration.

[0041] Monitoring is required to address drift or mismatches between conditions encountered by the UE in the field and the model's training data, which could affect functional performance. Additional requirements and tests may be considered to ensure the accuracy and consistency of monitoring data transmitted from the UE. Where applicable, the monitoring mechanism can also be used to manage fallback mechanisms, feature changes, model updates, model switching, and model transfer.

[0042] There remain many unresolved and undefined areas, particularly those related to the monitoring, testing, verification, and validation of ML-enabled solutions on the UE side. Discussions on these topics are expected to continue, with post-deployment validation increasingly recognized as a fundamental aspect of next-generation system development. The understandability and interpretability of AI / ML-enabled solutions in radio communications play a crucial role in establishing trust, enabling scalability, stability, reliability, robustness, and consistency across a wide range of multi-vendor deployment solutions in practice.

[0043] For newly proposed AI / ML-based solutions at the air interface (covering multiple protocol layers), a more complete verification framework for AIML-enabled functions rooted in interoperability principles is needed. ML-based solutions in next-generation air architectures will more likely require a shift towards more open and trusted designs (e.g., due to established benefits, legislation, regulations, etc.). Therefore, the practical process towards RAN needs to establish the credibility of ML-enabled solutions, which is beneficial to both UE and NW vendors.

[0044] For example, the anticipated deployment of 6G ML-enabled UE radio functions at L1 to L3, combined with network-side ML-enabled solutions, raises the need for expanded processes or mechanisms for handling ML-initiated fault detection and mitigation, a problem that already exists in current non-AI / ML 5G networks.

[0045] The UE reporting of information about / as an "explanation" of certain UE-side AI / ML behaviors (e.g., configured by the network (NW) on demand / periodically without explicit knowledge of the AI / ML model running on the UE side) requires allowing the gNB (or OAM) to resolve various UE-related AI / ML behaviors, which may degrade system performance.

[0046] One solution is to add "interpretation" information as a new information element with a pre-specified value to the existing RRC / MAC framework. This value is mapped to different interpretations, such as a new RRC "reason" element or an "interpretability characteristic" or "ML credibility indicator." However, this near-brute-force approach is not scalable when faced with the possible number of interpretations, UE types, UE conditions, etc., and it complicates the specification process and ultimately leads to significant signaling overhead (for both DL and UL).

[0047] Furthermore, the current 5G UE logging framework is insufficient / inappropriate because it results in significant latency in identifying the causes of UE failures. Integration with 5G-A UE-side data collection (training data and monitoring) would be more beneficial for MNO controllability, policy, and network analysis purposes.

[0048] The following section presents an overall signaling solution (ML-enabled algorithms for L1 to L3) that can carry “post-hoc,” “model-based model agnostic,” and quantitative contextual interpretation information from the UE to the network.

[0049] Figure 2 An example signaling flow 200 for trust management for ML is illustrated according to some embodiments and / or examples of this disclosure. The signaling flow can occur as follows: Figure 1 In the communication environment 100 shown. In some examples and / or embodiments, the operation of UE 110 and network element 120 may be interchangeable.

[0050] In operation, network element 120 sends (2010) a message to user equipment requesting a capability related to trust management of at least one ML-enabled feature associated with a radio network service. UE 110 receives (2020) the capability request message from network element 120 and sends (2030) a response notifying network element 120 of the capability. Network element 120 receives (2040) the response notifying UE 110 of the capability.

[0051] Based on capabilities and at least one ML-enabled feature, network element 120 determines (2050) one or more performance indicators and / or features of one or more performance indicators to be used as input metrics for at least one ML-enabled feature associated with trust management. Network element 120 sends (2060) a configuration message for a trust management report to UE 110. The configuration message includes information about the one or more performance indicators and / or features of one or more performance indicators. UE 110 receives (2070) the configuration message from network element 120. In the following text, the term "trust management report" may also be referred to as "interpretability report (ER)," and the two terms are used interchangeably.

[0052] In some examples, one or more performance indicators and / or the characteristics of one or more performance indicators may be associated with the following: user experience data rate, peak user data rate, latency, the ability to successfully transmit a predefined amount of data for a predetermined duration within a specified success probability, bit error rate, block error rate, packet loss, post-fault recovery time, reference signal strength indicator (RSSI), reference signal received power (RSRP), and / or reference signal received quality (RSRQ). In some examples, capabilities may be associated with UE 110 and / or host capabilities used to provide one or more performance indicators. In some examples, configuration messages for trust management reports may also indicate additional data pointer information for trust management reports. Additional data pointer information may be associated with the following: ML model inference speed, latency caused by computation of the ML model at the inference speed, ML-related capabilities (local, embedded, or provided by the network), and ML model capabilities used to perform in unseen scenarios and / or data distributions. UE 110 may include additional data pointer information in trust management reports.

[0053] In some examples, one or more characteristics of one or more performance indicators may include: trends and / or patterns in the time and space associated with one or more performance indicators, a time and space window for evaluation, a radio beam domain window for evaluation, a frequency domain window for evaluation, a geospatial window for evaluation, and / or the aggregation level of information in the trust management report. For simplicity, in the following text, the term "performance identifier" may be referred to as "Key Performance Indicator (KPI)," and "characteristics of one or more performance indicators" may be referred to as "KPI trends." In some examples, configuration messages may be associated with messages notifying the configuration of inference for at least one ML-enabled feature. UE 110 may perform inference for at least one ML model and send performance information obtained through inference for at least one ML model to network element 120.

[0054] Network element 120 detects (2080) a performance degradation with respect to at least one ML-enabled feature associated with the radio network service. In some examples, performance degradation detection can be performed by analyzing measurement data received from UE 110. In response to the detection of performance degradation, network element 120 sends a request for a trust management report to UE 110 (2090). UE 110 receives (2100) the request for a trust management report from network element 120. UE 110 then generates (2110) a trust management report based on its configuration and sends (2120) the trust management report to network element 120. Network element 120 receives (2130) the trust management report from UE 110.

[0055] Therefore, a robust, scalable, and flexible framework is provided to ensure the transparency and trustworthiness of AI / ML models, particularly for UE-side ML implementations in a trust management context. This approach ensures effective monitoring and management of ML functionality deployed at the UE, addressing the growing need for interpretability and transparency in AI / ML models.

[0056] The concepts disclosed herein can be referenced. Figure 3 To better understand, Figure 3 An overview of a trust management process 300 in ML according to some embodiments of this disclosure is shown. Assuming that in Figure 3 In the example, the ML model is implemented at UE 110. In operation 3010, UE 110 may advertise at least one of its own ER capabilities to network element 120. The ER capabilities of UE 110 may include, but are not limited to, a detailed set of configurations supported by UE 110, such as interpretation level, temporal-spatial trends, and time window size.

[0057] In operation 3020, network element 120 may send at least one ER configuration to UE 110 based on the capabilities received from UE 110. Additionally, network element 120 may send at least one of its own ER capabilities to the terminal device. The ER capabilities of network element 120 may include, but are not limited to, a detailed set of configurations supported by network element 120, such as (multiple) interpretation levels, (multiple) trends, and (multiple) time window sizes.

[0058] In operation 3030, network element 120 can detect any type of transmission performance degradation. To better understand the multiple causes of the performance degradation, network element 120 can decide to trigger an ER report from UE 110. In some examples, network element 120 may decide to trigger an ER report from UE 110 even if no performance degradation is detected.

[0059] In operation 3040, network element 120 may trigger an ER report from UE 110. In this operation, network element 120 may also send ER configuration based on the capabilities of the received UE 110.

[0060] In operation 3050, UE 110 may send an ER to network element 120 based on the configuration received from network element 120. As an example, for a non-real-time reporting configuration, the ER may include at least one trend across time frames. As an example, for a near real-time reporting configuration, the ER report may include at least one immediate report. Network element 120 can then use the ER to understand the multiple causes of performance degradation.

[0061] The Reporting Response (ER) may include three main components: a) multiple KPIs; b) multiple KPI trends and corresponding time-space windows; and c) multiple additional data pointers. The multiple KPIs and KPI trends may be mandatory components of the report, while the additional data pointers may be optional. This means that the ER must include at least one KPI and at least one KPI trend corresponding to at least one time-space window, and the ER may or may not include additional data pointers. These components can all be configured by network element 120 based on the multiple exposure capabilities of UE 110 associated with the multiple ML characteristics, and also based on the multiple capabilities and resources of network element 120.

[0062] Multiple KPIs can include various types of metrics, features, or statistics related to ER. These indicators represent real-time information. Multiple KPI trends can correspond to multiple time-space windows, which are also considered real-time information. Multiple KPI trends can include a set of general time-space trends to characterize the behavior of selected KPIs corresponding to configured time-space windows. Multiple additional data pointers can include information of the "address & protocol" type (e.g., RE characterizes a state transition (REST) ​​endpoint). Network element 120 can use multiple additional data pointers to retrieve additional non-real-time information, including extended context, metadata, conditions, and / or inference steps during the derivation of interpretation. Additional data pointers can be direct / explicit information. Alternatively, additional data pointers can be or include indexes from a pre-configured list of "address & protocol" information or from a pre-configured list of interpretation information. These three elements will be further explained below with examples.

[0063] The aforementioned ER architecture allows network element 120 to retrieve comprehensive (or complete) explanatory information for analyzing AI / ML behavior on the UE 110 side. Depending on latency requirements and operational needs, network element 120 can retrieve explanatory information in different ways. The first approach can be referred to as the low-latency approach. In this approach, explanatory information can be retrieved directly from UE 110 in real-time or near real-time. This supports fast processing and enables localized actions to address performance issues based on specific spatiotemporal contexts. The second approach can be referred to as the high-latency approach. In this approach, explanatory information can be obtained non-real-time from the Operation and Management (OAM) system or CN. The OAM system or CN can apply MNO policies, manage a large number of UEs (including UE 110 and other UEs), aggregate and store historical explanatory information, and provide long-term, large-scale mitigation actions. This also allows information retrieval / push to / from the UE provider server. The third approach can be referred to as the medium-latency approach. In this approach, explanatory information can be retrieved in semi-real-time. In the third method, the interpretation information from the UE (including UE 110 and other UEs) is stored in a multi-access edge computing (MEC) server, and then the network element 120 can retrieve the interpretation information with lower latency compared to OAM and / or CN. This method falls between the first and second methods in terms of latency and coverage of aggregated interpretations from multiple UEs, thus requiring a trade-off based on the preferences of the network element 120.

[0064] Network element 120 can provide indications to UE 110 (e.g., as an initial configuration) to indicate what type of interpretability information is expected to be reported and what the trigger (event and / or network signaling) for such reports is.

[0065] The UE-ER configuration is described in more detail below. The ER configuration can include three components (two mandatory components and one optional component), which have been mentioned above and will be described with examples. The first mandatory component is one or more KPIs. KPIs can include metrics, features, and / or statistics relevant to the UE interpretability report. They are considered real-time information and may differ from and / or be more detailed than the configured periodic / regular performance monitoring reports. For example, a KPI can be an additional set of upsampled measured KPI values. Alternatively or additionally, statistics derived from the measured KPIs can be included in the ER configuration, such as distributions, means, averages, quantiles, centroids, etc. In some examples, estimated combinations / aggregates of the measured KPIs / features (such as components from principal component analysis or similar techniques) can be included in the ER configuration.

[0066] The second mandatory component is the KPI trend and (multiple) time-space windows. The KPI trend may include a set of general time-space trends to characterize the behavior of the selected KPI, corresponding to the configured (multiple) time-space windows and aggregation levels considered as real-time information. For example, trend information may include indices from a pre-configured list of general trend types, independent of KPI scaling / range, and / or provide KPI-specific scaling / range through a procedural description of the trend (such as a function f(x) = y). Figure 4 Figure 400 illustrates examples of general KPI trends according to some examples and / or embodiments of this disclosure. The figure shows general examples of five KPI trend types that can be used to represent the behavior of KPIs over time and / or space. These trends are independent of KPI scaling or range, meaning they can be applied to any type of KPI regardless of its specific unit or range. Each of these trend types can be identified by a corresponding index value. The five trend types presented in the figure are: a) An increase indicates a continuous upward movement of the KPI, signifying improved performance; b) Decrease, which indicates a continuous downward shift in the KPI, suggesting a decline in performance; c) Maximum, which represents the maximum value of the KPI, indicating the position where the KPI reaches its highest point; d) Minimum, which represents the minimum value of the KPI, indicating the position where the KPI reaches its lowest point; e) Random, which represents irregular variations in KPIs.

[0067] It should be understood that Figure 4 The five trend types described are for illustrative purposes only and do not imply any limitations. A KPI trend can be any suitable trend type.

[0068] Temporal-spatial window information may include time windows (such as start timing and stop timing), geospatial windows (such as geographic bounding boxes), and / or beam / spatial domain windows (such as beam sets). Aggregate level information representing the level of precision may include single or aggregated interpretations of multiple UE inference implementations.

[0069] Optional components include additional data pointers, which contain "address & protocol" type information to be used by the network to retrieve additional non-real-time information. This pointer can be direct / explicit or simply an index from a pre-configured list. For example, the pointer can be or include an index from a pre-configured list of "address & protocol" information, or an index from a pre-configured list of interpretation information. For example, the format of the data pointer can include the UE vendor server IP address & data type (REST, etc.) or the UE data collection (e.g., Measurement Data Transmission (MDT)) session ID, which is used to record data potentially relevant to the interpretation. When deriving the interpretation, the pointer can include extended context, metadata, conditions, and / or inference steps. For example, such pointer information can include the method used to generate the interpretation (e.g., algorithm steps, etc.), additional datasets used to generate the interpretation, and / or information about whether the UE is using a more general or specialized model for the current configuration / conditions, which can help interpret the context without disclosing specific details about the AI / ML model. The pointer can also include details about aggregations, anomalies, visualizations, confidence levels, alternative predictions, etc.

[0070] UE-ER configuration can also include triggering (events and / or network signaling) for ERs. This configuration information can be set by the network based on how interpretability reports are used. This paper proposes at least two main levels (or types) of UE-ER. For UE-ER level 1, the inference output provided by the UE (reports configured for specific AI / ML features) can be directly used by the network, and the “recorded” UE-ER (configured by the network, as described above) can be used only for long-term monitoring purposes. Recorded UE-ERs can be used to explain performance degradation and / or trigger UE-specific model update actions. The collection of recorded UE-ERs allows for aggregation across multiple UEs (same or different vendors) and the generation of more global interpretations. For example, in AI / ML beam management, there may be possible outputs for different UE AI / ML enabled features (and the content of the reports). For example, the AI / ML model can directly predict Top-K beams (K is a positive integer), or the AI / ML model can first predict RSRP values ​​and then report Top-K beams based on the predicted RSRP values. The network will use the aforementioned set of "recorded" UE-ERs from multiple UEs to determine which type of output condition / configuration is superior to another type of output condition / configuration. The network can also use this interpretation to prioritize the activation of AI / ML features for these UEs.

[0071] For UE-ER Level 2, the inference output is combined with the "immediate" UE-ER for short-term monitoring purposes. The "immediate" UE-ER allows network decisions based on local UE-specific interpretations. For example, in AI / ML beam management, the inference output can be a beam ID, and the UE-ER can indicate the reason for providing the beam ID, such as the strongest beam and the next beam being separated by X dB. Therefore, the network can decide whether to use the reported beam ID or another beam, thereby facilitating load balancing when needed.

[0072] Figure 5 An example signaling flow 500 for trust management for ML according to some embodiments of this disclosure is shown. Signaling flow 500 can occur as follows: Figure 1 In the communication environment 100 shown. In some examples and / or embodiments, the operation of UE 110 and network element 120 may be interchangeable.

[0073] Signaling flow 500 includes operations identical or similar to those in signaling flow 200. In short, the operations in signaling flow 500 will not be detailed herein. The difference between signaling flow 500 and signaling flow 200 is that, in operation 5050, in addition to determining one or more performance indicators and / or the characteristics of one or more performance indicators, network element 120 also determines at least one request element associated with at least one inter-domain service to be used as an input metric for at least one ML-enabled feature associated with trust management. Therefore, the configuration message sent from network element 120 to UE 110 in operation 5060 also includes information about at least one request element associated with at least one inter-domain service. In some examples, at least one request element indicates the objective of the trust management report and / or at least one of the following: the method used, the dataset used, and at least one associated ML model. In some examples, the inter-domain service may be provided by a user equipment vendor server or a cloud server.

[0074] Figure 6 An example signaling flow 600 for trust management for ML according to some embodiments of this disclosure is shown. Signaling flow 600 can occur as follows: Figure 1 In the communication environment 100 shown. In some examples and / or embodiments, the operation of UE 110 and network element 120 may be interchangeable.

[0075] Signaling flow 600 includes operations identical or similar to those in signaling flow 200. In short, the operations in signaling 600 will not be detailed herein. The difference between signaling flow 600 and signaling flow 200 is that, in operation 6060, the configuration message sent from network element 120 to UE 110 additionally includes an indication of the transmission type for the trust management report. Therefore, in operation 6120, the trust management report will be sent from UE 110 to network element 120 according to the transmission type. In some examples, the indication of the transmission type for the trust management report may include an indication that assesses latency requirements associated with data collection regarding at least one ML-supported feature.

[0076] Figure 7A An example signaling flow 700A for trust management for ML is shown according to some embodiments of this disclosure. This signaling flow 700A can occur in situations such as... Figure 1 In the communication environment 100 shown. In some examples and / or embodiments, the operation of UE 110 and network element 120 may be interchangeable. Figure 7A All signaling operations between UE 110 and network element 120 as depicted can be specified and implemented as part of the access layer (AS) signaling process.

[0077] In operation 7010, network element 120 may request UE capabilities associated with a specific ML-enabled feature (configuration and conditions). In operation 7020, UE 110 may provide the requested capabilities, including supported UE-ER configurations, which may include level information. In operation 7030, network element 120 may use the UE-ER configuration along with its own capabilities, such as retrieving capacity from MEC / OAM, and the network element may configure the ER for the UE. In operation 7040, network element 120 may provide (multiple) UE-ER configurations, possibly along with other ML-enabled feature configurations. In operation 7050, UE 110 may apply ML-supporting feature configurations and initialize UE-ER generation. Operation 7050 may also include signaling procedures for applicability reporting. The UE-ER configuration in operation 7040 may be provided as a separate follow-up operation from the configuration of ML-enabled features. For example, the UE-ER configuration may be provided after operation 7050 and after the UE has indicated the applicability status of the configured ML functions / features. In operations 7060 to 7070, UE 110 can activate ML-enabled features based on the received configuration. UE 110 can perform the configured measurements and generate the configured reports. The UE can provide additional performance monitoring reports (when requested and available). In operation 7080, network element 120 can receive reports / feedback related to UE ML and make decisions based on them, such as when performance degradation is detected. Alternatively, network element 120 can notify UE 110 of triggering conditions for UE-ER and allow the UE to send an ER report as a proactive action to mitigate performance degradation. Operation 7080 can be initiated based on additional instructions received from UE 110 regarding the potential need for verification / testing of ML-enabled features following changes in UE-side conditions (radio conditions, model updates, etc.) or network-side conditions (associated ID).

[0078] This document then provides two options for network actions. For network option 1, network element 120 may decide to trigger UE-ER level 1, which may or may not include additional data pointer information available to UE 110. For network option 2, network element 120 may decide to trigger UE-ER level 2, which may not include additional data pointer information. Network option 1 and network option 2 can be combined such that, for example, network element 120 may initiate network option 2 within a given duration or number of received UE-ERs, and then initiate network option 1, or vice versa. The decision regarding how to use network option 1 and network option 2 is specific to the network implementation.

[0079] Network element 120 can decide to request a UE-ER from UE 110. In the example, performance degradation detection indicates performance changes over a longer time period (e.g., a longer UE RRC session). Therefore, as... Figure 7A As shown, network element 120 can optionally trigger UE-ER Level 1 (“Recorded” UE-ER), which can be a non-real-time report or a semi-real-time report, depending on the network element’s preferences and latency requirements in the use case. In operation 7090, network element 120 can request UE-ER Level 1, as configured in operation 7040.

[0080] Then, when selecting network option 1, two alternatives for UE actions are provided. For UE alternative 1, the server of UE 110 or its vendor can determine that it does not have any additional inference data available for the ER request from network element 110, or that it does not want to expose the data container of the ER request. For UE alternative 2, the server of UE 110 or its vendor can determine that it has additional inference data available for the ER request from network element 120. UE alternative 1 and alternative 2 can be combined such that, for example, UE 110 can select alternative 1 for a given duration or the number of UE-ERs sent, and then select alternative 2, or vice versa. The decision to use alternative 1 or alternative 2 is specific to the UE implementation, but must comply with the UE-ER configuration provided by network element 120 in operation 7040.

[0081] like Figure 7A As shown, UE 110 selects UE Alternate 1, meaning that inference data pointer information is unavailable. In operations 7100A and 7110A, UE 110 can generate a UE-ER, which includes the configured KPI type and KPI trend information but not the additional data pointer, and provide the UE-ER to network element 120. In operation 7120A, network element 120 can receive and use the UE-ER (without any additional interpretation data). The network element can then provide information to data server 130 to help update certain interpretations. This process can be based on NAS signaling, similar to the following... Figure 7B The operations described are 7110B and 7140B.

[0082] Figure 7B An example signaling flow 700B for trust management for ML is shown according to some embodiments of this disclosure. This signaling flow 700B can occur in situations such as... Figure 1 In the communication environment 100 shown. In some examples and / or embodiments, the operation of UE 110 and network element 120 may be interchangeable. Figure 7BAll signaling operations between UE 110 and network element 120 as depicted can be specified and implemented as part of the AS signaling process.

[0083] Operations 7010 to 7090 in signaling flow 700B are the same as or similar to operations 7010 to 7090 in signaling flow 700A, and will not be repeated here. However, as Figure 7B As shown, UE 110 can select UE Alternative Scheme 2, meaning that inference data pointer information is available. In operations 7100B and 7110B, an additional data pointer can be provided to UE 110 from data server 130 by an inference data server (RDS), from which the network element can extract additional information related to the interpretation included in the UE-ER. The RDS can be UE-specific or UE-free. In operations 7120B and 7130B, UE 110 can generate a UE-ER including configured KPI types and KPI trend information as well as the additional data pointer, and provide the UE-ER to network element 120. In operation 7140B, network element 120 can retrieve additional interpretation data from the provided additional data pointer from data server 130. In operation 7150B, network element 120 can receive and use the UE-ER, which includes the additional interpretation data retrieved from the provided additional data pointer. The network element can then provide information to data server 130 to help update certain interpretations. This process can be based on NAS signaling similar to that used in operating 7110B and 7140B.

[0084] Figure 7C An example signaling flow 700C for trust management for ML is shown according to some examples and / or embodiments of this disclosure. This signaling flow can occur as follows: Figure 1 In the communication environment 100 shown. In some examples and / or embodiments, the operation of UE 110 and network element 120 may be interchangeable. Figure 7C All signaling operations between UE 110 and network element 120 as depicted can be specified and implemented as part of the AS signaling process.

[0085] Operations 7010 to 7080 in signaling flow 700B are the same as or similar to operations 7010 to 7080 in signaling flow 700A, and will not be repeated here. However, as Figure 7CAs shown in the example, performance degradation detection only indicates performance changes over a short period of time (e.g., a few transmissions within a UE RRC session). Therefore, network element 120 can optionally trigger UE-ER level 2 (“immediate” UE-ER). In operation 7090C, the network element can request UE-ER level 2, as configured in operation 7040. In operation 7100C, UE 110 can use the configured ML-enabled features and generate a corresponding configuration report. In operation 7110C, UE 110 can attach UE-ER information with configured KPI type and KPI trend information to each report. In operation 7120C, network element 120 can receive and use UE-ER without any additional explanatory data. The network element can then provide information to data server 130 to help update certain explanatory data. This process can be based on NAS signaling similar to operations 7110B and 7140B.

[0086] Figure 8 This is a simplified block diagram of a device 800 suitable for implementing examples and / or embodiments of this disclosure. Device 800 may be provided to implement a communication device, such as... Figure 1 The UE 110 or network element 120 shown. As shown, the device 800 includes one or more processors 810, one or more memories 820 coupled to the processors 810, and one or more communication modules 840 coupled to the processors 810.

[0087] Communication module 840 is used for bidirectional communication. Communication module 840 has one or more communication interfaces to facilitate communication with one or more other modules or devices. The communication interface can represent any interface necessary for communication with other network elements. In some examples and / or embodiments, communication module 840 may include at least one antenna.

[0088] As a non-limiting example, processor 810 can be any type suitable for a local technology network and can include one or more of the following: general-purpose computer, special-purpose computer, microprocessor, digital signal processor (DSP), and processor based on a multi-core processor architecture. Device 800 can have multiple processors, such as application-specific integrated circuit (ASIC) chips that are time-subordinate to a clock that synchronizes the main processor.

[0089] Memory 820 may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, read-only memory (ROM) 824, electrically programmable read-only memory (EPROM), flash memory, hard disk, miniature optical disc (CD), digital video disc (DVD), optical disc, laser disc, and other magnetic and / or optical storage. Examples of volatile memories include, but are not limited to, random access memory (RAM) 822 and other volatile memories that will not persist during power-off periods.

[0090] Computer program 830 includes computer-executable instructions that are executed by an associated processor 810. The instructions of program 830 may include instructions for performing operations / actions of some examples and / or embodiments of this disclosure. Program 830 may be stored in memory (e.g., ROM 824). Processor 810 can perform any suitable actions and processes by loading program 830 into RAM 822.

[0091] Examples and / or embodiments of this disclosure may be implemented through procedure 830, such that device 800 can perform as described in the references. Figures 2 to 7C Any process discussed in this disclosure. Examples and / or embodiments of this disclosure may also be implemented by hardware or by a combination of software and hardware.

[0092] In some examples and / or embodiments, program 830 may be tangibly contained in a computer-readable medium, which may be included in device 800 (such as in memory 820) or other storage devices accessible by device 800. Device 800 may load program 830 from the computer-readable medium into RAM 822 for execution. In some examples and / or embodiments, the computer-readable medium may include any type of non-transitory storage medium, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc. As used herein, the term "non-transitory" is a limitation on the medium itself (i.e., tangible, not tactile), rather than a limitation on the persistence of data storage (e.g., RAM vs. ROM).

[0093] Figure 9 An example of a computer-readable medium 900 is shown, which may be in the form of a CD, DVD, or other optical storage disc. A program 830 is stored on the computer-readable medium 900.

[0094] Generally, the various embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software, which can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are illustrated and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0095] Some examples and / or embodiments of this disclosure also provide at least one computer program product tangibly stored on a computer-readable medium, such as a non-transitory computer-readable medium. The computer program product includes computer-executable instructions, such as those included in a program module, which execute in a device on a target physical or virtual processor to perform any of the methods described above. Typically, a program module includes routines, programs, libraries, objects, classes, components, data structures, etc., that perform a particular task or implement a particular abstract data type. In various examples and / or embodiments, the functionality of a program module can be combined or split among program modules as needed. The machine-executable instructions for a program module can execute within a local device or a distributed device. In a distributed device, the program module can reside in both local and remote storage media.

[0096] Program code used to perform the methods of this disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that, when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a stand-alone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this disclosure, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, etc.

[0098] Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable media can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media will include electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0099] Furthermore, although operations are described in a specific order, this should not be construed as requiring that such operations be performed in the specific order shown or sequentially, or that all shown operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Similarly, although several details of specific implementations have been presented in the above discussion, these details should not be construed as limiting the scope of this disclosure. Features described in the context of individual embodiments may also be implemented in combination in a single embodiment, and various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0100] Although this disclosure has been described in language specific to structural features and / or methodological actions, it should be understood that the disclosure as defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms of implementing the claims.

[0101] This disclosure includes, but is not limited to, the following example implementations.

[0102] Example 1. A user equipment, comprising: At least one processor; and At least one memory, which stores instructions that, when executed by at least one processor, enable the user device to: Receive a message from a network element requesting a capability, which is associated with the trust management of at least one machine learning-enabled feature, which is associated with an inter-domain service. A response to the ability to send notifications to network elements; Receive a configuration message for a trust management report from a network element. The configuration message includes information about one or more performance indicators and / or the characteristics of one or more performance indicators to be used as input metrics for at least one machine learning enabled feature, and information about at least one request element associated with at least one inter-domain service. The characteristics of one or more performance indicators and / or one or more performance indicators are adjusted based on the capabilities and at least one machine learning enabled feature. Receive requests for trust management reports from network components; Based on configuration messages, generate a trust management report, and Send trust management reports to network components.

[0103] Example 2. Based on the user equipment of Example 1, wherein the capability is associated with the user equipment and / or the host capability used to provide one or more performance indicators.

[0104] Example 3. Based on the user equipment of Example 1 or Example 2, one or more performance indicators and / or the characteristics of one or more performance indicators are associated with at least one of the following: User experience data rate, peak user data rate, latency, ability to successfully transmit a predefined amount of data within a predetermined duration with a specified success probability, bit error rate, block error rate, packet loss, reference signal strength indicator (RSSI), reference signal received power (RSRP), or reference signal received quality (RSRQ).

[0105] Example 4. For a user device according to any one of Examples 1 to 3, the configuration message for the trust management report further includes additional data pointer information associated with at least one of the following: Machine learning model inference speed, latency caused by computation at the inference speed of the machine learning model, machine learning-related capabilities, machine learning model capabilities for execution in unseen scenarios and / or data distributions, and whether machine learning-related capabilities are local, embedded, or network-provided.

[0106] Example 5. For a user device according to any one of Examples 1 to 4, one or more characteristics of one or more performance indicators include at least one of the following: The time-space trend and / or pattern associated with at least one of the performance indicators, the time-space window for evaluation, the radio beam domain window for evaluation, the frequency domain window for evaluation, the geospatial window for evaluation, or the aggregation level of information in the trust management report.

[0107] Example 6. A user equipment according to any one of Examples 1 to 5, wherein a configuration message is associated with a message notifying the configuration of inference for a machine learning model, the inference of the machine learning model being used for at least one machine learning enabling feature, and the user equipment is further configured to: Perform inference for at least one machine learning model, and Performance information is sent to network elements, and this performance information is obtained through inference from at least one machine learning model.

[0108] Example 7. A user device according to any one of Examples 1 to 6, wherein at least one request element indicates the objective of the trust management report and / or at least one of the following: the method used, the dataset used, and at least one associated machine learning model.

[0109] Example 8. A user equipment according to any of Examples 1 to 7, wherein inter-domain services are provided by a user equipment vendor server or a cloud server.

[0110] Example 9. A network element comprising: At least one processor; and At least one memory, which stores instructions that, when executed by at least one processor, enable network elements: A message is sent from a network element to a user equipment to request a capability, which is associated with the trust management of at least one machine learning-enabled feature, which is associated with an inter-domain service. Response to the ability to receive notifications from user equipment; Based on capabilities and at least one machine learning enabled feature, determine one or more performance indicators and / or one or more performance indicators to be used as input metrics for at least one machine learning enabled feature associated with trust management, and at least one request element associated with at least one inter-domain service. Send a configuration message for the trust management report to the user equipment. The configuration message includes information about one or more performance indicators and / or the characteristics of one or more performance indicators, and information about at least one request element associated with at least one inter-domain service; and In response to the detection of a performance degradation associated with radio network service and related to at least one machine learning-enabled feature, Send a request for a trust management report to the user equipment, and Receive trust management reports from user equipment.

[0111] Example 10. A network element according to Example 9, wherein a capability is associated with a user device and / or a host capability used to provide one or more performance indicators.

[0112] Example 11. According to the network element of Example 9 or Example 10, one or more performance indicators and / or the characteristics of one or more performance indicators are associated with at least one of the following: user experience data rate, peak user data rate, latency, ability to successfully transmit a predefined amount of data within a predetermined duration with a specified success probability, bit error rate, block error rate, packet loss, reference signal strength indicator (RSSI), reference signal received power (RSRP), and reference signal received quality (RSRQ).

[0113] Example 12. For any of Examples 9 to 11, the network element wherein the configuration message for the trust management report further includes additional data pointer information associated with at least one of the following: Machine learning model inference speed, latency caused by computation at the inference speed of the machine learning model, machine learning-related capabilities, machine learning model capabilities for execution in unseen scenarios and / or data distributions, and whether machine learning-related capabilities are local, embedded, or network-provided.

[0114] Example 13. A network element according to any one of Examples 9 to 12, wherein one or more characteristics of one or more performance indicators include at least one of the following: a trend and / or pattern in time and space associated with at least one of the performance indicators, a time and space window for evaluation, a radio beam domain window for evaluation, a frequency domain window for evaluation, a geospatial window for evaluation, and an aggregation level of information in a trust management report.

[0115] Example 14. A network element according to any one of Examples 9 to 13, wherein a configuration message is associated with a message that notifies the configuration of inference for a machine learning model, the inference of which is used for at least one machine learning enabling feature.

[0116] Example 15. A network element according to any of Examples 9 through 14, wherein the detection of performance degradation is performed by analyzing measurement data received from the user equipment.

[0117] Example 16. A network element according to any one of Examples 9 to 15, wherein at least one request element indicates the target of the trust management report.

[0118] Example 17. A method comprising: The user equipment receives a message from a network element requesting a capability, which is associated with trust management of at least one machine learning-enabled feature, which is associated with an inter-domain service. A response to the ability to send notifications to network elements; Receive a configuration message for a trust management report from a network element. The configuration message includes information about one or more performance indicators and / or the characteristics of one or more performance indicators to be used as input metrics for at least one machine learning enabled feature, and information about at least one request element associated with at least one inter-domain service. The characteristics of one or more performance indicators and / or one or more performance indicators are adjusted based on the capabilities and at least one machine learning enabled feature. Receive requests for trust management reports from network components; Based on configuration messages, generate a trust management report, and Send trust management reports to network components.

[0119] Example 18. According to the method of Example 17, the capability is associated with the user device and / or the host capability used to provide one or more performance indicators.

[0120] Example 19. According to the method of Example 17 or Example 18, one or more performance indicators and / or the characteristics of one or more performance indicators are associated with at least one of the following: User experience data rate, peak user data rate, latency, ability to successfully transmit a predefined amount of data within a predetermined duration with a specified success probability, bit error rate, block error rate, packet loss, recovery time after a fault, reference signal strength indicator (RSSI), reference received power (RSRP), or reference received quality (RSRQ).

[0121] Example 20. According to the method of any one of Examples 17 to 19, the configuration message for the trust management report also includes additional data pointer information associated with at least one of the following: Machine learning model inference speed, latency caused by computation at the inference speed of the machine learning model, machine learning-related capabilities, machine learning model capabilities for execution in unseen scenarios and / or data distributions, and whether machine learning-related capabilities are local, embedded, or network-provided.

[0122] Example 21. According to the method of any one of Examples 17 to 20, one or more characteristics of one or more performance indicators include at least one of the following: The time-space trend and / or pattern associated with at least one of the performance indicators, the time-space window for evaluation, the radio beam domain window for evaluation, the frequency domain window for evaluation, the geospatial window for evaluation, or the aggregation level of information in the trust management report.

[0123] Example 22. A method according to any one of Examples 17 to 21, wherein a configuration message is associated with a message notifying the configuration of inference for a machine learning model, the inference of the machine learning model being used for at least one machine learning enabling feature, and the method further includes: Perform inference for at least one machine learning model, and Performance information is sent to network elements, and this performance information is obtained through inference from at least one machine learning model.

[0124] Example 23. According to any one of Examples 17 to 22, at least one request element indicates the objective of the trust management report and / or at least one of the following: the method used, the dataset used, and at least one associated machine learning model.

[0125] Example 24. The method of any of Examples 17 to 23, wherein the inter-domain service is provided by a user equipment vendor server or a cloud server.

[0126] Example 25. A method comprising: A message is sent from a network element to a user equipment to request a capability, which is associated with the trust management of at least one machine learning-enabled feature, which is associated with an inter-domain service. Response to the ability to receive notifications from user equipment; Based on capabilities and at least one machine learning enabled feature, determine one or more performance indicators and / or one or more performance indicators to be used as input metrics for at least one machine learning enabled feature associated with trust management, and at least one request element associated with at least one inter-domain service. Send a configuration message for the trust management report to the user equipment. The configuration message includes information about one or more performance indicators and / or the characteristics of one or more performance indicators, as well as information about at least one request element associated with at least one inter-domain service; and In response to the detection of a performance degradation associated with radio network service and related to at least one machine learning-enabled feature, Send a request for a trust management report to the user equipment, and Receive trust management reports from user equipment.

[0127] Example 26. According to the method of Example 25, the capability is associated with the user device and / or the host capability used to provide one or more performance indicators.

[0128] Example 27. According to the method of Example 25 or Example 26, one or more performance indicators and / or the characteristics of one or more performance indicators are associated with at least one of the following: user experience data rate, peak user data rate, latency, ability to successfully transmit a predefined amount of data within a predetermined duration with a specified success probability, bit error rate, block error rate, packet loss, reference signal strength indicator (RSSI), reference signal received power (RSRP), and reference signal received quality (RSRQ).

[0129] Example 28. According to the method of any one of Examples 25 to 27, the configuration message for the trust management report also includes additional data pointer information associated with at least one of the following: Machine learning model inference speed, latency caused by computation at the inference speed of the machine learning model, machine learning-related capabilities, machine learning model capabilities for execution in unseen scenarios and / or data distributions, and whether machine learning-related capabilities are local, embedded, or network-provided.

[0130] Example 29. According to the method of any one of Examples 25 to 28, one or more characteristics of one or more performance indicators include at least one of the following: a trend and / or pattern in time and space associated with at least one of the performance indicators, a time and space window for evaluation, a radio beam domain window for evaluation, a frequency domain window for evaluation, a geospatial window for evaluation, and an aggregation level of information in the trust management report.

[0131] Example 30. A method according to any one of Examples 25 to 29, wherein a configuration message is associated with a message that notifies the configuration of inference for a machine learning model, the inference of which is used for at least one machine learning enabling feature.

[0132] Example 31. The method of any one of Examples 25 to 30, wherein the detection of performance degradation is performed by analyzing measurement data received from the user equipment.

[0133] Example 32. According to any one of Examples 25 to 31, at least one request element indicates the target of the trust management report.

[0134] Example 33. An apparatus comprising components for performing a method according to any one of Examples 17 to 32.

[0135] Example 34. A computer-readable medium including instructions stored thereon for causing a device to perform at least one of Examples 17 to 32.

Claims

1. A user equipment, comprising: At least one processor; as well as At least one memory, the at least one memory storing instructions that, when executed by the at least one processor, cause the user equipment to: Receive a message from a network element requesting a capability associated with trust management of at least one machine learning-enabled feature, which is associated with an inter-domain service. Send a response notifying the network element of the capability; The network element receives a configuration message for a trust management report, the configuration message including information about one or more performance indicators and / or the characteristics of the one or more performance indicators to be used as input metrics for the at least one machine learning enabled feature, and information about at least one request element associated with at least one of the inter-domain services, the one or more performance indicators and / or the characteristics of the one or more performance indicators being adjusted according to the capability and the at least one machine learning enabled feature; Receive a request for the trust management report from the network element; Based on the configuration message, the trust management report is generated, and Send the trust management report to the network element.

2. The user equipment of claim 1, wherein the capability is associated with the user equipment and / or a host capability used to provide the one or more performance indicators.

3. The user equipment according to claim 1 or claim 2, wherein the one or more performance indicators and / or the characteristics of the one or more performance indicators are associated with at least one of the following: User experience data rate, peak user data rate, latency, ability to successfully transmit a predefined amount of data within a predetermined duration with a specified success probability, bit error rate, block error rate, packet loss, reference signal strength indicator (RSSI), reference signal received power (RSRP), or reference signal received quality (RSRQ).

4. The user equipment according to any one of claims 1 to 2, wherein the configuration message for the trust management report further includes additional data pointer information associated with at least one of the following: Machine learning model inference speed, latency caused by computation at the inference speed of the machine learning model, machine learning-related capabilities, and machine learning model capabilities for execution in unseen scenarios and / or data distributions, wherein the machine learning-related capabilities are local, embedded, or network-provided.

5. The user equipment according to any one of claims 1 to 2, wherein the one or more characteristics of the one or more performance indicators include at least one of the following: The temporal and / or spatial trends and / or patterns associated with at least one of the one or more performance indicators, the temporal and spatial window for the evaluation, the radio beam domain window for the evaluation, the frequency domain window for the evaluation, the geospatial window for the evaluation, or the aggregation level of information in the trust management report.

6. The user equipment according to any one of claims 1 to 2, wherein the configuration message is associated with a message notifying configuration for inference of a machine learning model, the inference of the machine learning model being used for the at least one machine learning enabled feature, and the user equipment is further configured to: Perform the inference of the at least one machine learning model, and Performance information is sent to the network element, the performance information being obtained through the inference of the at least one machine learning model.

7. The user equipment according to any one of claims 1 to 2, wherein the at least one request element indicates the objective of the trust management report and / or at least one of the following: the method used, the dataset used, and at least one associated machine learning model.

8. The user equipment according to any one of claims 1 to 2, wherein the inter-domain service is provided by a user equipment vendor server or a cloud server.

9. A network element, comprising: At least one processor; as well as At least one memory storing instructions that, when executed by the at least one processor, cause the network element to: A network element sends a message to a user equipment requesting a capability associated with trust management of at least one machine learning-enabled feature, which is associated with an inter-domain service. Receive a response from the user equipment that notifies the capability; Based on the capabilities and the at least one machine learning enabled feature, determine one or more performance indicators and / or features of the one or more performance indicators to be used as input metrics for the at least one machine learning enabled feature associated with the trust management, and at least one request element associated with at least one of the inter-domain services. Send a configuration message for the trust management report to the user equipment, the configuration message including information about the one or more performance indicators and / or the characteristics of the one or more performance indicators, and information about the at least one request element associated with at least one of the inter-domain services; as well as In response to the detection of a performance degradation associated with the radio network service and related to the at least one machine learning-enabled feature, Send a request for the trust management report to the user equipment, and Receive the trust management report from the user equipment.

10. The network element of claim 9, wherein the capability is associated with the user equipment and / or host capabilities used to provide the one or more performance indicators.

Images