Devices, methods, apparatus, and computer-readable media for managing machine learning abstraction behavior

Abstract

Embodiments of the present disclosure relate to devices, methods, apparatuses and computer-readable storage media for ML abstract behavior management. A first device determines a first mapping from a network context to a set of abstract actions, the set of abstract actions representing actual actions of a machine learning entity; sends, to a second device, first information indicative of the first mapping; and receives, from the second device, second information associated with at least a second abstract action, the second abstract action corresponding to an actual action of the machine learning entity given the actual network context. The first device further monitors, based on the second information, a difference between a first abstract action determined based on the first mapping and the second abstract action.

Description

Technical Field

[0001] Various example embodiments of this disclosure generally relate to the field of communications, and more specifically, to methods, apparatuses, devices, and computer-readable storage media for managing abstract behaviors of machine learning (ML). Background Technology

[0002] In typical network operations, the operator configures and operates the ML application (APP) according to its manual. Typically, the operator knows the Configuration Management (CM) values ​​used to configure the MLApp, the CM values, Performance Management (PM) values, or Fault Management (FM) values ​​used as inputs to the MLApp to generate decisions and actions, and the PM or FM values ​​associated with the actions performed by the MLApp. However, the operator is usually unaware of the MLApp's internal decision-making details. MLApp vendors are interested in the internal aspects of their automation solutions' implementation that are hidden from the vendor. Furthermore, even when vendors are willing to expose those internal features and aspects, they often contain excessive and unnecessary detail that is not essential for the operator.

[0003] However, even without the internal details of the solution, operators need to operate the system alongside the automation solution. Specifically, operators need to instruct the MLApp solution and configure it to achieve the desired results. In some cases, the MLApp has specific actions it can take, and the operator also has operational actions it needs to take to guide the solution, such as shutting down the solution, reconfiguring the solution, and changing the solution inputs. It is necessary to match the operator's actions with the operational patterns or context of the automation solution. Summary of the Invention

[0004] In a first aspect of this disclosure, a first device is provided. The first device includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first device to at least: determine a first mapping from a network context to an abstract set of actions, the abstract set of actions representing actual actions of a machine learning entity; send first information indicating the first mapping to a second device; receive from the second device second information associated with at least a second abstract action, the second abstract action corresponding to an actual action of a machine learning entity given a real network context; and, based on the second information, monitor the difference between the first abstract action determined based on the first mapping and the second abstract action.

[0005] In a second aspect of this disclosure, a second device is provided. The second device includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second device to at least: receive first information from a first device, the first information indicating a first mapping from a network context to an abstract action set, the abstract action set representing actual actions of a machine learning entity; determine a first abstract action based on the first mapping and the actual network context used by the machine learning entity; determine a second abstract action corresponding to the actual action of the machine learning entity in a given actual network context based on a second mapping from the actual action of the machine learning entity to the abstract action set; monitor the difference between the first abstract action and the second abstract action; and send second information at least associated with the second abstract action to the first device.

[0006] In a third aspect of this disclosure, a third device is provided. The third device includes: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the third device to at least: receive a first registration request from a first device, the first registration request being used to store a first mapping from a network context to an abstract set of actions, the abstract set of actions representing actual actions of machine learning entities; and store the first mapping in association with an identifier of the machine learning entity.

[0007] In a fourth aspect of this disclosure, a fourth device is provided. The fourth device includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the fourth device to at least: receive a first message from a first device, the first message being used to initiate training of a machine learning entity based on an updated first mapping from a network context to an abstract action set, the abstract action set representing the actual actions of the machine learning entity; determine whether the training of the machine learning entity has been completed; and, based on the determination that the training has been completed, send a second message to the first device, the second message indicating a trained instance of the machine learning entity.

[0008] In a fifth aspect of this disclosure, a method is provided. The method includes: determining at a first device a first mapping from a network context to an abstract set of actions, the abstract set of actions representing actual actions of a machine learning entity; sending first information indicating the first mapping to a second device; receiving from the second device at least second information associated with a second abstract action, the second abstract action corresponding to an actual action of the machine learning entity given a real network context; and monitoring, based on the second information, a difference between the first abstract action determined based on the first mapping and the second abstract action.

[0009] In a sixth aspect of this disclosure, a method is provided. The method includes: receiving, at a second device, first information from a first device, the first information indicating a first mapping from a network context to an abstract action set, the abstract action set representing actual actions of a machine learning entity; determining a first abstract action based on the first mapping and an actual network context used by the machine learning entity; determining a second abstract action corresponding to an actual action of the machine learning entity in a given actual network context based on a second mapping from the actual actions of the machine learning entity to the abstract action set; monitoring a difference between the first abstract action and the second abstract action; and sending, to the first device, second information at least associated with the second abstract action.

[0010] In a seventh aspect of this disclosure, a method is provided. The method includes: receiving a first registration request from a first device at a third device, the first registration request being used to store a first mapping from a network context to an abstract action set, the abstract action set representing actual actions of a machine learning entity; and storing the first mapping in association with an identifier of the machine learning entity.

[0011] In an eighth aspect of this disclosure, a method is provided. The method includes: receiving, at a fourth device, a first message from a first device, the first message being used to initiate training of a machine learning entity based on an updated first mapping from a network context to an abstract action set, the abstract action set representing actual actions of the machine learning entity; determining whether training of the machine learning entity has been completed; and, based on determining that training has been completed, sending a second message to the first device, the second message indicating a trained instance of the machine learning entity.

[0012] In a ninth aspect of this disclosure, a first apparatus is provided. The first apparatus includes: means for determining a first mapping from a network context to an abstract set of actions, the abstract set of actions representing actual actions of a machine learning entity; means for sending first information indicating the first mapping to a second device; means for receiving from the second device at least second information associated with a second abstract action, the second abstract action corresponding to an actual action of the machine learning entity in a given actual network context; and means for monitoring, based on the second information, a difference between the first abstract action determined based on the first mapping and the second abstract action.

[0013] In a tenth aspect of this disclosure, a second apparatus is provided. The second apparatus includes: components for receiving first information from a first device, the first information indicating a first mapping from a network context to an abstract action set, the abstract action set representing actual actions of a machine learning entity; components for determining a first abstract action based on the first mapping and the actual network context used by the machine learning entity; components for determining a second abstract action corresponding to the actual action of the machine learning entity in a given actual network context based on a second mapping from the actual action of the machine learning entity to the abstract action set; components for monitoring a difference between the first abstract action and the second abstract action; and components for sending second information at least associated with the second abstract action to the first device.

[0014] In the eleventh aspect of this disclosure, a third apparatus is provided. The third apparatus includes: components for receiving a first registration request from a first device, the first registration request being used to store a first mapping from a network context to an abstract action set, the abstract action set representing actual actions of a machine learning entity; and components for storing the first mapping in association with an identifier of the machine learning entity.

[0015] In a twelfth aspect of this disclosure, a fourth apparatus is provided. The fourth apparatus includes: components for receiving a first message from a first device, the first message being used to initiate training of a machine learning entity based on an updated first mapping from a network context to an abstract action set, the abstract action set representing actual actions of the machine learning entity; components for determining whether training of the machine learning entity has been completed; and components for sending a second message to the first device based on the determination that training has been completed, the second message indicating a trained instance of the machine learning entity.

[0016] In a thirteenth aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to perform at least the method according to the fifth aspect.

[0017] In a fourteenth aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to perform at least the method according to a sixth aspect.

[0018] In a fifteenth aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to perform at least the method according to a seventh aspect.

[0019] In a sixteenth aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to perform at least the method according to an eighth aspect.

[0020] It should be understood that the summary portion is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] Some exemplary embodiments will now be described with reference to the accompanying drawings, in which:

[0022] Figure 1 An example communication environment in which example embodiments of this disclosure may be implemented is shown;

[0023] Figure 2 Example signaling diagrams of an ML abstract behavior management process according to some example embodiments of this disclosure are shown;

[0024] Figure 3 Another example signaling diagram of an ML abstract behavior management process according to some example embodiments of this disclosure is shown;

[0025] Figure 4 Another example signaling diagram of an ML abstract behavior management process according to some example embodiments of the present disclosure is shown;

[0026] Figure 5 An example signaling diagram also illustrates a retraining process according to some example embodiments of the present disclosure;

[0027] Figure 6A Example diagrams are shown illustrating information models used for abstract behavior when presented by artificial intelligence (AI) / ML functions, according to some example embodiments of the present disclosure;

[0028] Figure 6B Example diagrams are shown illustrating information models used for abstract behavior when represented by ML entities, according to some example embodiments of this disclosure;

[0029] Figure 6C An example diagram illustrating inheritance relationships for abstract behaviors according to some example embodiments of this disclosure is shown;

[0030] Figure 7 A flowchart is shown illustrating a method implemented at a first device according to some example embodiments of the present disclosure;

[0031] Figure 8 A flowchart illustrating a method implemented at a second device according to some example embodiments of the present disclosure is shown;

[0032] Figure 9 A flowchart is shown illustrating a method implemented at a third device according to some example embodiments of the present disclosure;

[0033] Figure 10A flowchart is shown illustrating a method implemented at a fourth device according to some example embodiments of the present disclosure;

[0034] Figure 11 A simplified block diagram of a device suitable for implementing example embodiments of the present disclosure is shown; and

[0035] Figure 12 A block diagram of an example computer-readable medium according to some example embodiments of the present disclosure is shown.

[0036] Throughout the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0037] The principles of this disclosure will now be described with reference to some exemplary embodiments. It should be understood that these embodiments are described for illustrative purposes only and to assist those skilled in the art in understanding and implementing this disclosure, and do not imply any limitation on the scope of this disclosure. The embodiments described herein can be implemented in various ways other than those described below.

[0038] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0039] References to "an embodiment," "embodiment," "example embodiment," etc., in this disclosure indicate that the described embodiments may include specific features, structures, or characteristics, but not every embodiment needs to include such specific features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in connection with an embodiment, it should be noted that those skilled in the art will recognize that such feature, structure, or characteristic can be implemented in combination with other embodiments, whether explicitly described or not.

[0040] It should be understood that although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0041] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, wherein a list of two or more elements combined with “and” or “or” means at least one of the elements, or at least any two or more of the elements, or at least all of the elements.

[0042] As used herein, unless explicitly stated otherwise, the “responding to A” execution step does not indicate that the step is executed immediately after “A” occurs, and may include one or more intermediate steps.

[0043] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms “comprising,” “including,” “having,” “having,” “containing,” and / or “comprising” as used herein specify the presence of the stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0044] As used in this application, the term "circuit system" may refer to one or more or all of the following:

[0045] (a) Hardware circuit implementation only (such as implementation in analog and / or digital circuits only), and

[0046] (b) A combination of hardware circuitry and software, such as (if applicable):

[0047] (i) A combination of (multiple) analog and / or digital hardware circuits and software / firmware, and

[0048] (ii) Any part of a hardware processor(s) having software (including (multiple) digital signal processors, software, and (multiple) memories, which work together to cause a device (such as a mobile phone or server) to perform various functions), and

[0049] (c) (Multiple) hardware circuits and / or (multiple) processors, such as (multiple) microprocessors or a portion thereof, which require software (e.g., firmware) for operation, but the software may be absent when it is not required for operation.

[0050] This definition of circuit system applies to all uses of the term in this application (including any claims). As another example, as used in this application, the term circuit system also covers implementations of only hardware circuitry or processors (or processors) or portions thereof and their accompanying software and / or firmware. For example, and if applicable to a particular claim element, the term circuit system also covers baseband integrated circuits or processor integrated circuits for mobile devices or similar integrated circuits in servers, cellular network devices or other computing or networking devices.

[0051] As used herein, the term "communication network" refers to a network that conforms to any suitable communication standard, such as New Radio (NR), Long Term Evolution (LTE), LTE-A Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed ​​Packet Access (HSPA), Narrowband Internet of Things (NB-IoT), etc. Furthermore, communication between terminal devices and network devices in a communication network can be performed according to any suitable generation of communication protocol, including but not limited to first-generation (1G), second-generation (2G), 2.5G, 2.75G, third-generation (3G), fourth-generation (4G), 4.5G, fifth-generation (5G) communication protocols and / or any other currently known or future-developed protocols. Embodiments of this disclosure can be applied to a variety of communication systems. Given the rapid development of communications, there will certainly be future types of communication technologies and systems that embody the future types of this disclosure. It should be understood that the scope of this disclosure is not limited to the foregoing systems.

[0052] It should be noted that any section / subsection headings provided herein are not intended to be limiting. Embodiments are described throughout this document, and any type of embodiment may be included under any section / subsection. Furthermore, embodiments disclosed in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0053] As used herein, the terms “training of ML entities” or “retraining of ML entities” may refer to the training or retraining of an ML entity or an ML model associated with an ML entity.

[0054] As briefly mentioned above, operators need to operate the system in conjunction with the MLApp's automation solution. Specifically, operators need to guide and configure the solution to achieve the desired results. For example, Table 1 illustrates three automation use cases: an MLApp that determines how users should be assessed for fire evacuation from a building; an autonomous driving use case (also known as "Robocar") that determines how to autonomously drive to a given location; and a load balancing (AutoLB) MLApp that determines how to distribute the load among networked objects. In all these cases, the MLApp has specific actions it can take, and the operator has operational actions it needs to take to guide the solution, such as shutting down the solution, reconfiguring the solution, and changing the solution inputs.

[0055] Table 1: Operability of Automation Solutions

[0056]

[0057]

[0058] Network automation functions (and other automation functions in general) typically do not expose detailed knowledge of the internal behavior of the automation function. However, the operational / operability actions that need to be taken by the operator (or at least require) are related to the internal actions and context considered in the decision-making process of the automation function.

[0059] Taking a fire evacuation use case as an example, the operator's decision to request a planned westward exit is linked to the knowledge of whether a westward / nearby door exists. In this case, assume there is no westward door and the solution does not consider other available doors. If the operator requests a westward exit, the solution might send people towards a wall because it only considers the building layout and not the exits. On the other hand, if the operator knows that the solution does not consider available exits and knows that a northwest exit exists, the operator could alternatively request a northward route, thus increasing the chances that people will be sent in a direction that facilitates exiting the building.

[0060] Exposing the internal context to the operators is beneficial, allowing them to set appropriate decisions. However, revealing relationships at this level would expose the business secrets of the MLApp. For example, for an MLApp employing model-free Q-learning, the relationship is the learned state-action policy within the MLApp.

[0061] Therefore, it is necessary to correlate the operator's actions with the internal, but abstract, context considered by the automation solution. The automation solution can find the best possible path out of the building, or request the operator to reconsider their actions if they lead to a dead end. For the operator to use the ML model or solution appropriately, the supplier needs to provide some minimum information about the model / solution's functionality. Sharing models / solutions between different parties (between different suppliers, between suppliers and operators, etc.) is generally understood as a method for addressing different use cases; for example, in mobility optimization, the model / solution can be shared between the gNB and the UE. It is crucial that this sharing be achieved without revealing the model provider's proprietary internal information, while enabling the consumer of the model / solution to utilize it appropriately and / or control / guide their behavior in the desired direction.

[0062] However, without knowledge of the model / solution details, operators may struggle to understand the overall behavior of the MLApp. Furthermore, if a part of the MLApp's decision / action is not preferred by the operator, the operator may not know how to instruct the MLApp to behave accordingly. Solutions are needed to enable operators to correlate operational actions with the AI / ML context of the MLApp, as well as to guide the implementation of AI / ML functionalities or automated solutions.

[0063] To address the above and other potential problems, exemplary embodiments of this disclosure propose a method for enabling AI / ML Management Service (MnS) consumers to utilize MLApps in a manner that allows them to control the behavior of MLApps in a preferred direction without knowing the internal details of the MLApp.

[0064] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0065] Example environment and working principle

[0066] Figure 1 An example communication environment 100 in which exemplary embodiments of the present disclosure may be implemented is shown. In the communication environment 100, there are a first device 110, a second device 120, and an ML entity 130.

[0067] The first device 110 can be a managed service (MnS) consumer, such as an AI / ML MnS consumer. In some example embodiments, the AI / ML MnS consumer can be an Operations Management and Maintenance (OAM) or Network Function (NF) function.

[0068] The second device 120 can be an MnS producer, such as an AI / ML MnS producer. In some example embodiments, the AI / ML MnS consumer can be a gNB / CU, another NF or OAM function different from the first device 110, and an ML entity (e.g., MLApp) is executed in the AI / ML MnS consumer.

[0069] ML entity 130 is associated with the second device 120. ML entity 130 may be an ML model, or it may contain ML models and metadata related to the ML model. ML entity 130 may be managed as a single composite entity. In some example embodiments, ML entity 130 may be implemented as an MLApp. It should be understood that this is for illustrative purposes only and does not imply any limitation on the embodiments of this disclosure.

[0070] According to example embodiments of this disclosure, a solution for trusted operation of ML entity 130 based on AI / ML context abstraction is proposed. In some example embodiments, the first device 110 may be an operator or an operator's management function (MnF), and may be implemented as, for example, an AI / ML MnS consumer. The second device 120 may act as a producer of management services, providing management services based on ML entity 130, and is referred to herein as an AI / ML management service producer or an AI / ML MnS producer.

[0071] The second device 120 can notify the first device 110 (e.g., the operator) of the abstract behavior of the ML entity 130 in a manner unknown to the ML entity, without exposing the internal characteristics of its ML entity 130 or AI / ML functions. The abstract behavior of the ML entity 130 may include abstract states representing the actual states associated with the ML entity 130 and abstract actions representing the actual actions of the ML entity 130.

[0072] The second device 120 enables the first device 110 (e.g., an authorized AI / ML MnS consumer (e.g., an operator)) to configure the behavior of the ML entity 130 in a manner agnostic to the ML entity, without exposing its internal characteristics. It allows the first device 110, acting as a management service consumer of the ML entity 130, to configure, manage, or guide the operation of the ML entity 130 through a set of abstract states and abstract actions. The ML entity 130 can then make its actions or decisions based on the operation of this management service consumer.

[0073] In some example embodiments, the second device 120 may have a set of candidate abstract states that can be notified to the first device 110. The first device 110 can configure abstract behavior by selecting an action to be taken in any of the abstract states.

[0074] The context and state / action of the second device 120 can be divided into an operating mode represented by an abstract state understood by both the first device 110 and the second device 120.

[0075] For example, a Robocar can be considered to have several (e.g., two) abstract states: a normal operating state and an external environment state. In the normal operating state, the Robocar can be given a destination and is permitted to act as it wishes. In the external environment state, representing unusual conditions such as an accident on the road ahead (e.g., learned from radio), there might be unusual street conditions, such as an unusually slippery street due to water splashing onto the street from a pipe, or a street power line bending into the road. In these situations, the operator's actions can be different, for example, requiring the car to stop suddenly or turn abruptly.

[0076] Similarly, for reinforcement learning (RL) solutions on load balancing, different RL state-action pairs can be mapped to different operating modes, which then become the abstract states of the automation solution.

[0077] The abstract state may need to be agreed upon between the second device 120 (represented by the solution provider) and the first device 110 (e.g., the solution operator). For example, the abstract state could be a standardized set of abstract states agreed upon among multiple potential developers and operators.

[0078] The expected number of abstract states can vary depending on the use case, but is typically small. The expected number can be normalized to a small value, but large enough to support most use cases (e.g., a set of states numbered 0-15 or 0-63).

[0079] The candidate set of abstract states and the possible actions in any such state can be set by the second device 120 and can be notified to the first device 110. The notification of the abstract state may also include defining the characteristics of the corresponding abstract state.

[0080] The second device 120 may allow the first device 110 to specify a subset of abstract states from a candidate set of abstract states that can be applied to the ML entity 130 providing management services. The operator or the first device 110 may decide how to derive the subset of abstract states from the features and feature values ​​that define the abstract states.

[0081] The first device 110 can define an abstract set of actions that maps to abstract states also defined by the first device 110. Use cases may require fewer states than the standardized set; that is, the first device 110 can set fewer abstract states than the already standardized number. In this case, only the required states are mapped, while unmapped states can take default actions, such as "NoAction".

[0082] The second device 120 may have a mapping function that maps the internal context and actions of the second device 120 to an abstract set of actions defined by the first device 110.

[0083] The mapping function can be a defined set of rules, or an ML mapping function to be trained by the second device 120 (or its supporting functions) to learn the mapping between the internal context and states / actions of the second device 120 and the defined set of abstract actions of the first device 110. The first device 110 can configure a specific abstract state ID as a specific abstract action itself. Such a configuration is then passed to the second device 120.

[0084] It should be understood that actual operation may require more states than the operators set. For example, in the Robocar use case, to limit the number of abstract states, the first device 110 may define a single abstract state called the "external environment," which actually aggregates multiple smaller internal states within the second device 120.

[0085] In all cases, the second device 120 maps its input context to the abstract action seen by the first device 110 using the mapping function given by the first device 110. The second device 120 takes an internal action for its input context. The second device 120 also maps its internal actions to the abstract actions it sees using its internal mapping function and compares whether the abstract actions of the two mappings are the same.

[0086] The first device 110 can observe the overall behavior of the second device 120 by monitoring the abstract actions exhibited by the second device 120 during operation. If it is necessary to retrain the ML entity 130 (triggered by the second device 120, the first device 110, or other entities), the ML entity 130 can be retrained at one of the OAM / network entities configured by the operator.

[0087] The above reference Figure 1 The general process is described. For a better understanding of the ML abstract behavior management in this disclosure, please refer to the following: Figures 2 to 5 Some example implementations are described.

[0088] In the following description, for illustrative purposes, some example embodiments are described in which the first device 110 operates as an MnS consumer, the second device 120 operates as an MnS producer, and the ML entity 130 is implemented as an MLApp. However, in some example embodiments, the operations described in connection with the first device 110 can be implemented on devices other than the MnS consumer, and the operations described in connection with the second device 120 can be implemented on devices other than the MnS producer.

[0089] Communication in communication environment 100 can be implemented according to any suitable communication protocol(s), including but not limited to cellular communication protocols such as first-generation (1G), second-generation (2G), third-generation (3G), fourth-generation (4G), fifth-generation (5G), and sixth-generation (6G), wireless local area network communication protocols such as IEEE 802.11, and / or any other currently known or future-developed protocols. Furthermore, communication can utilize any suitable wireless communication technology, including but not limited to: Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Frequency Division Duplex (FDD), Time Division Duplex (TDD), Multiple-Input Multiple-Output (MIMO), Orthogonal Frequency Division Multiple Access (OFDM), Discrete Fourier Transform Extended OFDM (DFT-s-OFDM), and / or any other currently known or future-developed technologies.

[0090] Example general process for managing abstract behaviors in ML

[0091] Figure 2Example signaling diagrams of an ML abstract behavior management process 200 according to some example embodiments of this disclosure are shown. For discussion purposes, reference will be made to... Figure 1 For example, process 200 can be discussed by using the first device 110 and the second device 120.

[0092] As shown in management process 200, first device 110 determines (205) a first mapping from network context to an abstract set of actions, which represents the actual actions of ML entity 130, and sends (210) first information indicative of the first mapping to second device 120. The network context may include any suitable attributes of the communication network. For example, the network context may include CM, PM, and FM values. Furthermore, the network context may include other types of management data, such as tracking.

[0093] In some example embodiments, the first mapping determined by the first device 110 may include two parts. A first part (hereinafter also referred to as the "third mapping") maps the network context to a first set of abstract states. A second part (hereinafter also referred to as the "fourth mapping") maps the first set of abstract states to the set of abstract actions. As an example, the third mapping may be implemented as an input state abstract function that maps CM / PM / FM attributes input by the ML entity 130 to abstract states, and this input state abstract function is defined by the first device 110 according to the ML entity. The fourth mapping may be implemented as a control action abstract function for mapping abstract states to abstract actions. This control action abstract function may also be defined by the first device 110 according to the ML entity.

[0094] In some example embodiments, the first abstract state set may be a subset of the second abstract state set. In other words, the second abstract state set may be as described in the reference. Figure 1 The aforementioned set of candidate abstract states. The first device 110 specifies or selects a first set of abstract states to be applied from the set of candidate abstract states. In the following text, the abstract states in the first set of abstract states may also be referred to as the applied abstract states, and the abstract states in the second set of abstract states may also be referred to as candidate abstract states. It should be understood that the operator and provider of the ML model of ML entity 130 can agree on the abstract state space and abstract actions for the ML model.

[0095] In some example embodiments, abstract behaviors may be associated with ML entity 130 or functions associated with ML entity 130. An abstract behavior may include one or more abstract states and corresponding abstract actions. Abstract states in a second set of abstract states, or in other words, candidate abstract states, may be associated with an identifier of that abstract state, a description of that abstract state, at least one abstract action available under that abstract state, etc.

[0096] As an example, to implement AI / ML abstract behavior, an ML entity or AI / ML function can have objects called abstract behaviors, which include the characteristics of the abstract behavior of the ML entity or AI / ML function. Abstract behaviors include two lists: a list of candidate abstract states and their candidate actions, and a list of selected and configured abstract states and their corresponding selected actions.

[0097] To this end, an Information Object Class (IOC) or data type, referred to as "abstractBehavior," can be introduced. In some example embodiments, such as those referenced... Figure 6A The description states that IOC can be a name contained in an ML entity or AI / ML function. An abstract behavior can have two attributes: "candidate abstract states" and "applied abstract states".

[0098] A candidate abstract state is a list of abstract states, and each state has a list of candidate abstract actions for that abstract state. Therefore, a data type called "CandidateAbstractState" can be introduced. Each state in a candidate abstract state can have an identifier, a human-readable description, and a list of possible actions that can be selected for that abstract state. Thus, a candidate abstract state can have an attribute for possible actions, called "possibleActions," which holds the possible actions for that state. The possibleActions attribute can be an enumeration of actions from which the Mns consumer can select the action to be applied. The abstract state to be applied is a list of state-action tuples. Each state can be represented by the identifier of the corresponding state listed in the candidateAbstractBehavior. Similarly, each action can be represented by the identifier of the corresponding action listed in the candidateAbstractBehavior.

[0099] Continuing in process 200, the second device 120 receives (215) first information from the first device 110. The second device 120 determines (220) a first abstract action based on the first mapping and the actual network context used by the ML entity 130. For example, the ML entity 130 can perform an actual action by using a given actual network context as input to the ML model. The second device 120 can then map the given actual network context to the first abstract action according to the first mapping. The first abstract action can be considered as an abstract action seen by the operator.

[0100] The second device 120 also determines (225) a second abstract action corresponding to the actual action of the ML entity 130 in a given actual network context, based on a second mapping from the actual actions of the ML entity 130 to a set of abstract actions. The second mapping is an internal mapping of the second device 120 and is not known to the first device 110. The second mapping can be defined by the provider of the ML model. For the example above, the second device 120 can map actual actions to a second abstract state based on the second mapping. The second abstract action can be considered an abstract action seen by the ML entity, for example, an abstract action seen by the MLApp.

[0101] Then, the second device 120 monitors (230) the difference between the first abstract action and the second abstract action. For example, the second device 120 may compare the first abstract action and the second abstract action to determine whether there is any conflict between the first abstract action and the second abstract action.

[0102] The second device 120 sends (235) second information, at least related to the second abstract action, to the first device 110. The first device 110 receives (240) the second information from the second device 120. Accordingly, the first device 110 monitors (245) the difference between the first abstract action and the second abstract action determined based on the first mapping, based on the second information.

[0103] The second information may include any suitable type of information from which the first device 110 can determine the difference. In some example embodiments, the second information may include an indication of a second abstract action. Accordingly, the first device 110 may determine a first abstract action on its own side based on a first mapping. The first device 110 may then compare the first abstract action and the second abstract action and monitor the difference.

[0104] Alternatively or additionally, in some example embodiments, the second information may include an indication of a first abstract action and a second abstract action. Accordingly, the first device 110 may compare the first abstract action and the second abstract action and monitor the difference. Alternatively or additionally, in some example embodiments, the second information may include an indication of whether a difference exists between the first abstract action and the second abstract action. Accordingly, the first device 110 may directly monitor the difference based on this indication.

[0105] The general process 200 has been described above. More details of example use cases will now be discussed in the following example embodiments of this disclosure. In these example embodiments, for illustrative purposes, the first device 110 may be referred to as, for example, an operator, an MnS consumer, an AI / ML MnS consumer, etc., and the second device 120 may be referred to as, for example, a supplier, an MnF producer, an AI / ML MnS producer, etc. As for the ML entity 130, it may also be referred to as an MLApp. It should be understood that this is only for discussion purposes and is not intended to imply any limitation.

[0106] In some example embodiments, the relevant abstract states and abstract actions may be standardized or defined by the first device 110 (e.g., the operator), but are also known to the second device 120 (e.g., the MLApp provider). Therefore, when interacting with abstract states and abstract actions, the first device 110 and the second device 120 can understand each other. The semantics of abstract states and abstract actions are typically use case or MLApp-specific (e.g., self-organizing network functionality), and they all share the same principles. Therefore, the two sets of IDs can be sufficient to be standardized to identify any number of finite abstract states and abstract actions. For example, Table 2 presents the abstract states and corresponding abstract actions of an MLApp for the mobility load balancing use case. Here, abstract actions may be associated with real actions such as switchover-triggered updates and need to be known to both the first device 110 and the second device 120.

[0107] Table 2

[0108]

[0109] Similarly, for further use cases (e.g., switchover optimization), the set of abstract states and actions can be provided in Table 3. Table 3 shows example abstract states and actions of the MLApp for the switchover optimization use case, where the complete table needs to be known by both the second device 120 and the first device 110.

[0110] Table 3

[0111]

[0112] It should be understood that the "*" in Tables 2 and 3 indicates that the first device 110 and the second device 120 have the same understanding of the abstract action defined by the first device 110. The first device 110 may also define the maximum step size value that the abstract action can take.

[0113] As can be seen from the above description, trusted MLApp operation is based on relevant abstract states and abstract actions defined jointly by the first device 110 and the second device 120 or standardized for MLApp. In some example embodiments, three mapping functions can be used: an input state abstract function that maps from the actual network context (the input of the ML model) to the abstract state; a control action abstract function that maps from the abstract state to the abstract action; and an MLApp action abstract function that maps from the actual actions / decision of the MLApp to the abstract actions seen by the MLApp.

[0114] The first device 110 indirectly controls the actual decisions / actions of the MLApp by providing the second device 120 with an input state abstract function and a control action abstract function. The second device 120 receives these two mapping functions from the first device 110. These two mapping functions map actual CM / PM / FM values ​​(using CM / PM / FM as an example, other types of management data, such as tracking) and / or other context values ​​related to the MLApp to abstract states and abstract actions seen by the operator. The MLApp action abstract function is unknown to the first device 110 and maps the CM / decision values ​​generated by the MLApp to the abstract actions seen by the MLApp, and presents the abstract actions seen by the MLApp to the first device 110 if requested.

[0115] The first device 110 can observe the overall behavior of the MLApp by monitoring the abstract actions seen by the MLApp during operation. That is, the overall behavior of the MLApp is effectively shown using all the abstract actions displayed by the MLApp. Given a specific real-world network context input (e.g., a real CM / PM / FM, etc.) to the MLApp, if the corresponding abstract action displayed by the MLApp differs from the abstract action mapped from the same input having an input state abstract function and a control action abstract function, then the MLApp's action / decision conflicts with the corresponding abstract action given by the first device 110. The MLApp can be retrained to align with the corresponding abstract actions configured by the operator when needed.

[0116] Any suitable mapping function can be used. In some example embodiments, as described above, input state abstraction functions, control action abstraction functions, and MLApp action abstraction functions can be used. Hereinafter, input state abstraction functions and control action abstraction functions can be collectively referred to as abstract mappings.

[0117] Input state abstract function (also represented as) F actual2asThis, discussed above as the third mapping, maps the CM / PM / FM attributes of the MLApp input (along with other types of administrative data, such as tracking, using CM / PM / FM as examples and / or other contextual values) to an abstract state defined by the operator according to MLApp. F actual2as It can be the ML function itself, which can learn the "optimal" mapping itself. For example, it can initially learn based on the given data from the supplier, and when in service, it learns based on the given data from the operator. The expression (1) below illustrates this. F actual2as Example mapping:

[0118]

[0119] Abstract function for controlling actions (also represented as) F as2aa This is discussed above as the fourth mapping, which maps abstract states to abstract actions. The control action abstract function can be defined by the operator according to MLApp (i.e., only by the operator from use case-specific tables of abstract states and abstract actions, such as Table 2 or Table 3). The expression (2) below illustrates this. F as2aa Example mapping:

[0120]

[0121] MLApp action abstract function (also represented as F ra2aa This maps the actual actions determined by the MLApp (the MLApp's actual CM / decision value) to the abstract actions determined / seen by the MLApp. It is MLApp-specific and initially defined by the MLApp provider, while the MLApp itself can be updated by the operator during MLApp operation. F actual2as and F as2aa The mapping function is then retrained internally. After the provider obtains the set of abstract actions defined / approved by the operator for MLApp, the MLApp provider can define this function to map from the actual actions of MLApp to the abstract actions, as follows:

[0122]

[0123] in F ra2aa This is known only to the second device 120 and is detached from access to the first device 110. Furthermore, such actual actions are determined based on the internal states known only to the second device 120. Abstract Actions "abstractAction （ * (Parameter: Value) Both the first device 110 and the second device 120 are known.

[0124] Then, the supplier can provide the defined [device] 120. F ra2aa And the F ra2aa It becomes part of the second device 120.

[0125] In this way, the second device 120 is based on two mappings. F actual2as and F as2aa It knows how to map from real input values ​​to abstract states and abstract actions. Additionally, the second device 120 knows how to base its mapping on internal mappings. F ra2aa The MLApp maps real actions to abstract actions it sees. Then, the MLApp is ready for operators to use. The MLApp determines its real actions / decision based solely on the real input values ​​to the MLApp.

[0126] Sometimes, the second device 120 may detect based on F actual2as and F as2aa The abstract actions of mapping are different from those based on mapping functions. F ra2aa The abstract action seen by the mapped MLApp. This situation may be due to the operator updating the mapping provided to the second device 120. F as2aa The conflict arises from an abstract action. This discrepancy indicates a conflict between the MLApp's actual action and the abstract action provided by the operator, corresponding to the app's actual input value. In this case, the MLApp can act according to the operator's given strategy for such a conflict. For example, the MLApp can abandon the conflicting actual action or alternatively take a non-conflicting actual action. The second device 120 can report the conflict or statistics about the conflict to the first device 110 (within a time period / range). The second device 120 or the first device 110 can then adjust the data based on updates from the operator. F as2aa Request retraining of MLApp. Mapping function. F ra2aa It can be updated during MLApp retraining. See below for reference. Figure 5 Describe the retraining process.

[0127] Using the example embodiments of this disclosure, a supplier can make an MLApp (e.g., an RL-based app) exhibit behavior and allow an operator to control its behavior at an abstract level without revealing any detailed state or internal design of the app to the operator. In this way, not only can the MLApp be controlled by the operator, but the intellectual property and commercial interests of the supplier of the MLApp can also be protected from the operator's influence.

[0128] Simplification is another key advantage. Using the example embodiments of this disclosure, operators can set constraints in a simplified manner, test compliance, and / or enforce them. Operators can also test conditions they deem important or necessary to gain insight into how the MLApp handles them, thereby building trust / confidence in the MLApp.

[0129] MLApp can also measure statistics on conflict occurrences over time within a given network scope after state-action pairs are updated. The criterion for retraining MLApp will be set, for example, that more than 5% of decisions conflict with abstract actions set by the operator for MLApp.

[0130] Example initialization process for managing abstract behaviors in ML

[0131] Figure 3 Another example signaling diagram of an ML abstract behavior management process 300 according to some example embodiments of this disclosure is shown. Reference will be made to this diagram for discussion purposes. Figure 1 The process 300 is discussed in relation to the first device 110, the second device 120, and a third device 301, which may be, for example, a repository. In some example embodiments, the third device 301 may be implemented as a repository function at the core network or a function at the OAM, for registering configuration files / metadata of ML entity instances. For example, process 300 may be a process of installing and activating an MLApp instance (such as MLApp1) along with the mapping described above.

[0132] In example procedure 300, compared with reference Figure 2 Similar to the description, the first device 110 determines (205) a first mapping from the network context to an abstract set of actions representing the actual actions of the ML entity 130. For example, the first device 110 may receive a request to install MLApp1. In response to this request, the first device 110 may generate an action for MLApp1 based on the network context. F actual2as and F as2aa Mapping instances.

[0133] First device 110 may send an instantiation request (310) to second device 120 to instantiate ML entity 130 using a first mapping. The instantiation request includes first information indicating the first mapping. For example, the instantiation request could be a request to install the provisioning management service (ProvMnS) for MLApp1. The ProvMnS request could include the APP-ID of MLApp1 and... F actual2as and F as2aa .

[0134] The second device 120 may receive an instantiation request (315) from the first device 110. In response to the instantiation request, the second device 120 may use the first mapping to instantiate (320) the ML entity 130. For example, in response to a ProvMnS request, the second device 120 may use... F actual2as and F as2aa Install MLApp1.

[0135] Then, the second device 120 can send (335) an instantiation response indicating that the instantiation of ML entity 130 is complete. The first device 110 can receive (340) the instantiation response from the second device 120. For example, the instantiation response can be a ProvMnS response indicating the installation of MLApp1, which may include the APP-ID of MLApp1.

[0136] In some example embodiments, the first device 110 may send an activation request (345) to the second device 120 to activate the ML entity 130. Upon receiving the activation request (350), the second device 120 may send an activation response (355) to the first device 110 to indicate the activity status of the ML entity 130. The first device 110 may receive an activation response (360) from the second device 120. For example, the activation request may be a ProvMnS request to activate MLApp1. The ProvMnS request may include the APP-ID of MLApp1. Accordingly, the activation response may be a ProvMnS response indicating that the service has been activated.

[0137] Additionally, in some example embodiments, the first device 110 may send (365) a first registration request to the third device 301 to store a first mapping for ML entity 130. Upon receiving (370) the first registration request, the third device 301 stores (375) the first mapping in association with the identifier of ML entity 130. For example, the first registration request may request the third device 130 to store a mapping for MLApp1. F actual2as andF as2aa The first registration request may include the APP ID of MLApp1. Accordingly, the third device 130 may store the APP ID in association with it. F actual2as and F as2aa .

[0138] Example mapping update process for ML abstract behavior management

[0139] The first mapping can be checked and updated whenever needed. For example, the second device 120 can trigger the check and update when it detects a conflict(s) between the first and second abstract actions. Similarly, the first device 110 can trigger the check and update when it notices unexpected behavior from the ML entity 130. An example procedure for checking and updating the first mapping will be referenced. Figure 4 discuss.

[0140] Figure 4 Further example signaling diagrams of an ML abstract behavior management process 400 according to some example embodiments of this disclosure are shown. For discussion purposes, reference will be made to... Figure 1 The process 400 is discussed in relation to the first device 110, the second device 120, and a third device 301, which may be, for example, a storage repository. In some example embodiments, the third device 301 may be implemented as a storage repository function at the core network or a function at the OAM, for registering configuration files / metadata for MLApp instances. As an example, process 400 may be a process for reviewing and updating abstract mappings for MLApp instances (such as MLApp1).

[0141] In some example embodiments, the checking and updating of the first mapping can be triggered by the second device 120, for example, by the ML entity 130 or the MnS producer. In such example embodiments, as Figure 4 As shown, the second device 120 can send an inspection request (402) to the first device 110 to inspect the first mapping for ML entity 130. If a difference is detected between the first abstract action and the second abstract action, the second device 120 can send the inspection request. For example, the inspection request may include the APP ID of MLApp1 and the mapping for MLApp1. F actual2as and F as2aa The first device 110 can receive a (404) inspection request from the second device 120.

[0142] Alternatively, in some example embodiments, the checking and updating of the first mapping may be triggered by the first device 110, for example, by the MnS consumer. For example, if a difference between a first abstract action and a second abstract action is detected, the first device 110 may trigger the checking and updating of the first mapping. In such an example embodiment, the first device 110 may send a (406) retrieval request to the third device 301 to retrieve the first mapping for ML entity 130. In response to receiving the (408) retrieval request, the third device 301 may send a (410) retrieval response to the first device 110 indicating the first mapping for ML entity 130. For example, the retrieval request may be a request for the current version of the abstract mapping for MLApp1. The retrieval request may include the current version for MLApp1. F actual2as and F as2aa The response.

[0143] In response to a trigger, the first device 110 can update the first mapping by inspecting at least a portion of it. The first device 110 can then send an update request to the second device 120, indicating an update to the first mapping. The second device 120 can update the first mapping accordingly and can send an update response to the first device 110, indicating that the update to the first mapping is complete.

[0144] As mentioned above, the first mapping may include a third mapping (such as...) F actual2as ) and the fourth mapping (such as F as2aa In some example embodiments, the first device 110 may check (414) the third mapping. The first device 110 may update (418) the third mapping and send (420) an update request to the second device 120 indicating the update of the third mapping. After receiving (422) the update request from the first device 110, the second device 120 may update (424) the third mapping locally. Then, the second device 120 may send (426) an update response for the third mapping to the first device 110 to indicate that the third mapping has been updated. Accordingly, the first device 110 may receive (428) an update response from the second device 120. For example, the mapping function F actual2as It can be viewed by the first device 110, and F actual2as At least a portion of it can be updated by the first device 110. The ProvMns request, as an update request, may include the updated... F actual2as And the APP ID of MLApp1. The second device 120 can update the mapping function locally.F actual2as And send a ProvMns response to the first device 110.

[0145] Alternatively or additionally, the first device 110 may inspect (416) the fourth mapping. After inspecting (416) the fourth mapping, the first device 110 may update (430) the fourth mapping and send (432) an update request for the fourth mapping to the second device 120. After receiving (434) the update request from the first device 110, the second device 120 may update (436) the fourth mapping locally. Then, the second device 120 may send (426) an update response for the fourth mapping to the first device 110 to indicate that the fourth mapping has been updated. The first device 110 may receive (440) an update response from the second device 120. For example, a mapping function F as2aa The mapping for one or more abstract states can be viewed by the first device 110, and can be updated by the first device 110. The ProvMns request, as an update request, may include the IDs of one or more abstract states and the updated mapping for those abstract states. The second device 120 can locally update the mapping for one or more abstract states and send a ProvMns response to the first device 110.

[0146] Additionally, in some example embodiments, the first device 110 may send a (442) registration request to the third device 301 to store an updated first mapping for ML entity 130. In response to receiving the (444) registration request, the third device 301 may store the updated first mapping in association with an identifier of ML entity 130. For example, the registration request may include the APP ID of MLApp1 and the current version of MLApp1. F actual2as and F as2aa .

[0147] Example retraining process for ML abstract behavior management

[0148] In the example embodiments of this disclosure, ML entity 130 can be retrained in various ways. Figure 5 An example retraining process 500 according to some example embodiments of this disclosure is illustrated. For the purposes of discussion, reference will be made to, for example... Figure 1 The process 500 is discussed using the first device 110, the second device 120, and the fourth device 501 related to machine learning training or model training.

[0149] If needed, the first device 110 or the second device 120 can trigger retraining of the ML entity 130. In some example embodiments, retraining can be triggered by the second device 120. In such example embodiments, such as Figure 5 As shown, the second device 120 can send (502) a training request to the first device 110 for training the ML entity 130. The first device 110 can receive (504) the training request from the second device 120 and execute the training process. When retraining is triggered, the second device 120 can also provide a reason in the training request. For example, the training request may include a reason indication such as "mapping update" or "too many conflicts".

[0150] In some example embodiments, retraining involves a fourth device 401. For example... Figure 5 As shown, the first device 110 can send (506) a first message to the fourth device 501 to initiate training of the ML entity 130 based on the updated first mapping. The fourth device 501 can receive (508) the first message and determine whether the training of the ML entity 130 is complete. Then, the fourth device 501 can send (510) a second message to the first device 110, which indicates the trained instance of the ML entity 130. The first device 110 can receive (512) the second message from the fourth device 501 and obtain (514) the trained instance of the ML entity 130 from the second message.

[0151] In some example embodiments, the fourth device 501 may include an ML training function. In this case, the first device 110 may send a (506) machine learning model training request to the fourth device 501. Then, the fourth device 501 may send a (510) machine learning model training report to the first device 110 to indicate the trained instance of the ML entity 130.

[0152] Alternatively, in some example embodiments, the fourth device 501 may include a network data analysis function (NWDAF) with a model training logic function (MTLF). In this case, the first device 110 may send (506) a subscription request for the machine learning model to the fourth device 501. Then, the fourth device 501 may send (510) a notification of machine learning model information to the first device 110 to indicate the trained instance of ML entity 130.

[0153] Using the knowledge of the training instances of ML entity 130, the first device 110 can send an update request (516) to the second device 120 to update the current instance of ML entity 130 to the trained instance. The second device 120 can receive the updated request (518) from the first device 110 and update ML entity 130 based on the received update request. Then, the second device 120 can send an update response (520) to the first device 110 indicating that the update of ML entity 130 is complete. Upon receiving the update response (522) from the second device 120, the first device 110 knows that the update of ML entity 130 is complete.

[0154] Alternatively, in some example embodiments, ML entity 130 may be directly trained by second device 120. Second device 120 may send (524) a training notification to first device 110. Upon receiving (526) the notification, first device 110 may know that ML entity 130 was trained by second device 120.

[0155] In some example embodiments, the first device 110 may send an activation request (528) to the second device 120 to activate the retrained ML entity 130. Upon receiving the activation request (530), the second device 120 may send an activation response (532) to the first device 110 to indicate the activity status of the retrained ML entity 130. The first device 110 may receive an activation response (534) from the second device 120. For example, the activation request may be a ProvMnS request for activating the retrained MLApp1. The ProvMnS request may include the APP-ID of the retrained MLApp1. Accordingly, the activation response may be a ProvMnS response indicating that the service has been activated.

[0156] Example procedure 500 has been described. Now return to steps 506-522. As described above, in some example embodiments, the fourth device 401 may include an ML training function, such as an AIML training function. In such an example embodiment, ML entity 130 (its ML model or the solution as a whole) may be trained by the ML training function. The first device 110 may send an AIML training request (506) to the second device 120 to request new training with a mapping function as the training context. The request may include or indicate the AIML entity ID (e.g., the APP ID of MLApp1), the APP construct, candidate training data resources, and the expected runtime context. The training context may be defined manually or learned from a separate analytics function. The request may also include an updated first mapping, for example, F actual2as and F as2aa For example, AIML training requests ( AIMLTrainingRequest The attribute "expected runtime context" in ) expectedRuntimeContext ") can be extended to carry (multiple) updated mapping functions, such as F actual2as and F as2aa The mapping function, as part of the expected runtime context, can be used for inference against non-reinforcement learning.

[0157] Once the second device 120 decides to begin training according to the training request, the second device 120 can instantiate one or more training procedures responsible for executing the training process, including training data collection, preparation and selection of training data, and actual training. For example, one or more AIML training procedures ( AIMLTrainingProcess Multiple MOIs can be instantiated. After training is complete, the second device 120 can send an AIML training report (510) with a new ID of the ML entity 130 to the first device 110. For example, the AIML training report ( AIMLTrainingReport The new AIML entity ID (AIMLEntityID) can be sent together. Furthermore, in steps 514 to 522, the first device 110 can provide the updated instance of the ML entity to the second device 120 and receive feedback from the second device 120.

[0158] Alternatively, in some example embodiments, the fourth device 401 may include an NWDAF with MTLF. In such example embodiments, the ML entity 1300 (its ML model or the solution as a whole) may be trained by an NWDAF with MTLF. The first device 110 may send (506) an Nnwdaf_MLModelProvision_Subscribe including the analysis ID and other parameters to the second device 120. In this way, the first device 110 subscribes to the MTLF to obtain the trained ML entity associated with the analysis ID. This subscription is initiated by the second device 120 as any NF and requesting the analysis results for a specific analysis ID, or if the first device 110 is mapped to AnLF, it can request the model provisioning directly from the MTLF. The subscription can be extended to carry updated mapping functions (such as F actual2as and F as2aa This indicates that retraining is required, as indicated by the signal already sent by the second device 120.

[0159] Upon receiving a (508) subscription, MTLF can determine whether retraining needs to be triggered for the existing trained ML model / solution. However, based on extensions in the subscription, the indication to trigger retraining will already be part of the subscription, as follows: if MTLF detects that the mapping in the subscription is different from the earlier mapping, MTLF can directly start from retraining.

[0160] At point 510, when the NWDAF with MTLF determines that a previously provided trained ML model / solution needs to be retrained and has already been retrained, the MTLF can invoke the Nnwdaf_MLModelProvision_Notify service operation to notify the available retrained ML model / solution. Utilizing this process in this step allows the MTLF to notify the first device 110 of the retrained MLApp instance.

[0161] Furthermore, in steps 514 to 522, the first device 110 may provide the second device 120 with an updated instance of the ML entity and receive feedback from the second device 120.

[0162] It should be noted that, Figure 5 In this configuration, the first device 110 (such as an AI / ML MnS consumer) is mapped to AnLF and can communicate directly with MTLF. Alternatively, the first device 110 (such as an AI / ML MnS consumer) can be mapped to NF (or OAM), which can request specific analytics from AnLF. Thus, AnLF can request model training / retraining from MTLF.

[0163] Example information model definition for ML abstract behavior management

[0164] The following example embodiments of this disclosure will discuss the information object classes (IOCs) and data types required to implement ML transfer learning, as well as the relationships between these IOCs and data types.

[0165] Figure 6A Example diagrams are shown illustrating information models used for abstract behavior when presented by AI / ML functions, according to some exemplary embodiments of this disclosure. For example... Figure 6A As shown, there are four classes: Managed Entity (601), AI / ML Function (602), ML Entity (603), and Abstract Behavior (604), where Abstract Behavior (604) is represented by AI / ML Function (602). The relationships between these classes are... Figure 6A As shown in the class diagram.

[0166] Figure 6B Example diagrams are shown illustrating an information model used for abstract behavior when represented by an ML entity, according to some example embodiments of the present disclosure. In these embodiments, abstract behavior 604 is represented by an ML entity 603, and Figure 6B The class illustrates the relationships between the managed entities (601), AI / ML functions (602), ML entities (603), and abstract behaviors (604).

[0167] Figure 6C An example diagram illustrating inheritance relationships for abstract behaviors according to some example embodiments of this disclosure is shown. Specifically, in Figure 6C The class diagram illustrates the relationships between classes AI / ML Function (602), ML Entity (603), Abstract Behavior (604), Top (605), and Function (606).

[0168] The following discusses some example implementations related to class definition. Specifically, the characteristics and attributes of a class are defined as follows.

[0169] AI / MLFunction <ioc>>

[0170] AI / MLFunction< <ioc>> represents an attribute of AI / MLFunction. Each AI / MLFunction 602 is a managed object that can be instantiated from the AI / MLFunction information object class, and its name is contained in the subnetwork, managed function, or management function. AI / MLFunction 602 is the type of managed function; that is, AI / MLFunction 602 is a subclass of the managed function and inherits the capabilities of the managed function.

[0171] Each AI / MLFunction 602 should be associated with one or more ML entities.

[0172] Each AI / MLFunction 602 can actually be associated with a candidate abstract behavior.

[0173] Each AI / MLFunction 602 can actually be associated with one or more instances of an abstract behavior, which are lists of candidate and selected state-action pairs, respectively.

[0174] An instance of the abstract behavior at AI / MLFunction 602 can also be associated with a specific ML entity.

[0175] Abstract behavior is conditionally enforced: if it is not associated with an mLEntity that is itself associated with AI / MLFunction, then it must be associated with AI / MLFunction.

[0176] AI / MLFunction IOC includes the following attributes as shown in Table 4.

[0177] Table 4

[0178]

[0179] MLEntity< <ioc>>

[0180] This IOC represents the characteristics of MLEntity 603. Each MLEntity 603 is a managed object contained in or associated with AI / MLFunction 602.

[0181] Each MLEntity 603 can actually be associated with an instance of AbstractBehavior, which is a list of pairs of state-action pairs containing both candidate and selected pairs.

[0182] AbstractBehavior is conditionally enforced; if it is not associated with AI / MLFunction 602 (MLEntity 603 is associated with the computation result of AI / MLFunction 602), then it must be associated with MLEntity 603.

[0183] MLEntity IOC includes the following properties as shown in Table 5.

[0184] Table 5

[0185]

[0186] abstractBehavior< <ioc>>

[0187] The IOC represents an attribute of abstract behavior.

[0188] Abstract behaviors are associated with MLEntities or AI / MLFunctions. Abstract behaviors contain the characteristics of abstract behaviors of MLEntities or ML functions.

[0189] Abstract behavior consists of two lists: a list of candidate abstract states and their candidate actions, and a list of selected and configured abstract states and their corresponding selected actions.

[0190] The ML Knowledge Request (MLKnowledgeRequest) IOC includes the following attributes as shown in Table 6.

[0191] Table 6

[0192]

[0193] Candidate Abstract State << Data Type>> <datatype>>)

[0194] This data type (dataType) represents the properties of the abstract state (abstractState). A candidate abstract state is a list of abstract states, and each state has a list of candidate abstract actions for that abstract state.

[0195] Each abstract state can be identified using an identifier. An abstract state can be characterized by a human-readable description that allows the human MnS consumer to know which features are grouped within that abstract state.

[0196] Each abstract state can have at least two possible actions that can be taken within that state. These are listed in the possibleActions property of the abstract state. The possible actions are an enumeration of possible actions, from which the MnS consumer can select the action to be applied.

[0197] Abstract State << Data Type>> (abstractState<< <datatype>>) Includes the following attributes in Table 7.

[0198] Table 7

[0199]

[0200] The applied abstract state << data type>> <datatype>>)

[0201] This data type (dataType) represents a property of the applied abstract state (appliedAbstractState). The applied abstract state is a list of state-action tuples.

[0202] Each applied abstract state has one action to be applied, which has been selected by the MnS producer or the MnS consumer.

[0203] Each state can be represented by the identifier of the corresponding state listed in the candidate abstract states.

[0204] Similarly, each action can be represented by an identifier of the corresponding action listed in the candidate abstract behaviors.

[0205] AbstractState <<Data Type>> <datatype>>) Includes the following attributes in Table 8.

[0206] Table 8

[0207]

[0208] Example Method

[0209] Figure 7 A flowchart of an example method 700 implemented at a first device according to some example embodiments of the present disclosure is shown. For purposes of discussion, [the following will be discussed]. Figure 1 The angle description method of the first device 110 in the 700.

[0210] At box 710, the first device 110 determines a first mapping from the network context to an abstract set of actions, which represents the actual actions of the machine learning entity.

[0211] At frame 720, the first device 110 sends first information indicating the first mapping to the second device.

[0212] At box 730, the first device 110 receives second information from the second device that is at least associated with a second abstract action, which corresponds to the actual action of the machine learning entity in a given real network context.

[0213] At frame 740, the first device 110 monitors the difference between a first abstract action and a second abstract action determined based on a first mapping, based on second information.

[0214] In some example embodiments, the first mapping includes: a third mapping from the network context to a first set of abstract states, which represents the actual states associated with the machine learning entity, and a fourth mapping from the first set of abstract states to a set of abstract actions.

[0215] In some example embodiments, the first set of abstract states is a subset of the second set of abstract states representing actual states, and a given abstract state in the second set of abstract states is associated with at least one of the following: an identifier of the given abstract state, a description of the abstract state, or at least one abstract action available in the abstract state.

[0216] In some example embodiments, the first abstract state set and the second abstract state set are included in an abstract behavior associated with at least one of the following: a machine learning entity, or a function associated with a machine learning entity.

[0217] In some example embodiments, method 700 further includes receiving an instantiation response from a second device indicating that the instantiation of the machine learning entity is complete.

[0218] In some example embodiments, the method further includes sending a first registration request to a third device, the first registration request being used to store a first mapping for a machine learning entity.

[0219] In some example embodiments, method 700 further includes: updating the first mapping by examining at least a portion of the first mapping; sending a first update request to a second device indicating an update to the first mapping; and receiving a first completion response from the second device indicating that the update to the first mapping is complete.

[0220] In some example embodiments, updating the first mapping includes updating at least one of the following: a mapping from the network context to abstract states in a first set of abstract states, which represents the actual states associated with a machine learning entity, or a mapping from abstract states in the first set of abstract states to one or more abstract actions in a set of abstract actions.

[0221] In some example embodiments, method 700 further includes receiving an inspection request from a second device for inspecting a first mapping for a machine learning entity.

[0222] In some example embodiments, method 700 further includes: in response to a difference being detected, sending a retrieval request to a third device for retrieving a first mapping for a machine learning entity; and receiving a retrieval response from the third device, the retrieval response indicating the first mapping for the machine learning entity.

[0223] In some example embodiments, the method further includes sending a second registration request to a third device for storing an updated first mapping for machine learning entities.

[0224] In some example embodiments, method 700 further includes: sending a first message to a fourth device to initiate training of a machine learning entity based on an updated first mapping; receiving a second message from the fourth device indicating a trained instance of the machine learning entity; sending a second update request to a second device to update the current instance of the machine learning entity to the trained instance; and receiving a second completion response from the second device indicating that the update of the machine learning entity is complete.

[0225] In some example embodiments, method 700 further includes receiving a request from a second device for training a machine learning entity.

[0226] In some example embodiments, the fourth device includes a machine learning training function, a first message including a machine learning model training request, and a second message including a machine learning model training report.

[0227] In some example embodiments, the fourth device includes a first network data analysis function with model training logic, a first message including a subscription request for the machine learning model, and a second message including a notification of machine learning model information.

[0228] In some example embodiments, the second information includes at least one of the following: an indication of a first abstract action and a second abstract action, an indication of a second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action.

[0229] Figure 8 A flowchart of an example method 800 implemented at a second device according to some example embodiments of the present disclosure is shown. For purposes of discussion, [the following will be discussed]. Figure 1 The second device 120 in the method of angle description 800.

[0230] At box 810, the second device 120 receives first information from the first device 110, the first information indicating a first mapping from the network context to an abstract action set, the abstract action set representing the actual actions of the machine learning entity.

[0231] At box 820, the second device 120 determines the first abstract action based on the first mapping and the actual network context used by the machine learning entity.

[0232] At box 830, the second device 120 determines a second abstract action corresponding to the actual action of the machine learning entity in a given actual network context, based on a second mapping from the actual actions of the machine learning entity to a set of abstract actions.

[0233] At frame 840, the second device 120 monitors the difference between the first abstract action and the second abstract action.

[0234] At frame 850, the second device 120 sends second information to the first device 110 that is at least associated with the second abstract action.

[0235] In some example embodiments, the first mapping includes the following: a third mapping from the network context to a first set of abstract states, which represents the actual states associated with machine learning entities, and a fourth mapping from the first set of abstract states to a set of abstract actions.

[0236] In some example embodiments, the first set of abstract states is a subset of the second set of abstract states representing actual states, and a given abstract state in the second set of abstract states is associated with at least one of the following: an identifier of the given abstract state, a description of the abstract state, or at least one abstract action available in the abstract state.

[0237] In some example embodiments, the first abstract state set and the second abstract state set are included in an abstract behavior associated with at least one of the following: a machine learning entity, or a function associated with a machine learning entity.

[0238] In some example embodiments, the method further includes: in response to an instantiation request, instantiating a machine learning entity using a first mapping; and sending an instantiation response to a first device, the instantiation response indicating that the instantiation of the machine learning entity is complete.

[0239] In some example embodiments, method 800 further includes: receiving a first update request from a first device, the first update request indicating an update to a first mapping; updating the first mapping based on the first update request; and sending a first completion response to the first device, the first completion response indicating that the update to the first mapping is complete.

[0240] In some example embodiments, updating the first mapping includes updating at least one of the following: a mapping from the network context to abstract states in a first set of abstract states, which represents the actual states associated with a machine learning entity, or a mapping from abstract states in the first set of abstract states to one or more abstract actions in a set of abstract actions.

[0241] In some example embodiments, method 800 further includes: in response to a difference being detected, sending an inspection request to a first device to inspect a first mapping for a machine learning entity.

[0242] In some example embodiments, method 800 further includes: receiving a second update request from a first device, the second update request being used to update the current instance of the machine learning entity to a trained instance of the machine learning entity, the trained instance being trained based on the updated first mapping; updating the machine learning entity based on the second update request; and sending a second update response to the first device indicating that the update of the machine learning entity is complete.

[0243] In some example embodiments, method 800 further includes sending a request to a first device for training a machine learning entity.

[0244] In some example embodiments, the second information includes at least one of the following: an indication of a first abstract action and an indication of a second abstract action, an indication of a second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action.

[0245] Figure 9 A flowchart of an example method 900 implemented at a third device according to some example embodiments of the present disclosure is shown. For the purposes of discussion, method 900 will be described from the perspective of third device 301.

[0246] At box 910, third device 301 receives a first registration request from first device 110. The first registration request is used to store a first mapping from network context to an abstract action set, which represents the actual actions of machine learning entities.

[0247] At box 920, the third device 301 stores the first mapping in association with the identifier of the machine learning entity.

[0248] In some example embodiments, the first mapping includes: a third mapping from the network context to a first set of abstract states, which represents the actual states associated with the machine learning entity, and a fourth mapping from the first set of abstract states to a set of abstract actions.

[0249] In some example embodiments, method 900 further includes: receiving a retrieval request from a first device for retrieving a first mapping for a machine learning entity; and sending a retrieval response to the first device, the retrieval response indicating the first mapping for the machine learning entity.

[0250] In some example embodiments, method 900 further includes: receiving a second registration request from a first device, the second registration request being used to store an updated first mapping for a machine learning entity; and storing the updated first mapping in association with an identifier of the machine learning entity.

[0251] Figure 10 A flowchart of an example method 1000 implemented at a fourth device according to some example embodiments of the present disclosure is shown. For the purposes of discussion, method 1000 will be described from the perspective of the fourth device 501.

[0252] At box 1010, fourth device 501 receives a first message from first device 110. The first message is used to initiate training of machine learning entities based on an updated first mapping from network context to an abstract action set, which represents the actual actions of the machine learning entities.

[0253] At box 1020, the fourth device 501 determines whether the training of the machine learning entity has been completed.

[0254] At box 1030, the fourth device 501, based on the determination that training has been completed, sends a second message to the first device 110, the second message indicating the trained instance of the machine learning entity.

[0255] In some example embodiments, the fourth device 501 includes a machine learning training function, a first message including a machine learning model training request, and a second message including a machine learning model training report.

[0256] In some example embodiments, the fourth device 501 includes a first network data analysis function with model training logic, a first message including a subscription request for a machine learning model, and a second message including a notification of machine learning model information.

[0257] Example devices, equipment and media

[0258] In some example embodiments, a first means capable of performing any of the methods 700 (e.g., Figure 1 The first device 110 may include components for performing the corresponding operations of method 700. These components may be implemented in any suitable form. For example, the device may be implemented in a circuit system or a software module. The first device may be implemented as or included in... Figure 1 The first device 110 in the middle.

[0259] In some example embodiments, the first device includes components for determining a first mapping from a network context to an abstract set of actions, the abstract set of actions representing actual actions of a machine learning entity; components for sending first information indicating the first mapping to a second device; components for receiving from the second device at least second information associated with a second abstract action, the second abstract action corresponding to an actual action of a machine learning entity given a real network context; and components for monitoring, based on the second information, the difference between the first abstract action determined based on the first mapping and the second abstract action.

[0260] In some example embodiments, the first mapping includes: a third mapping from the network context to a first set of abstract states, which represents the actual states associated with the machine learning entity, and a fourth mapping from the first set of abstract states to a set of abstract actions.

[0261] In some example embodiments, the first set of abstract states is a subset of the second set of abstract states representing actual states, and a given abstract state in the second set of abstract states is associated with at least one of the following: an identifier of the given abstract state, a description of the abstract state, or at least one abstract action available in the abstract state.

[0262] In some example embodiments, the first abstract state set and the second abstract state set are included in an abstract behavior associated with at least one of the following: a machine learning entity, or a function associated with a machine learning entity.

[0263] In some example embodiments, the first device further includes a component for receiving from the second device an instantiation response indicating that the instantiation of the machine learning entity has been completed.

[0264] In some example embodiments, the first device further includes a component for sending a first registration request to a third device, the first registration request being used to store a first mapping for a machine learning entity.

[0265] In some example embodiments, the first device further includes components for updating the first mapping by examining at least a portion of the first mapping; components for sending a first update request indicating an update to the first mapping to a second device; and components for receiving a first completion response from the second device indicating that the update to the first mapping is complete.

[0266] In some example embodiments, the component for updating the first mapping includes updating at least one of the following: a mapping from the network context to abstract states in a first set of abstract states, which represents the actual states associated with a machine learning entity, or a mapping from abstract states in the first set of abstract states to one or more abstract actions in a set of abstract actions.

[0267] In some example embodiments, the first device further includes a component for receiving an inspection request from a second device, the inspection request being for inspecting a first mapping for a machine learning entity.

[0268] In some example embodiments, the first device further includes means for sending a retrieval request to a third device in response to a difference being detected, the retrieval request being for retrieving a first mapping for a machine learning entity; and means for receiving a retrieval response from the third device, the retrieval response indicating the first mapping for the machine learning entity.

[0269] In some example embodiments, the first device further includes components for sending a second registration request to a third device, the second registration request being used to store an updated first mapping for machine learning entities.

[0270] In some example embodiments, the first apparatus further includes components for sending a first message to a fourth device, the first message being used to initiate training of a machine learning entity based on an updated first mapping; components for receiving a second message from the fourth device, the second message indicating a trained instance of the machine learning entity; components for sending a second update request to the second device, the second update request being used to update the current instance of the machine learning entity to the trained instance; and components for receiving a second completion response from the second device, the second completion response indicating that the update of the machine learning entity is complete.

[0271] In some example embodiments, the first device further includes a component for receiving a request from the second device for training a machine learning entity.

[0272] In some example embodiments, the fourth device includes a machine learning training function, a first message including a machine learning model training request, and a second message including a machine learning model training report.

[0273] In some example embodiments, the fourth device includes a first network data analysis function with model training logic, a first message including a subscription request for the machine learning model, and a second message including a notification of machine learning model information.

[0274] In some example embodiments, the second information includes at least one of the following: an indication of a first abstract action and a second abstract action, an indication of a second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action.

[0275] In some example embodiments, the first device further includes components for performing other operations in some example embodiments of method 700 or the first device 110. In some example embodiments, the components include: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause execution of the first device.

[0276] In some example embodiments, a second means capable of performing any of the methods 800 (e.g., Figure 1 The second device 120 may include components for performing the corresponding operations of method 800. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module. The second device may be implemented as or included in... Figure 1 The second device 120 in the middle.

[0277] In some example embodiments, the second apparatus includes components for receiving first information from the first device, the first information indicating a first mapping from a network context to an abstract action set, the abstract action set representing the actual actions of a machine learning entity; components for determining a first abstract action based on the first mapping and the actual network context used by the machine learning entity; components for determining a second abstract action corresponding to the actual action of the machine learning entity in a given actual network context based on a second mapping from the actual actions of the machine learning entity to the abstract action set; components for monitoring the difference between the first abstract action and the second abstract action; and components for sending second information to the first device that is at least associated with the second abstract action.

[0278] In some example embodiments, the first mapping includes the following: a third mapping from the network context to a first set of abstract states, which represents the actual states associated with machine learning entities, and a fourth mapping from the first set of abstract states to a set of abstract actions.

[0279] In some example embodiments, the first set of abstract states is a subset of the second set of abstract states representing actual states, and a given abstract state in the second set of abstract states is associated with at least one of the following: an identifier of the given abstract state, a description of the abstract state, or at least one abstract action available in the abstract state.

[0280] In some example embodiments, the first abstract state set and the second abstract state set are included in an abstract behavior associated with at least one of the following: a machine learning entity, or a function associated with a machine learning entity.

[0281] In some example embodiments, the second device further includes components for instantiating a machine learning entity using a first mapping in response to an instantiation request; and components for sending an instantiation response to the first device, the instantiation response indicating that the instantiation of the machine learning entity is complete.

[0282] In some example embodiments, the second apparatus further includes components for receiving a first update request from the first device, the first update request indicating an update to the first mapping; components for updating the first mapping based on the first update request; and components for sending a first completion response to the first device, the first completion response indicating that the update to the first mapping is complete.

[0283] In some example embodiments, the component for updating the first mapping includes a component for updating at least one of the following: a mapping from the network context to abstract states in a first set of abstract states, which represents the actual states associated with a machine learning entity, or a mapping from abstract states in the first set of abstract states to one or more abstract actions in a set of abstract actions.

[0284] In some example embodiments, the second device further includes a component for sending an inspection request to the first device in response to a difference being detected, the inspection request being for inspecting a first mapping for a machine learning entity.

[0285] In some example embodiments, the second apparatus further includes components for receiving a second update request from the first device, the second update request being used to update the current instance of the machine learning entity to a trained instance of the machine learning entity, the trained instance being trained based on the updated first mapping; components for updating the machine learning entity based on the second update request; and components for sending a second update response to the first device indicating that the update of the machine learning entity is complete.

[0286] In some example embodiments, the second device further includes a component for sending a request to the first device to train a machine learning entity.

[0287] In some example embodiments, the second information includes at least one of the following: an indication of a first abstract action and an indication of a second abstract action, an indication of a second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action.

[0288] In some example embodiments, the second device further includes components for performing other operations in some example embodiments of method 800 or second device 120. In some example embodiments, the components include: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause execution of the second device.

[0289] In some exemplary embodiments, a third means (e.g., a third device 301) capable of performing any of the methods 900 may include components for performing the corresponding operations of method 900. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module. The third means may be implemented as or included in the third device 301.

[0290] In some example embodiments, the third device includes components for receiving a first registration request from a first device, the first registration request being used to store a first mapping from a network context to an abstract set of actions, the abstract set of actions representing the actual actions of a machine learning entity; and components for storing the first mapping in association with an identifier of the machine learning entity.

[0291] In some example embodiments, the first mapping includes: a third mapping from the network context to a first set of abstract states, which represents the actual states associated with the machine learning entity, and a fourth mapping from the first set of abstract states to a set of abstract actions.

[0292] In some example embodiments, the third device may further include components for receiving a retrieval request from the first device for retrieving a first mapping for a machine learning entity; and components for sending a retrieval response to the first device, the retrieval response indicating the first mapping for the machine learning entity.

[0293] In some example embodiments, the third apparatus may further include components for receiving a second registration request from the first device, the second registration request being for storing an updated first mapping for a machine learning entity; and components for storing the updated first mapping in association with an identifier of the machine learning entity.

[0294] In some example embodiments, the third device further includes components for performing other operations in some example embodiments of method 900 or third device 301. In some example embodiments, the components include: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause execution of the third device.

[0295] In some exemplary embodiments, a fourth means (e.g., a fourth device 501) capable of performing any of the methods 1000 may include components for performing the corresponding operations of method 1000. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module. The fourth means may be implemented as or included in the fourth device 501.

[0296] In some example embodiments, the fourth device includes components for receiving a first message from a first device, the first message being used to initiate training of a machine learning entity based on an updated first mapping from a network context to an abstract action set, the abstract action set representing the actual actions of the machine learning entity; components for determining whether the training of the machine learning entity has been completed; and components for sending a second message to the first device based on the determination that the training has been completed, the second message indicating a trained instance of the machine learning entity.

[0297] In some example embodiments, the fourth device includes a machine learning training function, a first message including a machine learning model training request, and a second message including a machine learning model training report.

[0298] In some example embodiments, the fourth device includes a first network data analysis function with model training logic, a first message including a subscription request for the machine learning model, and a second message including a notification of machine learning model information.

[0299] In some example embodiments, the fourth device further includes components for performing other operations in some example embodiments of method 1000 or fourth device 501. In some example embodiments, the components include: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause execution of the fourth device.

[0300] Figure 11 This is a simplified block diagram of a device 1100 suitable for implementing exemplary embodiments of the present disclosure. The device 1100 can be provided to implement a communication device, such as... Figure 1 The first device 110 or the second device 120 shown in the figure. As shown, device 1100 includes one or more processors 1110, one or more memories 1120 coupled to processor 1110, and one or more communication modules 1140 coupled to processor 1110.

[0301] Communication module 1140 is used for bidirectional communication. Communication module 1140 has one or more communication interfaces to facilitate communication with one or more other modules or devices. The communication interface can represent any interface necessary for communication with other network elements. In some example embodiments, communication module 1140 may include at least one antenna.

[0302] As a non-limiting example, processor 1110 can be any type suitable for a local technology network and can include one or more of the following: general-purpose computer, special-purpose computer, microprocessor, digital signal processor (DSP), and processor based on a multi-core processor architecture. Device 1100 can have multiple processors, such as application-specific integrated circuit chips, which are time-dependent on the clock of a synchronous main processor.

[0303] Memory 1120 may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, read-only memory (ROM) 1124, electrically programmable read-only memory (EPROM), flash memory, hard disk, optical disc (CD), digital video disc (DVD), optical disc, laser disc, and other magnetic and / or optical storage. Examples of volatile memories include, but are not limited to, random access memory (RAM) 1122 and other volatile memories that will not persist during power outages.

[0304] Computer program 1130 includes computer-executable instructions that are executed by an associated processor 1110. The instructions of program 1130 may include instructions for performing operations / actions of some example embodiments of this disclosure. Program 1130 may be stored in memory, such as ROM 1124. Processor 1110 can perform any suitable actions and processes by loading program 1130 into RAM 1122.

[0305] The exemplary embodiments of this disclosure can be implemented by means of program 1130, enabling device 1100 to perform as described in the reference. Figures 2 to 10 Any process discussed in this disclosure. Exemplary embodiments of this disclosure may also be implemented in hardware or a combination of software and hardware.

[0306] In some example embodiments, program 1130 may be tangibly contained in a computer-readable medium, which may be included in device 1100 (such as in memory 1120) or in other storage devices accessible by device 1100. Device 1100 may load program 1130 from the computer-readable medium into RAM 1122 for execution. In some example embodiments, the computer-readable medium may include any type of non-transitory storage medium, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc. The term "non-transitory" as used herein refers to a limitation on the medium itself (i.e., tangible, not tactile), rather than a limitation on the persistence of data storage (e.g., RAM versus ROM).

[0307] Figure 12 An example of a computer-readable medium 1200, which may be in the form of a CD, DVD, or other optical storage disc, is shown. A program 1130 is stored on the computer-readable medium 1200.

[0308] Generally, the various embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while others may be implemented in firmware or software executable by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, apparatuses, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0309] Some exemplary embodiments of this disclosure also provide at least one computer program product tangibly stored on a computer-readable medium, such as a non-transitory computer-readable medium. The computer program product includes computer-executable instructions, such as those included in a program module that executes in a device on a target physical or virtual processor, to perform any of the methods described above. Typically, a program module includes routines, programs, libraries, objects, classes, components, data structures, etc., that perform a particular task or implement a particular abstract data type. In various embodiments, the functionality of a program module can be combined or split among program modules as needed. The machine-executable instructions for a program module can execute within a local or distributed device. In a distributed device, the program module can reside on both local and remote storage media.

[0310] The program code used to perform the methods of this disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code enables the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a stand-alone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0311] In the context of this disclosure, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, etc.

[0312] Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable media can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media will include electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0313] Furthermore, although operations are described in a specific order, this should not be construed as requiring that such operations be performed in the specific order shown or sequentially, or that all shown operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure, but rather as descriptions of features that may be specific to particular embodiments. Unless explicitly stated otherwise, certain features described in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, unless explicitly stated otherwise, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0314] Although this disclosure has been described in language specific to structural features and / or methodological actions, it should be understood that the disclosure as defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms for implementing the claims.< / datatype> < / datatype> < / datatype> < / datatype> < / ioc> < / ioc> < / ioc> < / ioc>

Claims

1. A first device for managing abstract behaviors in machine learning, comprising: At least one processor; as well as At least one memory storing instructions that, when executed by the at least one processor, cause the first device to perform at least the following: Determine a first mapping from the network context to an abstract set of actions, which represents the actual actions of machine learning entities; Send first information indicating the first mapping to the second device; The second device receives at least second information associated with a second abstract action, the second abstract action being an abstract action seen by the machine learning entity, determined by the second device based on a second mapping from the actual actions of the machine learning entity to the set of abstract actions, the second abstract action corresponding to the actual actions of the machine learning entity in a given actual network context, the second mapping being an internal mapping of the second device, and the second information including at least one of the following: an indication of the second abstract action, an indication of a first abstract action and the second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action; as well as Based on the second information, monitor the difference between the first abstract action determined based on the first mapping and the second abstract action.

2. The first device according to claim 1, wherein the first mapping comprises: A third mapping from the network context to a first abstract set of states, where the first abstract set of states represents the actual states associated with the machine learning entity, and A fourth mapping from the first set of abstract states to the set of abstract actions.

3. The first device according to claim 2, wherein the first set of abstract states is a subset of a second set of abstract states representing the actual states, and a given abstract state in the second set of abstract states is associated with at least one of the following: The identifier of the given abstract state, The description of the abstract state, or At least one abstract action is available in the abstract state.

4. The first device of claim 3, wherein the first abstract state set and the second abstract state set are included in an abstract behavior associated with at least one of the following: The machine learning entity, or The functions associated with the machine learning entity.

5. The first device of claim 1, wherein the first information indicating the first mapping is included in an instantiation request, the instantiation request being used to instantiate the machine learning entity using the first mapping, and further causing the first device to perform: Receive an instantiation response from the second device indicating that the instantiation of the machine learning entity is complete.

6. The first device according to claim 1, wherein the first device is further caused to perform: A first registration request is sent to a third device, the first registration request being used to store the first mapping for the machine learning entity.

7. The first device according to claim 1, wherein the first device is further caused to perform: The first mapping is updated by examining at least a portion of it; Send a first update request indicating an update to the first mapping to the second device; and Receive a first completion response from the second device indicating that the update of the first mapping is complete.

8. The first device according to claim 7, wherein updating the first mapping includes updating at least one of the following: A mapping from the network context to abstract states in a first set of abstract states, which represents the actual states associated with the machine learning entity. A mapping from abstract states in the first set of abstract states to one or more abstract actions in the set of abstract actions.

9. The first device according to claim 7, wherein the first device is further caused to perform: The second device receives an inspection request for inspecting the first mapping for the machine learning entity.

10. The first device according to claim 7, wherein the first device is further caused to perform: In response to the detection of the difference, a retrieval request is sent to a third device to retrieve the first mapping for the machine learning entity; and The third device receives a retrieval response indicating the first mapping for the machine learning entity.

11. The first device according to claim 10, wherein the first device is further caused to perform: A second registration request is sent to the third device, the second registration request being used to store the updated first mapping for the machine learning entity.

12. The first device according to claim 7, wherein the first device is further caused to perform: Send a first message to the fourth device, the first message being used to initiate training of the machine learning entity based on the updated first mapping; Receive a second message from the fourth device, the second message indicating a trained instance of the machine learning entity; A second update request is sent to the second device to update the current instance of the machine learning entity to the trained instance; as well as Receive a second completion response from the second device, the second completion response indicating that the update of the machine learning entity is complete.

13. The first device according to claim 12, wherein the first device is further caused to perform: Receive a request from the second device to train the machine learning entity.

14. The first device of claim 12, wherein the fourth device includes machine learning training functionality. The first message includes a machine learning model training request, and The second message includes a machine learning model training report.

15. The first device according to claim 12, wherein the fourth device includes a first network data analysis function having model training logic. The first message includes a subscription request for the machine learning model, and The second message includes a notification of machine learning model information.

16. The first device according to any one of claims 1 to 15, wherein the second information comprises at least one of the following: The instructions for the first abstract action and the second abstract action, The instruction of the second abstract action, or An indication of whether the difference exists between the first abstract action and the second abstract action.

17. A second device for managing abstract behaviors in machine learning, comprising: At least one processor; as well as At least one memory storing instructions that, when executed by the at least one processor, cause the second device to perform at least the following: Receive first information from a first device, the first information indicating a first mapping from a network context to an abstract set of actions, the abstract set of actions representing the actual actions of machine learning entities; Based on the first mapping and the actual network context used by the machine learning entity, a first abstract action is determined; Based on a second mapping from the actual actions of the machine learning entity to the set of abstract actions, a second abstract action is determined that corresponds to the actual actions of the machine learning entity in a given actual network context, wherein the second mapping is an internal mapping of the second device and the second abstract action is an abstract action seen by the machine learning entity. Monitor the differences between the first abstract action and the second abstract action; as well as Send at least one second piece of information associated with the second abstract action to the first device, the second piece of information including at least one of the following: an indication of the second abstract action, an indication of the first abstract action and the second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action.

18. The second device of claim 17, wherein the first mapping comprises: A third mapping from the network context to a first abstract set of states, where the first abstract set of states represents the actual states associated with the machine learning entity, and A fourth mapping from the first set of abstract states to the set of abstract actions.

19. The second device according to claim 18, wherein the first set of abstract states is a subset of a second set of abstract states representing the actual states, and a given abstract state in the second set of abstract states is associated with at least one of the following: The identifier of the given abstract state, The description of the abstract state, or At least one abstract action is available in the abstract state.

20. The second device of claim 19, wherein the first abstract state set and the second abstract state set are included in an abstract behavior associated with at least one of the following: The machine learning entity, or The functions associated with the machine learning entity.

21. The second device of claim 17, wherein the first information indicating the first mapping is included in an instantiation request for instantiating the machine learning entity using the first mapping, and further causes the second device to perform: In response to the instantiation request, the machine learning entity is instantiated using the first mapping; and An instantiation response is sent to the first device, indicating that the instantiation of the machine learning entity is complete.

22. The second device according to claim 17, wherein the second device is further caused to perform: Receive a first update request from the first device, the first update request indicating an update to the first mapping; Based on the first update request, update the first mapping; and Send a first completion response to the first device, the first completion response indicating that the update of the first mapping is complete.

23. The second device according to claim 22, wherein updating the first mapping includes updating at least one of the following: A mapping from the network context to abstract states in a first set of abstract states, which represents the actual states associated with the machine learning entity. A mapping from abstract states in the first set of abstract states to one or more abstract actions in the set of abstract actions.

24. The second device according to claim 22, wherein the second device is further caused to perform: In response to the detection of the difference, an inspection request is sent to the first device to inspect the first mapping for the machine learning entity.

25. The second device according to claim 22, wherein the second device is further caused to perform: The device receives a second update request to update the current instance of the machine learning entity to a trained instance of the machine learning entity, the trained instance being trained based on the updated first mapping. Update the machine learning entity based on the second update request; as well as Send a second update response to the first device, indicating that the update of the machine learning entity is complete.

26. The second device according to claim 25, wherein the second device is further caused to perform: Send a request to the first device to train the machine learning entity.

27. The second device according to any one of claims 17 to 26, wherein the second information includes at least one of the following: The indication of the first abstract action and the indication of the second abstract action, The instruction of the second abstract action, or An indication of whether the difference exists between the first abstract action and the second abstract action.

28. A third device for managing abstract behaviors in machine learning, comprising: At least one processor; as well as At least one memory storing instructions that, when executed by the at least one processor, cause the third device to perform at least the following: A first registration request is received from a first device, the first registration request being used to store a first mapping from a network context to an abstract action set, the abstract action set representing the actual actions of a machine learning entity; as well as The first mapping is stored in association with the identifier of the machine learning entity.

29. The third device of claim 28, wherein the first mapping comprises: A third mapping from the network context to a first abstract set of states, where the first abstract set of states represents the actual states associated with the machine learning entity, and A fourth mapping from the first set of abstract states to the set of abstract actions.

30. The third device according to any one of claims 28 to 29, wherein the third device is further caused to perform: Receive a retrieval request from the first device, the retrieval request being for retrieving the first mapping for the machine learning entity; and A retrieval response is sent to the first device, the retrieval response indicating the first mapping for the machine learning entity.

31. The third device according to claim 30, wherein the third device is further caused to perform: Receive a second registration request from the first device, the second registration request being used to store the updated first mapping for the machine learning entity; and The updated first mapping is stored in association with the identifier of the machine learning entity.

32. A fourth device for managing abstract behaviors in machine learning, comprising: At least one processor; as well as At least one memory storing instructions that, when executed by the at least one processor, cause the fourth device to perform at least the following: A first message is received from a first device, the first message being used to initiate training of a machine learning entity based on an updated first mapping from a network context to an abstract action set, the abstract action set representing the actual actions of the machine learning entity; Determine whether the training of the machine learning entity has been completed; as well as Upon determining that the training has been completed, a second message is sent to the first device, the second message indicating the trained instance of the machine learning entity.

33. The fourth device of claim 32, wherein the fourth device includes machine learning training functionality. The first message includes a machine learning model training request, and The second message includes a machine learning model training report.

34. The fourth device according to claim 32, wherein the fourth device includes a first network data analysis function having model training logic. The first message includes a subscription request for the machine learning model, and The second message includes a notification of machine learning model information.

35. A method for managing abstract behaviors in machine learning, comprising: At a first device, a first mapping from network context to an abstract set of actions, which represents the actual actions of machine learning entities, is determined. Send first information indicating the first mapping to the second device; The second device receives at least second information associated with a second abstract action, the second abstract action being an abstract action seen by the machine learning entity, determined by the second device based on a second mapping from the actual actions of the machine learning entity to the set of abstract actions, the second abstract action corresponding to the actual actions of the machine learning entity in a given actual network context, the second mapping being an internal mapping of the second device, and the second information including at least one of the following: an indication of the second abstract action, an indication of a first abstract action and the second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action; as well as Based on the second information, monitor the difference between the first abstract action determined based on the first mapping and the second abstract action.

36. A method for managing abstract behaviors in machine learning, comprising: At the second device, first information is received from the first device, the first information indicating a first mapping from network context to an abstract set of actions, the abstract set of actions representing the actual actions of machine learning entities; Based on the first mapping and the actual network context used by the machine learning entity, a first abstract action is determined; Based on a second mapping from the actual actions of the machine learning entity to the set of abstract actions, a second abstract action is determined that corresponds to the actual actions of the machine learning entity in a given actual network context, wherein the second mapping is an internal mapping of the second device and the second abstract action is an abstract action seen by the machine learning entity. Monitor the differences between the first abstract action and the second abstract action; as well as Send at least one second piece of information associated with the second abstract action to the first device, the second piece of information including at least one of the following: an indication of the second abstract action, an indication of the first abstract action and the second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action.

37. A method for managing abstract behaviors in machine learning, comprising: A first registration request is received from the first device at the third device. The first registration request is used to store a first mapping from a network context to an abstract action set, which represents the actual actions of a machine learning entity. as well as The first mapping is stored in association with the identifier of the machine learning entity.

38. A method for managing abstract behaviors in machine learning, comprising: At the fourth device, a first message is received from the first device, the first message being used to initiate the training of a machine learning entity based on an updated first mapping from a network context to an abstract action set, the abstract action set representing the actual actions of the machine learning entity; Determine whether the training of the machine learning entity has been completed; as well as Upon determining that the training has been completed, a second message is sent to the first device, the second message indicating the trained instance of the machine learning entity.

39. A first apparatus for managing abstract behaviors in machine learning, comprising: A component for determining a first mapping from network context to an abstract set of actions, which represents the actual actions of a machine learning entity; A component for sending first information indicating the first mapping to the second device; The second device receives at least second information associated with a second abstract action, the second abstract action being an abstract action seen by the machine learning entity, determined by the second device based on a second mapping from the actual actions of the machine learning entity to the set of abstract actions, the second abstract action corresponding to the actual actions of the machine learning entity in a given actual network context, the second mapping being an internal mapping of the second device, and the second information including at least one of the following: an indication of the second abstract action, an indication of a first abstract action and the second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action; as well as A component for monitoring the difference between a first abstract action determined based on the first mapping and a second abstract action, based on the second information.

40. A second apparatus for managing abstract behaviors in machine learning, comprising: A component for receiving first information from a first device, the first information indicating a first mapping from a network context to an abstract set of actions, the abstract set of actions representing the actual actions of a machine learning entity; A component for determining a first abstract action based on the first mapping and the actual network context used by the machine learning entity; A component for determining a second abstract action corresponding to the actual action of the machine learning entity in a given actual network context, based on a second mapping from the actual action of the machine learning entity to the set of abstract actions, wherein the second mapping is an internal mapping of the second device, and the second abstract action is an abstract action seen by the machine learning entity. A component used to monitor the difference between the first abstract action and the second abstract action; as well as A component for sending at least one second piece of information associated with the second abstract action to the first device, the second piece of information including at least one of the following: an indication of the second abstract action, an indication of the first abstract action and the second abstract action, or an indication of whether there is a difference between the first abstract action and the second abstract action.

41. A third device for managing abstract behaviors in machine learning, comprising: A component for receiving a first registration request from a first device, the first registration request being used to store a first mapping from a network context to an abstract set of actions, the abstract set of actions representing the actual actions of a machine learning entity; as well as A component for storing the first mapping in association with the identifier of the machine learning entity.

42. A fourth device for managing abstract behaviors in machine learning, comprising: A component for receiving a first message from a first device, the first message being used to initiate training of a machine learning entity based on an updated first mapping from a network context to an abstract action set, the abstract action set representing the actual actions of the machine learning entity; A component used to determine whether the training of the machine learning entity has been completed; as well as A component for sending a second message to the first device upon determining that the training has been completed, the second message indicating a trained instance of the machine learning entity.

43. A computer-readable medium comprising instructions stored thereon, the instructions being configured to cause a device to perform at least the method of claim 35, the method of claim 36, the method of claim 37, or the method of claim 38.

Images