IMPROVED JOINT TESTING OF MULTIPLE MACHINE LEARNING MODELS IN WIRELESS COMMUNICATIONS

Abstract

This disclosure describes systems, methods, and devices for jointly testing machine learning (ML) models in wireless networks. A service-based management architecture (SBMA) management service (MnS) generator setup can include a processing circuit arrangement coupled with storage to store information associated with the joint testing of machine learning (ML) models, wherein the processing circuit arrangement is designed to: define performance requirements for jointly testing a group of ML models; perform the joint testing of the group of ML models; and send the results of the joint testing to an MnS consumer, the results indicating whether the group of ML models meets the performance requirements.

Description

REFERENCE TO RELATED PATENT APPLICATION(S)

[0001] This application claims the benefit of preliminary U.S. application No. 63 / 516,736, filed on July 31, 2023, the disclosure of which is incorporated herein by reference as if fully set forth. TECHNICAL AREA

[0002] This disclosure relates generally to systems and methods for wireless communications and in particular to the joint testing of multiple machine learning models in wireless communications. BACKGROUND

[0003] Wireless devices are becoming increasingly prevalent and are using more and more wireless channels. The 3 rd The 3GPP Generation Partnership Program develops one or more standards for wireless communication. BRIEF DESCRIPTION OF THE DRAWINGS Fig.1 is an exemplary operational artificial intelligence / machine learning workflow according to one or more exemplary embodiments of the present disclosure. Fig. 2 is an exemplary functional overview and an exemplary service framework for machine learning training according to one or more exemplary embodiments of the present disclosure. Fig. Figure 3 illustrates an exemplary machine learning training requested by an administrative system consumer, according to one or more exemplary embodiments of the present disclosure. Fig. Figure 4 shows an exemplary test request and reporting procedure according to one or more embodiments of the present disclosure. Fig.Figure 5 shows a set of classes that encapsulates information relevant for machine learning model testing, according to one or more embodiments of the present disclosure. Fig. Figure 6 shows relationships of information models with respect to joint ML testing according to one or more exemplary embodiments of the present disclosure. Fig. Figure 7 represents an exemplary network architecture according to one or more exemplary embodiments of the present disclosure. Fig. Figure 8 schematically illustrates a wireless network according to one or more exemplary embodiments of the present disclosure. Fig. Figure 9 illustrates components that can read instructions from a machine-readable or computer-readable medium according to one or more exemplary embodiments of the present disclosure. Fig.Figure 10 illustrates an exemplary cellular network according to one or more exemplary embodiments of the present disclosure. Fig. Figure 11 shows exemplary network deployments, including an exemplary Next-Generation Fronthaul (NGF) deployment, according to one or more exemplary embodiments of the present disclosure. Fig. 12 represents an example of an Administrative Services (MnS) deployment, according to one or more exemplary embodiments of the present disclosure. Fig. Figure 13 shows an exemplary framework for providing MnS according to one or more exemplary embodiments of the present disclosure. Fig. Figure 14 represents an exemplary AI / ML-supported communication network according to one or more embodiments of the present disclosure. Fig.Figure 15 illustrates an exemplary neural network according to one or more embodiments. Fig. Figure 16 shows an RL architecture according to one or more exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0004] The following description and drawings sufficiently illustrate specific embodiments to enable a person skilled in the art to implement them. Other embodiments may include structural, logical, electrical, process-related, algorithm-related, and other modifications. Parts and features of some embodiments may be included in or replace other embodiments. Embodiments set forth in the claims include all available equivalents of those claims.

[0005] Wireless devices can operate as defined by technical standards. For cellular telecommunications, this is defined by the 3. rdGeneration Partnership Program (3GPP) communication techniques, including the use of artificial intelligence (AI) / machine learning (ML) techniques.

[0006] During the training phase of a machine learning (ML) entity (also called an ML model), the ML entity must be tested after training and validation to evaluate its performance when performing inference using the test data. Testing may involve interaction with third parties (e.g., in addition to the developer of the ML training function), and operators may use the ML training function or third-party systems / functions that rely on the inference results computed by the ML entity for testing.

[0007] If the test performance is not acceptable or does not meet the predefined requirements, an AI / ML test consumer (e.g., an MnS consumer) can request the ML training generator to retrain the ML entity with specific training data and / or performance requirements.

[0008] A group of ML entities can work in a coordinated manner for complex use cases. In such cases, an ML entity is just one step in the inference process of an AI / ML inference function, where the inference outputs of one ML entity become the inputs to the next ML entity.

[0009] As agreed in 3GPP DraftCR S5-234847, the group of ML entities may need to be tested together. The group, including all contained ML entities, must be tested.

[0010] ML testing can be requested by the MnS (Management Service) consumer or initiated by the MnS producer.

[0011] After the ML testing of the group, the MnS producer provides the test results to the consumer.

[0012] This disclosure provides a solution for co-testing multiple machine learning entities in 3GPP wireless communications using information object classes (IOCs) managed by the operations and notifications of the generic deployment management service defined in 3GPP TS 28.532. The IOC (information object class) labels and attribute names herein can be treated as examples, as they can be named differently.

[0013] 3GPP TS 28.105 may include network resource modeling techniques to support co-testing of multiple ML entities based on this disclosure. An IOC entity, referred to as an MLEntity, may represent the ML entity and may include several types of contexts: (1) TrainingContext, which is the context in which the ML entity was trained; (2) ExpectedRunTimeContext, which is the context in which an ML entity is expected to be used; and / or (3) RunTimeContext, which is the context in which the MLEntity is used. Table 1 below includes attributes for an MLEntity: Table 1: MLEntity Attributes: Attribute label Support Qualifier isReadable isWritable isInvariant(isInvariant) isNotifyable(isNotifyable) mLEntityld (ML entity ID) M T F F T inferenceType M T F F T mLEntityVersion(ML entity version) M T F F T expectedRunTimeContext O T T F T trainingContext(training context) CM T F F T runTimeContext (Runtime context) O T F F T mlEntityGroupids(ML-Entity group IDs) O T T F T

[0014] Table 2 below shows attribute restrictions for the MLEntity attributes from Table 1: Table 2: MLEntity attribute restrictions: Designation definition trainingContext SupportQualifier Condition: The trainingContext represents the status and conditions relating to the training and should be added when the training is complete.

[0015] An MLEntityGroup (ML entity group) can be an Information Order Group (IOC) representing a group of ML entities that perform ML inferences in a coordinated manner. How the ML entities in an MLEntityGroup coordinate is described by the relationships between the ML entities.

[0016] Table 3 below shows attributes of an MLEntityGroup: Table 3: MLEntity group attributes: Attribute designation nung Support Qualifier (Support qualification) (certifier) isRead able (istLes bear) isWritable (isScream) bbar) is invaria nt (istInvari ant) isNotifyable (isNotice tigbar) mlEntityGroup Profile (ML entity group) nprofil) M T F F T

[0017] There may be no attribute restrictions for an MLEntityGroup attribute.

[0018] MLEntitiesRelationPerLevel < <datatype>> (ML entity relationship per level < <datentyp>The `>` parameter can refer to a data type that represents the relationships between ML entities for each level of an MLEntityGroup. The relationship between ML entities within each level can be sequential or parallel. The output of one level of the ML entity group can be used as input for the next (e.g., higher) level. The level can be specified by a number, starting from one, where larger numbers represent higher levels of the ML entity group.

[0019] Table 4 below shows example attributes for a MLEntitiesRelationPerLevel < <datatype>>: Table 4: MLEntitiesRelationPerLevel < <datatype>> Attributes: Attribute name hung Support Qualifier (Support qualification) (certifier) isRead able (istLes bear) isWritable (iswriting bear) is invaria nt (istInvari ant) isNotifyable (isNotification igbar) levelNumber (Level Number) M T F F F mLEntitiesrelation (ML entity relationship) M T F F F attribute , the focusing on role refers to mLEntityrefList M T F F T

[0020] The attributes in Table 4 may not have any attribute restrictions.

[0021] ML TestingRequest (ML Test Request) can refer to an IOC that represents the ML entity test request generated by an ML Test MnS consumer.

[0022] The MLTestingRequest MOI can be enclosed within an MLTestingFunction MOI or ML TrainingFunction MOI, which represents the logical function that performs the ML entity tests. Each MLTestingRequest can be associated with at least one ML entity.

[0023] If the request is accepted, the ML test MnS generator decides when to start the ML tests. Once the MnS generator decides to start the test based on the request, it executes the following: • collects (more) data for testing if the test data is unavailable or the data is available but insufficient for testing; • prepares and selects the necessary test data; • tests the ML entity by performing an inference using the selected test data, and • Reports the performance of the ML entity when it performs on the selected test data.

[0024] The MLTestingRequest can have a requestStatus field to represent the status of the request. The attribute values ​​can be "NOT_STARTED", "TESTING_IN_PROGRESS", "SUSPENDED", "FINISHED", and "CANCELED".

[0025] Table 5 below shows attributes of MLTestingRequest: Table 5: MLTestingRequest attributes: Attribute label ung Support Qualifier (Support qua (liffer) isRead able (istLes bear) isWritable e (isScream) bbar) is invaria nt (istInvar iant) isNotifyable (isNotice tigbar) requestStatus (Request Status) M T F F T cancelRequest (cancellation request) O T T F T suspendRequest O T T F T mlEntityGroupToTest (ML entity groups to be tested) O T T F T attribute , the focus on the role refers to mLEntityToTestRef (reference of the ML entity to be tested) CM T F F T mlEntityGroupToTestRef (reference of the ML entity group to be tested) CM T F F T

[0026] The attributes in Table 5 may have the following attribute restrictions, as shown below in Table 6: Table 6: MLTestingRequest attribute restrictions: Designation definition mLEntityToTestRef SupportQualifier (Support qualifier for Condition: The MLTestingRequest MOI represents the request to test a single ML entity. (Reference of the ML entity to be tested) mlEntityGroupToTestRef SupportQualifier (Support qualifier for reference of the ML entity group to be tested) Condition: The MLTestingRequest MOI represents the requirement to jointly test a group of ML entities.

[0027] ML TestingReport can be an IOC representing the ML test report provided by the ML test MnS generator.

[0028] The MLTestingReport MOI can be enclosed within an MLTestingFunction MOI or ML TrainingFunction MOI, which represents the logical function that performs the ML entity tests.

[0029] For joint testing of a group of ML entities, the ML test report can include the test results for each ML entity in the group.

[0030] Table 7 below shows the MLTestingReport attributes: Table 7: ML TestingReport Attributes: Attribute designation g Support Qualifier (Support source) (alifier) isRead able (istLes bear) isWritable e (isScream) bbar) is invari ant (istInvar iant) isNotifyable (istBenachric (htigbar) modelPerformanceTesting(model performance testing) M T F F T mlTestingResult(ML-Testergebnis) M T F F T attribute , that me on the role refers to testingRequestRef (Test Request Reference) CM T F F T

[0031] Table 8 below shows attribute restrictions for the ML TestingReport attributes: Table 8: ML TestingReport attribute restrictions: Designation definition testingRequestRef SupportQualifier (Support Qualifier for Test Request Reference) Condition: The MLTestingReport-MOI represents the report for the ML model testing requested by the MnS consumer (via MLTestingRequest-MOI).

[0032] Table 9 below shows MLTestingReport attribute definitions: Table 9: MLTestingReport attribute definitions: Attribute label Documentation and permissible values Characteristics mLEntityld (ML entity ID) It identifies the ML entity. Type: String It is unique in every MnS generator. multiplicity: 1 allowedValues ​​(permitted values): n / a isOrdered: n / a isUnique: n / a defaultValue(Default value): None isNullable(isNullable): True performanceMetric It specifies the performance metric used to evaluate the performance of an ML entity, e.g., “accuracy”, “precision”, “F1score”, etc. Type: String multiplicity: 1isOrdered: n / a allowedValues ​​(permitted values): n / a isUnique: True defaultValue: None isNullable: False performanceScore (performance rating) It indicates the performance rating (in units of percent) of an ML entity when inference is performed on a specific dataset (Grade)(Note). Type: Real multiplicity: 1isOrdered: n / a Performance metrics can vary depending on the type of model used for different types of machine learning (ML) models. For example, the metric for numerical prediction might be accuracy; for classification, the metric might be a combination of precision and recall, such as the "F1 score". Allowed values: { 0..100}. isUnique: kAdefaultValue: NoneisNullable: False cancelRequest (cancellation request) It indicates whether the ML training MnS consumer cancels the ML training request. Setting this attribute to "TRUE" cancels the ML training request. Cancellation is possible if the requestStatus is "NOT_STARTED*", "TRAINING_IN_PROGRESS", or "SUSPENDED". Setting the attribute to "FALSE" has no observable result. The default value is set to "FALSE". AllowedValues: TRUE, FALSE. Type: Boolean multiplicity: 0..1 isOrdered: kAisUnique: kDefaultValue: FALSE isNullable: False suspendRequest It indicates whether the ML training MnS consumer suspends the ML training request. Setting this attribute to "TRUE" suspends the ML training request. Suspension is possible if the requestStatus is not "FINISHED". Setting the attribute to "FALSE" has no observable result. The default value is set to "FALSE". AllowedValues: TRUE, FALSE. Type: Boolean multiplicity: 0..1 isOrdered: kAisUnique: kDefaultValue: FALSE isNullable: False ML TestingRequest.requestStatus (ML-Testanforderung.Anforderungsstatus) It describes the status of a specific ML test request. allowedValues: NOT_STARTED, TESTING_IN_PROGRESS, CANCELLING, SUSPENDED, FINISHED, and CANCELLED. Type: Enummultiplicity: 1isOrdered: kAisUnique: kAdefaultValue: NoneisNullable: False mLEntityToTestRef (reference of the ML entity to be tested) It identifies the DN of the ML entity requested for testing. allowedValues: DN Type: DN (see 3GPP TS 32.156

[13] ) multiplicity: 1 isOrdered: False isUnique: True defaultValue: None isNullable: True modelPerformanceTesting(model performance testing) It indicates the performance rating of the ML entity when run on the test data. allowedValues: n / a Type: Model Performance Multiplicity: * is Ordered: k A is Unique: k Default Value: None isNullable(isNullable): False mlTestingResult (ML test result) It provides the address where the test result (including the inference result for each test data sample) will be delivered. The detailed test result format is vendor-specific. Allowed values: n / a Type: String multiplicity: 1isOrdered:FalseUnique:TruedefaultValue:NoneisNullable:True testingRequestRef (Test Request Reference) It identifies the DN of the ML TestingRequest-MOL.allowedValues ​​(allowed values): DN Type: DN (see 3GPP TS 32.156

[13] ) multiplicity: 1 isOrdered: False isUnique: True defaultValue: None isNullable: True mlEntityGroupProfile (ML entity group profile) It identifies the ML entity group that was initially trained or retrained. allowedValues: n / a Type: ML EntitiesRelationPerLevel (ML entity relationship per level) multiplicity: *isOrdered: False isUnique: True defaultValue: None isNullable: False levelNumber (level number) It specifies the level number of the ML entities within an ML entity group. allowedValues: allowedValues: { 1..100}. Type: Integer multiplicity: 1isOrdered:FalseUnique:noneDefaultValue:NoneNullable:False MlEntitiesRelation (ML entity relationship) It specifies the relationship between ML entities at one level of an ML entity group. The relationship could be "insequence" (sequential) or "in parallel" (parallel). If the relationship is "in sequence," then the sequence is represented by the order of the ML entities provided in the `LEntityrefList` attribute. Allowed values: IN_SEQUENCE, IN_PARALLEL Type: Enummultiplicity: 1isOrdered:FalseUnique:noneAdefaultValue:NoneNullable:False memberMLEntityrefList It identifies the list of member ML entities within a level of an ML entity group. `allowedValues`: `DN list` Type: DN (see 3GPP TS 32.156

[13] ) multiplicity: *isOrdered: True isUnique: True defaultValue: None isNullable: True mlEntityGroupToTestRef(Reference of the ML entity group to be tested) It identifies the DN of the MlEntityGroup requested for testing. allowedValues ​​(allowed values): DN Type: DN (see 3GPP TS 32.156

[13] ) multiplicity: 0..1 isOrdered: False isUnique: True defaultValue: None isNullable: True mlEntityGroupIds (ML entity group IDs) It identifies the list of ML entity group IDs to which the ML entity is a member. An ML entity can belong to multiple groups. AllowedValues: n / a Type: Integer multiplicity: *isOrdered: False Unique: True Default value: None Nullable: True mlEntityGroupToTest (ML entity groups to be tested) It provides the ID of the ML entity group to be tested. allowedValues ​​(allowed values): n / a Type: Integer multiplicity: 1isOrdered: False isUnique: True defaultValue: None isNullable: True

[0033] Table 10 below shows the ML test results that the MnS producer provides to the consumer: Table 10: ML test results provided by the MnS producer to the MnS consumer: requirement drawing Description Related Use cases e REQ-ML_TEST-4 The ML test MnS generator should have a capability to allow an authorized consumer to request testing of a group of ML entities. Joint testing of multiple ML entities

[0034] The descriptions above are for illustrative purposes only and should not be considered exhaustive. Numerous other examples, configurations, processes, algorithms, etc., may exist, some of which are described in more detail below. Exemplary embodiments are now described with reference to the accompanying figures.

[0035] Fig. 1 is an exemplary operational artificial intelligence / machine learning workflow 100 according to one or more exemplary embodiments of the present disclosure.

[0036] With reference to Fig. 1. The operational AI / ML workflow 100 can represent a lifecycle of an ML entity in a wireless network. The workflow 100 can include a training phase 102, during which ML training 104 and ML testing 106 are performed; an emulation phase 108, during which ML emulation 110 is performed; a deployment phase 112, during which ML entity loading 114 takes place; and an inference phase 116, during which AI / ML inference 118 takes place.

[0037] Training Phase 102 comprises ML Training 104 (MLT) and ML Testing 106 (or ML Model Testing). In some implementations, some or all of the operational tasks of MLT 104 and / or ML Testing 106 can be performed by an MLT-MnS-P, while in other implementations, at least some of the operational tasks of MLT 104 and / or ML Testing 106 are performed by an MLT-MnS-C. In Training Phase 102, the ML entity is created based on learning from training data, while performance and trustworthiness are evaluated using validation data.

[0038] ML Testing 106 involves testing the validated ML entity to evaluate its performance on test data. If the test result meets expectations, the ML entity can proceed to the next phase; otherwise, it may need to be retrained. Additionally or alternatively, ML Testing 106 (or model testing) includes performing one or more processes to validate the ML model's performance using test data (or a test dataset). If the trained ML entity's performance meets expectations on both training and validation data, it is then tested on test data to evaluate its performance.If the test result meets expectations, the ML entity can be considered a candidate for use in relation to the intended use case or task; otherwise, the ML entity may need to be further (re)trained.

[0039] Fig. 2 is an exemplary functional overview and an exemplary service framework 200 for machine learning training according to one or more exemplary embodiments of the present disclosure.

[0040] An MLT function, acting as the MLT-MnS-P, can consume various data for MLT purposes. Fig. Figure 2 shows an example of the MLT capability provided via MLT-MnS in the context of SBMA for the authorized consumer(s) by the MLT-MnS-P.

[0041] The operational steps in the AI / ML workflow (e.g., as in Fig. 1 and Fig. 2) are supported by the specific AI / ML management capabilities, as discussed below.

[0042] In the operational environment, before the ML entity is used to perform inference, the ML model associated with the ML entity must be trained (e.g., by an MLT function, which can be a separate entity or an external entity to the AI / ML inference function). The MLT can be the initial training of an ML entity or the retraining of an already trained ML entity.

[0043] The ML entity is trained by the MLT-MnS-P, and the training can be triggered by one or more requests from one or more MLT-MnS-Cs or initiated by the MLT-MnS-P (e.g., as a result of a model performance assessment).

[0044] Fig. Figure 3 illustrates an exemplary machine learning training 300 requested by an MnS consumer, according to one or more exemplary embodiments of the present disclosure.

[0045] Fig. Figure 3 presents an example of MLT requested by an MLT-MnS consumer (MnS-C). Here, an MLT-MnS producer (MnS-P) acts as an MLT function. MLT capabilities are provided by the MLT-MnS-P to one or more MLT consumers (e.g., MLT-MnS consumers) in Fig. 3 provided. Examples include one or more consumers, one or more network functions (NFs) (e.g., an NWDAF containing an analysis logic function (AnLF)), management functions (MFs), RAN functions (RANFs) (see, e.g., Fig. 11), Edge compute nodes (or Edge compute nodes), application functions (AFs) (e.g., AF 760 in Fig. 7), include an operator (or operator roles) and / or other functional differentiation.

[0046] After the machine learning (ML) entity has been trained, tests and / or validation are performed to ensure the training process is successful. However, even if validation is successfully performed during ML entity development, it may still be necessary to test and verify that the ML entity functions correctly under specific runtime contexts or constraints. Therefore, the ML entity can be tested using a test dataset. This testing may involve interaction with third parties (in addition to the MLT-MnS-P (MLT function)). For example, the operator may use the MLT function or third-party systems / functions that rely on the results calculated by the ML entity for testing purposes.

[0047] Upon completion of the ML entity training, and if the performance of the trained ML entity meets the expectations for both training and validation data, the ML entity is made available to one or more MLT-MnS-Cs via the MLT report (see, for example, MLTrainingReport-IOC, discussed below and / or in [TS28105]). Before the ML entity is applied to the target AI / ML inference function, the MLT-MnS-P may need to allow the MLT-MnS-C to evaluate the performance of the ML entity through the ML testing process using the test data provided by the MLT-MnS-C. The test data exhibits the same pattern as the input portion of the training data. If the performance and trustworthiness of the trained ML entity meets the expectations for both training and validation data, the ML entity will be made available to one or more MLT-MnS-Cs.

[0048] There are different ways in which a group of machine learning (ML) entities can be coordinated. One example is when the output of one ML entity can be used as input to another, forming a sequence of interconnected ML entities. Another example is when multiple ML entities provide output in parallel (either the same output type, where outputs can be combined (e.g., using weights), or their outputs are needed in parallel as input to another ML entity). The group of ML entities must be used in a coordinated manner to support an AI / ML inference function.

[0049] Therefore, it is desirable that these coordinated ML entities can be trained or retrained together, so that the group of these ML entities can complete a more complex task together with better performance.

[0050] The joint ML entity training can be initiated by the MnS-P or the MnS-C, with the grouping of ML entities being shared between the MnS-P and the MnS-C.

[0051] During the ML entity training phase, the ML entity must be tested after training and validation to evaluate its performance when performing inference using the test data. Testing may involve interaction with third parties (besides the developer of the ML function), such as operators using the ML function or third-party systems / functions that may rely on the inference results calculated by the ML entity for testing purposes.

[0052] If the test performance is not acceptable or does not meet the predefined requirements, the consumer can request the MLT manufacturer to retrain the ML entity with specific training data and / or performance requirements.

[0053] After receiving an MLT report about a trained ML entity from the MLT MnS P, the consumer can request the ML Test MnS P to test the ML entity before applying it to the target inference function. ML testing performs inference on the tested ML entity using the test data as the inference inputs and generates the inference output for each test data sample. The ML Test MnS P can be the same as or different from the MLT MnS P.

[0054] After the machine learning (ML) test is complete, the ML-Test-MnS-P provides the user with a test report indicating the success or failure of the ML test. For a successful ML test, the test report includes the test results, such as the inference output for each test dataset sample.

[0055] In some examples, the ML-Test-MnS-P has the capabilities to provide the services needed to allow the consumer to request tests and receive results about testing an ML entity. Additionally or alternatively, the ML-Test-MnS-P has the capability to allow an authorized consumer (e.g., ML-Test-MnS-C) to request testing of a specific ML entity. Additionally or alternatively, the ML-Test-MnS-P has the capability to report the performance of the ML entity when it performs inference on the test data.

[0056] The ML entity tests can also be initiated by the MnS-P after the ML entity has been trained and validated. A consumer (e.g., an operator) may still need to define the guidelines (e.g., allowable time window, maximum number of test iterations, etc.) for testing a given ML entity. The consumer can predefine performance requirements for the ML entity testing, allowing the MnS-P to decide whether retraining / validation needs to be triggered. Retraining can be triggered by the test MnS-P itself based on the performance requirements provided by the MnS-C.

[0057] In some examples, the ML-Test-MnS-P has the ability to trigger the testing of an ML entity and to allow the MnS consumer to set the policy for testing.

[0058] During the ML entity training phase, the ML entity must be tested after training and validation to evaluate its performance when making inferences or otherwise generating results using the test data. Testing may involve interaction with third parties (e.g., entities other than the developer of the ML function). For example, operators may use the ML function, or third-party systems / functions may rely on the inference results calculated by the ML entity for testing purposes.

[0059] If the test performance is not acceptable or does not meet predefined or configured requirements, the consumer can request the MLT manufacturer to retrain the ML entity with specific training data and / or performance requirements.

[0060] A group of machine learning (ML) entities can work in a coordinated manner for complex use cases. In such cases, an ML entity is just one step in the inference process of an AI / ML inference function, where the inference outputs of one ML entity become the inputs to the next. The group of ML entities is created by the MLT function. The group, including all its ML entities, must be tested.

[0061] Fig. Figure 4 shows an exemplary test request and reporting procedure 400 according to one or more embodiments of the present disclosure.

[0062] With reference to Fig. 4. Can ML testing be performed by an MnS-C (e.g., the AI / ML test consumer in Fig. 4) requested or by the MnS-P (e.g., the AI / ML test generator in Fig. 4) be initiated. After the ML testing of the group, the MnS-P provides the test results to the consumer.

[0063] In some examples, the use case involves ML entity testing during the training phase and is irrelevant to the test cases that deployed the ML entities. In other examples, the ML Test MnS-P has a capability that allows an authorized consumer to request testing of a group of ML entities.

[0064] This disclosure provides solutions for jointly testing multiple machine learning entities. In various embodiments, jointly testing multiple machine learning entities involves using information object classes (IOCs) that are managed by the operations and notifications of the generic deployment management service defined in [TS28532]. In this way, the aspects discussed herein enable the integration of machine learning, intelligence, and automation into 5GS 700.

[0065] The following discussion provides various exemplary names / labels for different parameters, attributes, information elements (IEs), information object classes (IOCs), managed object classes (MOCs), and other elements / data structures; however, the specific names used with respect to the various parameters, attributes, IEs, IOCs, MOCs, and / or the like are provided for the purpose of discussion and illustration rather than limitation. It should be noted that the various parameters, attributes, IEs, IOCs, MOCs, etc., may have alternative names to those provided below, and in additional or alternative embodiments, implementations, and / or iterations of the 3GPP specifications, the names may differ but still fall within the context of this description.

[0066] Fig. Figure 5 shows a set of classes 500 that encapsulates information relevant for machine learning model testing, according to one or more embodiments of the present disclosure.

[0067] With reference to Fig. 5 there is a < <informationobjectclass>> (Information object class) MLEntity repository for names of a < <informationobjectclass>> of an MLEntityGroup (ML entity group) and for a < <informationobjectclass>> an MLEntity (ML entity). 3GPP TS 32.156 provides UML semantics.

[0068] Fig. Figure 6 shows relationships of 600 of information models with respect to joint ML testing according to one or more exemplary embodiments of the present disclosure.

[0069] With reference to Fig. 6 represents a < <ProxyClass» (Proxy-Klasse) TestingFunction (Testfunktion) IOCs, wie etwa MLTestingFunction oder MLTrainingFunction. Namen von < <informationobjectclass>> MLTestingRequest (ML test request) and < <informationobjectclass>> MLTestingReport (ML test report) can refer to the TestingFunction. For the MLTestingRequest, a < <informationobjectclass>> MLEntity (ML entity) and a < <informationobjectclass>> Refer to the MLEntityGroup (ML entity group) in relation to the MLTestingRequest. A < <proxyclass>> MLTestingEntity (ML Test Entity) can represent IOCs, such as Subnetwork, ManagedFunction, or ManagedElement, and the name of a < <informationobjectclass>> MLTestingFunction (ML-Testfunktion) can refer to the MLTestingEntity.

[0070] Fig. Figure 7 presents an exemplary network architecture 700. The network 700 can operate in a manner consistent with 3GPP technical specifications for LTE or 5G / NR systems. However, the exemplary embodiments are not limited in this respect, and the described examples may apply to other networks that benefit from the principles described herein, such as future 3GPP systems or the like.

[0071] The Network 700 includes a UE 702, which is a mobile or stationary computing device capable of communicating with a RAN 704 via an over-the-air connection. The UE 702 communicates with the RAN 704 through a Uu interface, which is applicable to both LTE and NR systems. Examples of UE 702 devices include smartphones, tablet computers, and wearable devices (e.g., smartwatches).Smartwatch, fitness tracker, smart glasses, smart clothing / fabrics, head-mounted displays, smart shows and / or the like), desktop computer, workstation, laptop computer, in-vehicle infotainment system, in-vehicle entertainment system, instrument cluster, head-up display (HUD) device, on-board diagnostic device, mobile dashtop device, mobile data terminal, electronic engine management system, electronic / engine control unit, electronic / engine control module, embedded system, sensor, microcontroller, control module, engine management system, networked device, machine communication device, machine-to-machine (M2M), device-to-device (D2D), machine communication (MTC) device, Internet of Things (IoT) device, smart device, flying drone or unmanned aerial vehicle (UAV), terrestrial drone or autonomous vehicle, robot, electronic signage, single-board computer (SBC) (e.g.Raspberry Pi, Arduino, Intel Edison and the like), plug-in computers and / or any type of computing device, such as any of those discussed here.

[0072] The Network 700 can include a set of UEs 702 directly coupled to each other via a D2D, ProSe, PC5, and / or Sidelink (SL) interface, and / or any other suitable interface, such as any of those discussed here. In 3GPP systems, SL communication involves communication between two or more UEs 702 using 3GPP technology without passing through a network node. These UEs 702 can be M2M / D2D / MTC / IoT devices and / or vehicle systems communicating using an SL interface, which may include, for example, one or more logical SL channels (e.g., Sidelink Broadcast Control Channel (SBCCH), Sidelink Control Channel (SCCH), and Sidelink Traffic Channel (STCH)). one or more SL transport channels (e.g., shared sidelink channel (SL-SCH) and sidelink broadcast channel) (SL-BCH); and one or more physical SL channels (e.g.,This includes a physical shared sidelink channel (PSSCH), physical sidelink control channel (PSCCH), physical sidelink feedback channel (PSFCH), physical sidelink broadcast channel (PSBCH), and / or the like. The UE 702 can perform blind decoding attempts of SL channels / links according to the various examples herein.

[0073] In some examples, the UE 702 can additionally communicate with an AP 706 via an over-the-air (OTA) connection. The AP 706 manages a WLAN connection that can offload some or all of the network traffic from the RAN 704. The connection between the UE 702 and the AP 706 can be consistent with any IEEE 802.11 protocol. Additionally, the UE 702, the RAN 704, and the AP 706 can utilize cellular WLAN aggregation (integration, e.g., LWA / LWIP). Cellular WLAN aggregation can involve the UE 702 being configured by the RAN 704 to utilize both cellular radio resources and WLAN resources.

[0074] The RAN 704 includes one or more Access Network Nodes (ANs) 708. The ANs 708 terminate one or more air interfaces for the UE 702 by providing access stratum protocols, including RRC, PDCP, RLC, MAC, and PHY / L1 protocols. In this way, the AN 708 enables data / voice connectivity between the CN 720 and the UE 702. The ANs 708 can be a macrocell base station or a low-power base station for providing femtocells, picocells, or other similar cells with smaller coverage areas, smaller user capacities, or higher bandwidths compared to macrocells; or a combination thereof. In these implementations, an AN 708 is referred to as BS, gNB, RAN node, eNB, ng-eNB, NodeB, RSU, TRxP, and the like.

[0075] One exemplary implementation is a "CU / DU split" architecture, where the ANs 708 are configured as a gNB central unit (CU) communicatively coupled to one or more distributed gNB units (DUs), each DU being communicatively coupled to one or more radio units (RUs) (also referred to as RRHs, RRUs, or the like). In some implementations, the one or more RUs may be individual RSUs. In some implementations, the CU / DU split may include an ng-eNB CU and one or more ng-eNB DUs instead of, or in addition to, the gNB CU or gNB DUs.The ANs 708s used as CUs can be implemented in a discrete device or as one or more software entities running on server computers, for example, as part of a virtual network that includes a virtual baseband unit (BBU) or BBU pool, Cloud RAN (CRAN), Radio Equipment Controller (REC), Radio Cloud Center (RCC), Centralized RAN (C-RAN), Virtualized RAN (vRAN), and / or the like (although these terms may refer to different implementation concepts). Any other type of architecture, arrangement, and / or configuration can be used.

[0076] The set of ANs 708 is interconnected via their respective X2 interfaces if the RAN 704 is an LTE-RAN or Evolved Universal Terrestrial Radio Access Network (E-UTRAN) 710, or via their respective Xn interfaces if the RAN 704 is an NG-RAN 714. The X2 / Xn interfaces, which in some examples may be separated into control / user-level interfaces, allow the ANs to communicate information related to handovers, data / context transfers, mobility, load management, interference coordination, and the like.

[0077] The ANs of the RAN 704 can each manage one or more cells, cell groups, component carriers, and the like to provide the UE 702 with an air interface for network access. The UE 702 can be connected simultaneously to a set of cells provided by the same or different ANs 708 of the RAN 704. For example, the UE 702 and the RAN 704 can use carrier aggregation to allow the UE 702 to connect to a set of component carriers, each corresponding to a Pcell or Scell. In dual connectivity scenarios, a first AN 708 can be a master node providing an MCG, and a second AN 708 can be a secondary node providing an SCG. The first / second AN 708 can be any combination of eNB, gNB, ng-eNB, and the like.

[0078] The RAN 704 can provide the air interface over either a licensed or an unlicensed spectrum. To operate in the unlicensed spectrum, nodes can use LAA, eLAA, and / or feLAA mechanisms based on CA technology with PCells / Scells. Before accessing the unlicensed spectrum, nodes can perform medium / carrier acquisition operations based, for example, on a Listen-Before-Talk (LBT) protocol.

[0079] Additionally or alternatively, individual UEs 702 deliver radio information to one or more NANs 708 and / or one or more edge computing nodes (e.g., edge servers / hosts and the like).

[0080] In V2X scenarios, the UE 702 or the AN 708 can be, or function as, a Roadside Unit (RSU), which can refer to any transportation infrastructure entity used for V2X communications. An RSU can be implemented in or by a suitable AN or a stationary (or relatively stationary) UE. An RSU can be implemented in or by: a UE can be referred to as a "UE-type RSU"; an eNB can be referred to as an "eNB-type RSU"; a gNB can be referred to as a "gNB-type RSU"; and so on. In one example, an RSU is a computing device coupled with a high-frequency circuitry located at the roadside, providing connectivity support for passing vehicle UEs.The RSU can also include an internal data storage circuitry to store intersection map geometry, traffic statistics, media, and applications / software for capturing and controlling ongoing vehicle and pedestrian traffic. The RSU can provide very low-latency communications required for high-speed events such as collision avoidance, traffic alerts, and the like. Additionally or alternatively, the RSU can provide other cellular / WLAN communication services. The RSU components can be housed in a weatherproof enclosure suitable for outdoor installation and can include network interface control to provide a wired connection (e.g., Ethernet) to a traffic signal controller or backhaul network. Furthermore, one or more V2X RATs can be deployed, enabling V2X nodes to communicate directly with each other and with infrastructure equipment (e.g.,AN 708) and / or other devices / nodes. In some implementations, at least two distinct V2X RATs may be used, including WLAN V2X (W-V2X) RATs based on IEEE V2X technologies (e.g., DSRC for the US and ITS-G5 for Europe) and cellular V2X (C-V2X) RATs based on 3GPP V2X technologies (e.g., LTE-V2X, 5G / NR-V2X, and beyond). In one example, the C-V2X RAT may use a C-V2X air interface, and the WLAN V2X RAT may use a W-V2X air interface.

[0081] The W-V2X RATs include, for example, IEEE Guide for Wireless Access in Vehicular Environments (WAVE) Architecture, IEEE Standards Association, IEEE 1609.0-2019 (April 10, 2019) (“[IEEE16090]”), V2X Communications Message Set Dictionary, SAE Int'1 (July 23, 2020) (“[J2735_202007]”), Intelligent Transport Systems in the 5 GHz Frequency Band (ITS-G5), the [IEEE80211p] (which is the Layer 1 (L1) and Layer 2 (L2) part of WAVE, DSRC and ITS-G5), and / or IEEE Standard for Air Interface for Broadband Wireless Access Systems, IEEE Std 802.16-2017, p. 1-2726 (March 2, 2018) (“[WiMAX]”). The term “DSRC” refers to vehicle communications in the 5.9 GHz frequency band, which is generally used in the United States, while “ITS-G5” refers to vehicle communications in the 5.9 GHz frequency band in Europe.Since any number of different RATs are applicable (including [IEEE80211p] RATs) that may be used in any geographical or political area, the terms “DSRC” (used in the USA, among other areas) and “ITS-G5” (used in Europe, among other areas) may be used interchangeably throughout this disclosure. The access layer for the ITS-G5 interface is outlined in ETSI EN 302 663 V1.3.1 (01-2020) (hereinafter “[EN302663]”) and describes the access layer of the ITS-S reference architecture. The ITS-G5 access layer includes [IEEE80211] (which now includes [IEEE80211p]) as well as features for Decentralized Congestion Control (DCC) methods, which are discussed in ETSI TS 102 687 V1.2.1 (04-2018) (“[TS102687]”). The access layer for a 3GPP LTE V2X-based interface(s) is described, among other things, in ETSI EN 303 613 V1.1.1 (01-2020) and 3GPP TS 23.285 v16.2.0 (12-2019) outlined; and 3GPP 5G / NR-V2X is outlined, among other things, in 3GPP TR 23.786 v16.1.0 (06-2019) and 3GPP TS 23.287 v18.0.0 (31-03-2023) (“[TS23287]”).

[0082] In examples where the RAN 704 is an E-UTRAN 710 with one or more eNBs 712, the E-UTRAN 710 provides an LTE air interface (Uu) with the parameters and characteristics discussed at least in 3GPP TS 36.300 v17.2.0 (30-09-2022) (“[TS36300]”). In examples where the RAN 704 is a Next-Generation (NG) RAN 714 with a set of gNBs 716, each gNB 716 connects to 5G-enabled UEs 702 using a 5G NR air interface (which may also be referred to as a Uu interface) with parameters and characteristics discussed in [TS38300] among many other 3GPP standards. If the NG-RAN 714 includes a set of ng-eNBs 718, one or more ng-eNBs 718 connect to a UE 702 via the 5G-Uu and / or LTE-Uu interface. The gNBs 716 and the ng-eNBs 718 are connected to the 5GC 740 via their respective NG interfaces, which include an N2 interface, an N3 interface, and / or other interfaces.The gNB 716 and the ng-eNB 718 are interconnected via an Xn interface. Additionally, individual gNBs 716 and ng-eNBs 718 are interconnected via their respective Xn interfaces. In some examples, the NG interface can be divided into two parts: an NG user-level (NG-U) interface, which carries traffic data between the nodes of the NG-RAN 714 and a UPF 748 (e.g., N3 interface), and an NG control-level (NG-C) interface, which is a signaling interface between the nodes of the NG-RAN 714 and an AMF 744 (e.g., N2 interface).

[0083] The NG-RAN 714 can provide a 5G-NR air interface (which may also be referred to as a Uu interface) with the following characteristics: variable SCS; CP-OFDM for DL, CP-OFDM and DFT-s-OFDM for UL; polar, repeat, simplex, and Reed-Muller codes for control; and LDPC for data. The 5G-NR air interface may rely on CSI-RS and PDSCH / PDCCH-DMRS, similar to the LTE air interface. The 5G-NR air interface may not use CRS but may use PBCH-DMRS for PBCH demodulation; PTRS for phase tracking of PDSCH; and a tracking reference signal for timing. The 5G NR air interface can operate on FR1 bands, which include sub-6 GHz bands, or FR2 bands, which include bands from 24.25 GHz to 52.6 GHz. The 5G NR air interface can include an SSB, which is a portion of a downlink resource grid that includes PSS / SSS / PBCH.

[0084] The 5G NR air interface can utilize BWPs for various purposes. For example, BWP can be used for dynamic SCS adjustment. For instance, the UE 702 can be configured with multiple BWPs, each with a different SCS. When a BWP change is specified to the UE 702, the transmission's SCS is also modified. Another use case for BWP is related to power saving. Specifically, multiple BWPs for the UE 702 can be configured with varying amounts of frequency resources (e.g., PRBs) to support data transmission under different traffic load scenarios. A BWP containing fewer PRBs can be used for data transmission during periods of low traffic load, while allowing power savings on the UE 702 and, in some cases, on the gNB 716.A BWP that contains a larger number of PRBs can be used for scenarios with higher traffic loads.

[0085] In some implementations, individual gNBs 716 can contain a gNB CU and a set of gNB DUs. Additionally or alternatively, the gNBs 716 can contain one or more RUs. In these implementations, the gNB CU can be connected to each gNB DU via its respective F1 interface. In the case of shared network access with multiple cell ID broadcasts, each cell identity associated with a subset of PLMNs corresponds to a gNB DU, and the gNB CU to which it is connected shares the same physical layer cell resources. For resilience, a gNB DU can be connected to multiple gNB CUs through appropriate implementation. Additionally, a gNB CU can be separated into gNB CU control plane (gNB CU CP) and gNB CU user plane (gNB CU UP) functions.The gNB-CU-CP is connected to a gNB-DU via an F1 control plane interface (F1-C), the gNB-CU-UP is connected to the gNB-DU via an F1 user plane interface (F1-U), and the gNB-CU-UP is connected to the gNB-CU-CP via an E1 interface. In some implementations, a gNB-DU is connected to only one gNB-CU-CP, and a gNB-CU-UP is connected to only one gNB-CU-CP. For resilience, a gNB-DU and / or a gNB-CU-UP can be connected to multiple gNB-CU-CPs through appropriate implementation. A gNB-DU can be connected to multiple gNB-CU-UPs under the control of the same gNB-CU-CP, and a gNB-CU-UP can be connected to multiple DUs under the control of the same gNB-CU-CP. Data forwarding between gNB-CU-UPs during an intra-gNB-CU-CP handover within a gNB can be supported by Xn-U.

[0086] Similarly, individual ng-eNBs 718 can contain an ng-eNB-CU and a set of ng-eNB-DUs. In these implementations, the ng-eNB-CU and each ng-eNB-DU are interconnected via their respective W1 interfaces. An ng-eNB can contain an ng-eNB-CU-CP, one or more ng-eNB-CU-UP(s), and one or more ng-eNB-DU(s). An ng-eNB-CU-CP and an ng-eNB-CU-UP are interconnected via the E1 interface. An ng-eNB-DU is interconnected via the W1-C interface with an ng-eNB-CU-CP and via the W1-U interface with an ng-eNB-CU-UP. The general principle described herein with gNB aspects also applies to ng-eNB aspects and corresponding E1 and W1 interfaces, unless explicitly stated otherwise.

[0087] The node hosting a user-plane portion of the PDCP protocol layer (e.g., gNB-CU, gNB-CU-UP, and, for EN-DC, MeNB or SgNB depending on the carrier split) performs user inactivity monitoring and also informs the node that has a control plane connection to the core network (e.g., via E1, X2, or similar) of its inactivity or (re)activation. The node hosting the RLC protocol layer (e.g., gNB-DU) can also perform user inactivity monitoring and further inform the node hosting the control plane (e.g., gNB-CU or gNB-CU-CP) of its inactivity or (re)activation.

[0088] In these implementations, the NG-RAN 714 is layered into a radio network layer (RNL) and a transport network layer (TNL). The NG-RAN architecture 714 (e.g., the logical NG-RAN nodes and the interfaces between them) is part of the RNL. For each NG-RAN interface (e.g., NG, Xn, F1, and the like), the associated TNL protocol and functionality are specified. The TNL provides services for user-level transport and / or signaling transport. In NG-Flex configurations, each NG-RAN node is connected to all AMFs 744 of AMF sets within an AMF range that supports at least one slice, which is also supported by the NG-RAN node. The AMF set and AMF range are defined in [TS23501].

[0089] The RAN 704 is communicatively coupled to the CN 720, which contains network elements and / or network functions (NFs) to provide various functions to support data and telecommunications services for customers / subscribers (e.g., UE 702). The components of the CN 720 can be implemented in a single physical node or in separate physical nodes. In some examples, NFV can be used to virtualize any or all of the functions provided by the network elements of the CN 720 onto physical computing / storage resources in servers, switches, and the like. A logical instantiation of the CN 720 can be referred to as a network slice, and a logical instantiation of a portion of the CN 720 can be referred to as a network subslice.

[0090] In the example of Fig. 7. The CN 740 is a 5GC 740 760, which includes an Authentication Server Function (AUSF) 742, an Access and Mobility Management Function (AMF) 744, a Session Management Function (SMF) 746, a User Layer Function (UPF) 748, a Network Slice Selection Function (NSSF) 750, a Network Discovery Function (NEF) 752, a Network Repository Function (NRF) 754, a Policy Control Function (PCF) 756, a Unified Data Management (UDM) 758, and a Network Data Analysis Function (NWDAF) 762, which are coupled to each other via various interfaces, as shown. The NFs in the 5GC 740 are briefly described below.

[0091] The NWDAF 762 includes one or more of the following functionalities: support for data collection from NFs and AFs 760; support for data collection from OAM; NWDAF service registration and metadata discovery against NFs and AFs 760; support for providing analytical information to NFs and AFs 760; and support for machine learning model training and deployment to NWDAF(s) 762 (e.g., those containing analytical logic functionality). Some or all of the NWDAF functionalities can be supported in a single instance of an NWDAF 762. The NWDAF 762 also includes an analytical reporting capability, which comprises means that enable discovery of the type of analytics that can be consumed by an external party and / or the request to consume analytical information generated by the NWDAF 762.

[0092] The NWDAF 762 interacts with various entities for different purposes, such as one or more of the following: data collection based on a subscription of events provided by the AMF 744, SMF 746, PCF 756, UDM 758, NSACF, AF 760 (directly or via NEF 752), and OAM (not shown); analysis and data collection using the Data Collection Coordination Function (DCCF); retrieving information from data repositories (e.g., UDR via UDM 758 for subscriber-related information); data collection of location information from the LCS system; storing and retrieving information from an Analysis Data Repository Function (ADRF); analysis and data collection from a Messaging Framework Adaptor Function (MFAF); retrieving information about NFs (e.g.,of NF 754 for NF-related information); on-demand provision of analytics to consumers, as specified in clause 6 of [TS23288]; and / or provision of bulk data relating to analysis ID(s). NWDAF discovery and selection procedures are discussed in clause 6.3.13 in [TS23501] and clause 5.2 of [TS23288].

[0093] A single instance or multiple instances of the NWDAF 762 can be deployed in a PLMN. If multiple instances of the NWDAF 762 are deployed, the architecture supports the use of the NWDAF 762 as a central NF, as a collection of distributed NFs, or as a combination of both. If multiple instances of the NWDAF 762 are deployed, one NWDAF 762 can act as an aggregation point (e.g., Aggregator NWDAF 762) and collect analytical information from other NWDAFs 762s, which may have different service areas, to generate the aggregated analysis (e.g., per analysis ID), possibly with analyses generated by itself. If multiple NWDAFs 762s exist, not all need to be capable of providing the same type of analytical results. For example, some of the NWDAFs 762s may be specialized in providing specific types of analysis.An analysis ID information element is used to identify the type of supported analyses that the NWDAF 762 can generate. In some implementations, one or more instances of the NWDAF 762 may be collocated with a 5GS-NF. Additional aspects of the NWDAF 762's functionality are defined in 3GPP TS 23.288 v18.2.0 (21-06-2023) ("[TS23288]").

[0094] Different instances of the NWDAF 762 can exist within the 5GC 740, with possible specializations for each analysis type. The capabilities of an NWDAF 762 instance are described in the NWDAF profile, which is stored in the NRF 754. The NWDAF architecture allows multiple NWDAF 762 instances to be arranged in a hierarchy / tree with a flexible number of layers / branches. The number and organization of the hierarchy layers, as well as the capabilities of each NWDAF 762 instance, remain deployment options and can vary depending on the implementation and / or use case. In a hierarchical deployment, the NWDAFs 762 can provide a data acquisition discovery capability to generate analyses based on data collected by other NWDAFs 762 when the DCCFs 763 and / or MFAFs 765 are not present on the network.

[0095] The AUSF 742 stores data for authenticating the UE 702 and handles authentication-related functionality. The AUSF 742 can enable a common authentication framework for different access types.

[0096] The AMF 744 allows other functions of the 5GC 740 to communicate with the UE 702 and the RAN 704 and to subscribe to notifications about mobility events related to the UE 702. The AMF 744 is also responsible for registration management (e.g., registering the UE 702), connection management, reachability management, mobility management, lawful interception of AMF-related events, and access authentication and authorization. The AMF 744 provides transport for SM messages between the UE 702 and the SMF 746 and acts as a transparent proxy for routing SM messages. The AMF 744 also provides transport for SMS messages between the UE 702 and an SMSF. The AMF 744 interacts with the AUSF 742 and the UE 702 to perform various security anchor and context management functions.Furthermore, the AMF 744 is a termination point of a RAN-CP interface, which includes the N2 reference point between the RAN 704 and the AMF 744. The AMF 744 is also a termination point of the NAS(N1) signaling and performs NAS encryption and integrity protection.

[0097] The AMF 744 also supports NAS signaling with the UE 702 via an N3IWF interface. The N3IWF enables access to untrusted entities. The N3IWF can serve as a termination point for the N2 interface between the (R)AN 704 and the AMF 744 at the control plane and as a termination point for the N3 reference point between the (R)AN 704 and the 748 at the user plane. The AMF 744 handles N2 signaling from the SMF 746 and the AMF 744 for PDU sessions and QoS, encapsulates packets for IPSec and N3 tunneling, marks N3 packets at the user plane in the UL, and enforces QoS according to the N3 packet markup, taking into account the QoS requirements associated with such a markup received over N2.The N3IWF can also forward UL and DL control plane NAS signaling between the UE 702 and the AMF 744 via an N1 reference point between the UE 702 and the AMF 744, and forward UL and DL user plane packets between the UE 702 and the UPF 748. The N3IWF also provides mechanisms for establishing IPsec tunnels with the UE 702. The AMF 744 can have a Namf service-based interface and can provide a termination point for an N14 reference point between two AMF 744s and an N17 reference point between the AMF 744 and a 5G EIR (in ). Fig. (not shown in Figure 7). In addition to the functionality of the AMF 744 described herein, the AMF 744 can provide support for network slice constraints and network slice instance constraints based on NWDAF analysis.

[0098] The SMF 746 is responsible for SM (e.g., session setup, tunnel management between the UPF 748 and an AN 708); UE IP address assignment and management (including optional authorization); selection and control of a UP function; configuration of traffic control at the UPF 748 to route traffic to a suitable destination; completion of interfaces to policy control functions; control of part of policy enforcement, charge calculation, and QoS; lawful interception (for SM events and interface to the LI system); completion of SM portions of NAS messages; DL data notification; initiation of AN-specific SM information sent via the AMF 744 over N2 to the AN 708; and determination of a session's SSC mode. SM refers to the management of a PDU session, and a PDU session or "session" refers to a PDU connectivity service that provides or enables the exchange of PDUs between the UE 702 and the DN 736.The SMF 746 can also include the following functionalities to support edge computing enhancements (see, for example, [TS23548]): selection of the EASDF 761 and provision of its address to the UE as the DNS server for the PDU session; use of EASDF 761 services as defined in [TS23548]; and, to support the application layer architecture defined in [TS23558], provision and updates of ECS address configuration information to the UE. Discovery and selection procedures for EASDF 761 are discussed in [TS23501] Section 6.3.23.

[0099] The UPF 748 acts as an anchor point for intra-RAT and inter-RAT mobility, an external PDU session point for the interconnect with the 736 data network, and a branch point to support a multi-homed PDU session. The UPF 748 also performs packet routing and forwarding, packet inspection, enforcement of the user-level portion of policy rules, lawful packet interception (UP collection), traffic usage reporting, user-level QoS handling (e.g., packet filtering, gating, UL / DL rate enforcement), UL traffic verification (e.g., SDF-to-QoS flow mapping), transport-level packet marking in UL and DL, and DL packet buffering and DL data notification triggering. The UPF 748 can include a UL classifier to assist in routing traffic flows to a data network.

[0100] The NSSF 750 selects a set of network slice instances to serve the UE 702. The NSSF 750 also determines, if necessary, permissible NSSAIs and the mapping to the subscribed S-NSSAIs. The NSSF 750 also determines an AMF set to be used to serve the UE 702, or a list of candidate AMFs 744 based on a suitable configuration and possibly by querying the NRF 754. The selection of a network slice instance set for the UE 702 can be triggered by the AMF 744 with which the UE 702 is registered, by interacting with the NSSF 750; this can result in a change to the AMF 744. The NSSF 750 interacts with the AMF 744 via an N22 reference point. and can communicate with another NSSF in a visited network via an N31 reference point (not shown).

[0101] The NEF 752 uncovers services and capabilities provided by 3GPP NFs for third parties, internal discovery / re-discovery, AFs 760, edge computing networks / frameworks, and the like. In such examples, the NEF 752 can authenticate, authorize, or throttle AFs 760. The NEF 752 stores / retrieves information as structured data using the Nudr interface in a unified data repository (UDR). The NEF 752 also translates information exchanged with AF 760 and information exchanged with internal NFs. For example, the NEF 752 can translate between an AF service identifier and internal 5GC information, such as DNNs and S-NSSAI, as described in clause 5.6.7 of [TS23501]. In particular, the NEF 752 handles the masking of network and user-sensitive information to external AFs 760 according to the network policy.The NEF 752 also receives information from other NFs based on the capabilities of those NFs. This information can be stored on the NEF 752 as structured data or on a data storage NF using standardized interfaces. The stored information can then be re-uncovered by the NEF 752 to other NFs and AFs or used for other purposes, such as analytics. For example, NWDAF analyses can be securely uncovered by the NEF 752 for external parties, as specified in [TS23288]. Furthermore, data provided by an external party can be collected by the NWDAF 762 via the NEF 752 for the purpose of generating analyses. The NEF 752 handles and forwards requests and notifications between the NWDAF 762 and AF(s) 760, as specified in [TS23288].

[0102] The NRF 754 supports service discovery functions, receives NF discovery requests from NF instances, and provides information from discovered NF instances to the requesting NF instances. The NRF 754 also maintains NF profiles of available NF instances and their supported services. The NF profile of an NF instance held in the NRF 754 includes the following information: NF instance ID; NF type; PLMN ID in the case of PLMN, PLMN ID + NID in the case of SNPN; network slice-related identifier(s) (e.g., S-NSSAI, NSI ID). Network address(es) of an NF (e.g., FQDN, IP address, and / or the like), NF capacity information, NF priority information (e.g., for AMF selection), NF set ID, NF service set ID of the NF service instance; NF-specific service authorization information; names of supported services, if applicable; endpoint address(es) of instance(s) of each supported service; identification of stored data / information (e.g.,for UDR profile and / or other NF profiles); other service parameters (e.g., DNN or DNN list, LADN DNN or LADN DNN list, notification endpoint for each notification type the NF service is interested in receiving, and / or the like); location information for the NF instance (e.g., geographic location, data center, and / or the like); TAI(s); NF load information; routing indicator, Home Network Public Key identifier, for UDM 758 and AUSF 742; for UDM 758, AUSF 742, and NSSAAF in the case of accessing an SNPN using credentials belonging to a credential holder with an AAA server, identification of the credential holder (e.g.,the range of the network-specific identifier-based SUPI); for UDM 758 and AUSF 742, and if UDM 758 / AUSF 742 is used to access an SNPN using credentials belonging to a credential holder, identification of the credential holder (e.g., the range if network-specific identifier-based SUPI is used, or the MCC and MNC if IMSI-based SUPI is used); for AUSF 742 and NSSAAF in the case of SNPN onboarding using a DCS with an AAA server, identification of the DCS (e.g., the range of the network-specific identifier-based SUPI); For UDM 758 and AUSF 742, and if UDM 758 / AUSF 742 is used as the DCS in the case of SNPN onboarding, identification of the DCS (e.g., the range if network-specific identifier-based SUPI, or the MCC and MNC if IMSI-based SUPI); one or more GUAMI(s) in the case of the AMF 744; for the UPF 748, see clause 5.2.7.2.2 of [TS23502]; UDM group ID, range(s) of SUPIs, range(s) of GPSIs, range(s) of internal group identifiers, range(s) of external group identifiers for UDM 758; UDR group ID, range(s) of SUPIs, range(s) of GPSIs, range(s) of external group identifiers for UDR; AUSF group ID, range(s) of SUPIs for AUSF 742; PCF group ID, range(s) of SUPIs for PCF 756; HSS group ID, set(s) of IMPIs, set(s) of IMPUs, set(s) of IMSIs, set(s) of PSIs, set(s) of MSISDNs for HSS; Event ID(s) supported by AFs 760, in the case of NEF 752; Event Detection Service-supported event ID(s) by UPF 748; application identifier(s) supported by AFs 760, in the case of NEF 752; range(s) of external identifiers or range(s) of external group identifiers or the domain names served by NEF, in the case of NEF 752 (e.g.used when the NEF 752 AF reveals information for analysis purposes, as detailed in [TS23288]; Additionally, NRF 754 can store a mapping between UDM group ID and SUPI(s), UDR group ID and SUPI(s), AUSF group ID and SUPI(s), and PCF group ID and SUPI(s) to enable discovery of UDM 758, UDR, AUSF 742, and PCF 756 using SUPI, SUPI ranges as specified in clause 6.3 of [TS23501], and / or interact with UDR to resolve the UDM group ID / UDR group ID / AUSF group ID (PCF group ID) based on UE identity (e.g., SUPI); IP domain list, as specified in clause 6.1.6.2.21 of 3GPP TS 29.510 v18.2.0 (29-03-2023) (“[TS29510]”) described, range(s) of (UE) IPv4 addresses or range(s) of (UE) IPv6 prefixes, range(s) of SUPIs or range(s) of GPSIs or a BSF group ID, in the case of BSF; SCP domain to which NF belongs; DCCF service range information, NF data source types, NF data source set ID, if available, in the case of DCCF 763; supported DNAI list in the case of SMF 746; for SNPN, capability to support SNPN onboarding in the case of AMF, and capability to support user-level remote provisioning in the case of SMF 746; IP address range, DNAI for UPF 748; additional V2X-related NF profile parameters are defined in 3GPP TS 23.287; Additional ProSe-related NF profile parameters are defined in 3GPP TS 23.304; additional MBS-related NF profile parameters are defined in 3GPP TS 23.247; additional UAS-related NF profile parameters are defined in TS 23.256; discussed among many others in [TS23501].In some examples, service authorization information provided by an OAM system is also included in the NF profile if, for example, an NF instance has exceptional service authorization information.

[0103] For NWDAF 762, the NF profile includes: supported analysis ID(s), possibly per service; NWDAF service scope information (e.g., a list of TAIs for which NWDAF can provide services and / or data); supported analysis delay per analysis ID (if available); NF types of the NF data sources; NF set IDs of the NF data sources, if available; analysis aggregation capability (if available); analysis metadata provisioning capability (if available); ML model filter information parameter S-NSSAI(s) and area(s) of interest to the trained ML model(s) per analysis ID(s) (if available); federated learning (FL) capability type (e.g., FL server or FL client, if available); and time interval supported by FL (if available). The NWDAF service scope information 762 is common to all supported analysis IDs.The analysis IDs supported by NWDAF 762 can be associated with a supported analysis delay. For example, the analysis report can be generated with a time (including data collection and inference delays) less than or equal to the supported analysis delay. Determining the supported analysis delay and how NWDAF 762 avoids updating its supported analysis delay in NRF can often be specific to the NWDAF implementation.

[0104] The PCF 756 provides policy rules to control plane functions for enforcement and can also support a unified policy framework to govern network behavior. The PCF 756 can also implement a front end to access subscription information relevant to policy decisions in a UDR 759 or UDM 758. In addition to communicating with functions via reference points, as shown, the PCF 756 features a service-based Npcf interface.

[0105] The UDM 758 handles subscription-related information to support the management of communication sessions by network entities and stores subscription data from the UE 702. For example, subscription data can be communicated between the UDM 758 and the AMF 744 via an N8 reference point. The UDM 758 can comprise two parts: an application front end and a UDR. The UDR can store subscription and policy data for the UDM 758 and the PCF 756, and / or structured data for discovery and application data (including application detection PFDs and application request information for multiple UE 702s) for the NEF 752. The UDR can have the service-based Nudr interface to allow the UDM 758, PCF 756 and NEF 752 to access a special set of stored data, as well as to read and update notifications of relevant data changes in the UDR (e.g.The UDM 758 can add, modify, delete, and subscribe users. It can include a UDM frontend, which handles credential processing, site management, subscription management, and more. Multiple different frontends can serve the same user in different transactions. The UDM frontend accesses subscription information stored in the UDR and performs authentication credential processing, user identification handling, access authorization, registration / mobility management, and subscription management. In addition to communicating with other frontends via reference points, as shown, the UDM 758 can feature the service-based Nudm interface.

[0106] The Edge Application Server Discovery Function (EASDF) 761 has a service-based Neasdf interface and is connected to the SMF 746 via an N88 interface. One or more EASDF instances can be deployed within a PLMN, and interactions between 5GC-NF(s) and the EASDF 761 occur within a PLMN. The EASDF 761 includes one or more of the following functionalities: registering with the NRF 754 for discovery and selection of the EASDF 761; handling DNS messages according to the instruction from the SMF 746; and / or terminating DNS security, if used.Handling DNS messages according to the instructions from the SMF 746 includes one or more of the following functionalities: receiving DNS message handling rules and / or BaselineDNSPatters from the SMF 746; exchanging DNS messages to / with the UE 702; forwarding DNS messages to C-DNS or L-DNS for DNS querying; adding an EDNS Client Subnet (ECS) option to a DNS query for a FQDN; reporting information regarding received DNS messages to the SMF 746; and / or buffering / discarding DNS messages from the UE 702 or the DNS server. The EASDF has direct user-level connectivity (e.g., without any NAT) to the PSA-UPF over N6 for transmitting DNS signaling exchanged with the UE. The use of NAT between EASDF 761 and PSA-UPF 748 may or may not be supported. Additional aspects of EASDF 761 are discussed in [TS23548].

[0107] The AF 760 provides application influence over traffic routing, access to the NEF 752, and interacts with the policy framework for policy control. The AF 760 can influence (re)selection and traffic routing by the UPF 748. Depending on the operator's deployment, if the AF 760 is considered a trusted entity, the network operator may allow the AF 760 to interact directly with relevant network functions. In some implementations, the AF 760 is used for edge computing deployments.

[0108] An NF that needs to collect data from an AF 760 can subscribe to / unsubscribe from notifications regarding data collected by an AF 760 either directly from the AF 760 or via the NEF 752. The data collected by an AF 760 is used as input for analysis by the NWDAF 762. Details regarding data collected by an AF 760 and interactions between the NEF 752, the AF 760, and the NWDAF 762 are described in [TS23288].

[0109] The 5GC 740 can enable edge computing by selecting operator / third-party services that are geographically close to the point where the UE 702 connects to the network. This can reduce latency and network load. In edge computing implementations, the 5GC 740 can select a UPF 748 near the UE 702 and perform traffic routing from the UPF 748 to the DN 736 via the N6 interface. This can be based on UE subscription data, UE location, and information provided by the AF 760, allowing the AF 760 to influence UPF (re)selection and traffic routing.

[0110] The data network (DN) 736 can represent various network operator services, internet access, or third-party services that may be provided by one or more servers, including, for example, the application (app) / content server 738. The DN 736 can be, for example, an external public, a private PDN, or an internal operator packet data network for providing IMS services. In this example, the app server 738 may be coupled to an IMS via an S-CSCF or the I-CSCF. In some implementations, the DN 736 may represent one or more local DNs (LADNs), which are DNs 736 (or DN names (DNNs)) that a UE 702 can access in one or more specific scopes. Outside of these specific scopes, the UE 702 is unable to access the LADN / DN 736.

[0111] Additionally or alternatively, the DN 736 can be an Edge DN 736, which is a (local) DN that supports the architecture for enabling edge applications. In these examples, the App Server 738 can represent the physical hardware systems / devices that provide app server functionality and / or the application software that resides in the cloud or on an edge compute node that performs server function(s). In some examples, the App / Content Server 738 provides an edge hosting environment that offers the support required to run the edge application server.

[0112] The interfaces of the 5GC 740 include reference points and service-based interfaces. The reference points include: N1 (between UE 702 and AMF 744), N2 (between RAN 714 and AMF 744), N3 (between RAN 714 and UPF 748), N4 (between SMF 746 and UPF 748), N5 (between PCF 756 and AF 760), N6 (between UPF 748 and DN 736), N7 (between SMF 746 and PCF 756), N8 (between UDM 758 and AMF 744), N9 (between two UPF 748s), N10 (between UDM 758 and SMF 746), N11 (between AMF 744 and SMF 746), N12 (between AUSF 742 and AMF 744), N13 (between AUSF 742 and UDM 758). N14 (between two AMFs 744; not shown), N15 (between PCF 756 and AMF 744 in a non-roaming scenario, or between PCF 756 in a visited network and AMF 744 in a roaming scenario), N16 (between two SMFs 746; not shown), and N22 (between AMF 744 and NSSF 750). Others may also be present. Fig. Seven reference point representations, not shown, are used. The service-based representation in Fig. Section 7 represents NFs within the tax plane that allow other authorized NFs to access their services. Service-based interfaces (SBIs) include: Namf (SBI issued by AMF 744), Nsmf (SBI issued by SMF 746), Nnef (SBI issued by NEF 752), Npcf (SBI issued by PCF 756), Nudm (SBI issued by UDM 758), Naf (SBI issued by AF 760), Nnrf (SBI issued by NRF 754), Nnssf (SBI issued by NSSF 750), and Nausf (SBI issued by AUSF 742). Other service-based interfaces (e.g., Nudr, N5g-eir, and Nudsf) that are listed in Fig. The devices not shown in Figure 7 can also be used. In some examples, the NEF 752 can provide an interface to Edge Computing Nodes 736, which can be used to process wireless connections with the RAN 714.

[0113] Although this in Fig. Not shown in Figure 7, the Network 700 may also include NFs that are not shown, such as UDR, Unstructured Data Storage Function (UDSF), Network Slice Ingestion Control Function (NSACF), Network Slice Specific and Stand-alone Non-Public Network (SNPN) Authentication and Authorization Function (NSSAAF), UE Radio Capability Management Function (UCMF), 5G Device Identification Register (5G-EIR), Charge Calculation Function (CHF), Time-Sensitive Networking (TSN) AF 760, Time-Sensitive Communications and Time Synchronization Function (TSCTSF), Data Collection Coordination Function (DCCF), Analysis Data Repository Function (ADRF), Messaging Framework Adaptor Function (MFAF), Binding Support Function (BSF), Non-Seamless WLAN Offload Function (NSWOF), Service Communication Proxy (SCP), and Security Edge Protection Proxy (SEPP). Non-3GPP Interworking Function (N3IWF), Trusted Non-3GPP Gateway Function (TNGF),Wired Access Gateway Function (W-AGF) and / or Trusted WLAN Interworking Function (TWIF), as discussed in [TS23501].

[0114] Fig. Figure 8 schematically illustrates a Wireless Network 800. The Wireless Network 800 includes a UE 802 in wireless communication with an AN 804. The UE 802 and the AN 804 can be similar to and essentially interchangeable with the similarly named components described elsewhere herein.

[0115] The UE 802 can be communicatively coupled to the AN 804 via a Link 806. Link 806 is illustrated as an air interface to enable communicative coupling and can be consistent with cellular communication protocols, such as an LTE protocol or a 5G NR protocol operating at millimeter wave or sub-6 GHz frequencies.

[0116] The UE 802 includes a host platform 808 coupled to a modem platform 810. The host platform 808 includes an application processing circuit arrangement 812, which can be coupled to a protocol processing circuit arrangement 814 of the modem platform 810. The application processing circuit arrangement 812 can execute various applications for the UE 802 that produce / receive application data. The application processing circuit arrangement 812 can further implement one or more layer operations to transmit and receive application data to and from a data network. These layer operations include transport operations (for example, UDP) and internet operations (for example, IP).

[0117] The Protocol Processing Circuit Assembly 814 can implement one or more layer operations to enable the transmission or reception of data over the 806 link. The layer operations implemented by the Protocol Processing Circuit Assembly 814 include, for example, MAC, RLC, PDCP, RRC, and NAS operations.

[0118] The 810 modem platform can further include a digital baseband circuit arrangement 816, which can implement one or more layer operations that are “under” layer operations performed by the protocol processing circuit arrangement 814 in a network protocol stack. These operations include, for example, PHY operations, including one or more of HARQ-ACK functions, scrambling / descrambling, encoding / decoding, layer mapping / demapping, modulation symbol mapping, receive symbol / bit metric determination, multi-antenna port precoding / decoding, which includes one or more of spacetime, space frequency, or space coding, reference signal generation / detection, preamble sequence generation and / or decoding, synchronization sequence generation / detection, control channel signal blind decoding, and other related functions.

[0119] The modem platform 810 may further include a transmit circuit assembly 818, a receive circuit assembly 820, an RF circuit assembly 822, and an RF front end (RFFE) 824, which includes or is connected to one or more antenna panels 826. In short, the transmit circuit assembly 818 includes a digital-to-analog converter, a mixer, intermediate frequency (IF) components, and / or the like; the receive circuit assembly 820 includes an analog-to-digital converter, a mixer, IF components, and / or the like; the RF circuit assembly 822 includes a low-noise amplifier, a power amplifier, power tracking components, and / or the like. The RFFE 824 includes filters (for example, surface / volume acoustic wave filters), switches, antenna tuners, beamforming components (for example, phase array antenna components) and / or the like.The selection and arrangement of the components of the transmit circuit assembly 818, the receive circuit assembly 820, the RF circuit assembly 822, the RFFE 824, and the antenna panels 826 (generally referred to as "transmit / receive components") may depend on the specifics of a particular implementation, such as whether TDM or FDM communication is used, mmWave or sub-6 GHz frequencies, and / or the like. In some examples, the transmit / receive components may be arranged in multiple parallel transmit / receive chains, may be located on the same or different chips / modules, and / or the like.

[0120] In some examples, the protocol processing circuit arrangement 814 includes one or more instances of a control circuit arrangement (not shown) to provide control functions for the transmit / receive components.

[0121] UE reception can be established through and via the antenna panels 826, the RFFE 824, the RF circuit arrangement 822, the receive circuit arrangement 820, the digital baseband circuit arrangement 816, and the protocol processing circuit arrangement 814. In some examples, the antenna panels 826 can receive a transmission from the AN 804 by receiving beamforming signals received by a set of antennas / antenna elements of one or more antenna panels 826.

[0122] A UE transmission can be established through and via the protocol processing circuit arrangement 814, the digital baseband circuit arrangement 816, the transmit circuit arrangement 818, the RF circuit arrangement 822, the RFFE 824, and the antenna panels 826. In some examples, the transmit components of the UE 804 can apply a spatial filter to the data to be transmitted to form a transmit beam that is emitted by the antenna elements of the antenna panels 826.

[0123] Similar to the UE 802, the AN 804 includes a host platform 828 coupled to a modem platform 830. The host platform 828 includes an application processing circuit assembly 832 coupled to a protocol processing circuit assembly 834 of the modem platform 830. The modem platform may further include a digital baseband circuit assembly 836, a transmit circuit assembly 838, a receive circuit assembly 840, an RF circuit assembly 842, an RFFE circuit assembly 844, and antenna panels 846. The components of the AN 804 may be similar to, and substantially interchangeable with, the similarly named components of the UE 802.In addition to performing data transmission / data reception as described above, the components of the AN 808 can perform various logical functions, including, for example, RNC functions such as radio carrier management, dynamic uplink and downlink radio resource management, and data packet scheduling.

[0124] Examples of antenna elements for antenna panels 826 and / or antenna elements for antenna panels 846 include planar inverted F-antennas (PIFAs), monopole antennas, dipole antennas, loop antennas, patch antennas, Yagi antennas, parabolic antennas, omnidirectional antennas and / or the like.

[0125] Fig. Figure 9 illustrates components capable of reading instructions from a machine-readable or computer-readable medium (e.g., a non-volatile machine-readable storage medium) and executing one or more of the methodologies discussed herein. In particular, it shows Fig. 9 Hardware resources 900 including one or more processors (or processor cores) 910, one or more memory / storage devices 920, and one or more communication resources 930, each of which may be communicatively coupled via a bus 940 or other interface circuit arrangement. For examples where node virtualization (e.g., NFV) is used, a hypervisor 902 may be run to provide an execution environment for one or more network slices / subslices to utilize the hardware resources 900. In some examples, the hardware resources 900 may be implemented in or by a single compute node, which may be housed in an enclosure of various form factors.In other examples, the hardware resources 900 can be implemented through multiple compute nodes that can be deployed in one or more data centers and / or distributed across one or more geographical areas.

[0126] The 910 processors can, for example, include a 912 processor and a 914 processor. The 910 processors can be, for example, a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), such as a baseband processor, an ASIC, an FPGA, a radio frequency integrated circuit (RFIC), a microprocessor or microcontroller, a multi-core processor, a multi-threaded processor, an ultra-low voltage processor, an embedded processor, an xPU, a data processing unit (DPU), an infrastructure processing unit (IPU), a network processing unit (NPU), another processor (including any of those discussed herein), and / or any suitable combination thereof.

[0127] The memory / storage devices 920 can include main memory, disk storage, or any suitable combination thereof. The memory / storage devices 920 can include, among other things, any type of volatile, non-volatile, semi-volatile memory, and / or any combination thereof. Examples of memory / storage devices 920 include random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), conductive bridge random access memory (CB-RAM), spin transfer torque (STT) MRAM, phase-change RAM (PRAM), core memory, dual inline memory modules (DIMMs), microDIMMs, miniDIMMs, block-addressable memory device(s) (e.g., those based on NAND or NOR technologies).Single-Level Cell (SLC), Multi-Cell (MLC), Quad-Level Cell (QLC), Tri-Level Cell (TLC), or other NAND)) Read-only memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrical EPROM (EEPROM), Flash memory, Non-volatile RAM (NVRAM), Solid-state storage, Magnetic disk storage media, Optical storage media, Storage devices using chalcogenide glass, Multi-threshold NAND flash memory, NOR flash memory, Single- or multi-level phase-change memory (PCM) and / or phase-change memory with a switch (PCMS), NVM devices using chalcogenide phase-change material (e.g.chalcogenide glass), a resistive memory, nanowire memory, ferroelectric transistor random access memory (FeTRAM), antiferroelectric memory, magnetoresistive random access memory (MRAM) incorporating memristor technology, phase-change RAM (PRAM), resistive memory incorporating the metal oxide base, the oxygen vacancy base and conductive bridge random access memory (CB-RAM), or spin transfer torque (STT) MRAM, a spintronic magnetic junction memory-based device, a magnetic tunnel junction (MTJ)-based device, a domain wall (DW) and spin orbit transfer (SOT)-based device, a thyristor-based memory device and / or a combination of any of the above memory devices and / or other memory.

[0128] The communication resources 930 can include interlink or network interface controllers, components, or other suitable devices for communicating with one or more peripheral devices 904 or one or more databases 906 or other network elements over a network 908. For example, the communication resources 930 can include wired communication components (e.g., for coupling via USB, Ethernet, and / or the like), cellular communication components, NFC components, Bluetooth® (e.g., Bluetooth® Low Energy) components, WiFi® components, and other communication components.

[0129] The instructions 950 comprise software, program code, application(s), applet(s), app(s), firmware, microcode, machine code, and / or other executable code to cause any one of the processors 910 to perform one or more of the methodologies and / or techniques discussed herein. The instructions 950 may reside wholly or partially within at least one of the processors 910 (for example, within the processor's cache memory), the memory / storage devices 920, or any suitable combination thereof. Furthermore, any part of the instructions 950 may be transferred from any combination of the peripheral devices 904 or the databases 906 to the hardware resources 900. Accordingly, the memory of the processors 910, the memory / storage devices 920, the peripheral devices 904, and the databases 906 are examples of computer-readable and machine-readable media.

[0130] In some examples, the peripheral devices 904 can represent one or more sensors (also referred to as a "sensor circuit assembly"). The sensor circuit assembly includes devices, modules, or subsystems whose purpose is to detect events or changes in their environment and to transmit the information (sensor data) about the detected events to another device, module, subsystem, and / or the like. Individual sensors can be exteroceptive sensors (e.g., sensors that detect and / or measure environmental phenomena and / or external states), proprioceptive sensors (e.g., sensors that detect and / or measure internal states of a compute node or platform and / or individual components of a compute node or platform), and / or exproprioceptive sensors (e.g., sensors that detect, measure, or correlate internal and external states).Examples of such sensors include, but are not limited to, inertial measurement units (IMUs) comprising accelerometers, gyroscopes, and / or magnetometers; microelectromechanical systems (MEMS) or nanoelectromechanical systems (NEMS) comprising 3-axis accelerometers, 3-axis gyroscopes, and / or magnetometers; level sensors; flow sensors; temperature sensors (e.g., thermistors, including sensors for measuring the temperature of internal components and sensors for measuring a temperature outside the compute node or platform); pressure sensors; barometric pressure sensors; gravimeters; altimeters; image acquisition devices (e.g., cameras); LiDAR (light detection and distance measurement) sensors; proximity sensors (e.g., infrared radiation detectors and the like); depth sensors; ambient light sensors; optical light sensors; ultrasonic transceivers; microphones; and the like.

[0131] Additionally or alternatively, the peripheral devices 904 can represent one or more actuators that permit a compute node, platform, machine, device, mechanism, system, or other object to change its state, position, and / or orientation, or to move or control a compute node, platform, machine, device, mechanism, system, or other object. The actuators include electrical and / or mechanical devices for moving or controlling a mechanism or system and convert energy (e.g., electric current or moving air and / or fluid) into some kind of motion. For example, the actuators can be or include any number and combination of the following: soft actuators (e.g.,Actuators that change their shape in response to stimuli, such as mechanical, thermal, magnetic and / or electrical stimuli), hydraulic actuators, pneumatic actuators, mechanical actuators, electromechanical actuators (EMAs), microelectromechanical actuators, electrohydraulic actuators, linear actuators, linear motors, rotary motors, DC motors, stepper motors, servomechanisms, electromechanical switches, electromechanical relays (EMRs), circuit breakers, valve actuators, piezoelectric actuators and / or biomorphs, thermal biomorphs, solid-state actuators, solid-state relays (SSRs), shape memory alloy actuators, electroactive polymer actuators, integrated relay driver circuits (ICs), solenoids, imactive actuators / mechanisms (e.g.Jaws, claws, tweezers, clamps, hooks, mechanical fingers, humanoid dexterous robotic hands, and / or other gripping mechanisms that physically grasp an object by direct contact), drive actuators / mechanisms (e.g., wheels, axles, thrusters, propellers, motors, electric motors (e.g., those discussed previously), couplings, and the like), projectile actuators / mechanisms (e.g., mechanisms that launch or propel objects or elements), and / or acoustic tone generators, visual warning devices, and / or other similar electromechanical components. Additionally or alternatively, the actuators may include virtual instrumentation and / or virtualized actuator devices.Additionally or alternatively, the actuators may include various controllers and / or components of the compute node or platform (or components thereof), such as host controllers, cooling element controllers, baseboard management controller (BMC), platform control hub (PCH), uncore components (e.g., shared last-level cache (LLC), caching agent (Cbo), integrated memory controller (IMC), home agent (HA), power control unit (PCU), configuration agent (Ubox), integrated I / O controller (IIO), and interconnect (IX) link interfaces and / or controllers), and / or any other components, such as any of those discussed here.The compute node or platform can be designed to operate one or more actuators based on one or more captured events, instructions, control signals, and / or configurations received from a service provider, client device, and / or other components of the compute node or platform. Additionally or alternatively, the actuators are used to change the operating state (e.g., on / off, zoom or focus, and / or the like), position, and / or orientation of the sensors.

[0132] Fig. Figure 10 illustrates an exemplary cellular network 1000 according to one or more embodiments of the present disclosure.

[0133] The Network 1000 can operate in a manner consistent with 3GPP technical specifications or 6G system technical reports. In some examples, the Network 1000 can operate concurrently with the Network 700. For instance, in some examples, the Network 1000 can share one or more frequency or bandwidth resources with the Network 700. As a specific example, a UE (e.g., UE 1002) can be configured to operate in both the Network 1000 and the Network 700. Such a configuration can be based on a UE that incorporates a circuit arrangement designed to communicate using frequency and bandwidth resources of both the Network 700 and the Network 1000. In general, several elements of the Network 1000 can share one or more characteristics with elements of the Network 700. For brevity and clarity, such elements may not be repeated in the description of the Network 1000.

[0134] The Network 1000 can include a UE 1002, which can contain any mobile or non-mobile computing device designed to communicate with a RAN 1008 via an over-the-air connection. The UE 1002 can, for example, be similar to the UE 702.The UE 1002 can be, among other things, a smartphone, a tablet computer, a wearable computer device, a desktop computer, a laptop computer, in-vehicle infotainment, an in-vehicle entertainment device, a combination instrument, a head-up display device, an on-board diagnostic device, a mobile dashboard equipment, a mobile data terminal, an electronic engine management system, an electronic / engine control unit, an electronic / engine control module, an embedded system, a sensor, a microcontroller, a control module, an engine management system, a networked device, a machine-type communication device, an M2M or D2D device, an IoT device and / or the like.

[0135] Although in Fig. Not specifically shown in Figure 10, the Network 1000 may, in some examples, include a set of UEs directly coupled to each other via a sidelink interface. The UEs may be M2M / D2D devices communicating using physical sidelink channels, such as, but not limited to, PSBCH, PSDCH, PSSCH, PSCCH, PSFCH, and / or the like. Likewise, although this is shown in Figure 10, the network may also include a set of UEs directly coupled to each other via a sidelink interface. Fig. 10. Not specifically shown, UE 1002 may be communicatively coupled with an AP, such as AP 706, as with reference to Fig. 7 described. Although this in Fig. Unless specifically shown in section 10, RAN 1008 may additionally include one or more ANs, such as AN 708, in some examples, as shown in reference to Fig. 7 described. The RAN 1008 and / or the AN of the RAN 1008 may be referred to as a base station (BS), a RAN node, or by any other term or name.

[0136] The UE 1002 and the RAN 1008 can be configured to communicate over an air interface that may be referred to as a sixth-generation (6G) air interface. The 6G air interface may include one or more features, such as communication in a terahertz (THz) or sub-THz bandwidth, or shared communication and acquisition. As used herein, the term "shared communication and acquisition" may refer to a system that enables wireless communication as well as radar-based acquisition over various types of multiplexes. As used herein, THz or sub-THz bandwidths may refer to communication in the frequency ranges of 80 GHz and above. Such frequency ranges may additionally or alternatively be referred to as "millimeter wave" or "mmWave" frequency ranges.

[0137] The RAN 1008 enables communication between the UE 1002 and a 6G core network (CN) 1010. Specifically, the RAN 1008 allows the transmission and reception of data between the UE 1002 and the 6G-CN 1010. The 6G-CN 1010 can include various functions, such as NSSF 750, NEF 752, NRF 754, PCF 756, UDM 758, AF 760, SMF 746, and AUSF 742. The 6G-CN 1010 can additionally include UPF 748 and DN 736, as shown in [reference to relevant document / document]. Fig. 10 shown.

[0138] Additionally, the RAN 1008 can include various supplementary functions that are in addition to or alternative to the functions of a legacy cellular network, such as a 4G or 5G network. Two such functions can include a Compute Control Function (Comp-CF) 1024 and a Compute Service Function (Comp-SF) 1036. The Comp-CF 1024 and the Comp-SF 1036 can be parts or functions of the Compute Service Layer. The Comp-CF 1024 can be a control layer function that provides functionalities such as managing the Comp-SF 1036, creating and managing compute task contexts (e.g., creating, reading, modifying, deleting), interacting with the underlying compute infrastructure for compute resource management, and / or the like. The Comp-SF 1036 can be a user-level function that acts as the gateway to interface compute service users (such as the UE 1002) and compute nodes behind a Comp-SF instance.Some functionalities of the Comp-SF 1036 can include: parsing compute service data received from users to calculate tasks executable by compute nodes; maintaining a service mesh entry gateway or service API gateway; enforcing service and charge policies; performance monitoring and telemetry collection; and / or similar activities. In some examples, a single instance of the Comp-SF 1036 can act as the user-level gateway for a cluster of compute nodes. A single instance of the Comp-CF 1024 can control one or more instances of the Comp-SF 1036.

[0139] Two other such functions can include a Communication Control Function (Comm-CF) 1028 and a Communication Service Function (Comm-SF) 1038, which can be parts of the Communication Service Layer. The Comm-CF 1028 can be the control layer function for managing the Comm-SF 1038, creating / configuring / releasing communication sessions, and managing the communication session context. The Comm-SF 1038 can be a user layer function for data transport. The Comm-CF 1028 and the Comm-SF 1038 can be considered upgrades of the SMF 746 and the UPF 748, respectively, with respect to a 5G system in Fig. The upgrades provided by the Comm-CF 1028 and the Comm-SF 1038 can enable service-conscious transport. For legacy data transport (e.g., 4G or 5G), the SMF 746 and the UPF 748 can still be used.

[0140] Two other such functions can include a Data Control Function (Data-CF) 1022 and a Data Service Function (Data-SF) 1032, which can be parts of the data service layer. The Data-CF 1022 can be a control layer function and provides functionalities such as managing the Data-SF 1032, creating / configuring / sharing data services, managing the context of data services, and / or the like. The Data-SF 1032 can be a user layer function and act as the gateway between data service users (such as the UE 1002 and the various functions of the 6G-CN 1010) and data service endpoints behind the gateway. Specific functionalities can include: parsing data service user data and forwarding it to appropriate data service endpoints, generating charge data, and reporting the data service status.

[0141] Another such function is the Service Orchestration and Chaining Function (SOCF) 1020, which can discover, orchestrate, and chain communication / computing / data services provided by functions on the network. Upon receiving service requests from users, the SOCF 1020 can interact with one or more of the Comp-CF 1024, Comm-CF 1028, and Data-CF 1022 to identify instances of the Comp-SF 1036, Comm-SF 1038, and Data-SF 1032, configure service resources, and create the service chain. This chain could contain multiple instances of the Comp-SF 1036, Comm-SF 1038, and Data-SF 1032 and their associated compute endpoints. Workload processing and data movement can then be performed within the created service chain. SOCF 1020 can also be responsible for maintaining, updating, and releasing a created service chain.

[0142] Another such function is the Service Registry Function (SRF) 1012, which can act as a registry for system services provided at the user level, such as services provided by service endpoints behind gateways of the Comp-SF 1036 and the Data-SF 1032, and services provided by the UE 1002. The SRF 1012 can be considered a counterpart to the NRF 754, which can act as the registry for network functions.

[0143] Other such functions may include an Evolved Service Communications Proxy (eSCP) and a Service Infrastructure Control Function (SICF) 1026, which can provide a service communications infrastructure for control-plane and user-plane services. The eSCP may be related to the 5G Service Communications Proxy (SCP), adding user-plane service communications proxy capabilities. The eSCP is therefore expressed in two parts: eCSP-C 1012 and eSCP-U 1034 for control-plane and user-plane service communications proxy, respectively. The SICF 1026 can control and configure eCSP instances with respect to service traffic routing policies, access rules, load balancing configurations, performance monitoring, and / or the like.

[0144] Another such function is AMF 1044. AMF 1044 can be similar to 744, but with additional functionality. In particular, AMF 1044 can involve a potential functional repartitioning, such as moving the message forwarding functionality from AMF 1044 to RAN 1008.

[0145] Another such function is the Service Orchestration Disclosure Function (SOEF) 1018. The SOEF can be designed to disclose service orchestration and chaining services to external users, such as applications.

[0146] The UE 1002 can include an additional function called the Compute Client Service Function (Comp-CSF) 1004. The Comp-CSF 1004 can have both control-plane and user-plane functionalities and can interact with appropriate network-side functions, such as the SOCF 1020, Comp-CF 1024, Comp-SF 1036, Data-CF 1022, and / or Data-SF 1032, for service discovery, request / response, compute job workload exchange, and / or the like. The Comp-CSF 1004 can also work with network-side functions to decide whether a compute job should be executed on the UE 1002, the RAN 1008, and / or an element of the 6G-CN 1010.

[0147] The UE 1002 and / or the Comp-CSF 1004 can include a Service Mesh Proxy 1006. The Service Mesh Proxy 1006 can act as a proxy for service-to-service communication at the user level. Capabilities of the Service Mesh Proxy 1006 can include addressing, security, load balancing, and / or similar features.

[0148] Fig. Figure 11 shows exemplary network deployments, including an exemplary Next-Generation Fronthaul (NGF) deployment 1100a, in which a user device (UE) 1102 is connected via an air interface to an RU 1130 (also referred to as "remote radio unit 1130", "remote radio head 1130" or "RRH 1130"), the RU 1130 is connected via an NGF interface (NGFI)-I to a digital unit (DU) 1131, the DU 1131 is connected via an NGFI-II to a central unit (CU) 1132, the CU 1132 is connected via a backhaul interface to a core network (CN) 1142. In 3GPP-NG-RAN implementations (see e.g. [TS38401]), the DU 1131 can be a distributed entity (for the purposes of this disclosure, the term “DU” can refer to a digital entity and / or a distributed entity unless the context specifies otherwise).The UEs 1102 may be the same as or similar to the UE 702, the UE 802, the Hardware Resources 900, the UE 1002, the UE 1105 and / or any other UE described herein, and / or may share one or more features with them.

[0149] In some implementations, the NGF 1100a can be deployed in a distributed RAN (D-RAN) architecture, with the CU 1132, DU 1131, and RU 1130 located at a cell site and the CN 1142 located at a centralized site. Alternatively, the NGF 1100a can be deployed in a centralized RAN (C-RAN) architecture with centralized processing by one or more baseband units (BBUs) at the centralized site. In C-RAN architectures, the radio components are divided into discrete components that can be located at different locations. In an example C-RAN implementation, only the RU 1130 is located at the cell site, and the DU 1131, CU 1132, and CN 1142 are centralized or located at a central site.In another example C-RAN implementation, RU 1130 and DU 1131 are located at the cell site, and CU 1132 and CN 1142 are located at the centralized site. In another example C-RAN implementation, only RU 1130 is located at the cell site, DU 1131 and CU 1132 are located at a RAN hub site, and CN 1142 is located at the centralized site.

[0150] The CU 1132 is a central control unit that can serve or otherwise be connected to one or more DUs 1131 and / or multiple RUs 1130. The CU 1132 is a network (logic) node that hosts higher / upper layers of a network protocol function distribution. For example, in 3GPP-NG-RAN and / or O-RAN architectures, a CU 1132 hosts the Radio Resource Control (RRC), Service Data Adaptation Protocol (SDAP), and / or Packet Data Convergence Protocol (PDCP) layers of a next-generation NodeB (gNB), or hosts the RRC and PDCP protocol layers when contained in or operating as an E-UTRA-NR-gNB (en-gNB). The SDAP sublayer performs a mapping between QoS flows and data carriers (DRBs) and marks QoS flow IDs (QFI) in both DL and UL packets.The PDCP sublayer performs transfers of user-plane or control-plane data; maintains PDCP sequence numbers (SNS); performs header compression and decompression using the Robust Header Compression (ROHC) and / or Ethernet Header Compression (EHC) protocols; encrypts and decrypts; provides integrity protection and integrity checking; provides timer-based SDU discards; routing for split carriers; duplicating and discarding duplicates; reordering and order-preserving transmission; and / or out-of-order transmission. In various implementations, a CU 1132 terminates respective F1 interfaces connected to corresponding DUs 1131 (see, for example, [TS38401]).

[0151] A CU 1132 can contain a CU control plane (CP) entity (referred to here as "CU-CP 1132") and a CU user plane (UP) entity (referred to here as "CU-UP 1132"). The CU-CP 1132 is a logical node that hosts the RRC layer and the control plane portion of the PDCP protocol layer of the CU 1132 (e.g., a gNB-CU for an en-gNB or a gNB). The CU-CP terminates an E1 interface connected to the CU-UP and the F1-C interface connected to a DU 1131. The CU-UP 1132 is a logical node that hosts the user-level portion of the PDCP protocol layer (e.g., for a gNB-CU 1132 of an en-gNB) and the user-level portion of both the PDCP and SDAP protocol layers (e.g., for a gNB-CU 1132 of a gNB). The CU-UP 1132 terminates the E1 interface, which is connected to the CU-CP 1132, and the F1-U interface, which is connected to a DU 1131.

[0152] The DU 1131 controls radio resources, such as time and frequency bands, locally in real time and allocates resources to one or more UEs. DU 1131s are (logical) network nodes that host middle and / or lower layers of the network protocol function distribution. In 3GPP-NG-RAN and / or O-RAN architectures, for example, a DU 1131 hosts the Radio Link Control (RLC), Medium Access Control (MAC), and High-Physical (PHY) layers of the gNB or en-gNB, and its operation is at least partially controlled by the CU 1132. The RLC sublayer operates in a transparent mode (TM), an unacknowledged mode (UM), and / or an acknowledged mode (AM).The RLC sublayer performs transfer of PDUs from the upper layer; sequence numbering independent of that in PDCP (UM and AM); error correction via ARQ (AM only); segmentation (AM and UM) and resegmentation (AM only) of RLC-SDUs; SDU reassembly (AM and UM); duplicate detection (AM only); RLC-SDU discarding (AM and UM); RLC rebuild; and / or protocol error detection (AM only).The MAC sublayer performs mapping between logical channels and transport channels; multiplexing / demultiplexing of MAC-SDUs belonging to one or different logical channels into / from transport blocks (TBs) delivered to / from the physical layer on transport channels; scheduling of information messages; error correction via HARQ (one HARQ entity per cell in the case of CA); priority handling between UEs using dynamic scheduling; priority handling between logical channels of a UE using logical channel prioritization; priority handling between overlapping resources of a UE; and / or padding. In some implementations, a DU 1131 can host a Backhaul Adaptation Protocol (BAP) layer (see, for example, 3GPP TS 38.340 v17.5.0 (30-06-2023)) and / or an F1 Application Protocol (F1AP) (see, for example, 3GPP TS 38.470 v17.5.0 (29-06-2023)), as when the DU 1131 operates as an Integrated Access and Backhaul (IAB) node.A DU 1131 supports one or more cells, and a cell is supported by only one DU 1131. A DU 1131 terminates the F1 interface, which is connected to a CU 1132. Additionally or alternatively, the DU 1131 can be connected to one or more RRHs / RUs 1130.

[0153] The RU 1130 is a transmit / receive point (TRP) or other physical node that handles radio frequency (RF) processing functions. The RU 1130 is a network (logic) node that hosts lower layers based on a lower-layer functional partitioning. For example, in 3GPP-NG-RAN and / or O-RAN architectures, the RU 1130 hosts low-PHY layer functions and RF processing of the radio interface based on a lower-layer functional partitioning. The RU 1130 may be similar to the 3GPP transmit / receive point (TRP) or RRH, but specifically includes the low-PHY layer. Examples of low-PHY functions include fast Fourier transform (FFT), inverse FFT (iFFT), direct access channel (PRACH) extraction, and the like.

[0154] Each of the CUs 1132, DUs 1131, and RUs 1130 is connected by respective links, which can be any suitable wireless and / or wired (e.g., fiber, copper, and the like) links. In some implementations, various combinations of the CU 1132, DU 1131, and RU 1130 can correspond to one or more of the RAN 704, AN 708, AP 706, and / or any other NAN, such as any of those discussed here. Additional aspects of CUs 1132, DUs 1131, and RUs 1130 are discussed in [O-RAN], [TS38401], [TS38410], and [TS38300], the contents of which are hereby incorporated by reference in their entirety.

[0155] In some implementations, a fronthaul gateway (FHGW) function can be used between the DU 1131 and the RU / RRU 1130 (in Fig. (11 not shown) may be arranged, wherein the interface between the DU 1131 and the FHGW is an open fronthaul interface (e.g., Option 7-2x), and wherein the interface between the FHGW function and the RU / RRU 1130 is an open fronthaul interface (e.g., Option 7-2x) or any other suitable interface (e.g., Option 7, Option 8, or the like), including those that do not support open fronthaul (e.g., Option 7-2x). The FHGW may be packaged with one or more other functions (e.g., Ethernet switching and / or the like) in a physical device or appliance. In some implementations, a RAN controller may be communicatively coupled to the CU 1132 and / or the DU 1131.

[0156] The NGFI (also referred to as "xHaul" or similar) is a two-tier fronthaul architecture that separates the traditional RRU 1130 to BBU connectivity in the C-RAN architecture into two tiers, namely tiers I and II. Tier I connects the RU 1130 to the DU 1131 via the NGFI-I, and tier II connects the DU 1131 to the CU 1132 via the NGFI-II, as demonstrated by the use of the 1100a in Fig. Figure 11 shows that the NGFI-I and NGFI-II connections can be wired or wireless and can utilize any suitable RAT, such as any of those discussed here. The purpose of the two-tier architecture is to distribute (split) the RAN node protocol functions between the CU 1132 and DU 1131, thus reducing latency and providing greater deployment flexibility. Generally, NGFI-I is connected to the lower layers of the function split, which have strict delay and data rate requirements, while NGFI-II is connected to higher layers of the function split relative to the NGFI-I layers, thus reducing the requirements for the fronthaul link. Examples of NGFI fronthaul interfaces and function split architectures include O-RAN 7.2x Fronthaul (see, for example, [ORAN.XPSAAS] and [ORAN.CUS]), Enhanced Common Radio Interface (CPRI)-based C-RAN fronthaul (see, for example, Common Public Radio Interface: ECPRI Interface Specification, eCPRI Specification v2.0 (10-05-2019), Common Public Interface Radio Interface: Requirements for the eCPRI Transport Network, eCPRI Transport Network v1.2 (25-06-2018) and [ORAN.CUS]), Radio over Ethernet (RoE)-based C-RAN fronthaul (see, for example, IEEE Standard for Radio over Ethernet Encapsulations and Mappings, IEEE Standards Association, IEEE 1914.3-2018 (5 October 2018) ("[IEEE1914.3]") and / or similar. Additional aspects of NGFI are also addressed in [ORAN.XPSAAS], [ORAN.CUS], IEEE Standard for Packet-Based Fronthaul Transport Networks, IEEE Standards Association, IEEE 1914.1-2019 (April 21, 2020) ("[IEEE1914.1]"), [IEEE1914.3] and Nasrallah et al., Ultra-Low Latency (ULL) Networks: A Comprehensive Survey Covering the IEEE TSN Standard and Related ULL Research, arXiv:1803.07673v1 [cs.NI] (20.March 2018) (“[Nasrallah]”) discussed, the contents of which are hereby incorporated by reference in their entirety.

[0157] In one example, the 1100a deployment can implement a low-level split (LLS) (also known as "Lower Layer Functional Split 7-2x" or "Split Option 7-2x") that runs between the RU 1130 (e.g., an O-RU in O-RAN architectures) and the DU 1131 (e.g., an O-DU in O-RAN architectures) (see, e.g., [ORAN.IPC-HRD-Opt7-2], [ORAN.OMAC-HRD], [ORAN.OMC-HRD-Opt7-2], [ORAN.OMC-HRD-Opt7-2]). In this example implementation, the NGFI-I is the Open Fronthaul interface described in the O-RAN Open Fronthaul Specification (see, e.g., [ORAN.CUS]). Other LLS options can be used, such as the relevant interfaces described in other standards or specifications, such as the 3GPP NG-RAN Function Split (see, for example, [TS38401] and 3GPP TR 38.801 v14.0.0 (03-04-2017)), the Small Cell Forum for Split Option 6 (see, for example,5G small cell architecture and product definitions: Configurations and Specifications for companies deploying small cells 2020-2025, Small Cell Forum, Document 238.10.01 (July 5, 2020) (“[SCF238]”), 5G NR FR1 Reference Design: The case for a common, modular architecture for 5G NR FR1 small cell distributed radio units, Small Cell Forum, Document 251.10.01 (December 15, 2021) (“[SCF251]”) and [ORAN.IPC-HRD-Opt6], the contents of which are hereby incorporated in their entirety by reference) and / or in O-RAN White-Box Hardware Split Option 8 (e.g., [ORAN.IPC-HRD-Opt8]).

[0158] Additionally or alternatively, CUs 1132, DUs 1131, and / or RUs 1130 can be IAB nodes. IAB enables wireless forwarding in a NG-RAN, where a forwarding node (referred to as an "IAB node") supports access and backhauling over 3GPP-5G / New Radio (NR) links / interfaces. The NR backhaul termination node on the network side is referred to as an "IAB donor," which represents a RAN node (e.g., a gNB) with additional functionality to support IAB. Backhauling can occur over a single hop or multiple hops. All IAB nodes connected to an IAB donor via one or more hops form a Directed Acyclic Graph (DAG) topology with the IAB donor as its root. The IAB donor performs centralized resource, topology, and route management for the IAB topology. The IAB architecture is shown and described in [TS38300].

[0159] Although the NGF insert 1100a shows the CU 1132, the DU 1131, the RRH 1130, and the CN 1142 as separate entities, in other implementations some or all of these network nodes may be bundled, combined, or otherwise integrated into a single device or element, including collapsing some internal interfaces (e.g., F1-C, F1-U, E1, E2, and the like). At least the following implementations are possible: (i) integrating the CU 1132 and the DU 1131 (e.g., a CU-DU) connected to the RRH 1130 via the NGFI-I; (ii) integrating the DU 1131 and the integrated RRH 1130 (e.g.,(3) Integrating a RAN controller and the CU 1132, which is connected to the DU 1131 via NGFI-II; (4) Integrating the CU 1132, the DU 1131, and the RU 1130, which is connected to the CN 1142 via a backhaul interface; and (5) Integrating the network controller (or intelligent controller), the CU 1132, the DU 1131, and the RU 1130. Any of the above exemplary implementations involving the CU 1132 may also include integrating the CU-CP 1132 and CP-UP 1132.

[0160] Fig. Figure 11 also shows an exemplary RAN disaggregation deployment 1100b (also referred to as "disaggregated RAN 1100b"), in which the UE 1102 is connected to the RRH 1130, and the RRH 1130 is communicatively coupled to one or more of the RAN functions (RANFs) 1-N (where N is a number). The RANFs 1-N are disaggregated and geographically distributed across multiple component segments and network nodes. In some implementations, each RANF 1-N is a software (SW) element operated by a physical compute node, and the RRH 1130 incorporates a radio frequency (RF) circuit arrangement (e.g., an RF propagation module for a particular RAT and / or the like). In this example, RANF 1 is operated on a physical compute node located together with RRH 1130, and the other RANFs are located at locations farther away from RRH 1130.Additionally, in this example, the CN 1142 is also split into CN-NFs 1-x (where x is a number) in the same or a similar way as the RANFs 1-N, although the CN 1142 is not disaggregated in other implementations.

[0161] Network disaggregation (or disaggregated networking) involves separating network devices into functional components and allowing each component to be used individually. This can include separating software elements (e.g., network fields) from specific hardware elements and / or using APIs to enable software-defined networking (SDN) and / or network field virtualization (NFV). RAN disaggregation involves disaggregating and virtualizing different RANFs (e.g., RANFs 1-N in a network). Fig. 11) The RANFs 1-N can be deployed in a RAN deployment at different physical locations in various topologies, depending on the use case. This enables RANF distribution and deployment across different geographic areas and allows for RANF deployments to support various use cases (e.g., low-latency use cases) and flexible RAN implementations. Disaggregation provides a common or unified RAN platform capable of adopting a different profile depending on its deployment location. This results in fewer fixed-function devices and lower total cost of ownership compared to existing RAN architectures.Exemplary RAN disaggregation frameworks are provided by Telecom Infra Project (TIP) OpenRAN™, Cisco® Open vRAN™, [O-RAN], Open Optical & Packet Transport (OOPT), Reconfigurable Optical Add Drop Multiplexer (ROADM) and / or the like.

[0162] Fig. Figure 11 also shows various function distribution options for the 1100c for both DL and UL directions. The traditional RAN is an integrated network architecture based on a distributed RAN (D-RAN) model, where D-RAN integrates all RANFs into a few network elements. As previously indicated, the disaggregated RAN architecture provides flexible function distribution options to overcome several limitations of the D-RAN model. The disaggregated RAN divides the integrated network system into multiple functional components, which can then be rearranged individually as needed without hindering their ability to work together to provide holistic network services. The 1100c distribution options are primarily divided between the CU 1132 and the DU 1131, but can also include a division between the CU 1132, the DU 1131, and the RU 1130.For each Option 1100c, protocol entities on the left side of the figure are contained in the RANF implementing CU 1132, and the protocol entities on the right side of the figure are contained in the RANF implementing DU 1131. For example, Option 2 function splitting involves separating non-RT processing (e.g., RRC and PDCP layers) from RT processing (e.g., RLC, MAC, and PHY layers), with the RANF implementing CU 1132 performing network functions of the RRC and PDCP layers, and the RANF implementing DU 1131 performing baseband processing functions of the RLC (including high-RLC and low-RLC), MAC (including high-MAC and low-MAC), and PHY layers. In some implementations, the PHY layer is further divided between the DU 1131 and the RU 1130, with the RANF implementing the DU 1131 performing the functions of the high-PHY layer and the RU 1130 handling the functions of the low-PHY layer.In some implementations, the low-PHY entity can be operated by the RU 1130 independently of the selected function distribution option. With Option 2 distribution, the RANF implementing the CU 1132 can be connected to multiple DU 1131s (e.g., the CU 1132 is centralized), which eliminates RRC and PDCP anchor changes during a handover across the DUs 1131 and allows the centralized CU 1132 to pool resources across multiple DUs 1131. In this way, Option 2 function distribution can improve resource efficiency. The specific function distribution option used can vary depending on service requirements and network deployment scenarios and may be implementation-specific. It should also be noted that in some implementations, all function distribution options can be selected, with each protocol stack entity operated by a separate RANF (e.g.,A first RANF operates the RRC layer, a second RANF operates the PDCP layer, a third RANF operates the high-RLC layer, and so on, until an eighth RANF operates the low-PHY layer. Other partitioning options are possible, such as those discussed in [ORAN.IPC-HRD-Opt6], [ORAN.IPC-HRD-Opt7-2], [ORAN.IPC-HRD-Opt8], [ORAN.OMAC-HRD], and [ORAN.OMC-HRD-Opt7-2].

[0163] Fig. 12 represents an example of an Administrative Services (MnS) deployment 1200, according to one or more exemplary embodiments of the present disclosure.

[0164] MnS is a service-based management architecture (SBMA). An MnS is a set of offered management capabilities (e.g., capabilities for managing and orchestrating (MANO) networks and services). The entity that creates an MnS is called an MnS producer (MnS-P), and the entity that consumes an MnS is called an MnS consumer (MnS-C). An MnS provided by an MnS-P can be consumed by any entity with appropriate authorization and authentication. As in Fig. As shown in Figure 1, the MnS-P offers its services via a standardized service interface consisting of individually specified MnS components (e.g., MnS-C).

[0165] A Managed Network Service (MnS) is specified using several independent components. A concrete MnS includes at least two of these components. Three different component types are defined, including MnS component type A, MnS component type B, and MnS component type C. An MnS component type A is a group of management operations and / or notifications that are agnostic with respect to the managed entities. The operations and notifications themselves therefore do not contain any information about the managed network. These operations and notifications are referred to as generic or network-agnostic. For example, operations to create, read, update, and delete managed object instances, where the managed object instance to be manipulated is specified only in the operation's signature, are generic.An MnS component type B refers to management information represented by information models that represent the managed entities. An MnS component type B is also referred to as a network resource model (NRM) (see, for example, [TS28622], [TS28541]). MnS component type C is performance information and fault information of the managed entity. Examples of management service component type C include alarm information (see, for example, [TS28532] and [TS28545]) and performance data (see, for example, [TS28552], [TS28554], and [TS32425]).

[0166] An MnS-P is described by a set of metadata called the MnS-P profile. The profile contains information about the supported MnS components and their version numbers. This can also include information about support for optional features. For example, a read operation on a complete subtree of managed object instances might support applying filters to the scoped set of objects as an optional feature. In this case, the MnS profile should include information indicating whether filtering is supported.

[0167] Fig. Section 12 also presents an exemplary use of a Management Function (MnF) 1210. The MnF is a logical entity that performs the roles of MnS-C and / or MnS-P. An MnF with the role of Management Service Discovery Governance is referred to as a "Discovery Governance Management Function" or "Discovery Governance MnF." An MnS generated by an MnF 1210 can have multiple consumers. The MnF 1210 can consume multiple MnS from one or more MnS-Ps. In the MnF 1210 use, the MnF performs both roles (e.g., MnS-P and MnS-C). An MnF can be used as a separate entity or embedded in a Network Function (NF) to provide MnS(s). For example, MnF deployment scenario 1220 shows an example where the MnF is used as a separate entity to provide MnS(s), and MnF deployment scenario 1230 shows where an MnF is embedded in an NF to provide MnS(s).In these examples, the MnFs can interact by consuming MnS generated by other MnFs.

[0168] The 3GPP management system is also capable of consuming an NFV-MANO interface (e.g., OS-Ma-nfvo, VE-Vnfm-em, and VE-Vnfm-vnf reference points). A MnS-P can consume management interfaces provided by NFV-MANO for at least the following purposes: network service LCM; and VNF LCM, PM, FM, CM on resources that support VNF.

[0169] Fig. Figure 13 shows an exemplary framework for the deployment of MnS, where an MnS-P is connected to an ETSI-NFV-MANO to support the lifecycle management of VNFs.

[0170] Machine learning (ML) involves programming computing systems to optimize a performance criterion using sample (training) data and / or past experience. ML refers to the use and development of computer systems capable of learning and adapting without following explicit instructions by using algorithms and statistical models to analyze patterns in data and draw inferences from them. ML includes using algorithms to perform one or more specific tasks without explicit instructions for performing those tasks, instead relying on learned patterns and / or inferences. ML uses statistics to create one or more mathematical models (also called "ML models" or simply "models") to make predictions or decisions based on sample data (e.g.,The model is defined as having a set of parameters, and learning is the execution of a computer program to optimize the model's parameters using the training data or past experience. The trained model can be a predictive model, which makes predictions based on an input dataset; a descriptive model, which gains knowledge from an input dataset; or both predictive and descriptive. Once the model is learned (trained), it can be used to make inferences (e.g., predictions).

[0171] Fig. Figure 14 represents an exemplary AI / ML-supported communication network according to one or more embodiments of the present disclosure.

[0172] The AI / ML-enabled communication network involves communication between an ML Function (MLF) 1402 and an MLF ax04. Specifically, as described in more detail below, AI / ML models can be used or leveraged to enable wired and / or over-the-air communication between the MLF 1402 and the MLF 1404. In this example, the MLF 1402 and the MLF 1404 operate in a manner consistent with 3GPP technical specifications and / or technical reports for 5G and / or 6G systems. In some examples, the communication mechanisms between the MLF 1402 and the MLF 1404 include any suitable access technologies and / or RATs, such as any of those discussed here. Additionally, the communication mechanisms can be further customized. Fig. 14. Part of the components, devices, systems, networks and / or deployments from Fig. 1, 2, 3, 4, 7, 8, 10, 11, 12-13 and / or some other components, devices, systems, networks and / or inserts described herein, or operating simultaneously with them.

[0173] MLFs 1402 and 1404 can correspond to any of the entities / elements discussed here. For example, MLF 1402 corresponds to an MnF and / or an MnS-P, and MLF 1404 corresponds to a Consumer 310, an MnS-C, or vice versa. Additionally or alternatively, MLF 1402 corresponds to a set of MLFs of the Fig. 12-13 and the MLF 1404 corresponds to another set of the MLFs of the Fig. 12-13. In this example, the sets of MLFs can be mutually exclusive, or some or all of the MLFs in each set of MLFs can overlap or be used together. In another example, MLF 1402 and / or MLF 1404 are implemented by respective UEs (e.g., UE 702, UE 802). Additionally or alternatively, MLF 1402 and / or MLF 1404 are implemented by the same UE or by different UEs. In yet another example, MLF 1402 and / or MLF 1404 are implemented by respective RANs (e.g., RAN 704) or respective NANs (e.g., AP 706, NAN 714, NAN 804).

[0174] As in Fig. As shown in Figure 14, MLF 1402 and MLF 1404 include various AI / ML-related components, functions, elements, or entities that may be implemented as hardware, software, firmware, and / or any combination thereof. In some examples, one or more of the AI / ML-related elements are implemented as part of the same hardware (e.g., IC, chip, or multiprocessor chip), software (e.g., program, process, engine, and / or the like), or firmware as at least one other component, function, element, or entity. The AI / ML-related elements of MLF 1402 may be the same as or similar to the AI / ML-related elements of MLF 1404. For brevity, a description of the various elements is provided from the perspective of MLF 1402; however, it is understood that such a description also applies to similarly named / numbered elements of MLF 1404 unless explicitly stated otherwise.

[0175] Data Repository 1415 is responsible for data collection and storage. For example, Data Repository 1415 can collect and store RAN configuration parameters, NF configuration parameters, measurement data, RLM data, key performance indicators (KPIs), SLAs, model capability metrics, knowledge base data, ground truth data, ML model parameters, hyperparameters, and / or other data for model training, updates, and inference. In some examples, a data collection function (not shown) is part of or associated with Data Repository 1415. The data collection function provides input data to MLTF 1425 and Model Inference Function 1445. AI / ML algorithm-specific data preparation (e.g., data preprocessing and cleaning, formatting, and transformation) may or may not be performed in Data Collection Function 1405.Examples of input data may include measurements from UEs 702, RAN nodes 714, and / or additional or alternative network entities; feedback from actuator 1620; and / or output(s) from one or more AI / ML models. The input data fed into the MLTF 1425 is training data, and the input data fed into the model inference function 1445 is inference data.

[0176] The collected data is stored in / by repository 1415, and the stored data can be discovered by other elements and extracted from data repository 1415. For example, the inference data selector / filter 1450 can retrieve data from data repository 1415 and provide this data to the inference engine 1445 for generating / determining inferences. In various examples, MLF 1402 is designed to discover and request data from data repository 1415 in MLF 1404, and / or vice versa. In these examples, data repository 1415 of MLF 1402 can be communicatively coupled with data repository 1415 of MLF 1404, so that the respective data repositories 1415 can share collected data with each other.Additionally or alternatively, the MLF 1402 and / or the MLF 1404 are designed to discover and request data from one or more external sources and / or data storage systems / devices.

[0177] The training data selector / filter 1420 is designed to generate training, validation, and test datasets for machine learning (MLT) training (or machine learning model training). One or more of these datasets can be extracted or otherwise obtained from the data repository 1415. Data can be selected / filtered based on the specific AI / ML model to be trained. Data can optionally be transformed, augmented, and / or preprocessed (e.g., normalized) before being loaded into datasets. The training data selector / filter 1420 can label data in datasets for supervised learning, or the data can remain unlabeled for unsupervised learning. The generated datasets can then be fed into the MLT function (MLTF) 1425.

[0178] The MLTF 1425 is responsible for training and updating (e.g., tuning and / or retraining) AI / ML models. A selected model (or set of models) can be trained (including training, validation, and testing) using the datasets provided by the training data selection / filtering process (1420). The MLTF 1425 generates trained and tested AI / ML models that are ready for use. These models can be stored in a model repository (1435). Additionally or alternatively, the MLTF 1425 performs AI / ML model training, validation, and testing. The MLTF 1425 can generate model performance metrics as part of the model testing procedure and / or as part of the model validation procedure. Examples of model performance metrics are discussed below. The MLTF 1425 can also be used for data preparation (e.g.The MLTF 1425 is responsible for data preprocessing and cleaning, formatting, and transformation based on training data supplied by the data collection function and / or the training data selection / filter, if required. The MLTF 1425 performs model deployment and updates, initially deploying a trained, validated, and tested AI / ML model to the model inference function 1445 and / or supplying one or more updated models to the model inference function 1445. Examples of model deployment and updates are discussed below.

[0179] Model Repository 1435 is responsible for storing and disclosing AI / ML models (both trained and untrained). Various types of model data can be stored in Model Repository 1435. This data can include, for example, one or more trained / updated models, model parameters, hyperparameters, and / or model metadata, such as model performance metrics, hardware platform / configuration data, model execution parameters / conditions, and / or the like. In some examples, the model data may also include inferences made while running the ML model. Examples of AI / ML models and other ML modeling aspects are discussed below in relation to… Fig. 14 and Fig. Section 15 discusses this. Model data can be discovered and requested by other MLF components (e.g., the training data selection / filter 1420 and / or the MLTF 1425). In some examples, MLF 1402 can discover and request model data from the model repository 1435 of MLF 1404. Additionally or alternatively, MLF 1404 can discover and / or request model data from the model repository 1435 of MLF 1402. In some examples, MLF 1404 can configure models, model parameters, hyperparameters, model execution parameters / conditions, and / or other ML model aspects in the model repository 1435 of MLF 1402.

[0180] The Model Management Function (Model mgmt Function) 1440 is responsible for managing (mgmt) the AI / ML model generated by the MLTF 1425. Such mgmt functions may include deploying a trained model, monitoring ML entity performance, reporting ML entity validation and / or performance data, and / or the like. In model deployment, the Model mgmt 1440 may allocate and schedule hardware and / or software resources for inference based on received trained and tested models. For the purposes of this disclosure, the term "inference" refers to the process of using one or more trained AI / ML models to perform statistical inferences, predictions, decisions, probabilities and / or probability distributions, actions, configurations, policies, data analyses, results, optimizations, and / or the like based on new, unseen data (e.g.,to generate "input inference data"). In some examples, the inference process may involve feeding input inference data into the ML model (e.g., the Inference Engine 1445), forward routing of the input inference data through the architecture / topology of the ML model, where the ML model performs computations on the data using its learned parameters (e.g., weights and biases), and generating the prediction output. In some examples, the inference process may include pre-forward data transformation, where the input inference data is pre-processed or transformed to conform to the format required by the ML model. For performance monitoring, the Model Management 1440 may decide, based on model performance KPIs and / or metrics, to terminate the running model, initiate model retraining and / or tuning, select a different model, and / or similar actions.For example, the Model-mgmt 1440 of MLF 1404 may be able to configure Model-mgmt policies in MLF 1402, and vice versa.

[0181] The inference data selection / filter 1450 is responsible for generating datasets for model inference at the inference engine 1445, as described below. For example, inference data can be extracted from the data repository 1415. The inference data selection / filter 1450 can select and / or filter the data based on the AI / ML model used. Data can be transformed, augmented, and / or preprocessed in the same or a similar way as the transformation, extension, and / or preprocessing of the training data selection / filter, as described in relation to the training data selection filter 1420. The generated inference dataset can be fed into the inference engine 1445.

[0182] The Inference Engine 1445 (also referred to as "Model Inference Function 1445" and / or the like) is responsible for performing / generating inferences as described herein. The Inference Engine 1445 consumes an inference dataset provided by the Inference Data Selector / Filter 1450 and produces an AI / ML model inference output that includes one or more inferences. The inferences may be, for example, statistical inferences, predictions, decisions, probabilities and / or probability distributions, actions, configurations, policies, data analyses, results, optimizations, and / or the like. The inference(s) / result(s) may be provided to the Performance Measurement Function 1430. The model inference function 1445 can provide model performance feedback to the MLTF 1425 and / or the performance measurement function 1430, if applicable.Model performance feedback can include various performance metrics (e.g., any of those discussed here) related to inference generation. Model performance feedback, if available, can be used to monitor the performance of the AI / ML model. The Model Inference Function 1445 can also be responsible for data preparation (e.g., data preprocessing and cleaning, formatting, and transformation) based on inference data supplied by the Data Collection Function and / or the Inference Data Selection / Filter function ax50, if required. The Model Inference Function 1445 produces inference output, which is the inferences generated or otherwise produced when the Model Inference Function 1445 operates the AI / ML model using the inference data(s).Details of the inference output are use case specific and may be based on the specific type of AI / ML model used (see e.g. ). Fig. 15-16).

[0183] In some examples, the model inference function 1445 provides the inference output to an actuator (not shown). The actuator is a function, engine, component, device, system, network, and / or other entity that receives an inference output from the model inference function 1445 and triggers or otherwise performs one or more appropriate actions based on the inference output. The actuator can trigger actions directed at other entities and / or itself. In some examples, the actuator is a network energy saving (NES) function, a mobility robustness optimization (MRO) function, a load balancing optimization (LBO) function, and / or another self-organizing network (SON) function.Additionally or alternatively, the inference output relates to NES, MRO, and / or LBO, and the actuator is one or more RAN nodes 714 that perform various NES, MRO, and / or LBO operations based on the inferences. The actuator can also provide feedback to the performance measurement function ax30 and / or the data collection function / data repository 1415 for storage. The actuator feedback includes information regarding the actions performed by the actuator. For example, the feedback includes any information that may be needed to derive training data, test data, and / or validation data; inference data; and / or data for monitoring the performance of the AI / ML model and its impact on the network by updating KPIs, performance counters, and / or the like.

[0184] The performance measurement function 1430 is designed to measure model performance metrics (e.g., accuracy, momentum, precision, quantile, recall / sensitivity, model bias, runtime latency, resource consumption, and / or other suitable metrics / measures, such as any of those discussed herein) of deployed and executing models based on the inference(s) for monitoring purposes. Model performance data can be stored in the ax15 data repository and / or reported according to the validation reporting mechanisms discussed herein.

[0185] The performance metrics that can be measured and / or predicted by Performance Measurement Function 1430 can be based on the specific AI / ML task and the other inputs / parameters of the ML entity. These performance metrics can include model-based and platform-based metrics. Model-based metrics are metrics regarding the performance of the model itself and / or without considering the underlying hardware platform. Platform-based metrics are metrics regarding the performance of the underlying hardware platform when running the ML model.

[0186] Model-based metrics can be based on the specific type of AI / ML model and / or the AI / ML domain. For example, regression-related metrics can be predicted for regression-based ML models. Examples of regression-related metrics include error value, mean error, mean absolute error (MAE), mean reciprocal rank (MRR), mean squared error (MSE), root mean squared error (RMSE), correlation coefficient (R), coefficient of determination (R²), Golbraikh and Tropsha criteria, and / or other similar regression-related metrics, such as those discussed in Naser et al., “Insights into Performance Fitness and Error Metrics for Machine Learning”, arXiv:2006.00887v1 (May 17, 2020) (“[Naser]”), which is hereby incorporated by reference in its entirety.

[0187] Fig. Figure 15 illustrates an exemplary neural network according to one or more embodiments.

[0188] The NN 15100 can be used by one or more of the computing systems (or subsystems) of the various implementations discussed herein, some of which are implemented by a hardware accelerator, and / or the like. The NN 1500 can be a deep neural network (DNN) used as an artificial brain of a computing node or network of computing nodes to handle very large and complex observation spaces.Additionally or alternatively, the NN 1500 can be any other topology type (or a combination of topologies), such as a convolutional NN (CNN), a deep CNN (DCN), a recurrent NN (RNN), a long short-term memory (LSTM) network, an unfolding NN (DNN), a gated recurrent unit (GRU), a deep belief NN, a forward-linked NN (FFN), a deep FNN (DFF), a deep stacking network, a Markov chain, a perception NN, a Bayesian network (BN) or Bayesian NN (BNN), a dynamic BN (DBN), a linear dynamic system (LDS), a switching LDS (SLDS), optical NNs (ONNs), a reinforcement learning (RL) NN and / or deep RL (DRL) NN and / or the like. NNs are typically used for supervised learning, but can be used for unsupervised learning and / or RL.

[0189] The NN 1500 can encompass a variety of ML techniques, with a collection of connected artificial neurons 1510, which model (loosely) neurons in a biological brain, transmitting signals to other neurons / nodes 1510. The neurons 1510 can also be referred to as nodes 1510, processing elements (PEs) 1510, or the like. The connections 1520 (or edges 1520) between the nodes 1510 are (loosely) modeled at synapses in a biological brain and transmit the signals between the nodes 1510. It should be noted that in Fig. For the sake of clarity, not all neurons 1510 and edges 1520 are labeled.

[0190] Fig. Figure 16 shows a learning logic (RL) architecture 1600, comprising an agent 1610 and an environment 1620. The agent 1610 (e.g., a software agent or AI agent) is the learner and decision-maker, and the environment 1620 encompasses everything outside of the agent 1610 with which the agent 1610 interacts. The environment 1620 is typically specified in the form of a Markov decision process (MDP), which can be described using dynamic programming techniques. An MDP is a discrete-time stochastic control process that provides a mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision-maker.

[0191] Learning-reward (RL) is goal-oriented learning based on interaction with the environment. RL is a machine learning (ML) paradigm that deals with how software agents (or AI agents) should perform actions in an environment to maximize a numerical reward signal. Generally, RL involves an agent performing actions in an environment that are interpreted as a reward and a representation of a state, which is then fed back to the agent. In RL, an agent aims to optimize a long-term goal by interacting with the environment based on a trial-and-error process. In many RL algorithms, the agent receives a reward in the next time step (or epoch) to evaluate its previous action. Examples of RL algorithms include Markov Decision Process (MDP) and Markov chains, associative RL, inverse RL, safe RL, Q-learning, multi-armed bandit learning, and deep RL.

[0192] Agent 1610 and environment 1620 interact continuously, with agent 1610 selecting actions A to be performed and environment 1620 responding to these actions and presenting agent 1610 with new situations (or states S). Action A encompasses all possible actions, tasks, movements, and / or the like that agent 1610 can perform in a given context. State S is a current situation, such as a complete description of a system, a unique configuration of information in a program or machine, a snapshot of a measure of various conditions in a system, and / or the like. In some implementations, agent 1610 selects an action A to be performed based on a policy π. The policy π is a strategy that agent 1610 uses to determine the next action A based on the current state S.The environment 1620 also generates rewards R, which are numerical values ​​that Agent 1610 seeks to maximize over time through their choice of actions.

[0193] Environment 1620 starts by sending a state St to agent 1610. In some implementations, environment 1620 also sends an initial reward Rt to agent 1610 with the state St. Agent 1610 takes an action based on its knowledge in response to this state St (and reward Rt, if present). The action At is reported back to environment 1620, and environment 1620 sends a state-reward pair, including a next state St+1 and a next reward Rt+1 based on the action At, to agent 1610. Agent 1610 updates its knowledge with the reward Rt+1 returned by environment 1620 to evaluate its previous action(s). The process repeats until environment 1620 sends a final state S, which ends the process or episode. Additionally or alternatively, Agent 1610 can take a specific action A to optimize a value V.The value V is an expected long-term return with a discount, as opposed to the short-term reward R. Vπ(S) is defined as the expected long-term return of the current state S under the policy π.

[0194] Q-learning is a model-free RL algorithm that learns the value of an action in a given state. Q-learning does not require a model of an environment and can handle problems with stochastic transitions and rewards without requiring adjustments. The "Q" in Q-learning refers to the function that the algorithm computes, which is the expected reward(s) for an action A taken in a given state S. In Q-learning, a Q-value is computed using the state St and the action At at time t, using the function Qπ(St, At). Qπ(St, At) is the long-term return of a current state when taking action A under policy π. For any finite MDP (FMDP), Q-learning finds an optimal policy π in the sense of maximizing the expected value of the total reward over any and all successive steps, starting from the current state S.Additionally, examples of value-based Deep-RL include Deep-Q Network (DQN), Double-DQN, and Dueling-DQN. A DQN is created by replacing the Q function of Q-Learning with an artificial neural network (ANN), such as a convolutional neural network (CNN).

[0195] The following examples refer to further embodiments.

[0196] Example 1 can include a Service-Based Management Architecture (SBMA) Management Service (MnS) producer setup, wherein the setup includes a processing circuitry coupled with storage to store information associated with the joint testing of machine learning (ML) models, the processing circuitry being designed to: define performance requirements for jointly testing a group of ML models; perform the joint testing of the group of ML models; and send results of the joint testing to an MnS consumer, the results indicating whether the group of ML models meets the performance requirements.

[0197] Example 2 may include the setup of Example 1 and / or any other example herein, wherein the setup performs the co-testing without the MnS consumer requesting the co-testing.

[0198] Example 3 may include the setup of Example 1 and / or any other example herein, wherein the processing circuit arrangement is further configured to receive a request from the MnS generator to perform the joint testing, the performance of the joint testing being based on the request from the MnS generator.

[0199] Example 4 may include the setup of Example 2 or 3 and / or any other example herein, where the group of ML models is identified by an Information Object Class (IOC) ML Model Coordination Group.

[0200] Example 5 may include the setup of Example 4 and / or any other example herein, where the joint testing is based on an ML test requirement IOC that identifies the ML model coordination group IOC.

[0201] Example 6 may include the setup of Example 5 and / or any other example herein, with the ML test request IOC further identifying a status of the joint testing as either not started, in progress, suspended, completed or aborted.

[0202] Example 7 may include the setup of Example 1 and / or any other example herein, wherein the processing circuit arrangement is further designed to retrain the group of ML models if the results do not meet the performance requirements.

[0203] Example 8 may include a computer-readable medium that stores computer-executable instructions for jointly testing machine learning (ML) models, which, when executed by one or more processors of a Service-Based Management Architecture (SBMA) management service (MnS) producer, result in the performance of operations that include: defining performance requirements for jointly testing a group of ML models; performing the joint testing of the group of ML models; and sending results of the joint testing to an MnS consumer, the results indicating whether the group of ML models meets the performance requirements.

[0204] Example 9 can include the computer-readable medium of Example 8 and / or any other example herein, wherein one or more processors perform the joint testing without the MnS consumer requesting the joint testing.

[0205] Example 10 may include the computer-readable medium of Example 1 and / or any other example herein, wherein the operations further include receiving a request from the MnS producer to perform the joint testing, wherein the performance of the joint testing is based on the request from the MnS producer.

[0206] Example 11 may include the computer-readable medium of any of Example 9 or 10 and / or any other example herein, wherein the group of ML models is identified by an Information Object Class (IOC) ML Model Coordination Group.

[0207] Example 12 may include the computer-readable medium of Example 11 and / or any other example herein, wherein the performance of joint testing is based on an ML test requirement IOC that identifies the ML model coordination group IOC.

[0208] Example 13 may include the computer-readable medium of Example 12 and / or any other example herein, wherein the ML test request IOC further identifies a joint testing status as either not started, in progress, suspended, completed or aborted.

[0209] Example 14 may include a procedure for jointly testing machine learning (ML) models, wherein the procedure comprises: defining, through a processing circuit arrangement of a Service-Based Management Architecture (SBMA) Management Service (MnS) producer, capability requirements for jointly testing a group of ML models; performing, through the processing circuit arrangement, the joint testing of the group of ML models; and sending, through the processing circuit arrangement, results of the joint testing to an MnS consumer, the results indicating whether the group of ML models meets the capability requirements.

[0210] Example 15 may include the procedure of Example 14 and / or any other example herein, wherein the processing circuit arrangement performs the joint testing without the MnS consumer requesting the joint testing.

[0211] Example 16 may include the procedure of Example 14 and / or any other example herein, further comprising receiving a request from the MnS producer to perform the joint testing, wherein the performance of the joint testing is based on the request from the MnS producer.

[0212] Example 17 may include the procedure of any of Example 15 or 16 and / or any other example herein, wherein the group of ML models is identified by an Information Object Class (IOC) ML Model Coordination Group.

[0213] Example 18 may include the procedure of Example 17 and / or any other example herein, wherein the joint testing is performed based on an ML test requirement IOC that identifies the ML model coordination group IOC.

[0214] For one or more embodiments, at least one of the components shown in one or more of the preceding figures can be configured to perform one or more operations, one or more techniques, one or more processes, and / or one or more methods, as set out in the example section below. For example, the baseband circuit, as described above in conjunction with one or more of the preceding figures, can be configured to operate according to one or more of the examples below. For another example, a circuit arrangement associated with a UE, base station, network element, etc., as described above in conjunction with one or more of the preceding figures, can be configured to operate according to one or more of the examples set out in the example section below.

[0215] The word "exemplary" is used herein to mean "serving as an example, case, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferable or advantageous over other embodiments. The terms "computing device," "user device," "communication station," "station," "handheld device," "mobile device," "wireless device," and "user device" (UE), as used herein, refer to a wireless communication device, such as a cellular phone, smartphone, tablet, netbook, wireless terminal, laptop computer, femtocell, HDR subscriber station (HDR: High Data Rate), access point, printer, point-of-sale device, access terminal, or other PCS device (PCS: Personal Communication System).The device can be either mobile or stationary.

[0216] As used in this document, the term "communicate" is intended to include transmitting or receiving, or both. This can be particularly useful in claims describing the organization of data transmitted by one device and received by another, where only the functionality of one of these devices is required to infringe the claim. Similarly, the bidirectional exchange of data between two devices (both devices transmitting and receiving during the exchange) can be described as "communicating" if only the functionality of one of these devices is claimed. The term "communicate," as used herein in relation to a wireless communication signal, includes transmitting and / or receiving the wireless communication signal.For example, a wireless communication unit capable of communicating a wireless communication signal may include a wireless transmitter to transmit the wireless communication signal to at least one other wireless communication unit, and / or a wireless communication receiver to receive the wireless communication signal from at least one other wireless communication unit.

[0217] The use of the order adjectives “first”, “second”, “third”, etc., as employed herein to describe a common object, unless otherwise stated, merely indicates that reference is made to different instances of similar objects, and does not imply that the objects so described must be present in a given sequence, neither temporally, nor spatially, in order, nor in any other way.

[0218] As used here, the term "access point" (AP) can refer to a fixed station. An access point may also be referred to as an access node, a base station, an evolved Node B (eNodeB), or any other similar terminology known from the prior art. An access terminal may also be referred to as a mobile station, a user device (UE), a wireless communication device, or any other similar terminology known from the prior art. The embodiments disclosed herein relate generally to wireless networks. Some embodiments may relate to wireless networks operating in accordance with one of the IEEE 802.11 standards.

[0219] Some embodiments can be used in conjunction with various devices and systems, for example, a personal computer (PC), a desktop computer, a mobile computer, a laptop computer, a notebook computer, a tablet computer, a server computer, a handheld computer, a handheld device, a personal digital assistant (PDA) device, a handheld PDA device, an on-board device, an off-board device, a hybrid device, a vehicle device, a non-vehicle device, a mobile or portable device, a consumer device, a non-mobile or non-portable device, a wireless communication station, a wireless communication device, a wireless access point (AP), a wired or wireless router, a wired or wireless modem, a video device, an audio device, an audio-video (A / V) device,a wired or wireless network, a wireless network, a wireless video network (WVAN), a local area network (LAN), a wireless LAN (WLAN), a personal network (WPAN), a wireless PAN (WPAN), and the like.

[0220] Some embodiments may be used in conjunction with one-way and / or two-way radio communication systems, cellular radio telephone communication systems, a mobile phone, a cellular telephone, a wireless telephone, a PCS device (PCS: Personal Communication Systems), a PDA device integrating a wireless communication device, a mobile or portable Global Positioning System (GPS) device, a device integrating a GPS receiver or transceiver or chip, a device integrating an RFID element or RFID chip, a MIMO transceiver (MIMO: Multiple Input Multiple Output) or a MIMO device, a SIMO transceiver (SIMO: Single Input Multiple Output) or a SIMO device,a MISO transceiver (MISO: Multiple Input Single Output) or MISO device, a device having one or more internal and / or external antennas, digital video broadcast (DVB) devices or systems, multi-standard radio devices or systems, a wired or wireless handheld device, e.g. a smartphone, a wireless application protocol (WAP) device or the like.

[0221] Some embodiments can be used in conjunction with one or more types of wireless communication signals and / or systems that follow one or more wireless communication protocols, for example, radio frequency (RF), infrared (IR), frequency division multiplexing (FDM), orthogonal FDM (OFDM), time division multiplexing (TDM), time division multiplexing (TDMA), enhanced TDMA (E-TDMA), general packet radio service (GPRS), enhanced GPRS, code division multiplexing (CDMA), wideband CDMA (WCDMA), CDMA 2000, single-carrier CDMA, multi-carrier CDMA, multi-carrier modulation (MDM), discrete multi-tone (DMT), Bluetooth®, global positioning system (GPS), Wi-Fi, Wi-Fi Max, ZigBee, ultra-wideband (UWB), GSM (global system for mobile communications), 2G, 2.5G, 3G, 3.5G, 4G, Fifth generation (5G) mobile networks, 3GPP, LTE (Long Term Evolution), LTE-Advanced, EDGE (enhanced data rates for GSM Evolution) or the like.Other embodiments can be used in various other devices, systems and / or networks.

[0222] Various designs are described below.

[0223] Embodiments according to the disclosure are disclosed in particular in the appended claims, which relate to a method, a storage medium, a device, and a computer program product, wherein any feature mentioned in one claim category, e.g., method, may also be claimed in another claim category, e.g., system. The dependencies or cross-references in the appended claims are selected for formal reasons only. However, any subject matter of the invention resulting from a targeted reference back to any prior claims (in particular, multiple dependencies) may also be claimed, so that any combination of claims and their features is disclosed and may be claimed independently of the dependencies chosen in the appended claims.The subject matter that may be claimed includes not only the combinations of features as set forth in the appended claims, but also any other combination of features in the claims, wherein each feature mentioned in the claims may be combined with any other feature or any other combination of other features in the claims. Furthermore, any embodiments and features described or illustrated herein may be claimed in a separate claim and / or in any combination with any embodiment or feature described or illustrated herein or with any features of the appended claims.

[0224] The foregoing description of one or more implementations provides an illustration and description, but is not intended to be exhaustive or to limit the scope of protection of the embodiments to the exact disclosed form. Modifications and variations are possible in light of the above teachings or may be obtained from the exercise of different embodiments.

[0225] Certain aspects of the disclosure are described above with reference to block and flowcharts of systems, procedures, facilities, and / or computer program products according to various implementations. It is understood that one or more blocks of the block and flowcharts, and combinations of blocks in the block and flowcharts, can be implemented by computer-executable program instructions. Likewise, some blocks of the block and flowcharts may not necessarily have to be executed in the order shown, or may not have to be executed at all.

[0226] These computer-executable program instructions can be loaded onto a special computer or other special machine, processor or other programmable data processing device to create a special machine such that the instructions executed on the computer, processor or other programmable data processing device generate means for implementing one or more functions specified in the flowchart block(s).These computer program instructions may also be stored in a computer-readable storage medium or memory that can instruct a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable storage medium produce a manufactured item that includes instructional means implementing one or more functions specified in the flowchart block(s). As an example, certain implementations may provide a computer program product that includes a computer-readable storage medium with computer-readable program code or program instructions implemented therein, the computer-readable program code being designed to be executed to implement one or more functions specified in the flowchart block(s).The computer program instructions can also be loaded onto a computer or other programmable data processing device to cause a series of operational elements or steps to be performed on the computer or other programmable device in order to create a computer-implemented process, such that the instructions executed on the computer or other programmable device provide elements or steps for implementing the functions specified in the flowchart block(s).

[0227] Accordingly, blocks in block diagrams and flowcharts support combinations of means for performing the specified functions, combinations of elements or steps for performing the specified functions, and program instruction means for performing the specified functions. It is also understood that each block in block diagrams and flowcharts, and combinations of blocks in block diagrams and flowcharts, can be implemented by special hardware-based computer systems that perform the specified functions, elements, or steps, or by combinations of special hardware and computer instructions.

[0228] Conditional phrases, such as "may," "might," "might," or "possibly," are intended, unless specifically stated otherwise or understood differently in the context, to generally convey that certain implementations may include certain features, elements, and / or operations, while other implementations do not. Thus, such conditional phrases are generally not intended to imply that features, elements, and / or operations are in any way required for one or more implementations, or that one or more implementations necessarily include logic for deciding, with or without user input or prompting, whether these features, elements, and / or operations are included or to be performed in any particular implementation.

[0229] Many modifications and other implementations of the disclosure set forth herein become apparent with the benefit of the teachings set forth in the preceding descriptions and the associated drawings. It is therefore understood that the disclosure is not intended to be limited to the specific implementations disclosed and that modifications and other implementations within the scope of protection of the attached claims are intended to be included. Although specific terms are used herein, they are employed only in a generic and descriptive sense and not for the purpose of limitation.

[0230] For the purposes of this document, the following terms and definitions shall apply to the examples and embodiments discussed herein.

[0231] As used herein, the term “circuit arrangement” refers to hardware components, such as an electronic circuit, a logic circuit, a processor (shared, dedicated, or group) and / or memory (shared, dedicated, or group), an application-specific integrated circuit (ASIC), a field-programmable device (FPD) (e.g., a field-programmable gate array (FPGA), a programmable logic device (PLD), a complex PLD (CPLD), a high-capacity PLD (HCPLD), a structured ASIC, or a programmable SoC), digital signal processors (DSPs), etc., designed to provide, is part of, or includes the described functionality. In some embodiments, the circuit arrangement may execute one or more software or firmware programs to provide at least some of the described functionality.The term "circuit arrangement" can also refer to a combination of one or more hardware elements (or a combination of circuits used in an electrical or electronic system) with the program code used to execute the functionality of that program code. In these embodiments, the combination of hardware elements and program code can be described as a particular type of circuit arrangement.

[0232] The term "processor circuit arrangement," as used herein, refers to, is part of, or includes a circuit arrangement capable of sequentially and automatically performing a sequence of arithmetic or logical operations, or recording, storing, and / or transmitting digital data. The processing circuit arrangement may include one or more processing cores for executing instructions and one or more memory structures for storing program and data information.The term "processor circuit arrangement" can refer to one or more application processors, one or more baseband processors, a physical central processing unit (CPU), a single-core processor, a dual-core processor, a triple-core processor, a quad-core processor, and / or any other device capable of executing or otherwise operating computer-executable instructions, such as program code, software modules, and / or functional processes. The processing circuit arrangement may include multiple hardware accelerators, which may be microprocessors, programmable processing devices, or the like. The one or more hardware accelerators may, for example, include computer vision (CV) and / or deep learning (DL) accelerators.The terms “application circuit arrangement” and / or “baseband circuit arrangement” can be considered synonymous with “processor circuit arrangement” and can be referred to as such.

[0233] The term "interface circuit arrangement," as used herein, refers to, is part of, or includes a circuit arrangement that enables the exchange of information between two or more components or devices. The term "interface circuit arrangement" may refer to one or more hardware interfaces, such as buses, I / O interfaces, peripheral component interfaces, network interface cards, and / or the like.

[0234] The term "user device" or "UE," as used herein, refers to a device with radio communication capabilities and can describe a remote user of network resources in a communications network. The term "user device" or "UE" may be considered synonymous with, and referred to as, client, handset, mobile device, mobile terminal, user terminal, mobile unit, mobile station, mobile user, subscriber, user, remote station, access agent, user agent, receiver, radio, reconfigurable radio, reconfigurable mobile device, etc. Furthermore, the term "user device" or "UE" may include any type of wireless / wired device or any computing device with a wireless communication interface.

[0235] The term "network element," as used herein, refers to a physical or virtualized device and / or physical or virtualized infrastructure used to provide wired or wireless communication network services. The term "network element" may be considered synonymous with, and / or referred to as, a networked computer, networking hardware, network device, network node, router, switch, hub, bridge, wireless network controller, RAN device, RAN node, gateway, server, virtualized VNF, NFVI, and / or the like.

[0236] The term "computer system," as used here, refers to any type of interconnected electronic devices, computer equipment, or their components. Additionally, the term "computer system" and / or "system" can refer to various components of a computer that are communicatively coupled. Furthermore, the term "computer system" and / or "system" can refer to multiple computer equipment and / or multiple computing systems that are communicatively coupled and configured to share computing and / or networking resources.

[0237] The terms “device,” “computer device,” or the like, as used herein, refer to a computer device or computer system with program code (e.g., software or firmware) specifically designed to provide a particular computing resource. A “virtual device” is a virtual machine image to be implemented by a hypervisor-equipped device that virtualizes or emulates a computer device or is otherwise dedicated to providing a specific computing resource.

[0238] As used herein, the term “resource” refers to a physical or virtual device, a physical or virtual component within a computing environment, and / or a physical or virtual component within a particular device, such as computer devices, mechanical devices, storage space, processor / CPU time, processor / CPU utilization, processor and accelerator loads, hardware time or utilization, electrical power, input / output operations, ports or network sockets, channel / link allocation, throughput, memory utilization, storage, network, database and applications, workload units, and / or the like. A “hardware resource” may refer to computing, storage, and / or networking resources provided by one or more physical hardware elements.A "virtualized resource" can refer to compute, storage, and / or network resources provided by virtualization infrastructure to an application, device, system, etc. The term "network resource" or "communication resource" can refer to resources that computer devices / systems can access via a communication network. The term "system resources" can refer to any type of shared entity used to provide services and may include compute and / or network resources. System resources can be viewed as a set of coherent functions, network data objects, or services accessible by a server, where such system resources reside on a single host or multiple hosts and are uniquely identifiable.

[0239] The term "channel," as used herein, refers to any tangible or intangible transmission medium used to communicate data or a data stream. The term "channel" may be synonymous with and / or equivalent to "communication channel," "data communication channel," "transmission channel," "data transmission channel," "access channel," "data access channel," "link," "data link," "carrier," "radio frequency carrier," and / or any other similar term denoting a path or medium over which data is communicated. Furthermore, the term "link," as used herein, refers to a connection between two devices via a RAT for the purpose of transmitting and receiving information.

[0240] The terms "instantiate," "instantiation," and the like, as used herein, refer to the creation of an instance. An "instance" also refers to a concrete occurrence of an object, which can occur, for example, during the execution of program code.

[0241] The terms "coupled" and "communicatively coupled," along with derivatives thereof, are used herein. The term "coupled" can mean that two or more elements are in direct physical or electrical contact with each other, that two or more elements are in indirect contact with each other but nevertheless cooperate or interact with each other, and / or that one or more other elements are coupled between or connected between the elements considered to be coupled. The term "directly coupled" can mean that two or more elements are in direct contact with each other. The term "communicatively coupled" can mean that two or more elements can be in contact with each other through a means of communication, including via a wire or other interconnect connection, a wireless communication channel or link, and / or the like.

[0242] The term "information element" refers to a structural element that contains one or more fields. The term "field" refers to individual pieces of content within an information element or a data element that contains content.

[0243] Unless otherwise stated herein, terms, definitions, and abbreviations may be consistent with terms, definitions, and abbreviations defined in 3GPP-TR 21.905 v16.0.0 (06-2019) and / or any other 3GPP standard. For the purposes of this document, the following abbreviations (shown in Table 11) may apply to the examples and embodiments discussed herein. Table 11: Abbreviations 3GPP Third Generation Partnership Project IBE In-band emission PUSCH Shared physical uplink channel 4G Fourth generation IEEE Institute of Electrical and Electronics Engineers QAM Quadrature amplitude modulation 5G Fifth generation IEI Information element identifier QCI QoS identifier class 5GC 5G core network IEIDL Information element identifier data length QCL Quasi-colocalization AC Application Client IETF Internet Engineering Task Force QFI QoS flow ID, QoS flow identifier ACK Confirmation IF Infrastructure QoS Service quality ACID Application Client Identification IN THE Interference measurement, intermodulation, IP multimedia QPSK Quadrature (quaternary) phase shift keying AF Application function IMC IMS authorization certificates QZSS quasi-zenith satellite system AM Confirmed mode IMEI International Mobile Device Identity RA-RNI Random-Access RNT1 AMBR Aggregated maximum bitrate IMGI International Mobile Group Identity RAB Radio access carrier,random access burst AMF Access and mobility management function IMPI Private IP multimedia identity RACH Random-access channel TO Access network IMPU Public IP multimedia identity RADIUS Remote AuthenticationDial In User Service ANR Automatic Neighborhood Relationship IMS IP Multimedia Subsystem RAN Wireless access network AP Application protocol, antenna connection, access point IMSI International Mobile Subscriber Identity EDGE Random number (used for authentication) API Application Programming Interface IoT Internet of Things Rare Random access response APN Access point name IP Internet Protocol COUNCIL Wireless access technology ARP Allocation and retention priority IPsec IP security, Internet protocol security ROUGH Routing area update ARQ Automatic retry request IP-CAN IP connectivity access network RB Resource block, radio carrier AS Access Stratum IP-M IP Multicast RBG Resource block group ASP Application service provider IPv4 Internet Protocol Version 4 REG Resource element group ASN.1 Abstract Syntax Notation One IPv6 Internet Protocol Version 6 Rel Release EXECUTION Authentication server function IR Infrared REQ Requirement AWGN Additive white Gaussian noise IS Synchronous RF High frequency BAP Backhaul adaptation protocol IRP Integration reference point RI Ranking indicator BCH Broadcast channel ISDN Digital network for integrated services RIV Resource indicator value BER Bit error ratio ISIM IM service identity module RL Wireless link BFD Beam failure detection ISO International Organization for Standardization RLC Radio link control, radio link control layer BLER Block error rate ISP Internet service provider RLC AM RLC Confirmed Mode BPSK Binary Phase Shift Keying IMF Interworking Function RLC UM RLC Unconfirmed Mode BRAS Broadband remote access server I-WLAN Interworking Wi-Fi RLF Wireless link failure BSS Business support system Folding locks Restriction length of the code, individual USIMl RLM Radio link monitoring BS Base station kB Kilobyte (1000 bytes) RLM-RS Reference signal for RLM BSR Buffer status report kbps Kilobits per second RM Registration Management BW bandwidth Kc Encryption key RMC Reference measurement channel BWP bandwidth portion Ki Individual participant authentication key RMSI Remaining MSI, remaining minimum system information C-RNTI Temporary cell network identifier KPI Key Performance Indicator RN Relay node CA Carrier aggregation, certification body KQI Key quality indicator RNC Wireless network control CAPEX Investment expenditure KSI Key set identifier RNL Wireless network layer CBRA Conflict-based random access ksps Kilo symbols per second RNTI Temporary radio network identifier CC Component carrier, country code, cryptographic checksum KVM Virtual kernel machine ROHC Robust header compression CCA Free channel rating L1 Layer 1 (Bit transmission layer) RRC Radio Resource Control, Radio Resource Control Layer CCE Control channel element L1-RSRP Layer 1 reference signal reception power RRM Radio Resource Management CCCH common control channel L2 Layer 2 (Data Link Layer) RS Reference signal CE Improved coverage L3 Layer 3 (Network Layer) RSRP Reference signal reception power CDM Content delivery network LAA Licensed supported access RSRQ Reference signal reception quality CDMA Code multiplexing multiple access LAN Local network RSSI Receive signal strength indicator CFRA Unrivaled random access SHOP Local Area Data Network RSU Roadside unit CG cell group LBT Listen Before Talk RSTD Reference signal time difference CCF Fee calculation gateway function LCM Lifecycle management RTP Real-time protocol CHF Fee calculation function LCR Low chip rate RTS Readiness to transmit CI Cell identity LCS Location services RTT Cycle time CID Cell ID (e.g., positioning method) LCID Logic channel ID Rx Reception, receiving, receiver CIM Common Information Model LI Shift indicator S1AP S1 application protocol CIR Carrier-to-disturbance ratio LLC Logic link control, low-layer compatibility S1-MMES1 for the tax level CK Encryption key LPLMN Local PLMN S1-U S1 for the user level CM Connection management, conditionally mandatory LPP LTE positioning protocol S-GW Supply gateway CMAS Commercial mobile warning service LSB Least significant bit S-RNTI Temporary SRNC radio network identity CMD command LTE Long Term Evolution S-TMSI Temporary SAE mobile station identifier CMS Cloud management system LWA LTE-WLAN aggregation SA Standalone operating mode CO Conditionally optional LWIP LTE / WLAN radio plane integration with IPsec tunnel SAE System architecture development CoMP Coordinated multiple point LTE Long Term Evolution SAP Service access point CORESE Tax resource rate M2M Machine-to-machine SAPD Service Access Point Descriptor COTS Commercial serial product MAC Media access control (protocol layer context) SAPI Service access point identifier CP Control plane, cyclic prefix, connection point MAC Message authentication code (security / encryption context) SCC Secondary component carrier, secondary CC CPD Connection point descriptor MAC-A MAC for authentication and key agreement (Context TSG T WG3) Scell Secondary cell CPE Equipment at the customer's site MAC-I MAC for data integrity of signaling messages (Context TSG T WG3) SCEF Fitness-for-duty detection function CPICH Joint pilot channel MANO Management and Orchestration SC-FDM 1A Single Carrier Frequency Division Multiple Access CQI Channel quality indicator MBMS Multimedia broadcast and multicast service SCG Secondary cell group CPU CSI processing unit, central processing unit MBSFN Multimedia Broadcast Multicast Service Single Frequency Network SCM Security context management C / R Command / response field bit MCC Mobile network country code SCS subbeam spacing CRAN Cloud radio access network, Cloud RAN MCG Master cell group SCTP Stream control transmission protocol CRB Shared Resource Block MCOT Maximum channel occupancy time SDAP Service Data Adaptation Protocol, Service Data Adaptation Protocol Layer CRC Cyclic redundancy check MCS Modulation and coding scheme SDL Additional downlink CRI Channel state information resource indicator, CSI-RS resource indicator MDAF Management data analysis function SDNF Network function with structured data storage C-RNTI Cellular RNTI MDAS Management data analysis service SDP Meeting description protocol CS Line-switched MDT Minimizing drive tests SDSF Function for structured data storage CSAR Cloud service archive ME Mobile device SDU Service Data Unit CSI Channel condition information MeNB Master eNB SEAF Safety anchor function CSI-IM CSI interference measurement MER Message error ratio SeNB Secondary eNB CSI-RS CSI reference signal MGL Measurement gap length SEPP Security Edge Protection Proxy CSI-RSRP CSI reference signal reception performance MRP Measurement gap repetition period SFI Slot format specification CSI-RSR Q CSI reference signal reception quality MIB Master information block, management information base SFTD Spatial frequency-time diversity, SFN and frame timing difference CSI-SINE 1 CSI signal-to-noise ratio MIMO Multiple input / multiple output SFN System Frame Number CSMA Carrier capture - multiple access MLC Mobile Site Center SgNB Secondary gNB CSMA / C. CSMA with collision avoidance MM Mobility Management SGSN Operating GPRS support node CSS Shared search space, cell-specific search space MME Mobility management entity S-GW Supply gateway CTF Fee calculation trigger function MN Master node SI System information CTS Readiness to transmit MNO Mobile network operators SI-RNTI System Information RNT1 CW Code word MO Object of measurement, mobile origin SIB System information block CWS Competing window size MPBCH Physical MTC broadcast channel SIM Participant identity module D2D Device-to-device MPDCCH Physical MTC downlink control channel SIP Meeting-initiated minutes DC Dual connectivity, direct current MPDSCH Shared physical MTC downlink channel SIP System-in-Package DCI Downlink control information MPRACH Physical MTC random access channel SL Sidelink DF Deployment Flavor MPUSCH Shared physical MTC uplink channel SLA Service Level Agreement (SLA) DL Downlink MPLS MultiProtocol LabelSwitching SM Meeting management DMTF Distributed Management Working Group MS Mobile station SMF Meeting management function DPDK Data Layer Development Kit MSB Highest-order bit SMS Short message service DM-RS, DMRS demodulation reference signal MSC Mobile phone exchange SMSF SMS function DN Data network MSI Minimum system information, MCH planning information SMTC SSB-based measurement timing configuration DNN Data network name MSID Mobile station identifier SN Secondary node, sequence number DNAI Data network access identifier MSIN Mobile station identification number SOC System-on-Chip DRB Data radio carrier MSISDN Mobile subscriber ISDN number SON Self-organizing network DRS Discovery reference signal MT Mobile contract completed, mobile contract SpCell Special cell DRX Discontinuous reception MTC Machine type communication SP-CSI- RNTI Semi-Persistent CSI-RNTI DSL Domain-specific language Digital participant management mMTC Massive MTC, Massive Machine Type Communication PLC Semi-persistent planning DSLAM DSL access multiplexer MU-MIMO Multi-user MIMO SQN Sequence number DwPTS Downlink pilot time slot MWUS MTC wake-up signal, MTC-WUS SR Planning requirement E-LAN Ethernet local area network NACK Negative confirmation SRB Signaling radio carrier E2E End-to-end NAI Network access identifier SRS Probing reference signal ECCA Extended Free Channel Rating, Extended CCA NAS Non-access stratum, Non-access stratum layer SS Synchronization signal ECCE Improved control channel element, improved CCE NCT Network connectivity topology SSB Synchronization signal block ED Energy detection NC-JT incoherent joint transmission SSID Service code identifier EDGE Improved data rates for GSMEvolution (GSMEvolution) NEC Network capability discovery SS / PBC H Block EAS Edge application server NE-DC NR-E-UTRA dual connectivity SSBRI SS / PBCH block resource indicator, synchronization signal, block resource indicator EASID Edge application server identification NEF Network discovery function SSC Meeting and service continuity ECS1 Edge configuration server NF Network function SS-RSRP Synchronization signal-based reference signal reception performance ECSP Edge computing service providers NFP Network forwarding path SS-RSRQ Synchronization signal-based reference signal reception quality EDN Edge data network NFPD Network forwarding path descriptor SS-SINR Synchronization signal-based signal-to-noise ratio EEC Edge Enabler Client NFV Network function virtualization SSS Secondary synchronization signal EECID Edge enabler client identification NFVI NFV infrastructure SSSG search space set group EES Edge Enabler Server NFVO NFV Orchestra SSSIF Search space set indicator EESID Edge enabler server identification NG Next generation, Next-Gen SST Slice / Service types BEFORE Edge hosting environment NGEN-DC NG-RAN-E-UTRA-NR dual connectivity SU-MIMO Single-user MIMO EGMF Disclosure Governance Management Function NM Network manager SUL Additional uplink EGPRS Improved GPRS NMS Network management system TA Timing Advance, Tracking Area EIR Equipment identity register N-PoP Network presence point TAC Tracking area code eLAA Extended Licensed Assisted Access, Extended LAA NMIB -MIB Narrowband-MIB DAY Timing Advance Group EM Element Manager NPBCH Physical narrowband broadcast channel TAI Tracking area identity eMBB extended mobile broadband NPDCCH Physical narrowband downlink control channel DEW Tracking area update EMS Element management system NPDSCH Shared physical narrowband downlink channel TB Transport block eNB Evolved NodeB, E-UTRAN-NodeB NPRACH Physical narrowband random access channel TBS Transport block size EN-DC E-UTRA-NR dual connectivity NPUSCH Shared physical narrowband uplink channel TBD Still to be defined EPC Evolved Packet Core NPSS Narrowband primary synchronization signal TCI Transmission configuration indicator EPDCCH Extended PDCCH, extended physical downlink control channel NSSS Narrowband secondary synchronization signal TCP Transmission communication protocol EPRE Energy per resource element NR New radio, neighborhood relationship TDD Time duplex EPS Developed package system NRF NF repository function TDM Time-division multiplex EREG Extended REG, extended resource element groups NRS narrowband reference signal TDMA Time-division multiple access ETSI EuropeanTelecommunicationsStandards Institute NS Network service TE terminal device ETWS Earthquake and tsunami warning system NSA Non-standalone operating mode TEID Tunnel endpoint identifier eUICC embedded UICC, embedded universal integrated circuit board NSD Network Service Descriptor TFT Traffic flow template E-UTRA UTRA developed NSR Network Service Recording TMSI Temporary Mobile Subscriber Identity E-UTRAZ Developed UTRAN NSSAI Network slice selection assistant information TNL Network transport layer EV2X Enhanced V2X S-NNSAI Single NSSAI TPC Transmit power control F1AP F1 application protocol NSSF Network slice selection function TPMI Transmitted precoding matrix indicator F1-C F1 control plane interface NW network TR Technical Report F1-U F1 user interface NWUS Narrowband alarm signal, narrowband WUS TRP, TRxP Transmission receiving point FACCH Fast associated control channel NZP Non-zero power TRS Tracking reference signal FACCH / F Faster associated control channel / full rate O&M Operation and maintenance TRX Transmitter / receiver FACCH / H Faster associated control channel / half rate ODU2 Optical Channel Data Unit - Type 2 TS Technical specifications, technical standard ACADEMIC SUBJECT Forward access channel OFDM Orthogonal frequency division multiplexing TTI Transmission time interval FAUSCH Fast uplink signaling channel OFDMA Orthogonal Frequency Division Multiple Access Tx transmission, transmitted, sender FB Function block OOB Out-of-Band U-RNTI Temporary UTRAN radio network identity FBI Feedback information OOS Out of sync UART Universal asynchronous receiver and transmitter FCC Federal Communications Commission OPEX Operating costs UCI Uplink control information FCCH Frequency correction channel OSI Other system information UE user device FDD Frequency duplex OSS Operational support system UDM Unified data management FDM Frequency division multiplex OTA Over-the-Air UDP User datagram log FDMA Frequency division multiple access PAPR Peak-to-average performance ratio UDSF Network function for unstructured data storage FE Frontend PAR Peak-to-average ratio UICC Universal integrated circuit board FEC Forward error correction PBCH Physical broadcast channel UL Uplink FFS for further investigation PC Performance control, personal computer UM Unconfirmed mode FFT Fast Fourier transform PCC Primary component carrier, primary CC UML Unified modeling language feLAA Further expanded licensed supported access, further expanded LAA Pcell Primary cell UMTS Universal mobile telecommunications system FN Frame number PCI Physical cell ID, Physical cell identity UP User level FPGA field programmable gate array PCEF Policy and fee enforcement function UPF User-level function FR Frequency range PCF Policy control function URI Uniform Resource Identifier FQDN fully qualified domain name PCRF Policy control and fee calculation rule function URL Unified Resource Locator G-RNTI Temporary GERAN radio network identity PDCP Packet Data Convergence Protocol, Packet Data Convergence Protocol Layer URLLC Ultra-reliable and low latency GERAN GSM EDGE RAN,GSM EDGE radio access network k PDCCH Physical downlink control channel USB Universal Serial Bus GGSN Gateway GPRS support node PDCP Packet Data Convergence Protocol USIM Universal Participant Identity Module GLONAS S GLObal'nayaNAvigatsionnayaSputnikovaya Sistema(German: Global Satellite Navigation System) PDN Packet data network, public data network USS EU-specific search space gNB Next-generation NodeB PDSCH Shared physical downlink channel UTRA Terrestrial UMTS radio access gNB-CU gNB central processing unit, next-generation NodeB central processing unit PDU Protocol data unit UTRAN Universal Radio Access Network gNB-DU distributed gNB unit, next generation distributed NodeB unit PEI Permanent equipment identifiers UwPTS Uplink pilot time slot GNSS Global Satellite Navigation System PDF Package flow description V2I Vehicle-to-infrastructure GPRS General packet-oriented radio service P-GW PDN Gateway V2P Vehicle-to-pedestrian GPSI Generic public participant identification PHICH Physical Hybrid ARQ Indicator Channel V2V Vehicle-to-vehicle GSM Global System for Mobile Communications, Groupe SpecialMobile PHY Physical layer V2X Vehicle-to-Everything GTP GPRS tunnel protocol PLMN Public terrestrial mobile network VIM Virtualized Infrastructure Manager GTP-U GPRS tunnel protocol for user level PIN Personal Identification Number VL Virtual link GTS Go To Sleep Signal (related to WUS) PM Performance measurement VLAN Virtual LAN, Virtual Local Area Network GUMMY Globally unique MME identifier PMI Precoding matrix indicator VM Virtual machine GUTI Globally unique temporary EU identity PNF Physical network function VNF Virtualized network function HARQ Hybrid ARQ, hybrid automatic retry request PNFD Physical Network Function Descriptor VNFFG VNF forwarding graph HANDO Handover PNFR Physical network function recording VNFFGD VNF forwarding graph descriptor HFN HyperFrame number POC PTT-over-Cellular VNFM VNF Manager HO Hard handover PP, PTP Point-to-point VoIP Voice over IP, Voice over Internet Protocol HLR Home Location Register PPP Point-to-point protocol VPLMN Visited public terrestrial mobile network HN Home network PRACH Physical RACH VPN Virtual Private Network HO Handover PRB Physical resource block VRB Virtual Resource Block HPLMN Public terrestrial mobile home network PRG Physical resource block group WiMAX Worldwide Interoperability for Microwave Access HSDPA High-speed downlink packet access ProSe Proximity services, proximity-based service Wi-Fi Wireless Local Network HSN Jump sequence number PRS Positioning reference signal WMAN Wireless urban network HSPA High-speed package access PRR Packet receiving radio WPAN Wireless personal network HSS Home participant server PS Parcel services X2-C X2 control plane HSUPA High-speed uplink packet access PSBCH Physical Sidelink Broadcast Channel X2-U X2 user level HTTP Hypertext Transfer Protocol PSDCH Physical sidelink-downlink channel XML Extensible Markup Language HTTPS Hypertext Transfer Protocol Secure (https is http / 1.1 over SSL, i.e., port 443) PSCCH Physical sidelink control channel XRES Expected user reaction I-Block Information block PSSCH Shared physical sidelink channel XOR Exclusive OR ICCID Integrated circuit card identification PScell Primary cell ZC Zadoff-Chu IAB Integrated access and backhaul PSS Primary synchronization signal ZP Zero Po ICIC Inter-cell disruption coordination PSTN Public telephone exchange network ID Identity, identifier PT-RS Phase tracking reference signal IDFT Inverse Discrete Fourier Transform PTT Push-to-Talk IE Information element PUCK Physical uplink control channel QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 516,736

[0001] Cited non-patent literature

[0000] Standard for Air Interface for Broadband Wireless Access Systems, IEEE Std 802.16-2017, pp. 1-2726 (March 2, 2018

[0081] < / informationobjectclass> < / proxyclass> < / informationobjectclass> < / informationobjectclass> < / informationobjectclass> < / informationobjectclass> < / informationobjectclass> < / informationobjectclass> < / informationobjectclass> < / datatype> < / datatype> < / datentyp> < / datatype>

Claims

Establishment of a Service-Based Management Architecture (SBMA) Management Service (MnS) producer, wherein the establishment comprises a processing circuit arrangement coupled with a storage for storing information associated with the joint testing of machine learning (ML) models, wherein the processing circuit arrangement is designed to: define performance requirements for jointly testing a group of ML models; perform the joint testing of the group of ML models; and send results of the joint testing to an MnS consumer, wherein the results indicate whether the group of ML models meets the performance requirements. Device according to claim 1, wherein the device performs the joint testing without the MnS consumer requesting the joint testing. Device according to claim 1, wherein the processing circuit arrangement is further configured to receive a request from the MnS generator to perform the joint testing, wherein the performance of the joint testing is based on the request from the MnS generator. Device according to claim 2 or claim 3, wherein the group of ML models is identified by an information object class (IOC) ML model coordination group. The device according to claim 4, wherein the performance of the joint testing is based on an ML test requirement IOC that identifies the ML model coordination group IOC. Device according to claim 5, wherein the ML test request IOC further identifies a status of the joint testing as either not started, in progress, suspended, completed or aborted. Device according to claim 1, wherein the processing circuit arrangement is further designed to retrain the group of ML models if the results do not meet the performance requirements. A computer-readable medium that stores computer-executable instructions for jointly testing machine learning (ML) models, which, when executed by one or more processors of a Service-Based Management Architecture (SBMA) management service (MnS) producer, result in the performance of operations that include: defining performance requirements for jointly testing a group of ML models; performing the joint testing of the group of ML models; and sending results of the joint testing to an MnS consumer, the results indicating whether the group of ML models meets the performance requirements. Computer-readable medium according to claim 8, wherein the one or more processors perform the joint testing without the MnS consumer requesting the joint testing. Computer-readable medium according to claim 1, wherein the operations further comprise receiving a request from the MnS producer to perform the joint testing, wherein the performance of the joint testing is based on the request from the MnS producer. Computer-readable medium according to one of claims 9 or 10, wherein the group of ML models is identified by an information object class (IOC) ML model coordination group. Computer-readable medium according to claim 11, wherein the performance of joint testing is based on an ML test requirement IOC that identifies the ML model coordination group IOC. Computer-readable medium according to claim 12, wherein the ML test request IOC further identifies a status of the joint testing as either not started, in progress, suspended, completed or aborted. A method for jointly testing machine learning (ML) models, wherein the method comprises: defining, through a processing circuit arrangement of a Service-Based Management Architecture (SBMA) Management Service (MnS) producer, capability requirements for jointly testing a group of ML models; performing, through the processing circuit arrangement, the joint testing of the group of ML models; and sending, through the processing circuit arrangement, results of the joint testing to an MnS consumer, wherein the results indicate whether the group of ML models meets the capability requirements. Method according to claim 14, wherein the processing circuit arrangement performs the joint testing without the MnS consumer requesting the joint testing. The method of claim 14, further comprising receiving a request from the MnS producer to perform the joint testing, wherein the performance of the joint testing is based on the request from the MnS producer. Method according to one of claims 15 or 16, wherein the group of ML models is identified by an information object class (IOC) ML model coordination group. Method according to claim 17, wherein the performance of the joint testing is based on an ML test requirement IOC that identifies the ML model coordination group IOC. A computer-readable storage medium comprising instructions for carrying out the method according to any one of claims 14-18. Equipment comprising means for carrying out the method according to one of claims 14 - 18 .

Images