Artificial intelligence / machine learning client selection

Abstract

An apparatus configured to generate, for transmission to an artificial intelligence / machine learning enabler (AIMLE) server, a request for AIMLE client discovery, wherein the request comprises one or more discovery criteria that registered AIMLE clients are to satisfy, wherein registered AIMLE clients are registered with the AIMLE server and process, based on signaling received from the AIMLE server, a response to the request comprising an indication of whether the request was successfully completed, partially completed or failed to complete, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request.

Description

Attorney Docket No . 30164 / 102702Ref . No . P71143WO1Artificial Intelligence / Machine Learning Client Selection Inventors : Walter Featherstone, David Manuel Gutierrez Estevez ,Haij ing Hu and Sudeep Manithara VamananPriority / Incorporation By Reference

[0001] This application claims priority to US Provisional Application Serial No . 63 / 755, 683 filed on February 7 , 2025, entitled "Artificial Intelligence / Machine Learning Client Selection, " the entirety of which is incorporated by reference herein .Background

[0002] Federated learning ( FL) is a machine learning technique used to train an Artificial Intelligence / Machine Learning (AI / ML) model across multiple decentralized edge nodes and / or clients . For example, an application layer FL among user equipment (UE ) (and / or network layer FL among a distributed deployment of Network Data Analytics Functions (NWDAFs ) ) may be used to train the AI / ML models . Each node may perform local model training using local data samples .

[0003] Part of the FL process is selecting the nodes from a set of registered nodes to participate in the FL process . When a node is not selected, the requestor may not understand the reasons why one or more nodes are not selected to be part of the FL process .Summary

[0004] Some example embodiments are related to an apparatus comprising processing circuitry coupled to memory, the processing circuitry configured to generate, for transmission toAttorney Docket No . 30164 / 102702Ref . No . P71143WO1an artificial intelligence / machine learning enabler (AIMLE) server, a request for AIMLE client discovery, wherein the request comprises one or more discovery criteria that registered AIMLE clients are to satisfy, wherein registered AIMLE clients are registered with the AIMLE server and process, based on signaling received from the AIMLE server, a response to the request comprising an indication of whether the request was successfully completed, partially completed or failed to complete, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request .

[0005] Other example embodiments are related to an apparatus comprising processing circuitry coupled to memory, the processing circuitry configured to process, based on signaling received from a requestor, a request for artificial intelligence / machine learning enabler (AIMLE) client discovery, wherein the request comprises one or more discovery criteria that registered AIMLE clients are to satisfy, determine registered AIMLE clients that satisfy the one or more discovery criteria and generate, for transmission to the requestor, a response to the request comprising an indication of whether the request was successfully completed, partially completed or failed to complete, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1Brief Description of the Drawings

[0006] Fig . 1 shows an example network arrangement according to various example embodiments .

[0007] Fig . 2 shows an example network architecture according to various example embodiments

[0008] Fig . 3 shows a first example signaling diagram for selecting AI / ML clients according to various example embodiments .

[0009] Fig . 4 shows a second example signaling diagram for selecting AI / ML clients according to various example embodiments .

[0010] Fig . 5 shows an example AIML enabler (AIMLE ) client discovery request according to various example embodiments .

[0011] Fig . 6 shows an example AIMLE client discovery response according to various example embodiments

[0012] Fig . 7 shows an example AIMLE client discovery criteria exclusion attribute according to various example embodiments .

[0013] Fig . 8 shows an example AIMLE client discovery criteria excluded client attribute according to various example embodiments .

[0014] Fig . 9 shows an example network arrangement according to various example embodiments .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0015] Fig. 10 shows an example user equipment (UE) according to various example embodiments .

[0016] Fig. 11 shows an example vertical application layer (VAL) server according to various example embodiments .

[0017] Fig. 12 shows an example AIMLE server according to various example embodiments .Detailed Description

[0018] The example embodiments may be further understood with reference to the following description and the related appended drawings, wherein like elements are provided with the same reference numerals . The example embodiments are related to AIMLE client discovery. Specifically, the example embodiments are related to information that may be provided to the requestor or AIMLE clients related to AIMLE client selection.

[0019] The example embodiments are described with regard to a 5G New Radio (NR) network. However, reference to a 5G NR network is merely provided for illustrative purposes . The example embodiments may be utilized with any network that may establish a connection to a wireless device and exchange information and data with the wireless device (e . g. , 5G-Advanced networks, 6G networks, 7G networks, etc . ) .

[0020] The example embodiments are described with reference to a vertical application layer (VAL) server that operates as the requestor for the AIMLE client selection. However, while the example operations are described with reference to the VALAttorney Docket No . 30164 / 102702Ref . No . P71143WO1server and the application layer, the example embodiments are not limited to this type of device or the application layer, e . g . , the example operations performed by the VAL server may be performed by other devices / entities .

[0021] Similarly, the example embodiments are described with reference to an AI / ML enablement (AIMLE) server that operates as the device determining whether registered AIMLE clients satisfy criteria to be selected as AIMLE clients for a particular FL task . However, while the example operations are described with reference to the AIMLE server and the enablement layer, the example embodiments are not limited to this type of device or the application layer, e . g . , the example operations performed by the AIMLE server may be performed by other devices / entities .

[0022] The example embodiments are also described with reference to a UE executing one or more AIMLE clients . However, the AIMLE clients are not limited to AIMLE clients running on UEs . Other nodes ( e . g . , base stations , routers , edge components , etc . ) may run AIMLE clients and may be selected as an AIMLE client for a particular FL task .

[0023] According to some aspects , the example embodiments introduce manners for a requestor ( e . g . , VAL server) to indicate whether the requestor accepts a partial success in response to a request for identifying AIMLE clients for a particular task . In addition, the example embodiments also provide various information that may be provided back to the requestor when a request for AIMLE clients is not fully successful . This information may be used by the requestor to modify selection criteria or continue with the FL task even though the selectionAttorney Docket No . 30164 / 102702Ref . No . P71143WO1of AIMLE clients was not fully successful . The example embodiments further include analytics data related to the selection of AIMLE clients that may be used by the requestor and / or the AIMLE clients . These and other example embodiments will be described in greater detail below .

[0024] Fig . 1 shows an example network arrangement 100 according to various example embodiments . The network arrangement comprises a user equipment 110 , a 3GPP network system 120, a vertical application layer (VAL) server 130 , an AI / ML enablement (AIMLE ) server 140 and a machine learning (ML) repository 150 .

[0025] The 3GPP network system 120 may be any type of network . As described above, the example embodiments are described with reference to a 5G new radio (NR) radio access network (RAN) . However, the example embodiments may apply to other types of networks ( e . g . , 5G Advanced, sixth generation ( 6G) RAN, 5G cloud RAN, a next generation RAN (NG-RAN) , a longterm evolution (LTE ) RAN, a legacy cellular network, a wireless local area network (WLAN) , etc . ) .

[0026] The UE 110 may be an AI / ML client or potential client . In this example, one UE 110 is illustrated . However, in an actual network arrangement there may be multiple UEs that are AI / ML clients or potential AI / ML clients . The term "potential client" means that the UE 110 may be selected as an AI / ML client for the purposes of FL, e . g . , the UE 110 may be registered as a discoverable AI / ML client . However, as will be described in greater detail below, there may be criteria for selecting a UE as an AI / ML client . In some cases, the UE 110 may qualify underAttorney Docket No . 30164 / 102702Ref . No . P71143WO1these criteria and be selected as an AI / ML client . In other cases, the UE 110 may not qualify under these criteria and may not be selected as an AI / ML client . In the example of Fig . l , the UE 110 is executing one AIMLE client . However, the UE 110 may execute any number of AIMLE clients .

[0027] The VAL server 130 is an application layer server that may connect to a VAL client 113 resident on the UE 110 via a VAL Uu connection shown in Fig . 1 . As will be described in greater detail below, the VAL server 130 may leverage the services of the AIMLE server 140 for the purposes of FL . The AIMLE server 140 may also connect to an AIMLE client 117 resident on the UE 110 via an AIMLE Uu connection shown in Fig . 1 . The AIMLE server 140 may also interface with the 3GPP network system 120 through various network interfaces . Various operations of the VAL server 130 and the AIMLE server 140 are described in greater detail below .

[0028] The ML repository 150 may store various information related to the FL for the AI / ML models . For example, AI / ML capable UEs may register to an AIMLE server and provide an AIMLE client profile and / or a list of supported services . Thus, the ML repository 150 may store these client profiles so that the AI / ML capable UEs that have registered may be discovered as AI / ML clients . Some examples of this discovery process are described in further detail below with reference to Figs . 3 and 4 .

[0029] Fig . 2 shows an example network architecture 200 according to various example embodiments . The network architecture 200 shows the various network functions operated by the core network of the 3GPP network system 120 and theAttorney Docket No . 30164 / 102702Ref . No . P71143WO1interfaces between these core network functions and other components o f the architecture , e . g . , UE , RAN, data network ( DN) , etc .

[0030] The purpose o f the example network architecture 200 i s to place the AIMLE server 140 in relation to the 3GPP network system 120 . In some example embodiments , the AIMLE server 140 may be located in the data network ( DN) behind the N6 (user plane ) interface . However , to consume the services o f fered by the core network ( e . g . , 5GC ) , the AIMLE server 140 may behave as an appl ication function (AF) and uti l i ze the 3GPP service based inferences ( SBA) . Such services may be consumed through the network exposure function (NEE) , or i f the AF was cons idered trusted ( e . g . , Mobi le Network Operator (MNO ) operated) , the AIMLE server 140 may be able to consume services directly from speci f ic network functions ( e . g . , Pol icy Control Function ( PCF) , Uni fied Data Management (UDM) , etc . ) . The network architecture 200 i s only an example and the example embodiments may be implemented in other network architectures .

[0031] Fig . 3 shows a f irst example s ignal ing diagram 300 for selecting AI / ML cl ients according to various example embodiments . In these example embodiments , the VAL server 130 selects the AI / ML cl ients .

[0032] In 310 , the VAL server 130 sends a cl ient di scovery request to the AIMLE server 140 . Thi s cl ient di scovery request may include a selection criteria ( e . g . , characteristics o f the potential cl ient ) , a number o f required cl ients , etc .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0033] In 320, the AIMLE server 140 may respond with a client discovery response that includes a list of client IDs that meet the VAL server 130 provided client selection criteria . The AIMLE server 140 may determine the list of clients based on information stored in the ML repository 150. In some example embodiments, client discovery response may also include other information such as AIML supported tasks .

[0034] In 330, the VAL server 130 may select one or more of the client IDs provided in the list of client IDs from 320 and send a client selection request with the list of selected clients . The AIMLE server 140 may treat the list of selected clients in the client selection request as candidate AIMLE clients .

[0035] In this example, the UE 110 may be one of the candidate AIMLE clients . Thus, in 340, the AIMLE server 140 and the UE 110 may perform an AIMLE client participation procedure where the UE is either accepted or rejected as an AI / ML client . This AIMLE client participation procedure may be performed for each candidate AIMLE client .

[0036] In 350, the AIMLE server 140 may send a client selection response that includes information concerning the candidate AIMLE clients . As will be described in greater detail below, this information may include an AIMLE client set identifier that identifies those candidate AIMLE clients that have been successfully activated as AIMLE clients . However, the information may also include identification and other information related to those candidate AIMLE clients that have not been successfully activated as AIMLE clients .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0037] Fig. 4 shows a second example signaling diagram 400 for selecting AI / ML clients according to various example embodiments . In these example embodiments, the AIMLE server 140 selects the AI / ML clients .

[0038] In 410, the VAL server 130 sends a client selection request to the AIMLE server 140. This client selection request may include a selection criteria (e . g. , characteristics of the potential client) , a number of required clients, etc .

[0039] The AIMLE server 140 may select candidates based on the client selection criteria provided by the VAL server 130. For example, the AIMLE server 140 may compare the client selection criteria to information stored for potential AIMLE clients in the ML repository 150. The potential AIMLE clients that satisfy the client selection criteria may be determined by the AIMLE server 140 to be candidate AIMLE clients .

[0040] In this example, the UE 110 may be one of the candidate AIMLE clients . In 420, the AIMLE server 140 and the UE 110 may perform an AIMLE client participation procedure where the UE is either accepted or rejected as an AI / ML client . This AIMLE client participation procedure may be performed for each candidate AIMLE client .

[0041] In 430, the AIMLE server 140 may send a client selection response that includes information concerning the candidate AIMLE clients . As will be described in greater detail below, this information may include an AIMLE client set identifier that identifies those candidate AIMLE clients thatAttorney Docket No . 30164 / 102702Ref . No . P71143WO1have been successfully activated as AIMLE clients . However, the information may also include identification and other information related to those candidate AIMLE clients that have not been successfully activated as AIMLE clients .

[0042] The above example embodiments provided examples of the VAL server 130 being the requestor for the AIMLE client selection . In other example embodiments an AIMLE server may be the requestor . For example, an AIMLE server may not identify enough AIMLE clients . In this case, the AIMLE server may make a request to another AIMLE server to discover clients from that server, e . g . , the first AIMLE server is also a requestor . In further example embodiments, an AIMLE client ( e . g . , the UE 110 ) may be the requestor .

[0043] The above example embodiments provided a request / response framework for selecting AIMLE clients . In other example embodiments a publish / subscribe framework for selecting AIMLE clients , e . g . , a VAL server may subscribe to an AIMLE server with a request to be informed when a number of potential AIMLE clients become available that satisfy a selection criteria . The general flow of publish / subscribe framework and the information exchanged between the VAL server and the AIMLE server is similar to that described above in the request / response framework . Thus, the example embodiments may also be implemented in systems where the publish / subscribe framework is used.

[0044] The above examples provided manners of AIMLE client selection . The example embodiments further provide information and insight into the selection of AIMLE clients , e . g . , why was aAttorney Docket No . 30164 / 102702Ref . No . P71143WO1AIMLE client selected or not selected. This information may be valuable on the server side . For example, on the server side, this information may be used to modify the client selection criteria, e . g. , the selection criteria provided by the VAL server 130 in operation 310 or 410. This may enable more clients to be selected through a suitable relaxation of the selection criteria . This information may also indicate why a particular set of clients were selected. For example, this may enable selection criteria to be modified to ensure similardisparity / variance / standard deviation between the criteria and the clients' actual values . This may allow a degree of flexibility to the criterion and create greater balance across the members of a candidate client set .

[0045] In some example embodiments, the information and insight may be related to the number of requested clients . As described above, the client discovery request (310) or the client selection request (410) may include a number of required AIMLE clients . This is further described with reference to Figs .5 and 6 .

[0046] Fig. 5 shows an example AIMLE client discovery request 500 according to various example embodiments . The AIMLE discovery request 500 includes various fields or information elements ( IES ) . As described above, the AIMLE client discovery request 500 may be sent from the VAL server 130 to the AIMLE server 140 (e . g. , operation 310 of Fig. 3) . A first IE is a Requestor Identity IE 510 that identifies the VAL server 130 making the request . The second IE is an AIMLE client discovery criteria IE 520 that includes the selection criteria for the AIMLE clients as defined by the VAL server 130. A third IE is aAttorney Docket No . 30164 / 102702Ref . No . P71143WO1Number of Required AIMLE clients IE 530 that identifies the number of AIMLE clients that the VAL server 130 requires to be in the set of AIMLE clients .

[0047] A fourth IE is a Minimum Number of Required AIMLE clients IE 540 that identifies a minimum number of required AIMLE clients . For example, the IE 540 may indicate to the AIMLE server 140 an indication of whether a response with fewer than the number of the required AIMLE clients identified by the IE 530 is acceptable . To provide an example use case, the IE 530 may indicate the number of the required AIMLE clients is 100 and the IE 540 may indicate a minimum number of required AIMLE clients is 50 . If the AIMLE server 140 successfully discovers 75 candidate AIMLE clients , the AIMLE server 140 may provide a partially successful response in the client discover response, e . g . , operation 330 of Fig . 3 . In such a case, when the VAL server 130 receives the partially successful response, the VAL server 130 may use this information to determine a further course of action . For example, the VAL server 130 may relax the selection criteria to attempt to increase the number from a partially successful response to a fully successful response .

[0048] In the above example embodiments , an AIMLE client discovery criteria IE 520 was described. In some example embodiments , an AIMLE client selection criteria IE may also be provided, e . g . , in the client selection request 330 of Fig . 3 or the client selection request 410 of Fig . 4 . The AIMLE client selection criteria IE may include information similar to that described for the AIMLE client discovery criteria IE, e . g . , selection criteria similar to the discovery criteria . In cases where both the AIMLE client discovery criteria IE 520 and theAttorney Docket No . 30164 / 102702Ref . No . P71143W01AIMLE client selection criteria IE is provided ( e . g . , in client discovery request 310 and client selection request 330 of Fig . 3 , respectively) , the discovery criteria and the selection criteria may be the same or may vary . In addition, a client selection request may also include a minimum number of AIMLE clients for AIMLE client selection that operates in a similar manner as the minimum number of required AIMLE clients IE 540.

[0049] Fig . 6 shows an example AIMLE client discovery response 600 according to various example embodiments . The AIMLE discovery response 600 includes various fields or IES . As described above, the AIMLE client discovery response 600 may be sent from the AIMLE server 140 to the VAL server 130 in response to the client discovery request (e . g . , operation 320 of Fig . 3 ) . A first IE is a Status IE 610 that identifies the status of the request . As shown in Fig . 6, the status may be "Success", "Partial Success" or "Fail" . To carry through with the above example where the client discovery request has a required number AIMLE clients of 100 and a minimum number of required AIMLE clients of 50 , when the AIMLE server 140 discovers greater than or equal to 100 potential AIMLE clients, the value of IE 610 may be "Success" . When the AIMLE server 140 discovers greater than or equal to 50 but less than 100 potential AIMLE clients, the value of IE 610 may be "Partial Success" . When the AIMLE server 140 discovers less than 50 potential AIMLE clients, the value of IE 610 may be "Fail" .

[0050] The values provided above are only examples to illustrate how the different IEs may be interpreted by the VAL server 130 and the AIMLE server 140 . Any other values may be used. In some example embodiments, the value for the IE 530 andAttorney Docket No . 30164 / 102702Ref . No . P71143WO1the IE 540 may be set to the same value , meaning that the operation i s a fai lure unles s the number o f the required AIMLE cl ients indicated by the IE 530 are identi f ied by the AIMLE server 140 .

[0051] In some example embodiments , the IE 540 may al so be a partial response acceptable Boolean f lag . I f thi s IE 540 i s set to TRUE , the AIMLE server may return any AIMLE client IDs in the di scovery response that sati s f ied the selection criteria even i f the number o f the required AIMLE cl ients indicated by the IE 530 i s not sati s f ied . These AIMLE clients may be identi f ied to the VAL server 130 in the IE 620 o f the AIMLE client di scovery response that may include the l ist o f AIMLE cl ient IDs that satis f ied the selection criteria .

[0052] The VAL server 130 may use thi s information, e . g . , Partial Succes s , for various purposes . For example , the VAL server 130 may continue with the AI / ML related operations or modi fy the requirements o f the operations to be able to run with a lower number of cl ients .

[0053] The above example IE S may be applicable to an AIMLE cl ient di scovery request ( e . g . , operation 310 o f Fig . 3 ) , an AIMLE cl ient selection request ( e . g . , operation 410 o f Fig . 4 ) , an AIMLE cl ient selection subscription request , an AIMLE cl ient di scovery response ( e . g . , operation 320 o f Fig . 3 ) , an AIMLE cl ient selection response 430 ( e . g . , operation 430 o f Fig . 4 ) and / or an AIMLE cl ient selection subscription response .

[0054] In the above examples , the response that included information regarding the AIMLE cl ient selection was based onAttorney Docket No . 30164 / 102702Ref . No . P71143WO1whether a number of requested AIMLE clients was satisfied.However, a response that includes additional information about client selection does not need to be predicated on a number of requested AIMLE . For example, the requestor (e . g. , VAL server 130) may receive feedback regarding registered AIMLE clients that were not selected and / or discovery criteria that were not satisfied by the AIMLE clients that were not selected for every request, regardless of the outcome of the request (e . g. , success, partial success, fail) . In other example embodiments, this feedback may only be provided when the outcome of the request is partial success or fail .

[0055] In some example embodiments, the request may include an indication of when the requestor would like to receive feedback regarding the request, e . g. , always, only upon partial success or failure, etc . In other example embodiments, when the feedback is provided may be determined based on a configuration of the AIMLE server . Some examples of the feedback that may be provided to the requestor are described in greater detail below.

[0056] When the VAL server 130 receives a response identifying the AIMLE clients or potential AIMLE clients, the VAL server 130 determines that the AIMLE server 140 will only select clients that meet the selection criteria defined by the VAL server 130. However, when the operation is a failure or only partially successful, the VAL server 130 may want to determine the criteria that ruled out certain clients . For example, the AIMLE server 140 may return all clients that satisfied the selection criteria and the issue is that there are not enough registered clients . In another example, there may be enoughAttorney Docket No . 30164 / 102702Ref . No . P71143WO1registered cl ients , it i s j ust that the selection criteria was too restrictive and excluded many registered cl ients .

[0057] The example embodiments may provide indications to the VAL server 130 so that the VAL server 130 may determine the reason that the selection operation was a fai lure or only partial ly success ful . In some example embodiments , the information provided back to the VAL server 130 by the AIMLE server 140 in response to a request for AIMLE cl ient selection may include information related to the reasons why registered cl ients were excluded . In some example embodiments , this information may be provided back to the VAL server 130 us ing an attribute ( e . g . , AIMLE cl ient discovery criteria exclusion ) that may be provided to the VAL server 130 in, for example , an AIMLE cl ient di scovery response ( e . g . , operation 320 o f Fig . 3 ) , an AIMLE cl ient selection response 430 ( e . g . , operation 430 o f Fig .4 ) and / or an AIMLE client selection subscription response .

[0058] Fig . 7 shows an example AIMLE cl ient discovery criteria exclusion attribute 700 according to various example embodiments . This attribute 700 may be returned for each registered cl ient that failed to meet the selection criteria speci f ied by the VAL server 130 . As shown in Fig . 7 , the attribute 700 has a data type 705 o f di scoverycriteria that compri ses an array o f the di scovery criteria that the regi stered cl ient failed to satis fy . The discoverycriteria wil l be described in greater detail below . The attribute 700 may identi fy as a key-value pair 710 the AIMLE cl ient ID ( e . g . , res iding on a UE ) and the di scovery criteria that were not satis f ied, e . g . , <aimle-client- id23 , { di scoverycriteria, di scoverycriteria } > .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0059] The discoverycriteria 720 is shown in more detail in Fig. 7. In this example, the AIMLE client failed to satisfy two (2 ) criteria . The first criteria is Service ID. In this example, the selection criteria defined by the VAL server 130 may be that the AIMLE support a Service ID value 101. The AIMLE client having the ID aimle-client-id23 does not satisfy this selection criteria so the discoverycriteria 720 lists the key value pair <Service ID, 101> as a selection criteria that the AIMLE client does not satisfy. Similarly, the second criteria is AIMLE client velocity. In this example, the selection criteria defined by the VAL server 130 may be that the AIMLE support a AIMLE client velocity value of 12. The AIMLE client having the ID aimle-client-id23 does not satisfy this selection criteria so the discoverycriteria 720 lists the key value pair <AIMLE client velocity, 12> as a selection criteria that the AIMLE client does not satisfy. Again, these are only examples and the attribute 700 may include any criteria that the AIMLE client does not satisfy .

[0060] The VAL server 130, when receiving this information, may then determine the reasons why various AIMLE clients have not satisfied the selection criteria . This may allow the VAL server 130 to relax the selection criteria to allow an excluded AIMLE client to be selected. For example, if an AIMLE client was only excluded for a single criteria, e . g. , <AIMLE client velocity, 12>, the VAL server 130 may relax the selection criteria for the AIMLE client velocity to a value that is less stringent than 12, e . g. , 10, 8, etc . This less stringent selection criteria may enable the previously excluded AIMLE client to be selected.Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0061] In some example embodiments, the discoverycriteria 720, instead of listing the value of the selection criteria, may list the actual capability of the AIMLE client . For example, the AIMLE client velocity selection criteria may have a value of 12 as described in the examples above . The AIMLE client may not satisfy this criteria but instead of indicating the value of the AIMLE client velocity selection criteria in the discoverycriteria 720 (e . g. , the VAL server 130 may be already aware of the value because the VAL server determined the value) , the discoverycriteria 720 may indicate the actual value associated with the AIMLE client . For example, if the actual value supported by the AIMLE client for AIMLE client velocity is 10, the discoverycriteria 720 may indicate the key value pair <AIMLE client velocity, 10>. In this way, the VAL server 130 has additional information if the VAL server 130 wants to relax the selection criteria . As described above and further below, a user of the AIMLE client may have to opt in to these type of example embodiments where the actual capabilities of the AIMLE client are exposed.

[0062] In other example embodiments, the information provided back to the VAL server 130 may be on a per discovery criteria basis . This information may be provided back to the VAL server 130 using an attribute (e . g. , AIMLE client discovery criteria excluded clients) that may be provided to the VAL server 130 in, for example, an AIMLE client discovery response (e . g. , operation 320 of Fig. 3) , an AIMLE client selection response 430 (e . g. , operation 430 of Fig. 4 ) and / or an AIMLE client selection subscription response .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0063] Fig. 8 shows an example AIMLE client discovery criteria excluded client attribute 800 according to various example embodiments . This attribute 800 may be returned for each discovery criteria specified by the VAL server 130 that excluded registered clients . As shown in Fig. 8, the attribute 800 includes a list of the discovery criteria . In the example of Fig. 8, the example criteria that excluded clients include the Service ID (e . g. , serviceidExClients 810) and the AIMLE client velocity (e . g. , clientVelocityExClients 820) . Again, the use of these two discovery criteria is only an example and the attribute 800 may include any discovery criteria that was used to exclude AIMLE clients .

[0064] The discovery criteria listed in the attribute 800 have a data type of Clientcount (e . g. , as shown by 815 and 825) . The ClientCount is shown in more detail in 830 but is a number of AIMLE clients that were excluded based on the specific discovery criteria . For example, if the Service ID discovery criteria excluded 5 AIMLE clients from being selected, the attribute 810 may identify as a key-value pair 817, the discovery criteria and the number of excluded AIMLE clients, e . g. , <ServiceID, 5>. In this manner, the VAL server 130 understands how many AIMLE clients each discovery criteria excluded .

[0065] In some example embodiments, rather than the client count, the attribute 800 may include a list of the AIMLE clients that were excluded based on the individual discovery criteria (e . g. , listed by AIMLE client ID) . In these example embodiments, the VAL server 130 may determine whether one AIMLE client is failing to meet all criteria or whether each criteria is rulingAttorney Docket No . 30164 / 102702Ref . No . P71143WO1out a different AIMLE client . For example, if the attribute 800 lists the same AIMLE client ID for both the Service ID and the Client velocity discovery criteria, the VAL server 130 may understand that only relaxing one of those criteria may not increase the number of AIMLE clients because the AIMLE client associated with the AIMLE client ID will still not satisfy the other discovery criteria . On the other hand, if different AIMLE client IDs are listed for the Service ID and the Client velocity discovery criteria, the VAL server 130 may understand that only relaxing one of those criteria may increase the number of AIMLE clients or relaxing both may result in an additional 2 AIMLE clients being selected. Again, these are only examples and any number of AIMLE client IDs may be listed for any criteria depending on whether the AIMLE client has satisfied the listed discovery criteria .

[0066] In some example embodiments , an AIML client selection analytics service may be performed through Artificial Intelligence Machine Learning Enablement, or Application Data Analytics Enablement, e . g . , by an AIMLE server or an Application Data Analytics Enablement Service (ADAE) server . For example, a mechanism may be made available from the enabler layer ( e . g . , offered by an ADAE or AIMLE server) by leveraging the information at the enabler layer and provided by the 5GC in the Network Exposure Function (NEF) Member UE Selection Assistance Notify IE to generate analytics relating to member selection . The query (or subscription) for analytics may be across all candidate registered clients or limited to the list of clients provided by the VAL server 130 . For example, a procedure may be defined between the VAL server 130 and the ADAE (or AIMLE ) server (or between ADAE servers ) to support AIMLE clientAttorney Docket No . 30164 / 102702Ref . No . P71143WO1selection analytics . These analytics may include, but are not limited to, AIMLE client selection analytics subscription request / response and associated notification, AIMLE client selection analytics request / response, etc . This information may allow the VAL server 130 to select or modify one or more client selection criteria . The analytics related information may be passed between the ADAE / AIMLE servers . These servers may be deployed in a distributed manner to ensure that a VAL server only receives information relatinq to AIMLE clients to which it is authorized, e . g . , clients that have provided consent to be FL members .

[0067] The VAL server 130 may use the analytics for various purposes . For example, the VAL server 130 may revoke a group member' s preferential status ( e . g . , relating to a service level contract ) if the group member is continuously rej ecting FL learning requests even after being selected . In another example, the analytics may reveal a fairness issue, e . g . , one AIMLE client may be targeted for inclusion in group ( s ) more than another AIMLE client . The VAL server may use this analytics information to address the issue .

[0068] On the client side, the analytics information may provide insight to the candidate clients on their selection or exclusion . This may enable clients to modify the client profile to influence the likelihood of selection . In some example embodiments , a registered AIMLE client may query (or subscribe ) for analytics relating to their selection . These analytics may also include predictions . For example, a registered AIMLE client may have a service contract where the AIMLE client contributes to the FL for a reduced fee or other benefit . The analytics mayAttorney Docket No . 30164 / 102702Ref . No . P71143WO1inform the user of the AIMLE client information such as the AIMLE client is not being selected or is not contributing to the FL task. This may indicate to the user that the benefit may be taken away. This information may provide a client with insight into whether it was being selected proportionately to other members, e . g. , fairly as compared to other participants . The analytics may also provide the AIMLE client with warnings, e . g. , the client' s contribution is below an expected commitment and suggestions for increasing contributions, e . g. , moving to a service area, enabling Wi-Fi / 3GPP, charging battery, etc.

[0069] In one example, the AIMLE client may report obtained analytics to the application layer as evidence of a contribution (e . g. , charging or service contract related) , or to provide evidence that the AIMLE client was available for selection but not selected. This may be feedback to the VAL server 130 via the VAL client from the AIML client .

[0070] In some example embodiments, the analytics service may be an application layer service that is performed by the VAL server 130 rather than at the enablement / enabler layer) .

[0071] When the analytics are provided at the enabler layer, some of the reported information that was described above may not be provided. For example, if the VAL server 130 is subscribed to the analytics service of the AIMLE server 140, when the AIMLE server 140 is providing a client discovery response, the information described above (e . g. , partial success, identification of AIMLE clients not satisfying discovery criteria, etc . ) may not be provided in the responsesAttorney Docket No . 30164 / 102702Ref . No . P71143WO1because the VAL server 130 may query or subscribe to this information as part of the analytics service .

[0072] Some examples of the analytics may include reports based on past selection (e . g. , over a period of time) or predictions relating to future selection. In another example, reports may be relative to the group selected for a task / service, e . g. , of the clients that met the selection criteria for the required task: a client / UE selected more (> 1 / "client pool") or less (< 1 / "client pool") than the expected average . In a further example, reports may be relative to the group of registered clients, to the group of UEs / clients that have provided their consent to be considered as members, etc . The analytics may be provided on the clients' contribution to the overall task / service . The analytics may include feedback as to reason (s) a client was not selected, e . g. , insufficient battery level, out of service area, too close to other clients, etc . This feedback may include information on thedisparity / variance / standard deviation of clients discarded for each specific criterion when the requester is able to tolerate such variance between a criteria and the clients' actual values . Again, any information that is exposed to a client should not expose identifiable information of the other registered clients, e . g. , the analytics should be generic in nature . These are only examples of the types of analytics information that may be provided to the VAL server 130 and / or AIMLE clients . Other types of analytics information may also be provided.

[0073] Fig. 9 shows an example network arrangement 900 according to various example embodiments . The example network arrangement 100 includes a UE 110. The UE 110 may be any typeAttorney Docket No . 30164 / 102702Ref . No . P71143WO1of electronic component that is configured to communicate via a network, e . g. , mobile phones, tablet computers, desktop computers, smartphones, embedded devices, wearables, Internet of Things ( loT) devices, etc . An actual network arrangement may include any number of UEs being used by any number of users .Thus, the example of a single UE 110 is merely provided for illustrative purposes .

[0074] The UE 110 may be configured to communicate with one or more networks . In the example of the network arrangement 900, the network with which the UE 110 may wirelessly communicate is a 5G NR radio access network (RAN) 920. The UE 110 may also communicate with other types of networks (e . g. , 5G cloud RAN, a next generation RAN (NG-RAN) , a legacy cellular network, etc . ) and the UE 110 may also communicate with networks over a wired connection. With regard to the example embodiments, the UE 110 may establish a connection with the 5G NR RAN 920. Therefore, the UE 110 may have, at least, a 5G NR chipset to communicate with the NR RAN 920.

[0075] The 5G NR RAN 920 may be portions of a cellular network that may be deployed by a network carrier (e . g. , Verizon, AT&T, T-Mobile, etc. ) . The 5G NR RAN 920 may include cells or base stations that are configured to send and receive traffic from UEs that are equipped with the appropriate cellular chip set . In this example, the 5G NR RAN 120 includes the gNB 920A. However, reference to a gNB is merely provided for illustrative purposes, any appropriate base station or cell may be deployed (e . g. , Node Bs, eNodeBs, HeNBs, eNBs, gNBs, gNodeBs, macrocells, microcells, small cells, femtocells, etc . ) .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0076] Any association procedure may be performed for the UE 110 to connect to the 5G NR RAN 920 . For example, as discussed above, the 5G NR RAN 920 may be associated with a particular network carrier where the UE 110 and / or the user thereof has a contract and credential information (e . g . , stored on a SIM card) . Upon detecting the presence of the 5G NR RAN 920, the UE 110 may transmit the corresponding credential information to associate with the 5G NR RAN 920. More specifically, the UE 110 may associate with a specific cell ( e . g . , gNB 920A) .

[0077] The network arrangement 900 also includes a cellular core network 930, the Internet 940 , an IP Multimedia Subsystem ( IMS ) 950 , and a network services backbone 960 . The cellular core network 930 manages the traffic that flows between the cellular network and the Internet 940 . The IMS 950 may be generally described as an architecture for delivering multimedia services to the UE 110 using the IP protocol . The IMS 950 may communicate with the cellular core network 930 and the Internet 940 to provide the multimedia services to the UE 110 . The network services backbone 960 is in communication either directly or indirectly with the Internet 940 and the cellular core network 130. The network services backbone 960 may be generally described as a set of components (e . g . , servers , network storage arrangements , etc . ) that implement a suite of services that may be used to extend the functionalities of the UE 110 in communication with the various networks .

[0078] The network arrangement 900 may also include a VAL server 970 and an AIMLE server 980. Example manners of the VAL server 970 and the AIMLE server communicating with and through the cellular core network 930 were described above . In addition,Attorney Docket No . 30164 / 102702Ref . No . P71143WO1example operations performed by the VAL server 970 and an AIMLE server 980 for implementing the example embodiments were described in detail above .

[0079] Fig . 10 shows an example UE 110 according to various example embodiments . The UE 110 may represent any electronic device and may include a processor 1005, a memory arrangement 1010, a display device 1015, an input / output ( I / O) device 1020 , a transceiver 1025, and other components 1030. The other components 1030 may include, for example, an audio input device, an audio output device, a battery that provides a limited power supply, a data acguisition device, ports to electrically connect the UE 110 to other electronic devices, sensors to detect conditions of the UE 110 , etc .

[0080] The processor 1005 may be configured to execute a plurality of engines for the UE 110 . For example, the engines may include a FL engine 1035 for performing operations related to FL . The operations include, but are not limited to, executing one or more VAL client ( s ) , executing one or more AIMLE client ( s ) and providing user authorization for various FL operations .

[0081] The above referenced engine being an application ( e . g . , a program) executed by the processor 1005 is only example . The functionality associated with the engines may also be represented as a separate incorporated component of the UE 110 or may be a modular component coupled to the UE 110, e . g . , an integrated circuit with or without firmware . For example, the integrated circuit may include input circuitry to receive signals and processing circuitry to process the signals and other information . The engines may also be embodied as oneAttorney Docket No . 30164 / 102702Ref . No . P71143WO1application or separate applications . In addition, in some UEs, the functionality described for the processor 1005 is split among two or more processors such as a baseband processor and an applications processor. The example embodiments may be implemented in any of these or other configurations of a UE .

[0082] In some examples, inputs may be fed to the FL engine 1035. The FL engine 1035 may include one or more learning-based and / or non-learning-based models for perceiving, synthesizing, and inferring information. Persons skilled in the art will appreciate that the FL engine 1035 can include any suitable number of processes to train the models based on the input data .

[0083] Persons of ordinary skill in the art will appreciate that FL engine 1035 can include any suitable machine learning models that are well-known or widely available such as regression techniques, classification techniques, neural networks, and deep learning networks . In instances where FL engine 1035 comprises a machine-learning based model, the FL engine 1035 can be trained using one or more well-known or widely available training techniques such as supervised learning, semi-supervised learning, unsupervised learning, and / or reinforcement learning techniques . The training data can include the aforementioned input data .

[0084] The memory arrangement 1010 may be a hardware component configured to store data related to operations performed by the UE 110. The display device 1015 may be a hardware component configured to show data to a user while the I / O device 1020 may be a hardware component that enables the user to enter inputs . The display device 1015 and the I / O deviceAttorney Docket No . 30164 / 102702Ref . No . P71143WO11020 may be separate components or integrated together such as a touchscreen .

[0085] The transceiver 1025 may be a hardware component configured to establ ish a connection with the 5G NR-RAN 120 , an LTE-RAN (not pictured) , a legacy RAN (not pictured) , a wireles s local area network (WLAN) (not pictured) , etc . Accordingly, the transceiver 1025 may operate on a variety o f di f ferent frequencies or channel s ( e . g . , set o f consecutivefrequencies ) . The transceiver 1025 includes circuitry conf igured to transmit and / or receive s ignals ( e . g . , control s ignal s , data s ignal s ) . Such signals may be encoded with information implementing any one of the methods described herein . The proces sor 1005 may be operably coupled to the transceiver 1025 and conf igured to receive from and / or transmit s ignal s to the transceiver 1025 . The proces sor 1005 may be conf igured to encode , decode and / or proces s s ignal s ( e . g . , s ignal ing from a base station o f a network ) for implementing any one o f the methods described herein .

[0086] In the example o f Fig . 10 , the proces sor 1005 and the radio frequency ( RF) circuitry ( e . g . , transceiver 1025 ) are i l lustrated as separate components . However , in some example embodiments , the RF circuitry and the process ing circuitry may be integrated into the same chip, e . g . , a system on chip that includes a baseband processor and RF circuitry .

[0087] Fig . 11 shows an example VAL server 1100 according to various example embodiments . The VAL server 1100 may represent an appl ication layer device that that implements FL operations as described in detail above . In some examples , the VAL serverAttorney Docket No . 30164 / 102702Ref . No . P71143WO11100 may be implemented in a server device such as illustrated in the example of Fig. 11. In other examples, the VAL server 1100 may be implemented in a distributed manner such as in a cloud computing implementation or as a network function.

[0088] The VAL server 1100 may include a processor 1105, a memory arrangement 1110, an input / output ( I / O) device 1115, a transceiver 1120, and other components 1125. The other components 1125 may include, for example, an audio input device, an audio output device, a battery, a data acquisition device, ports to electrically connect the VAL server 1100 to other electronic devices and / or power sources, etc.

[0089] The processor 1105 may be configured to execute a plurality of engines for the VAL server 1100. For example, the engines may include an FL engine 1130 for performing operations related to FL including, but not limited to, generating client discovery requests including discovery criteria and reporting criteria for responding to the discovery requests, selecting AIMLE clients, and subscribing to AIMLE client selection analytics .

[0090] The memory 1110 may be a hardware component configured to store data related to operations performed by the VAL server 1100. The I / O device 1115 may be a hardware component or ports that enable a user to interact with the VAL server 1100.

[0091] The transceiver 1120 may be a hardware component configured to exchange data with the UE 110, the AIMLE server 140 and any other component in the example network arrangements described herein, either directly or indirectly. Because the VALAttorney Docket No . 30164 / 102702Ref . No . P71143WO1server 1100 is typically resident within a data network, the transceiver 1120 may include a wired network interface such as an Ethernet or other wired type network interface . In some examples , the transceiver 1120 may include a wireless interface and operate on a variety of different frequencies or channels ( e . g . , set of consecutive frequencies ) . Therefore, the transceiver 1120 may include one or more components ( e . g . , radios ) to enable the data exchange with the various networks and UEs . The transceiver 1120 includes circuitry configured to transmit and / or receive signals (e . g . , control signals , data signals ) . Such signals may be encoded with information implementing any one of the methods described herein . The processor 1105 may be operably coupled to the transceiver 1120 and configured to receive from and / or transmit signals to the transceiver 1120. The processor 1105 may be configured to encode and / or decode signals for implementing any one of the methods described herein .

[0092] Fig . 12 shows an example AIMLE server 1200 according to various example embodiments . The AIMLE server 1200 may represent an enabler layer device that that implements FL operations as described in detail above . In some examples , the AIMLE server 1200 may be implemented in a server device such as illustrated in the example of Fig . 12 . In other examples, the AIMLE server 1200 may be implemented in a distributed manner such as in a cloud computing implementation or as a network function .

[0093] The AIMLE server 1200 may include a processor 1105, a memory arrangement 1210 , an input / output ( I / O) device 1215, a transceiver 1220, and other components 1225. The otherAttorney Docket No . 30164 / 102702Ref . No . P71143WO1components 1225 may include, for example, an audio input device, an audio output device, a battery, a data acquisition device, ports to electrically connect the AIMLE server 1200 to other electronic devices and / or power sources, etc.

[0094] The processor 1205 may be configured to execute a plurality of engines for the AIMLE server 1200. For example, the engines may include an FL engine 1230 for performing operations related to FL including, but not limited to, determining AIMLE clients that meet / do not meet discovery criteria, report information regarding the AIMLE clients that do not meet the discover criteria to a VAL server and host an analytics service related to AIMLE client selection.

[0095] The memory 1210 may be a hardware component configured to store data related to operations performed by the AIMLE server 1200. The I / O device 1215 may be a hardware component or ports that enable a user to interact with the AIMLE server 1200.

[0096] The transceiver 1220 may be a hardware component configured to exchange data with the UE 110, the VAL server 130 and any other component in the example network arrangements described herein, either directly or indirectly. Because the AIMLE server 1200 is typically resident within a data network, the transceiver 1220 may include a wired network interface such as an Ethernet or other wired type network interface . In some examples, the transceiver 1220 may include a wireless interface and operate on a variety of different frequencies or channels (e . g. , set of consecutive frequencies) . Therefore, the transceiver 1220 may include one or more components (e . g. , radios) to enable the data exchange with the various networksAttorney Docket No . 30164 / 102702Ref . No . P71143WO1and UEs . The transceiver 1220 includes circuitry configured to transmit and / or receive signals (e . g . , control signals , data signals ) . Such signals may be encoded with information implementing any one of the methods described herein . The processor 1205 may be operably coupled to the transceiver 1220 and configured to receive from and / or transmit signals to the transceiver 1220. The processor 1205 may be configured to encode and / or decode signals for implementing any one of the methods described herein .Examples

[0097] In a first example, a method, comprising generating, for transmission to an artificial intelligence / machine learning enabler (AIMLE ) server, a request for AIMLE client discovery, wherein the request comprises one or more discovery criteria that registered AIMLE clients are to satisfy, wherein registered AIMLE clients are registered with the AIMLE server and processing, based on signaling received from the AIMLE server, a response to the request comprising an indication of whether the request was successfully completed, partially completed or failed to complete, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request .

[0098] In a second example, the method of the first example, wherein the request comprises an AIMLE client discovery request, an AIMLE client selection request or an AIMLE client selection subscription request .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0099] In a third example, the method of the first example, wherein the response comprises an AIMLE client discovery response, an AIMLE client selection response or an AIMLE client selection subscription response .

[0100] In a fourth example, the method of the first example, wherein the request further comprises a number of AIMLE clients for AIMLE client discovery and a minimum number of AIMLE clients for AIMLE client discovery, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request and greater than or equal to the minimum number of AIMLE clients for AIMLE client discovery in the request .

[0101] In a fifth example, the method of the fourth example, wherein the request further comprises a second request comprising an AIMLE selection criteria or an AIMLE selection criteria .

[0102] In a sixth example, the method of the fifth example, wherein the second request further comprises a minimum number of AIMLE clients for AIMLE selection, wherein the indication of success indicates that a number of AIMLE clients that satisfied the selection criteria is greater than or equal to the minimum number of AIMLE clients for AIMLE client selection in the request .

[0103] In a seventh example, the method of the first example, wherein the request further comprises a number of AIMLE clients for AIMLE client selection and a minimum number of AIMLE clientsAttorney Docket No . 30164 / 102702Ref . No . P71143WO1for AIMLE client selection, wherein the indication of partially successful indicates that a number of AIMLE clients that satisfied the selection criteria is less than the number of AIMLE clients identified in the request and greater than or equal to the minimum number of AIMLE clients for AIMLE client selection in the request .

[0104] In an eighth example, the method of the first example 1, wherein the request further comprises an indication that partially completed is an acceptable response to the request .

[0105] In a ninth example, the method of the first example, wherein the response further comprises an indication of registered AIMLE clients that did not satisfy the one or more discovery criteria .

[0106] In a tenth example, the method of the ninth example, wherein the indication of registered AIMLE clients that did not satisfy the one or more discovery criteria is provided in the response when the response comprises an indication of partially completed or failed to complete .

[0107] In an eleventh example, the method of the ninth example, wherein the request further comprises an indication that the response is to include the indication of registered AIMLE clients that did not satisfy the one or more discovery criteria when one of i) the response indicates successfully completed, partially completed or failed to complete or ii) the response indicates partially completed or failed to complete .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0108] In a twelfth example, the method of the ninth example, wherein the response further comprises, for each registered AIMLE client that did not satisfy the one or more discovery criteria, an identification of the one or more discovery criteria not satisfied.

[0109] In a thirteenth example, the method of the twelfth example, wherein the response further comprises, for each registered AIMLE client that did not satisfy the one or more discovery criteria, an actual capability of each registered AIMLE client relative to the one or more discovery criteria not satisfied .

[0110] In a fourteenth example, the method of the first example, wherein the response further comprises an indication of each of the one or more discovery criteria that were not satisfied and a number of registered AIMLE clients that did not satisfy each of the one or more discovery criteria.

[0111] In a fifteenth example, the method of the fourteenth example, wherein the response further comprises, for each of the one or more discovery criteria that were not satisfied, an identification of the registered AIMLE clients that did not satisfy the one or more discovery criteria .

[0112] In a sixteenth example, the method of the fourteenth example, wherein the indication of each of the one or more discovery criteria that were not satisfied is provided in the response when the response comprises an indication of partially completed or failed to complete .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0113] In a seventeenth example, the method of the fourteenth example, wherein the request further comprises an indication that the response is to include each of the one or more discovery criteria that were not satisfied when one of i) the response indicates successfully completed, partially completed or failed to complete or ii) the response indicates partially completed or failed to complete .

[0114] In an eighteenth example, the method of the first example, further comprising generating a query or a subscription request associated with one or more requests for AIMLE client discovery and processing analytics data received in response to the query or a subscription request, wherein the analytics data is related to the one or more requests for AIMLE client discovery .

[0115] In a nineteenth example, the method of the eighteenth example, wherein the analytics data is provided by the AIMLE server or an Application Data Analytics Enablement Service .

[0116] In a twentieth example, the method of the eighteenth example, wherein the analytics data comprises one of previous registered AIMLE client selection, a prediction related to future AIMLE client selection, registered AIMLE clients selected for a task or service, characteristics of registered AIMLE clients that satisfied discovery criteria, or characteristics of the registered AIMLE clients .

[0117] In a twenty first example, a processor configured to perform any of the methods of the first through twentieth examples .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0118] In a twenty second example, a user equipment configured to perform any of the methods of the first through twentieth examples .

[0119] In a twenty third example, a method, comprising processing, based on signaling received from a requestor, a request for artificial intelligence / machine learning enabler (AIMLE) client discovery, wherein the request comprises one or more discovery criteria that registered AIMLE clients are to satisfy, determining registered AIMLE clients that satisfy the one or more discovery criteria and generating, for transmission to the requestor, a response to the request comprising an indication of whether the request was successfully completed, partially completed or failed to complete, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request .

[0120] In a twenty fourth example, the method of the twenty third example, wherein the requestor comprises one of a vertical application layer (VAL) server or an AIMLE server.

[0121] In a twenty fifth example, the method of the twenty third example, wherein the request comprises an AIMLE client discovery request, an AIMLE client selection request or an AIMLE client selection subscription request .

[0122] In a twenty sixth example, the method of the twenty third example, wherein the response comprises an AIMLE clientAttorney Docket No . 30164 / 102702Ref . No . P71143WO1discovery response, an AIMLE client selection response or an AIMLE client selection subscription response .

[0123] In a twenty seventh example, the method of the twenty third example, wherein the request further comprises a number of AIMLE clients for AIMLE client discovery and a minimum number of AIMLE clients for AIMLE client discovery, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request and greater than or equal to the minimum number of AIMLE clients for AIMLE client discovery in the request .

[0124] In a twenty eighth example, the method of the twenty seventh example, wherein the request further comprises a second request comprising an AIMLE selection criteria or an AIMLE selection criteria .

[0125] In a twenty ninth example, the method of the twenty eighth example, wherein the second request further comprises a minimum number of AIMLE clients for AIMLE selection, wherein the indication of success indicates that a number of AIMLE clients that satisfied the selection criteria is greater than or equal to the minimum number of AIMLE clients for AIMLE client selection in the request .

[0126] In a thirtieth example, the method of the twenty third example, wherein the request further comprises a number of AIMLE clients for AIMLE client selection and a minimum number of AIMLE clients for AIMLE client selection, wherein the indication of partially successful indicates that a number of AIMLE clientsAttorney Docket No . 30164 / 102702Ref . No . P71143WO1that satisfied the selection criteria is less than the number of AIMLE clients identified in the request and greater than or equal to the minimum number of AIMLE clients for AIMLE client selection in the request .

[0127] In a thirty first example, the method of the twenty third example, wherein the request further comprises an indication that partially completed is an acceptable response to the request .

[0128] In a thirty second example, the method of the twenty third example, wherein the response further comprises an indication of registered AIMLE clients that did not satisfy the one or more discovery criteria .

[0129] In a thirty third example, the method of the thirty second example, wherein the indication of registered AIMLE clients that did not satisfy the one or more discovery criteria is provided in the response when the response comprises an indication of partially completed or failed to complete .

[0130] In a thirty fourth example, the method of the thirty second example, wherein the request further comprises an indication that the response is to include the indication of registered AIMLE clients that did not satisfy the one or more discovery criteria when one of i) the response indicates successfully completed, partially completed or failed to complete or ii) the response indicates partially completed or failed to complete .Attorney Docket No . 30164 / 102702Ref . No . P71143WO1

[0131] In a thirty fifth example, the method of the thirty second example, wherein the response further comprises, for each registered AIMLE client that did not satisfy the one or more discovery criteria, an identification of the one or more discovery criteria not satisfied.

[0132] In a thirty sixth example, the method of the thirty fifth example, wherein the response further comprises, for each registered AIMLE client that did not satisfy the one or more discovery criteria, an actual capability of each registered AIMLE client relative to the one or more discovery criteria not satisfied .

[0133] In a thirty seventh example, the method of the twenty third example, wherein the response further comprises an indication of each of the one or more discovery criteria that were not satisfied and a number of registered AIMLE clients that did not satisfy each of the one or more discovery criteria .

[0134] In a thirty eighth example, the method of the thirty seventh example, wherein the indication of each of the one or more discovery criteria that were not satisfied is provided in the response when the response comprises an indication of partially completed or failed to complete .

[0135] In a thirty ninth example, the method of the thirty seventh example, wherein the request further comprises an indication that the response is to include each of the one or more discovery criteria that were not satisfied when one of i) the response indicates successfully completed, partiallyAttorney Docket No . 30164 / 102702Ref . No . P71143WO1completed or failed to complete or ii ) the response indicates partially completed or failed to complete .

[0136] In a fortieth example, the method of the thirty seventh example, wherein the response further comprises, for each of the one or more discovery criteria that were not satisfied, an identification of the registered AIMLE clients that did not satisfy the one or more discovery criteria .

[0137] In a forty first example, the method of the twenty third example, further comprising generating analytics data related to the one or more requests for AIMLE client discovery.

[0138] In a forty second example, the method of the forty first example, further comprising processing, based on signaling from the requestor, a query or a subscription request related to the analytics data and generating, for transmission to the requestor, a response comprising the analytics data .

[0139] In a forty third example, the method of the forty first example, further comprising processing, based on signaling from a registered AIMLE client, a query or a subscription request related to the analytics data and generating, for transmission to the registered AIMLE client, a response comprising the analytics data .

[0140] In a forty fourth example, the method of the forty first example, wherein the analytics data comprises one of previous registered AIMLE client selection, a prediction related to future AIMLE client selection, registered AIMLE clients selected for a task or service, characteristics of registeredAttorney Docket No . 30164 / 102702Ref . No . P71143WO1AIMLE clients that satisfied discovery criteria, or characteristics of the registered AIMLE clients .

[0141] In a forty fifth example, a processor configured to perform any of the methods of the twenty third through forty fourth examples .

[0142] In a twenty second example, a server device configured to perform any of the methods of the twenty third through forty fourth examples .

[0143] Those skilled in the art will understand that the above-described example embodiments may be implemented in any suitable software or hardware configuration or combination thereof . An example hardware platform for implementing the example embodiments may include, for example, an Intel x86 based platform with compatible operating system, a Windows OS, a Mac platform and MAC OS , a mobile device having an operating system such as iOS , Android, etc . The example embodiments of the above described method may be embodied as a program containing lines of code stored on a non-transitory computer readable storage medium that, when compiled, may be executed on a processor or microprocessor .

[0144] Although this application described various embodiments each having different features in various combinations , those skilled in the art will understand that any of the features of one embodiment may be combined with the features of the other embodiments in any manner not specifically disclaimed or which is not functionally or logicallyAttorney Docket No . 30164 / 102702Ref . No . P71143WO1inconsistent with the operation of the device or the stated functions of the disclosed embodiments .

[0145] As described above, one aspect of the present technology is the gathering and use of data available from specific and legitimate sources to improve the delivery to users of invitational content or any other content that may be of interest to them. The present disclosure contemplates that in some instances, this gathered data may include personal information data that uniquely identifies or can be used to identify a specific person. Such personal information data can include demographic data, location-based data, online identifiers, telephone numbers, email addresses, home addresses, data or records relating to a user' s health or level of fitness (e . g. , vital signs measurements, medication information, exercise information) , date of birth, or any other personal information .

[0146] The present disclosure recognizes that the use of such personal information data, in the present technology, can be used to the benefit of users . For example, the personal information data can be used to deliver targeted content that may be of greater interest to the user in accordance with their preferences . Accordingly, use of such personal information data enables users to have greater control of the delivered content . Further, other uses for personal information data that benefit the user are also contemplated by the present disclosure .

[0147] The present disclosure contemplates that those entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal informationAttorney Docket No . 30164 / 102702Ref . No . P71143WO1data will comply with well-established privacy policies and / or privacy practices . In particular, such entities would be expected to implement and consistently apply privacy practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy of users . Such information regarding the use of personal data should be prominent and easily accessible by users and should be updated as the collection and / or use of data changes . Personal information from users should be collected for legitimate uses only. Further, such collection / sharing should occur only after receiving the consent of the users or other legitimate basis specified in applicable law. Additionally, such entities should consider taking any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures . Further, such entities can subj ect themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices . In addition, policies and practices should be adapted for the particular types of personal information data being collected and / or accessed and adapted to applicable laws and standards, including urisdiction-specific considerations that may serve to impose a higher standard. For instance, in the US, collection of or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA) ; whereas health data in other countries may be subj ect to other regulations and policies and should be handled accordingly.

[0148] Despite the foregoing, the present disclosure also contemplates embodiments in which users selectively block theAttorney Docket No . 30164 / 102702Ref . No . P71143WO1use of, or access to, personal information data . That is, the present disclosure contemplates that hardware and / or software elements can be provided to prevent or block access to such personal information data . For example, such as in the case of advertisement delivery services, the present technology can be configured to allow users to select to "opt in" or "opt out" of participation in the collection of personal information data during registration for services or anytime thereafter . In another example, users can select not to provide mood-associated data for targeted content delivery services . In yet another example, users can select to limit the length of time mood-associated data is maintained or entirely block the development of a baseline mood profile . In addition to providing "opt in" and "opt out" options, the present disclosure contemplates providing notifications relating to the access or use of personal information. For instance, a user may be notified upon downloading an app that their personal information data will be accessed and then reminded again just before personal information data is accessed by the app.

[0149] Moreover, it is the intent of the present disclosure that personal information data should be managed and handled in a way to minimize risks of unintentional or unauthorized access or use . Risk can be minimized by limiting the collection of data and deleting data once it is no longer needed. In addition, and when applicable, including in certain health related applications, data de-identif ication can be used to protect a user' s privacy. De-identif ication may be facilitated, when appropriate, by removing identifiers, controlling the amount or specificity of data stored (e . g. , collecting location data at city level rather than at an address level) , controllingAttorney Docket No . 30164 / 102702Ref . No . P71143WO1how data is stored (e . g . , aggregating data across users ) , and / or other methods such as differential privacy .

[0150] The refore, although the present disclosure broadly covers use of personal information data to implement one or more various disclosed embodiments , the present disclosure also contemplates that the various embodiments can also be implemented without the need for accessing such personal information data . That is , the various embodiments of the present technology are not rendered inoperable due to the lack of all or a portion of such personal information data . For example, content can be selected and delivered to users based on aggregated non-personal information data or a bare minimum amount of personal information, such as the content being handled only on the user' s device or other non-personal information available to the content delivery services .

[0151] Some embodiments described herein can include use of learning and / or non-learning-based process (es ) . The use can include collecting, pre-processing, encoding, labeling, organizing, analyzing, recommending and / or generating data .Entities that collect, share, and / or otherwise utilize user data should provide transparency and / or obtain user consent when collecting such data . The present disclosure recognizes that the use of the data in the FL processes can be used to benefit users .

[0152] For example, the data can be used to train models that can be deployed to improve performance, accuracy, and / or functionality of applications and / or services . Accordingly, the use of the data enables the FL processes to adapt and / orAttorney Docket No . 30164 / 102702Ref . No . P71143WO1optimize operations to provide more personalized, efficient, and / or enhanced user experiences . Such adaptation and / or optimization can include tailoring content, recommendations, and / or interactions to individual users, as well as streamlining processes, and / or enabling more intuitive interfaces . Further beneficial uses of the data in the FL processes are also contemplated by the present disclosure .

[0153] The present disclosure contemplates that, in some embodiments, data used by FL processes includes publicly available data . To protect user privacy, data may be anonymized, aggregated, and / or otherwise processed to remove or to the degree possible limit any individual identification. As discussed herein, entities that collect, share, and / or otherwise utilize such data should obtain user consent prior to and / or provide transparency when collecting such data . Furthermore, the present disclosure contemplates that the entities responsible for the use of data, including, but not limited to data used in association with FL processes, should attempt to comply with well-established privacy policies and / or privacy practices .

[0154] It will be apparent to those skilled in the art that various modifications may be made in the present disclosure, without departing from the spirit or the scope of the disclosure . Thus, it is intended that the present disclosure cover modifications and variations of this disclosure provided they come within the scope of the appended claims and their eguivalent .

Claims

Attorney Docket No . 30164 / 102702Ref . No . P71143WO1What is claimed:1 . An apparatus comprising processing circuitry coupled to memory, the processing circuitry configured to :generate, for transmission to an artificial intelligence / machine learning enabler (AIMLE) server, a reguest for AIMLE client discovery, wherein the request comprises one or more discovery criteria that registered AIMLE clients are to satisfy, wherein registered AIMLE clients are registered with the AIMLE server; andprocess , based on signaling received from the AIMLE server, a response to the request comprising an indication of whether the request was successfully completed, partially completed or failed to complete, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clients identified in the request .2 . The apparatus of claim 1 , wherein the request comprises an AIMLE client discovery request, an AIMLE client selection request or an AIMLE client selection subscription request .3 . The apparatus of claim 1 , wherein the response comprises an AIMLE client discovery response, an AIMLE client selection response or an AIMLE client selection subscription response .4 . The apparatus of claim 1 , wherein the request further comprises a number of AIMLE clients for AIMLE client discovery and a minimum number of AIMLE clients for AIMLE client discovery, wherein the indication of partially completed indicates that a number of AIMLE clients that satisfied the discovery criteria is less than the number of AIMLE clientsAttorney Docket No . 30164 / 102702Ref . No . P71143WO1identified in the request and greater than or equal to the minimum number of AIMLE clients for AIMLE client discovery in the request .

5. The apparatus of claim 4 , wherein the request further comprises a second request comprising an AIMLE selection criteria or an AIMLE selection criteria .

6. The apparatus of claim 5, wherein the second request further comprises a minimum number of AIMLE clients for AIMLE selection, wherein the indication of success indicates that a number of AIMLE clients that satisfied the selection criteria is greater than or equal to the minimum number of AIMLE clients for AIMLE client selection in the request .7 . The apparatus of claim 1 , wherein the request further comprises a number of AIMLE clients for AIMLE client selection and a minimum number of AIMLE clients for AIMLE client selection, wherein the indication of partially successful indicates that a number of AIMLE clients that satisfied the selection criteria is less than the number of AIMLE clients identified in the request and greater than or equal to the minimum number of AIMLE clients for AIMLE client selection in the request .8 . The apparatus of claim 1 , wherein the request further comprises an indication that partially completed is an acceptable response to the request .Attorney Docket No . 30164 / 102702Ref . No . P71143WO19. The apparatus of claim 1, wherein the response further comprises an indication of registered AIMLE clients that did not satisfy the one or more discovery criteria .

10. The apparatus of claim 9, wherein the indication of registered AIMLE clients that did not satisfy the one or more discovery criteria is provided in the response when the response comprises an indication of partially completed or failed to complete .

11. The apparatus of claim 9, wherein the request further comprises an indication that the response is to include the indication of registered AIMLE clients that did not satisfy the one or more discovery criteria when one of i) the response indicates successfully completed, partially completed or failed to complete or ii) the response indicates partially completed or failed to complete .

12. The apparatus of claim 9, wherein the response further comprises, for each registered AIMLE client that did not satisfy the one or more discovery criteria, an identification of the one or more discovery criteria not satisfied.

13. The apparatus of claim 12, wherein the response further comprises, for each registered AIMLE client that did not satisfy the one or more discovery criteria, an actual capability of each registered AIMLE client relative to the one or more discovery criteria not satisfied.

14. The apparatus of claim 1, wherein the response further comprises an indication of each of the one or more discoveryAttorney Docket No . 30164 / 102702Ref . No . P71143WO1criteria that were not satisfied and a number of registered AIMLE clients that did not satisfy each of the one or more discovery criteria .

15. The apparatus of claim 14 , wherein the response further comprises , for each of the one or more discovery criteria that were not satisfied, an identification of the registered AIMLE clients that did not satisfy the one or more discovery criteria .

16. The apparatus of claim 14 , wherein the indication of each of the one or more discovery criteria that were not satisfied is provided in the response when the response comprises an indication of partially completed or failed to complete .17 . The apparatus of claim 14 , wherein the request further comprises an indication that the response is to include each of the one or more discovery criteria that were not satisfied when one of i ) the response indicates successfully completed, partially completed or failed to complete or ii ) the response indicates partially completed or failed to complete .18 . The apparatus of claim 1 , wherein the processing circuitry is further configured to :generate a query or a subscription request associated with one or more requests for AIMLE client discovery; andprocess analytics data received in response to the query or a subscription request, wherein the analytics data is related to the one or more requests for AIMLE client discovery.Attorney Docket No . 30164 / 102702Ref . No . P71143WO119. The apparatus of claim 18 , wherein the analytics data is provided by the AIMLE server or an Application Data Analytics Enablement Service .20 . The apparatus of claim 18 , wherein the analytics data comprises one of previous registered AIMLE client selection, a prediction related to future AIMLE client selection, registered AIMLE clients selected for a task or service, characteristics of registered AIMLE clients that satisfied discovery criteria, or characteristics of the registered AIMLE clients .

Images