Model selection method, network-side device and readable storage medium

Abstract

The present application discloses a model selection method, an apparatus and a network side device, which belong to the technical field of communication. The model selection method according to the embodiment of the present application includes: a first network element records model information of N candidate models, and the N candidate models are applied to a data analysis task corresponding to a first analysis identifier; the first network element receives a first model request message from a second network element; and if the model information of M candidate models among the N candidate models matches the model information corresponding to the first model request message, the first network element sends model information of P candidate models among the M candidate models to the second network element, where N, M and P are all positive integers, and N≧M≧P.

Description

[Technical Field]

[0001] The present application relates to the technical field of communications, and in particular to a model selection method, apparatus, and network-side device. [Background technology]

[0002] In a communication network, some network elements (e.g., Network Data Analytics Function (NWDAF)) can perform intelligent data analysis to generate data analysis results for some tasks, which can assist devices inside and outside the network in making policy decisions, thereby improving the degree of intelligence of device policy decisions.

[0003] The Model Training Logical Function (MTLF) in the NWDAF trains an artificial intelligence (AI) / machine language (ML) model based on training data to obtain a model applicable to a specific AI data analysis task. The Analytics Logical Function (AnLF) in the NWDAF performs model inference based on the AI / ML model and inference input data to obtain data analysis results (also called inference results, or analytics) for a specific AI data analysis task. The Policy Control Function (PCF) in the network can then intelligently implement corresponding policies based on the data analysis results. For example, it can formulate intelligent user stay policies based on the analysis results of user service behavior to improve the user's service experience. The Access and Mobility Management Function (AMF) can intelligently perform mobility management operations based on the data analysis results. For example, it can intelligently paging users based on the analysis results of user movement trajectories to improve paging reachability.

[0004] However, in the above process, whether devices inside and outside the network can make correct policy decisions based on data analysis results is a prerequisite. If the accuracy of the data analysis results is low, incorrect information may be provided to devices inside and outside the network for reference, which could lead to incorrect policy decisions or inappropriate operations being performed by devices inside and outside the network. Therefore, how to ensure the data analysis performance (e.g., accuracy) of AI / ML models has become an urgent issue. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments of the present application provide a model selection method, apparatus, and network-side device that can solve the problem of poor data analysis performance of conventional AI / ML models. [Means for solving the problem]

[0006] In a first aspect, a model selection method is provided, the method including: a first network element recording model information of N candidate models, wherein the N candidate models are applied to a data analysis task corresponding to a first analysis identifier; a first network element receiving a first model request message from a second network element; and, if model information of M candidate models among the N candidate models matches model information corresponding to the first model request message, the first network element sending model information of P candidate models among the M candidate models to the second network element, where N, M, and P are all positive integers, and N≧M≧P.

[0007] In a second aspect, there is provided a model selection device including: a recording module used to record model information of N candidate models, where the N candidate models are applied to a data analysis task corresponding to a first analysis identifier; a first receiving module used to receive a first model request message from a second network element; and a first transmitting module used to transmit model information of P candidate models among the M candidate models to the second network element if the model information of M candidate models among the N candidate models matches the model information corresponding to the first model request message, where N, M, and P are all positive integers, and N≧M≧P.

[0008] In a third aspect, there is provided a network side device including a processor and a memory, wherein the memory stores programs or commands executable by the processor, and when the programs or commands are executed by the processor, the steps of the method according to the first aspect are realized.

[0009] In a fourth aspect, there is provided a network side device including a processor and a communication interface, wherein the processor is used to record model information of N candidate models, and the N candidate models are applied to a data analysis task corresponding to a first analysis identifier, and the communication interface is used to receive a first model request message from a second network element, and if model information of M candidate models among the N candidate models matches model information corresponding to the first model request message, transmit model information of P candidate models among the M candidate models to the second network element, where N, M, and P are all positive integers, and N≧M≧P.

[0010] In a fifth aspect, there is provided a readable storage medium having stored thereon a program or commands which, when executed by a processor, cause the steps of the model selection method according to the first aspect to be implemented.

[0011] In a sixth aspect, there is provided a chip including a coupled processor and a communication interface, the processor executing programs or commands to implement the model selection method described in the first aspect.

[0012] In a seventh aspect, there is provided a computer program / program product stored on a storage medium and configured to, when executed by at least one processor, implement the steps of the model selection method described in the first aspect. [Effects of the Invention]

[0013] In an embodiment of the present application, a first network element records model information of N candidate models to be applied to a data analysis task corresponding to a first analysis identifier, receives a first model request message from a second network element, and if the model information of M candidate models among the N candidate models matches the model information corresponding to the first model request message, transmits model information of P candidate models among the M candidate models to the second network element, where N, M, and P are all positive integers, and N≧M≧P. With this solution, the first network element can record the model information of the candidate models to be applied to the data analysis task. After receiving the first model request message, the first network element can determine whether the model information of the recorded candidate models matches the model information corresponding to the first model request message, thereby determining an optimal model that meets the first model request message or a model that meets the performance requirements. In this way, the performance of the model in data analysis can be ensured. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram of a wireless communication system provided by an embodiment of the present application; [Figure 2] 1 is a schematic flow chart of a model selection method provided by an embodiment of the present application; [Figure 3] 1 is a first schematic diagram of the application flow of the model selection method provided by the embodiment of the present application. [Figure 4] FIG. 2 is a second schematic diagram of the application flow of the model selection method provided by the embodiment of the present application. [Figure 5] FIG. 1 is a schematic diagram of the structure of a model selection device provided by an embodiment of the present application; [Figure 6] FIG. 2 is a schematic diagram of the structure of a network-side device provided by an embodiment of the present application; [Figure 7] FIG. 2 is a hardware schematic diagram of a network-side device provided by an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0015] In the following, the technical solutions in the embodiments of the present application will be clearly described with reference to the drawings in the embodiments of the present application, and it should be understood that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments, and all other embodiments obtained by those skilled in the art based on the embodiments of the present application are all within the scope of protection of the present application.

[0016] The terms "first," "second," etc., used in the specification and claims of this application are not intended to describe a particular order or sequence, but rather to distinguish between similar objects. It should be understood that terms used in this manner may be interchangeable in some cases, allowing the embodiments of this application to be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first model may be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the symbol " / " generally indicates that the related objects before and after are in an "or" relationship.

[0017] It should be noted that the techniques described in the embodiments of the present application are not limited to use in Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but may also be used in other wireless communication systems such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-Carrier Frequency-Division Multiple Access (SC-FDMA), and other systems. The technical terms "system" and "network" in the embodiments of the present application are often used interchangeably, and the techniques described may be used in not only the above systems and wireless technologies, but also other systems and wireless technologies. In the following description, for illustrative purposes, a 5G system will be described and 5G terminology will be used in much of the following description, however, these techniques may be applied outside of 5G system applications, such as in 6th Generation (6G) communication systems.

[0018] 1 shows a block diagram of a wireless communication system to which an embodiment of the present application can be applied. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal is also called user equipment (UE), and the terminal 11 may be a terminal-side device such as a mobile phone, a tablet personal computer (Tablet Personal Computer), a laptop computer (Laptop Computer, also called Notebook Computer), a personal digital assistant (PDA), a palmtop computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, a vehicle-mounted equipment (VUE), a pedestrian-mounted equipment (PUE), a smart home (home appliances with wireless communication capabilities such as a refrigerator, a television, a washing machine, or furniture), a game console, a personal computer, an automated teller machine, or a kiosk. The wearable device includes a smart watch, a smart bracelet, a smart earphone, a smart glass, a smart accessory (such as a smart bangle, a smart chain bracelet, a smart ring, a smart necklace, a smart anklet, etc.), a smart wristband, a smart wear, etc. It should be noted that the embodiments of the present application are not limited to a specific type of the terminal 11. The network side device 12 may include an access network device or a core network device, and the access network device 12 is also called a radio access network device, a radio access network (RAN), a radio access network function, or a radio access network unit.The access network device 12 may include a base station, a WLAN access point, a WiFi node, etc., and the base station may also be called a Node B, an evolved Node B (eNB), an access point, a base transceiver station (BTS), a radio base station, a radio transceiver, a basic service set (BSS), an extended service set (ESS), a home Node B, a home evolved Node B, a transmitting and receiving point (TRP), or any other suitable technical term in the field. The base station is not limited to a specific technical term as long as the same technical effect is achieved. It should be noted that in the embodiments of the present application, only base stations in an NR system are described as examples, and the specific type of base station is not limited. The core network device may be a core network node, a core network function, or the like. , Ne network dataNetwork Data Analytics Function (NWDAF), analytics data storage function (ADRF), Mobility Management Entity (MME), Access and Mobility Management Function (AMF), Session Management Function (SMF), User Plane Function (UPF), Policy Control Function (PCF), Policy and Charging Rules Function (PCRF), Edge Application Server Discovery Function (EASDF), Unified Data Management (UDM), Unified Data Repository (UDR), Home Subscriber Server (HSS), Centralized network configuration (CNC), Network Repository Function (NRF), Network Exposure Function (NEF), Local NEF The core network device may include, but is not limited to, at least one of a Binding Support Function (NEF or L-NEF), a Binding Support Function (BSF), an Application Function (AF), etc. It should be noted that in the embodiments of the present application, only core network devices in a 5G system are described as examples, and the specific type of the core network device is not limited.

[0019] In the embodiments of the present application, the MTLF and the AnLF may be independent network elements or logical function modules within the NWDAF, and may be located in the same NWDAF, or may be located separately in different NWDAFs, or may be located outside the NWDAF.

[0020] Hereinafter, the model selection method provided by the embodiments of the present application will be described in detail with reference to the drawings through several embodiments and their application scenarios.

[0021] As shown in FIG. 2, an embodiment of the present application provides a model selection method, which includes the following steps 201 to 203.

[0022] Step 201: A first network element records model information of N candidate models.

[0023] The N candidate models may be applied to a data analysis task corresponding to a first analytic ID. The data analysis task may also be referred to as an inference task.

[0024] It should be noted that the analytic ID may be used to identify a data analysis task and may itself be a type, i.e., the analytic ID may be used to indicate the type of the data analysis task. It can be understood that the data analysis task corresponding to the above first analytic ID may be the data analysis task identified by the first analytic ID.

[0025] For example, if the analytic ID is UE mobility, the data analysis task identified by the analytic ID is predicting the trajectory of the UE. If the analytic ID is UE communication, the data analysis task identified by the analytic ID is predicting the communication performance and behavior of the UE.

[0026] Optionally, in the embodiment of the present application, the first network element may include a MTLF or an analytics data storage function (ADRF).

[0027] In some embodiments, the model information for the N candidate models recorded by the MTLF may be model information determined by the MTLF itself, and the model information for the N candidate models recorded by the ADRF may be model information obtained from the MTLF.

[0028] Optionally, in the embodiments of the present application, the above N candidate models may be multiple different models, or may be different models obtained by training or retraining one model multiple times. Specific details can be determined according to actual usage needs, and are not limited in the embodiments of the present application.

[0029] Optionally, in an embodiment of the present application, the model information of the candidate model may include at least one of the following a. to f.:

[0030] a. A model identifier (model ID) for the candidate model, used to uniquely identify the candidate model.

[0031] b. Model file information of the candidate model, in which any information regarding the candidate model, such as the network structure, weight parameters, input / output data, etc. of the candidate model, can be stored.

[0032] c. Download address information of the candidate model, which is used to indicate the storage address of the model file of the candidate model, i.e., the download address of the model file of the candidate model. For example, if the candidate model is stored in the ADRF, the download address information may be the address information of the ADRF.

[0033] d. An analysis identifier for the candidate model, for example, a first analysis identifier, that is used to identify the data analysis task to which the candidate model is to be applied.

[0034] e. Usage scope information of the candidate model, which may indicate at least one of a usage area range, a usage time range, and a usage target range.

[0035] The target of use may include one UE, multiple UEs, or any UE. The specific details can be determined according to actual usage needs and are not limited in the embodiments of the present application. The UE may be identified by a UE ID, UE address information, a UE group ID, etc.

[0036] f. Model performance information for candidate models.

[0037] In embodiments of the present application, model performance information may be used to characterize the degree of agreement or matching between the model output (i.e., predicted or statistical value) and the true value, where matching refers to the deviation between the predicted value (or statistical value) and the true value being within a predetermined range.

[0038] In one embodiment, the degree of matching can be obtained by the results of one or more comparisons, for example, by comparing the predicted value with the true value multiple times, or in another embodiment, the degree of matching can be a value representing the ratio of the number of times the predicted value and the true value coincide or match to the total number of predictions.

[0039] Optionally, in the embodiments of the present application, the performance indicated by the model performance information includes at least one of model accuracy and model mean absolute error (MAE). Of course, in actual implementation, the model performance information may further include other information that can reflect the performance of the model, which is specifically determined according to actual usage needs.

[0040] It should be noted that the performance indicated by the model performance information may be expressed in any possible form, such as a number, a level, a percentage, etc. Nobility When the performance indicated by the information includes model accuracy, the model accuracy may be expressed as an exact numerical value.

[0041] In the embodiments of the present application, the model information of the candidate model can reflect the performance performance and corresponding usage scope of the candidate model in the actual usage process. Therefore, when another network element (e.g., the second network element in the embodiments of the present application) requests a model, the first network element can use this information to determine an optimal model that matches the model request message (e.g., the first model request message in the embodiments of the present application) sent by the other network element. For example, if the model information includes usage scope information, the first network element can provide different models for different usage scopes, thereby improving the model's performance in data analysis in the network. If the model information includes model performance information, the first network element can select a model that meets the conditions according to the performance requirements of the requested model, thereby improving the model's performance in data analysis in the network.

[0042] It should be noted that the above actual use process may include the process of actual use and data inference (or data analysis) on the network.

[0043] Step 202: A first network element receives a first model request message from a second network element.

[0044] Optionally, in the present embodiment, the second network element may include an AnLF or an MTLF.

[0045] In the present embodiment, the first network element is a MTLF and the second network element is an AnLF, or the first network element is an ADRF and the second network element is an AnLF or a MTLF, or the first network element is MTLF 1 and the second network element is MTLF 2. MTLF 1 and MTLF 2 may be different MTLFs.

[0046] It should be noted that in the embodiments of the present application, the first network element is illustratively described as an MTLF and the second network element is illustratively described as an AnLF.

[0047] Optionally, in the embodiment of the present application, the first model request message may include at least one of the following items A to E:

[0048] A. A second analysis identifier that can be used to identify the data analysis task to which the requested model is applied, for example, the second analysis identifier is UE mobility, in which case the data analysis task to which the requested model is applied is predicting a user's movement trajectory.

[0049] B. Model filtering information that can be used to indicate conditions that the requested model must meet, such as area of ​​interest (AOI), single network slice selection assistance information (S-NSSAI), data network name (DNN), etc.

[0050] C. Model target information that can be used to indicate the training target of the requested model, which can be one UE, multiple UEs, or any UE.

[0051] D. Model reporting information may include at least one of the following: a required model reporting method, an application time, or a reporting time. The model reporting method may include periodic reporting or conditional reporting. Of course, in actual implementation, the model reporting information may further include any other possible information, and the embodiments of the present application are not specifically limited.

[0052] E. Model performance requirement information that can be used to indicate the performance that the required model must meet, e.g., minimum accuracy, maximum MAE, etc. that the model must reach.

[0053] Step 203: If the model information of M candidate models among the N candidate models matches the model information corresponding to the first model request message, the first network element sends the model information of P candidate models among the M candidate models to the second network element.

[0054] Here, N, M, and P are all positive integers, and N≧M≧P.

[0055] It can be understood that the above P candidate models may be M candidate models or may be a part of the M candidate models.

[0056] In an embodiment of the present application, after the first network element receives the first model request message, the first network element can determine, from the N candidate models, candidate models whose model information matches the model information corresponding to the first model request message, for example, the M candidate models. After the M candidate models are determined, the first network element can send model information of some or all of the M candidate models (i.e., the P candidate models) to the second network element. Thus, the second network element can perform a data analysis task corresponding to the first analysis identifier based on the P candidate models to obtain a corresponding data analysis result.

[0057] Optionally, in the embodiment of the present application, the model information of the above M candidate models matching the model information corresponding to the first model request message may include at least one of the following:

[0058] (1) The first analysis identifier and the second analysis identifier are the same. If the first analysis identifier and the second analysis identifier are the same, it can be understood that the data analysis task corresponding to the first analysis identifier and the data analysis task corresponding to the second analysis identifier are the same.

[0059] (2) The usage region ranges included in the model usage range information of the M candidate models match the model filtering information included in the first model request message. For example, the S-NSSAI, region of interest, and DNN included in the model usage range information of the M candidate models are the same as the S-NSSAI, region of interest, or DNN in the model filtering information, or the S-NSSAI, region of interest, or DNN included in the model usage range information of the M candidate models include the S-NSSAI, region of interest, or DNN in the model filtering information. The model usage range information of the M candidate models may include multiple DNNs, i.e., the model usage range information may correspond to a DNN list, in which case the DNN in the model filtering information may be a DNN in the DNN list.

[0060] (3) The usage target range included in the model usage range information of the M candidate models matches the model target information included in the first model request message. For example, the usage target range included in the model usage range information of the M candidate models may include multiple targets (e.g., multiple UEs), that is, the usage target range may correspond to one target list, and in this case, the target indicated by the model target information may be a target in the target list.

[0061] (4) The usage time range included in the model usage range information of the M candidate models matches the model report information included in the first model request message. For example, the usage time range included in the model usage range information of the M candidate models is the same as the application time of the model corresponding to the model report information, or the application time of the model corresponding to the model report information is included in the usage time range.

[0062] (5) The model performance information of the M candidate models satisfies the performance indicated by the model performance requirement information included in the first model request message. For example, the performance of the models corresponding to the model performance information of the M candidate models is higher than the performance indicated by the model performance requirement information.

[0063] Optionally, in an embodiment of the present application, if the first network element is an MTLF and the second network element is an AnLF, the AnLF and the MTLF request and obtain model information through Nnwdaf_MLModelProvision_Subscribe and Nnwdaf_MLModelProvision_Notify, or the AnLF and the MTLF request and obtain model information through Nnwdaf_MLModelInfo_Request message and Nnwdaf_MLModelInfo_Response message.

[0064] Optionally, if the first network element is an ADRF and the second network element is an AnLF, the AnLF and the ADRF request and obtain the model information by Nadrf_DataManagement_RetrievalRequest message and Nadrf_DataManagement_RetrievalResponse message, or the AnLF and the ADRF request and obtain the model information by Nadrf_DataManagement_RetrievalSubscribe and Nadrf_DataManagement_RetrievalNotify.

[0065] In an embodiment of the present application, the first network element can record model information of a candidate model to be applied to a data analysis task, so that after the first network element receives the first model request message, it can determine whether the recorded model information of the candidate model matches the model information corresponding to the first model request message sent by the second network element, thereby determining an optimal model that meets the first model request message or a model that meets the performance requirements, thereby ensuring the performance of the model in data analysis.

[0066] Optionally, before the above step 201, the model selection method provided by the embodiment of the present application may further include the following step 204.

[0067] Step 204: The first network element determines N candidate models that satisfy the predetermined condition from the K models according to the model performance information of the K models.

[0068] The K models may be applied to the data analysis task corresponding to the first analysis identifier, where K is a positive integer. The performance indicated by the model performance information is the highest among the K models. the performance indicated by the model performance information is higher than the first predetermined performance.

[0069] It should be noted that the performance indicated by the model performance information may be expressed in any possible form, such as a number, a level, a percentage, etc. Nobility When the performance indicated by the information includes a model accuracy, the model accuracy may be expressed as an exact numerical value. It is understood that the first predetermined performance may also be a numerical value, a level, or a percentage.

[0070] Optionally, in embodiments of the present application, the K models may include multiple candidate models to be applied to the data analysis task corresponding to the first analysis identifier, or the K models may include multiple models obtained by training multiple times using the data analysis task corresponding to the first analysis identifier.

[0071] Optionally, in an embodiment of the present application, if the above K models include multiple models obtained by performing training multiple times using a data analysis task corresponding to the first analysis identifier, the model selection method provided by the embodiment of the present application may further include the following steps 205 and 206.

[0072] Step 205: If the performance of the first model sent by the first network element to the fourth network element is lower than the second default performance, the first network element obtains a second model by retraining the first model or reselecting a model according to the usage range information of the first model.

[0073] The above second model is a model among K models. The above fourth network element may be an AnLF, for example, the fourth network element may be AnLF 1.

[0074] Optionally, in the present embodiment, the fourth network element may be the same as the second network element. That is, the model request message for obtaining the first model and the first model request message may be sent by the same network element, for example, by ANLF 1. For example, after ANLF 1 loses the optimal model due to an abnormality, it can re-obtain the optimal model by sending a model request message to the MTLF or ADRF.

[0075] Optionally, in the embodiment of the present application, the above first model usage range information may be obtained from a fourth network element.

[0076] In the embodiment of the present application, the above-mentioned "the first network element retrains the first model according to the usage range information of the first model" refers to retraining the first model for the usage range of the first model. Specifically, taking the first network element as an example, MTLF, and the fourth network element as AnLF 1, the MTLF collects label data corresponding to the usage range of the first model as a training dataset, and retrains the model based on the training dataset. The training data in the training dataset may include label data obtained from a data provider and label data obtained by the MTLF from AnLF 1. It can be understood that the training data used by the MTLF includes label data.

[0077] Based on the retraining of the first model, the first network element can update and repeat the first model according to the performance of the first model in the current network, record the performance of the model each time, and search for and record the optimal model or the most appropriate model corresponding to the usage range information of the first model. Then, when another network element (e.g., the above-mentioned fourth network element or fifth network element) requests to obtain a model, the first network element can provide the optimal model or the most appropriate model according to the usage range of the requested model.

[0078] The above-mentioned model reselection refers to the following: after a first network element transmits model information of a first model to a fourth network element, if there are multiple selectable candidate models (including the first model already transmitted) for a model request message (e.g., a second model request message in an embodiment of the present application) of the fourth network element, and if the performance of the first model is lower than the second default performance, the first network element can reselect another candidate model (e.g., the second model) from the multiple candidate models and transmit it to the fourth network element.

[0079] Based on the above model reselection, the first network element can obtain and record the performances of the above multiple models in the actual use process and their corresponding use ranges. Then, when another network element (e.g., the fourth network element or the fifth network element) requests to obtain a model, the first network element can provide the optimal model or the most appropriate model according to the use range of the requested model.

[0080] Step 206: The first network element sends the model information of the second model to the fourth network element or the fifth network element.

[0081] Optionally, in the embodiment of the present application, the above-mentioned fifth network element may be an AnLF, and it can be understood that the fifth network element and the above-mentioned fourth network element are different AnLFs, for example, the fourth network element is AnLF 1, and the fifth network element is AnLF 2.

[0082] Optionally, in the embodiment of the present application, the above-mentioned sending of model information of the second model by the first network element to the fourth network element may include two possible embodiments. In one embodiment, after the first network element obtains the second model, the first network element actively pushes the model information of the second model to the fourth network element. In another embodiment, after the first network element obtains the second model, the first network element can send the model information of the second model to the fourth network element upon receiving a model request message (e.g., a third model request message) retransmitted by the fourth network element. Optionally, the usage range of the requested model corresponding to the third model request message is the same as the usage range of the second model.

[0083] In an embodiment of the present application, if the second model is a model obtained by retraining the first model and the second model is an updated replacement of the first model, the model identifier of the second model is the same as the model identifier of the first model (e.g., model ID1). If the second model is a model obtained by performing model reselection or a new model obtained by retraining the first model (without replacing the first model), the model identifier of the second model corresponds to a new model identifier (e.g., model ID2), which may be different from the model identifier of the first model.

[0084] Optionally, the model selection method provided by the embodiment of the present application may further include the following step 207:

[0085] Step 207: The first network element sends the usage range information of the second model to the fourth network element or the fifth network element.

[0086] The usage range indicated by the usage range information of the second model is the same as the usage range indicated by the usage range information of the first model, or the usage range indicated by the usage range information of the second model is determined by the usage range indicated by the usage range information of the first model.

[0087] It should be noted that the above-mentioned second model usage range information being determined by the first model usage range information may be understood as the usage range indicated by the second model usage range information including the usage range indicated by the first model usage range information, or the usage range indicated by the second model usage range information intersecting with the usage range indicated by the first model usage range information, which may include at least one of a usage time range, a usage target range, and a usage area range.

[0088] In the embodiment of the present application, when the first network element sends the second model to the fourth network element, the first network element can send the usage range information of the second model to the fourth network element. When the first network element sends the second model to the fifth network element, the first network element can send the usage range information of the second model to the fifth network element.

[0089] In the embodiment of the present application, after the first network element obtains the model performance information of the first model, the first model can determine whether the performance of the first model is lower than the second predetermined performance according to the model performance information of the first model, and can thus determine whether to retrain the first model. Based on this, the model selection method provided by the embodiment of the present application may further include any one of the following steps 208, 209, or 210.

[0090] Step 208: The first network element calculates model performance information of the first model.

[0091] The model performance information of the first model is used to indicate the performance of the first model.

[0092] For example, the first network element is an MTLF. The MTLF obtains a corresponding validation dataset (including label data, ground truth, etc.) according to the usage range information of the first model, and calculates the output result performance of the first model, for example, the output result accuracy of the first model, based on the validation dataset and the data analysis output result of the first model itself, thereby obtaining model performance information of the first model.

[0093] Optionally, in the embodiment of the present application, based on the above step 208, the first network element can record usage range information of the first model, which indicates the range of data task analysis or model performance measurement by the first model, including the usage area range of the first model, the usage time range of the first model, the usage target range of the first model, etc.

[0094] Step 209: The first network element receives model performance information of the first model from the fourth network element.

[0095] For example, taking the fourth network element as an AnLF, the AnLF obtains a corresponding validation dataset (including label data, ground truth, etc.) according to the usage range information of the first model, and calculates the output result performance of the first model, for example, the output result accuracy of the first model, based on the validation dataset and the data analysis output result set of the first model, to obtain model performance information of the first model, and sends the model performance information of the first model to the first network element, for example, the MTLF.

[0096] Optionally, in the embodiment of the present application, based on the above step 209, the fourth network element can send the first model usage range information to the first network element, so that the first network element can receive the first model usage range information from the fourth network element. For details of the first model usage range information, please refer to the relevant description in the above embodiment, and to avoid duplication, the description will be omitted here.

[0097] Optionally, in embodiments of the present application, the fourth network element may include multiple network elements, such that the multiple network elements can jointly use the same first model and use the first model in the same or intersecting usage scopes, and the multiple network elements can each report model performance information of the first model, such as the accuracy of the output result of the first model (i.e., model accuracy).

[0098] Step 210: A first network element receives model performance information for a first model from a data analysis consumer.

[0099] Optionally, in the embodiments of the present application, the above-mentioned data analysis consumer may be any other possible data analysis consumer, such as PCF, AMF, etc. The specifics can be determined according to actual usage needs, and are not limited in the embodiments of the present application.

[0100] In an embodiment of the present application, the data analysis consumer may obtain a corresponding validation dataset (including label data, ground truth, etc.) according to the data analysis result of the first model obtained from a fourth network element (e.g., AnLF) and the extent to which the data analysis result of the model is used by the data analysis consumer, and calculate the output result performance of the first model, for example, the output result accuracy of the first model, based on the validation dataset and the data analysis result of the first model, thereby obtaining model performance information of the first model, and sending the model performance information of the first model to a first network element (e.g., MTLF). The data analysis consumer may send the model performance information of the first model directly to the first network element, or may send it to the first network element via the fourth network element.

[0101] Optionally, in the embodiment of the present application, there may be a plurality of data analysis consumers, i.e., a plurality of data analysis consumers that use the data analysis results, and these multiple data analysis consumers may jointly calculate and report model performance information of the first model, such as the output result accuracy of the first model (i.e., model accuracy).

[0102] Optionally, in the embodiment of the present application, based on the above step 210, the data analysis consumer can send the usage range information of the first model to the first network element, so that the first network element can receive the usage range information of the first model from the data analysis consumer. The data analysis consumer may send the usage range information of the first model directly to the first network element, or may send it to the first network element via a fourth network element. For details of the usage range information of the first model, please refer to the related description in the above embodiment, and to avoid duplication, the description will be omitted here.

[0103] It should be noted that in an embodiment of the present application, when the above K models include multiple candidate models to be applied to the data analysis task corresponding to the above first analysis identifier, the first network element can obtain model performance information of the multiple candidate models through the above steps 208, 209 or 210.

[0104] Optionally, in an embodiment of the present application, before the above step 205, the model selection method provided by an embodiment of the present application may further include the following steps 211 and 212.

[0105] Step 211: The first network element receives a second model request message from the fourth network element.

[0106] Step 212: The first network element sends model information of the first model to the fourth network element through a second model request message.

[0107] The above-mentioned first model may include one model or multiple models. For details of the second model request message, please refer to the detailed description of the first model request message in the above embodiment, and the description will be omitted here.

[0108] Optionally, in an embodiment of the present application, if the above first model includes multiple models, before the above step 212, the model selection method provided by the embodiment of the present application may further include the following step 213 or step 214.

[0109] Step 213: The first network element determines whether the fourth network element supports the acquisition of multiple models.

[0110] Step 214: The first network element determines the model usage capability of the fourth network element.

[0111] In an embodiment of the present application, if there are multiple models that match the model information corresponding to the second model request message, the first network element can determine whether the fourth network element supports the acquisition of multiple models or whether the fourth network element supports the use of multiple models by determining the model use capability of the fourth network element; if the fourth network element can support the use of multiple models, the first network element will send all model information of the multiple models to the fourth network element; otherwise, the first network element will send only one of the multiple models to the fourth network element, that is, the first network element will not send model information of the multiple models to the fourth network element simultaneously.

[0112] Optionally, in the embodiment of the present application, if the above first network element is an ADRF, the model selection method provided by the embodiment of the present application may further include the following step 215:

[0113] Step 215: The first network element obtains model information of the N candidate models from the third network element.

[0114] In an embodiment of the present application, the third network element may be a Multi-Tunnel-Layer-Federation (MTLF). Thus, before the ADRF records the model information of the N candidate models, the ADRF can first obtain the model information of the N candidate models from the MTLF.

[0115] It can be understood that if the first network element is a MTLF, the MTLF may obtain the model information of the above N candidate models by itself, and then record the model information of the N candidate models.

[0116] Optionally, in the embodiment of the present application, before the above step 201 (the first network element records the model information of the N candidate models), the model selection method provided by the embodiment of the present application may further include the following step 216:

[0117] Step 216: The first network element obtains the usage range information and / or model performance information of each candidate model among the N candidate models.

[0118] In an embodiment of the present application, before the first network element records the model information of the above-mentioned N candidate models, the first network element can first obtain the usage range information and / or model performance information of each of the N candidate models, and then record the model information of the N candidate models in association with each other.

[0119] In this embodiment, it can be understood that the first network element does not perform screening on the N candidate models, but only records information corresponding to the N candidate models (including usage range information, model performance information, etc.), and then, after receiving a model request message sent by another network element, can select an appropriate model according to the information and send it to the network element.

[0120] Optionally, in the embodiment of the present application, the above step 216 may be specifically realized by the following step 216a, step 216b, or step 216c.

[0121] Step 216a: The first network element determines usage range information for each candidate model among the N candidate models and / or calculates model performance information for each candidate model among the N candidate models.

[0122] Step 216b: The first network element receives usage range information and / or model performance information of each candidate model among the N candidate models from the seventh network element.

[0123] The seventh network element may include one or more network elements, and the seventh network element may be an AnLF.

[0124] Optionally, the seventh network element may be the same as or different from the second network element in the above embodiment, and can be determined according to actual usage needs, and is not limited in the embodiment of the present application.

[0125] Step 216c: The first network element receives from the data analysis consumer usage range information and / or model performance information for each candidate model among the N candidate models.

[0126] It should be noted that for details of the first network element obtaining the usage range information and / or model performance information of each candidate model among the N candidate models, reference can be made to the detailed description of the first network element obtaining the usage range information and / or model performance information of the first model in the above embodiment, and the description will be omitted here to avoid duplication.

[0127] Optionally, in the embodiment of the present application, if the above-mentioned first network element is an MTLF, after the above step 201 (the first network element records the model information of the N candidate models), the model selection method provided by the embodiment of the present application may further include the following step 217:

[0128] Step 217: The first network element stores the model information of the N candidate models in the sixth network element.

[0129] The sixth network element may include an ADRF or a unified data repository (UDR).

[0130] Optionally, in the embodiment of the present application, based on the above step 217, the above step 203 can be specifically realized by the following step 203a.

[0131] Step 203a: The first network element sends first information to the second network element, where the first information is used to indicate that model information of the P candidate models is stored in the sixth network element.

[0132] The first information may include at least one of the following: identifier information, a fully qualified domain name (FQDN), and address information of the sixth network element. Of course, the first information may also include other information corresponding to the sixth network element, which can be determined according to actual usage needs and is not limited in the embodiments of the present application.

[0133] It can be understood that at least one of the identifier information, FQDN, or address information of the sixth network element can be download address information of the P candidate models, thereby allowing the second network element to download model information of the P candidate models through at least one of the identifier information, FQDN, or address information of the sixth network element.

[0134] In an embodiment of the present application, after the MTLF stores the model information of the N candidate models in the sixth network element, when the second network element sends the first model request message to the first network element, if the model information of M candidate models among the N candidate models matches the model information corresponding to the first model request message, the first network element can send the first information to the second network element, so that after the second network element receives the first information, the second network element can download the model information of the P candidate models from the sixth network element.

[0135] Optionally, in an embodiment of the present application, after the first network element stores the model information of the above N candidate models in the sixth network element, the eighth network element can obtain the model information of the models from the sixth network element by directly sending a model request message to the sixth network element, where the eighth network element can be configured to obtain the model information of the models from the sixth network element, and the eighth network element can be an AnLF, for example, an AnLF 3.

[0136] Hereinafter, a model selection method provided by an embodiment of the present application will be described by way of example with reference to the drawings.

[0137] Example 1 As shown in FIG. 3, in step 1a, AnLF 1 can send a model request message (e.g., request 1) to MTLF, and the model request message can include an analytic ID, model filter information 1, etc.

[0138] In step 1b, the MTLF can provide model information of an ML model (e.g., a model to be applied to AI task 1) to the AnLF 1, which can be an initial model with a model identifier of model ID1.

[0139] In step 2, the MTLF / AnLF 1 / Data Analytics Consumer performs analytics performance evaluation on the ML model from step 1b. It records the model's model identifier, the data analysis task to which the model is applied, and the scope of model use. After AnLF 1 / Data Analytics Consumer performs performance evaluation on the model, it reports the model performance information and the scope of use information of the model to the MTLF.

[0140] In step 3, if the performance of the ML model is lower than the default performance, the MTLF reselects or retrains a new ML model according to the model usage scope, which may be different from the initial model or may be an updated version of the initial model.

[0141] In step 4a (optionally), AnLF 1 sends a model request message (e.g., request 2) to the MTLF.

[0142] In step 4b, the MTLF sends model information of the new ML model to the AnLF 1. The model information may include a model identifier and a model usage scope. If the new ML model is a model different from the initial model, the model identifier may be model ID2. If the new ML model is an updated version of the initial model, the model identifier may be model ID1.

[0143] In step 5, the MTLF / AnLF 1 / Data Analytics Consumer performs performance measurement / evaluation on the new ML model from step 4b. For details, see the explanation in step 2 above.

[0144] In step 6, the MTLF determines a target model based on the model performance information corresponding to the multiple models obtained in steps 2 to 5. The target model may be the one with the highest / best performance among the multiple models, or the target model may be the one with performance higher / better than a predetermined performance among the multiple models. There may be one target model or multiple target models.

[0145] In step 7, AnLF 2 sends a model request message (e.g., request 3) to the MTLF to request a model corresponding to the analytic ID, which may be the same as the analytic ID in step 1a. The model request message includes model filter information (e.g., model filter information 2), etc.

[0146] In step 8, the MTLF determines whether the model information corresponding to the model request message in step 7 matches the model information of the target model, and if so, executes step 9 below.

[0147] In step 9, the MTLF sends the model information of the target model to the AnLF 2.

[0148] Example 2 Still referring to Figure 3, this embodiment differs from embodiment 1 in that in step 1b, if there are multiple candidate models in the MTLF that meet the requirements for the model request message sent by AnLF 1, the MTLF can simultaneously publish these multiple candidate models to AnLF 1. Optionally, before publishing the models, the MTLF must first determine whether the AnLF supports acquiring or has the ability to use multiple models; if not, the MTLF will not simultaneously send multiple candidate models to the AnLF.

[0149] In step 2, performance measurements can be performed on the multiple candidate models in a manner similar to step 2 in the first embodiment.

[0150] Example 3 As shown in FIG. 4, this embodiment differs from Embodiments 1 and 2 in that in step 6b, the MTLF stores the model information of the acquired target model in a database or an integrated data platform (e.g., ADRF, UDR, etc.).

[0151] In step 7, AnLF 2 sends a model acquisition request message to a database or an integrated data platform.

[0152] In step 8, the database or integrated data platform determines whether the model information corresponding to the model request message in step 7 matches the model information of the target model, and if so, performs step 9 below.

[0153] The matching method in step 8 above may be similar to the matching method in step 8 in the first embodiment.

[0154] In step 9, the database or integrated data platform sends the model information of the target model to AnLF 2.

[0155] It should be noted that for steps not described in Example 2 and Example 3, reference can be made to the relevant descriptions in Example 1, and the descriptions will be omitted here to avoid duplication.

[0156] Example 4 Steps 1 to 6 are the same as steps 1 to 6 in the first embodiment above.

[0157] In step 6a, the MTLF stores the model information of the target model (i.e., the candidate model in the embodiment of the present application) in the ADRF, and records the corresponding ADRF information (e.g., the identifier information of the ADRF, the address information of the ADRF, etc.).

[0158] Steps 7 and 8 are the same as steps 7 and 8 in the first embodiment above.

[0159] In step 9, the MTLF sends the ADRF information stored by the matched target model to the AnLF 2, so that the AnLF 2 can obtain the model information of the target model from the ADRF.

[0160] The model selection method provided by the embodiment of the present application may be executed by a model selection device. In the embodiment of the present application, the model selection device provided by the embodiment of the present application will be described by taking the execution of the model selection method by the model selection device as an example.

[0161] As shown in FIG. 5 , an embodiment of the present application provides a model selection device 300, which includes a recording module 301, a first receiving module 302, and a first sending module 303. The recording module 301 is used to record model information of N candidate models, where the N candidate models are applied to a data analysis task corresponding to a first analysis identifier. The first receiving module 302 is used to receive a first model request message from a second network element. The first sending module 303 is used to send model information of P candidate models among the M candidate models to the second network element if the model information of M candidate models among the N candidate models matches the model information corresponding to the first model request message, where N, M, and P are all positive integers, and N≧M≧P.

[0162] Optionally, in an embodiment of the present application, the model selection device further includes a determination module used to determine N candidate models that satisfy a predetermined condition from the K models according to model performance information of the K models, where the K models are applied to the data analysis task corresponding to the first analysis identifier, and K is a positive integer. The predetermined condition includes any one of the following: the performance indicated by the model performance information is the highest among the K models; or the performance indicated by the model performance information is higher than a first predetermined performance.

[0163] Optionally, in an embodiment of the present application, the model selection device further includes a first acquiring module used for acquiring model information of the N candidate models from a third network element.

[0164] Optionally, in an embodiment of the present application, the model information of the candidate model may include: a model identifier for the candidate model; Model file information of the candidate model, download address information of the candidate model, which is used to indicate the storage address of the model file of the candidate model; an analysis identifier for the candidate model, which is used to identify the data analysis task to which the candidate model is to be applied; Use range information of the candidate model; Model performance information of candidate models It includes at least one of the following:

[0165] Optionally, in an embodiment of the present application, the usage range information may include: Use area range, Usage time range, Scope of use Indicate at least one of the following:

[0166] Optionally, in an embodiment of the present application, the performance indicated by the model performance information includes at least one of: model accuracy, model mean absolute error.

[0167] Optionally, in an embodiment of the present application, the model selection apparatus further includes a second acquisition module used to acquire usage range information and / or model performance information of each candidate model among the N candidate models.

[0168] Optionally, in an embodiment of the present application, the second acquisition module is specifically used to determine usage range information of each candidate model and / or calculate model performance information of each candidate model, or the second acquisition module is specifically used to receive usage range information and / or model performance information of each candidate model from the seventh network element, or the second acquisition module is specifically used to receive usage range information and / or model performance information of each candidate model from the data analysis consumer.

[0169] Optionally, in an embodiment of the present application, the first model request message comprises: a second analysis identifier used to identify the data analysis task to which the requested model is to be applied; Model filtering information, which is used to indicate the conditions that the requested model must satisfy; model target information used to indicate the training targets of the required model; Model reporting information, including at least one of the required model reporting method, application time, or reporting time; Model performance requirement information used to dictate the performance that a required model must meet It includes at least one of the following:

[0170] Optionally, in the embodiment of the present application, the model information of the M candidate models matches the model information corresponding to the first model request message by: the first analysis identifier and the second analysis identifier are the same; The usage area ranges included in the model usage range information of the M candidate models match the model filtering information; The usage target ranges included in the model usage range information of the M candidate models match the model target information; The usage time ranges included in the model usage range information of the M candidate models match the model report information; The model performance information of the M candidate models satisfies the performance indicated by the model performance requirement information. It includes at least one of the following:

[0171] Optionally, in embodiments of the present application, the K models include a plurality of candidate models to be applied to the data analysis task corresponding to the first analysis identifier, or the K models include a plurality of models obtained by training multiple times using the data analysis task corresponding to the first analysis identifier.

[0172] Optionally, in an embodiment of the present application, when the K models include a plurality of models obtained by performing training multiple times by the data analysis task corresponding to the first analysis identifier, the model selection device may: 1st The network further includes an execution module used to obtain a second model, which is a model among the K models, by retraining the first model or reselecting a model based on usage range information of the first model when the performance of the first model transmitted by the transmission module to the fourth network element is lower than the second default performance, and a second transmission module used to transmit model information of the second model to the fourth network element or the fifth network element.

[0173] Optionally, in an embodiment of the present application, the model selection device further includes a third sending module used to send usage range information of the second model to the fourth network element or the fifth network element, and the usage range indicated by the usage range information of the second model is the same as the usage range indicated by the usage range information of the first model, or the usage range indicated by the usage range information of the second model is determined by the usage range indicated by the usage range information of the first model.

[0174] Optionally, in an embodiment of the present application, the model selection device further includes a calculation module used to calculate model performance information of the first model, or a second receiving module also used to receive model performance information of the first model from the fourth network element, or a third receiving module also used to receive model performance information of the first model from the data analysis consumer, wherein the model performance information of the first model is used to indicate the performance of the first model.

[0175] Optionally, in an embodiment of the present application, the model selection device further includes a storage module used for storing model information of the N candidate models in the sixth network element.

[0176] Optionally, in an embodiment of the present application, the first sending module is specifically used for sending first information to the second network element, and the first information is used for indicating that the model information of the P candidate models is stored in the sixth network element, and the first information includes at least one of identifier information, FQDN, and address information of the sixth network element.

[0177] Optionally, in an embodiment of the present application, the sixth network element includes an ADRF or a UDR.

[0178] Optionally, in an embodiment of the present application, the second network element comprises an AnLF or an MTLF.

[0179] The model selection device provided by the embodiment of the present application can record model information of candidate models applied to a data analysis task, so that after the model selection device receives a first model request message, it can determine whether the model information of the recorded candidate models matches the model information corresponding to the first model request message, thereby determining an optimal model that meets the first model request message or a model that meets the performance requirements, thereby ensuring the performance of the model in data analysis.

[0180] The model selection device in the embodiments of the present application may be a network-side device, such as a network-side device with an operating system, or may be a component of the network-side device, such as an integrated circuit or chip. The network-side device may be an NWDAF, a MTLF, an AnLF, an ARDF, etc., and is not specifically limited in the embodiments of the present application.

[0181] The model selection device provided by the embodiments of the present application can realize each process realized by the embodiments of the model selection method described above and achieve the same technical effects, and to avoid duplication, the description will be omitted here.

[0182] Optionally, as shown in FIG. 6, the embodiment of the present application further provides a network-side device 400, including a processor 401 and a memory 402, wherein the memory 402 stores a program or command executable by the processor 401, such as ,So When the program or command is executed by the processor 401, it realizes each step of the above-mentioned embodiment of the model selection method and achieves the same technical effect, and therefore, to avoid repetition, the description will be omitted here.

[0183] Specifically, an embodiment of the present application further provides a network-side device 500. As shown in Fig. 7, the network-side device 500 includes a processor 501, a network interface 502, and a memory 503. The network interface 502 is, for example, a common public radio interface (CPRI).

[0184] Specifically, the network-side device 500 according to the embodiment of the present invention further includes commands or programs stored in the memory 503 and executable by the processor 501, and the processor 501 invokes the commands or programs in the memory 503 to execute the methods performed by each module shown in FIG. 6, and achieves the same technical effects. To avoid duplication, the description will be omitted here.

[0185] The embodiments of the present application further provide a readable storage medium storing a program or command, which, when executed by a processor, realizes each process of the above-mentioned model selection method embodiments and achieves the same technical effects. To avoid repetition, the description will be omitted here.

[0186] The processor is the processor in the terminal in the above embodiments. The readable storage medium includes, for example, a computer readable storage medium such as a computer read only memory ROM, a random access memory RAM, a magnetic disk, or an optical disk.

[0187] The embodiments of the present application also provide a chip, which includes a coupled processor and a communication interface, and the processor is for implementing each process of the above-mentioned embodiment of the model selection method by executing a program or command, and can achieve the same technical effects, so that the description will be omitted here to avoid duplication.

[0188] It should be understood that the chip described in the embodiments of the present application may also be referred to as a system on a chip, a system chip, a chip system, or an SoC.

[0189] The embodiments of the present application also provide a computer program / program product, which is stored in a storage medium and executed by at least one processor to realize each process of the above-mentioned model selection method embodiments and achieve the same technical effects. To avoid repetition, the description will be omitted here.

[0190] It should be noted that, in this specification, the terms "comprises," "has," and any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus comprising a set of elements includes not only those elements but also other elements not expressly stated or inherent in such process, method, article, or apparatus. Unless otherwise specified, an element limited by a phrase "comprises a..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element. It should also be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed herein, but may further include performing functions substantially simultaneously or in the reverse order, depending on the functionality involved. For example, the methods described may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described with reference to some examples may be combined with other examples.

[0191] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be realized in the form of a combination of software and a required general-purpose hardware platform, and of course, hardware implementation is also possible, but in many cases the former is a more preferred embodiment. Based on this view, the technical solution of the present application can be substantially embodied or the part that contributes to the prior art can be embodied as a computer software product, which is stored in a storage medium (e.g., ROM / RAM, magnetic disk, optical disk) and includes several commands for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network-side device, etc.) to execute the method according to each embodiment of the present application.

[0192] Although the examples of the present application have been described above with reference to the drawings, the present application is not limited to the specific embodiments described above, which are merely illustrative and not limiting. Based on the suggestions of the present application, many forms that a person skilled in the art can make without departing from the spirit of the present application and the scope of protection of the claims are all within the scope of protection of the present application.

[0193] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to a Chinese patent application filed in China on May 5, 2022, bearing application number 202210483837.0, the entire contents of which are incorporated herein by reference.

Claims

1. a first network element recording model information for N candidate models, the N candidate models being applied to a data analysis task corresponding to a first analysis identifier; receiving a first model request message from a second network element by the first network element; If model information of M candidate models among the N candidate models matches model information corresponding to the first model request message, the first network element sends model information of P candidate models among the M candidate models to the second network element; where N, M, and P are all positive integers, and N≧M≧P; Before the step of the first network element recording model information of the N candidate models, The method further includes a step of: determining, by the first network element, the N candidate models that satisfy a predetermined condition from the K models according to model performance information of the K models, wherein the K models are applied to a data analysis task corresponding to the first analysis identifier, where K is a positive integer; The predetermined condition is: The performance indicated by the model performance information is the highest among the K models; and The performance indicated by the model performance information is higher than a first predetermined performance; including any one of Model selection method.

2. The method further includes the step of: the first network element obtaining model information of the N candidate models from a third network element. The model selection method of claim 1 .

3. The model information of the candidate model is a model identifier for the candidate model; Model file information of the candidate model, download address information of the candidate model, which is used to indicate the storage address of the model file of the candidate model; an analysis identifier for the candidate model, which is used to identify the data analysis task to which the candidate model is to be applied; Use range information of the candidate model; Model performance information of candidate models The method of claim 1 , further comprising at least one of:

4. The method of claim 1 , wherein the performance indicated by the model performance information includes at least one of model accuracy, model mean absolute error.

5. Before the step of the first network element recording model information of the N candidate models, The method further includes the step of the first network element obtaining usage range information and / or model performance information of each candidate model among the N candidate models; The step of the first network element obtaining usage range information and / or model performance information of each candidate model among the N candidate models comprises: the first network element determining usage range information for each of the candidate models and / or calculating model performance information for each of the candidate models; receiving, by the first network element, usage range information and / or model performance information for each of the candidate models from a seventh network element; and receiving, by the first network element, usage range information and / or model performance information for each of the candidate models from a data analysis consumer. The model selection method of claim 1 .

6. The first model request message includes: a second analysis identifier used to identify the data analysis task to which the requested model is to be applied; Model filtering information, which is used to indicate the conditions that the requested model must satisfy; model target information used to indicate the training targets of the required model; model reporting information, including at least one of the required model reporting method, application time, or reporting time; Model performance requirement information used to dictate the performance that a required model must meet The method of claim 1 , further comprising at least one of:

7. the model performance requirement information includes at least one of a minimum accuracy and a maximum MAE that the model must achieve; The model selection method according to claim 6.

8. The model information of the M candidate models matches the model information corresponding to the first model request message, the first analysis identifier and the second analysis identifier are the same; The usage area ranges included in the model usage range information of the M candidate models match the model filtering information; the usage target ranges included in the model usage range information of the M candidate models match the model target information; the usage time ranges included in the model usage range information of the M candidate models match the model report information; The model performance information of the M candidate models satisfies the performance indicated by the model performance requirement information. The method of claim 6 , further comprising at least one of:

9. 2. The model selection method of claim 1, wherein the K models include a plurality of candidate models to be applied to the data analysis task corresponding to the first analysis identifier, or the K models include a plurality of models obtained by performing training multiple times using the data analysis task corresponding to the first analysis identifier.

10. When the K models include a plurality of models obtained by performing training multiple times using a data analysis task corresponding to the first analysis identifier, When the performance of the first model transmitted by the first network element to the fourth network element is lower than a second predetermined performance, the first network element retrains the first model or reselects a model according to the usage range information of the first model, thereby obtaining a second model that is a model among the K models; the first network element sending model information of the second model to the fourth network element or the fifth network element; The model selection method of claim 9 further comprising:

11. The method further includes a step of transmitting usage range information of the second model from the first network element to the fourth network element or the fifth network element; The usage range indicated by the usage range information of the second model is the same as the usage range indicated by the usage range information of the first model, or the use range indicated by the range information is determined by the use range indicated by the use range information of the first model; The model selection method of claim 10.

12. The method of claim 11, further comprising: calculating model performance information for the first model by the first network element; receiving, by the first network element, model performance information of the first model from the fourth network element; the first network element receiving model performance information of the first model from a data analytics consumer; the model performance information of the first model is used to indicate performance of the first model. The model selection method of claim 10.

13. After the step of the first network element recording model information of the N candidate models, The method further includes the step of the first network element storing model information of the N candidate models in a sixth network element; The step of transmitting model information of P candidate models among the M candidate models from the first network element to the second network element comprises: transmitting first information from the first network element to the second network element, the first information being used to indicate that model information for the P candidate models is stored in the sixth network element; the first information includes at least one of identifier information, a fully qualified domain name (FQDN), and address information of the sixth network element; the sixth network element comprises an analytical data storage function ADRF or a unified data repository UDR; The model selection method of claim 1 .

14. the first network element comprises a model training logic function MTLF or an analytical data storage function ADRF; the second network element comprises an analysis logic function AnLF or MTLF; The model selection method of claim 1 .

15. A network side device including a processor and a memory, wherein the memory stores a program or command executable by the processor, and when the program or command is executed by the processor, the steps of the model selection method described in any one of claims 1 to 14 are realized.

16. A readable storage medium storing a program or commands that, when executed by a processor, implements the steps of the model selection method according to any one of claims 1 to 14.

Images