A method and apparatus for evaluating a function

Abstract

The embodiment of the application discloses a discovery method and device for evaluating functions, and is used for improving the accuracy of analysis results obtained by a network data function (NWDAF) network element. The method of the embodiment of the application comprises the following steps: a network storage function network element receives a discovery request sent by a first network element, wherein the discovery request is used for requesting to discover a network element supporting evaluation requirements of the first network element. The network storage function network element sends a discovery response to the first network element, wherein the discovery response comprises information of a candidate network element, the candidate network element supports the evaluation requirements of the first network element, and the candidate network element is used for determining a target network element, and the target network element is used for performing an evaluation task.

Description

Technical Field

[0001] This application relates to the field of communications, and more particularly to a method and apparatus for discovering evaluation functions. Background Technology

[0002] With the development of artificial intelligence (AI) technology, AI technology has been widely used in the field of mobile communications. For example, mobile communication systems can perform network data analysis based on AI technology, analyze massive amounts of information in the network and output valuable information to assist operators in policy formulation and network resource adjustment.

[0003] In current network data analytics-based solutions, the 3rd Generation Partnership Project (3GPP) standard defines a network architecture based on network data analytics function (NWDAF) network elements. In this architecture, NWDAF network elements can perform service experience analysis. Specifically, NWDAF network elements can collect network data from other network elements. For example, NWDAF can collect service experience information from application function (AF) network elements, user equipment location information from access and mobility management function (AMF) network elements, and bit rate and latency information from user plane function (UPF) network elements. Subsequently, NWDAF network elements can analyze the collected data to obtain recommended quality of service (QoS) data.

[0004] In current solutions, NWDAF network elements often generate analysis results (e.g., recommended QoS parameters) based on built-in models when performing service experience analysis. However, these analysis results often lack evaluation based on live network data. Furthermore, NWDAF network elements cannot assess the potential impact of using these analysis results on the network, resulting in poor accuracy and reliability of the analysis results generated by NWDAF network elements. Summary of the Invention

[0005] This application provides a method for discovering evaluation functions, which improves the accuracy of analysis results obtained from Network Data Functional Analysis (NWDAF) of network elements. This application also provides a corresponding device, network equipment, communication system, computer-readable storage medium, and computer program product for discovering evaluation functions.

[0006] In a first aspect, embodiments of this application provide a method for discovering evaluation functions. This method can be executed by a network device, or by a component of the network device, such as a processor, chip, or chip system, or by a logic module or software capable of implementing all or part of the network device's functions. Taking a network storage function element as an example, the method provided in the first aspect includes: the network storage function element receiving a discovery request sent by a first network element. The discovery request requests the discovery of network elements within the network storage function element that support the evaluation requirements of the first network element. The network storage function element stores registration information for the evaluation function. The first network element is, for example, an NWDAF network element. The network storage function element sends a discovery response to the first network element. The discovery response indicates information about candidate network elements, such as an APSF network element. The candidate network elements support the evaluation requirements of the first network element. The candidate network elements are used to determine a target network element, and the target network element is used to perform the evaluation task.

[0007] In this embodiment, the network storage function network element can register an evaluation function of a third network element. This allows the first network element to discover the registered evaluation function during data analysis and evaluate its analysis results based on the evaluation function, thereby updating the data analysis results of the network storage function network element. Compared to the prior art where the NWDAF network element directly generates the data analysis results, in this embodiment, the first network element can discover the registered evaluation function in the network storage function network element and use the evaluation function to update its analysis results, thus improving the accuracy and reliability of the analysis results generated by the first network element.

[0008] In one possible implementation, the network storage function network element receives a registration request sent by a third network element. The registration request includes one or more of the following: assessment type and assessment capability information. The assessment capability information includes one or more of the following: assessment capability and assessment capability range. The assessment capability includes the specific assessment capability value registered by the third network element, and the assessment capability range includes the range of assessment capability values ​​registered by the third network element.

[0009] In this embodiment, the network storage function network element can store the evaluation type and evaluation capability information registered by the third network element. Therefore, when the first network element requests to discover the evaluation function, the network storage function network element can provide candidate network elements based on the evaluation capability information registered by the third network element, thereby improving the feasibility of the first network element discovering the evaluation function. In one possible implementation, the network storage function network element receives a first registration request sent by the third network element. The registration information in the first registration request includes the evaluation type and evaluation capability. The third network element is used to determine the target network element, i.e., the target network element can be selected from the third network elements, such as the APSF network element.

[0010] In this embodiment, the third network element can register evaluation types and evaluation capabilities in the network storage function network element, so that the first network element can discover the target network element based on the evaluation types and evaluation capabilities registered by the third network element, thereby improving the accuracy of the first network element in discovering the target network element. Based on the target network element, the analysis results of the first network element are evaluated, further improving the accuracy and reliability of the analysis results of the first network element.

[0011] In one possible implementation, a network storage function (FSF) element receives a first discovery request from a first network element. The first discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capabilities. Based on the first discovery request, the FSF element determines a first candidate network element. The evaluation capabilities supported by the first candidate network element satisfy the required evaluation type, and the evaluation capability information supported by the first candidate network element satisfies the required evaluation capability information. The FSF element then sends a first discovery response to the first network element, which includes information about the first candidate network element.

[0012] In this embodiment, the first network element can send a discovery request to the network storage function network element, and carry the required evaluation type and evaluation capability through the discovery request. The network storage function network element can directly match the target network element based on the registered evaluation type and evaluation capability, thereby improving the discovery efficiency of the first network element for the target network element.

[0013] In one possible implementation, the network storage function network element receives a second registration request sent by a third network element. The second registration request does not carry registration information or carries one or more of the following registration information: evaluation type and evaluation capability.

[0014] In this embodiment, the third network element can register specific evaluation types and evaluation capabilities with the network storage function network element during the registration phase, and build evaluation capabilities in real time during the evaluation function discovery phase, thereby improving the flexibility of the third network element in registering evaluation functions.

[0015] In one possible implementation, the network storage function (FSF) network element receives a second discovery request sent by a first network element. The second discovery request does not carry indication information, or carries one or more of the following indication information: first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. Based on the second discovery request, the FSF network element determines a second candidate network element. When the second discovery request does not carry indication information, the second candidate network element includes one or more network elements that have registered evaluation functions in the FSF network element. When the second discovery request carries indication information, the evaluation type of the second candidate network element supports the required evaluation type, and the evaluation capability of the second candidate network element supports the required evaluation capability. The FSF network element sends a second discovery response to the first network element. The second discovery response includes information about the second candidate network element, which is used to determine the target network element.

[0016] In this embodiment, the first network element can send a discovery request to the network storage function network element, and carry the required evaluation type and evaluation capability through the discovery request. The network storage function network element can match multiple second candidate network elements based on the registered evaluation type and evaluation capability, and the first network element can select the target network element from the second candidate network elements, thereby improving the feasibility of the first network element in determining the target network element.

[0017] In one possible implementation, the network storage function network element receives a third registration request sent by a third network element, and the registration information in the third registration request includes the evaluation type and the evaluation capability range.

[0018] In this embodiment, the third network element can register an evaluation capability range in the network storage function network element, so that the first network element can determine multiple third candidate network elements based on the evaluation capability range registered by the third network element, and then the first network element and the third candidate network elements negotiate the specific evaluation capability. Since the third network element can register an evaluation capability range in the network storage function network element, the frequent updating of the evaluation capability by the third network element to the network storage function network element is avoided.

[0019] In one possible implementation, the network storage function element receives a third discovery request from a first network element. The third discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capabilities. Based on the third discovery request, the network storage function element determines a third candidate network element. The evaluation type of the third candidate network element supports the required evaluation type, and the evaluation capability range of the third candidate network element includes the required evaluation capabilities. The network storage function element sends a third discovery response to the first network element. The third discovery response includes information about the third candidate network element, which is used to determine the target network element.

[0020] In this embodiment, the first network element can send a discovery request to the network storage function network element, and carry the required evaluation type and evaluation capability through the discovery request. The network storage function network element can match multiple third candidate network elements based on the registered evaluation type and evaluation capability range, and the first network element can select the target network element from the third candidate network elements, thereby improving the feasibility of the first network element in determining the target network element.

[0021] In one possible implementation, the network storage function network element receives a fourth registration request sent by a third network element. The registration information in the fourth registration request includes the evaluation type, and the registration information in the fourth registration request also includes the evaluation capability range or evaluation capability.

[0022] In one possible implementation, the network storage function network element receives a fourth discovery request sent by a first network element. The fourth discovery request includes first indication information and third indication information. The first indication information indicates the required evaluation type, and the third indication information indicates the required evaluation capability range, or the third indication information further indicates the required evaluation capability and corresponding error range. Based on the fourth discovery request, the network storage function network element determines a fourth candidate network element. The evaluation type of the fourth candidate network element supports the required evaluation type, and the evaluation capability range or evaluation capability corresponding to the fourth candidate network element is within the required evaluation capability range, or within the error range corresponding to the required evaluation capability. The network storage function network element sends a fourth discovery response to the first network element. The fourth discovery response includes a fourth candidate network element, which is used to determine the target network element.

[0023] In this embodiment, the first network element can send a discovery request to the network storage function network element and carry the required evaluation capability range in the discovery request. The network storage function network element can match multiple fourth candidate network elements based on the registered evaluation capability or evaluation capability range, and the first network element can select the target network element from the fourth candidate network elements, thereby improving the feasibility of the first network element in determining the target network element.

[0024] In one possible implementation, the network storage function network element receives a registration information update request sent by a third network element. The registration information update request includes one or more of the following: the updated evaluation type, the updated evaluation capability, and the updated evaluation capability range.

[0025] In this embodiment, the third network element can send a registration information update request to the network storage function network element, thereby updating the evaluation capability registered by the third network element. This enables the first network element to discover the target network element based on the latest registration information, thereby improving the accuracy of the first network element in discovering the target network element and further improving the accuracy and reliability of the analysis results generated by the first network element.

[0026] Secondly, embodiments of this application provide a method for discovering evaluation capabilities. This method can be executed by a network device, a component of the network device (such as its processor, chip, or chip system), or a logic module or software capable of implementing all or part of the network device's functions. Taking a network storage function network element as an example, the method provided in this second aspect includes: the network storage function network element receiving a discovery request sent by a first network element. The discovery request includes a first evaluation capability indication, and the discovery request is used to discover a network data analysis function network element that supports the evaluation capability. The network storage function network element sends a discovery response to the first network element. The discovery response includes information about the network data analysis function network element, which supports the evaluation capability.

[0027] In this embodiment, the evaluation function can be provided by the evaluation function module of the network element. The first network element can discover the target evaluation function module based on the registration information of the evaluation function module, and trigger the target evaluation function module to evaluate the analysis results of the first network element. Based on the evaluation results, the analysis results are updated, thereby improving the accuracy and reliability of the analysis results of the first network element.

[0028] In one possible implementation, the network storage function network element receives a registration request sent by the network data analysis function network element. The registration request includes a second evaluation capability indication, which is used to indicate that the network data analysis function network element supports the evaluation capability.

[0029] In this embodiment, the evaluation function module can indicate that it supports the evaluation function through the evaluation capability identifier. The first network element can discover the target evaluation function module based on the registration information of the evaluation function module, thereby improving the accuracy of the first network element in discovering the target evaluation function module.

[0030] Thirdly, embodiments of this application provide a method for discovering evaluation functions. This method can be executed by a network device, or by a component of the network device, such as a processor, chip, or chip system, or by a logic module or software capable of implementing all or part of the network device's functions. Taking a first network element as an example, the method provided in the third aspect includes: the first network element sending a discovery request to a network storage function network element. The discovery request is used to request the discovery of network elements that support the evaluation requirements of the first network element. The first network element is, for example, an NWDAF network element. The network storage function network element is used to store registration information for the evaluation function. The network storage function network element is, for example, an NRF network element. The first network element receives a discovery response sent by the network storage function network element. The discovery response is used to indicate information about candidate network elements. The candidate network elements are used to determine a target network element. The target network element is used to perform the evaluation task. The target network element is, for example, an APSF network element.

[0031] In this embodiment, the first network element can discover and register an evaluation function with the network storage function network element, and evaluate the analysis results of the first network element based on the evaluation function, thereby updating the data analysis results of the network storage function network element, thus improving the accuracy and reliability of the analysis results generated by the first network element.

[0032] In one possible implementation, a first network element sends a first discovery request to a network storage function network element. The first discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The first network element receives a first discovery response from the network storage function network element. The first discovery response includes information about a first candidate network element, the evaluation type of which supports the required evaluation type, and the evaluation capability of which supports the required evaluation capability.

[0033] In this embodiment of the application, the first network element can discover the first candidate network element based on the registered evaluation type and evaluation capability, determine the target network element based on the first candidate network element, and evaluate the analysis results of the first network element based on the target network element, thereby improving the accuracy and reliability of the analysis results of the first network element.

[0034] In one possible implementation, a first network element sends a second discovery request to a network storage function network element. The second discovery request either does not carry indication information or carries one or more of the following: first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The first network element receives a second discovery response from the network storage function network element. The second discovery response includes second candidate network elements. When the second discovery request does not carry indication information, the second candidate network elements include one or more network elements that have registered evaluation functions in the network storage function network element. When the second discovery request carries indication information, the evaluation type of the second candidate network element supports the required evaluation type, and the evaluation capability of the second candidate network element supports the required evaluation capability.

[0035] In this embodiment, the discovery request sent by the first network element to the network storage function network element may not carry indication information. When the discovery request does not carry indication information, the network storage function network element can determine all third network elements with registered evaluation functions as second candidate network elements. Then, the first network element selects the target network element from the second candidate network elements, thereby improving the feasibility of the first network element in determining the target network element.

[0036] In one possible implementation, a first network element determines a target network element from one or more second candidate network elements. The first network element sends a first request to the target network element, the first request including a first desired evaluation capability. The first network element receives a first response from the target network element, the first response including the target network element's constructed evaluation capabilities.

[0037] In this embodiment, the third network element can build evaluation capabilities in real time and negotiate the evaluation capabilities supported by the first network element. The first network element can choose the evaluation capabilities built by the target network element, thereby improving the accuracy and reliability of the analysis results of the first network element and also improving the flexibility of the third network element in building evaluation capabilities.

[0038] In one possible implementation, the first network element sends a third discovery request to the network storage function network element. The third discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The first network element receives a third discovery response from the network storage function network element. The third discovery response includes a third candidate network element, the evaluation type of the third candidate network element supports the required evaluation type, and the evaluation capability range of the third candidate network element includes the required evaluation capability.

[0039] In this embodiment, the first network element sends a third discovery request to the network storage function network element. The third discovery request can register the evaluation capability range during the evaluation function registration and discover a third candidate network element. The evaluation capability range registered by the third candidate network element includes the required evaluation capabilities, thereby improving the feasibility of the first network element discovering the third candidate network element.

[0040] In one possible implementation, a first network element sends a second request to one or more third candidate network elements, the second request including a second desired evaluation capability. The first network element receives a second response from the one or more third candidate network elements, the second response including first information indicating the result of the one or more third candidate network elements meeting the second desired capability. The first network element determines a target network element from the one or more third candidate network elements based on the first information.

[0041] In one possible implementation, the first network element sends a second request to one or more third candidate network elements, the second request including actual assessment capability report indication information. The first network element receives a second response from one or more third candidate network elements, the second response including the actual assessment capability of the one or more third candidate network elements. The first network element determines the target network element from the one or more third candidate network elements based on the actual assessment capability.

[0042] In this embodiment, the first network element can determine multiple third candidate network elements based on the evaluation capability range registered by the third network element. Then, the first network element and the third candidate network elements negotiate the supported evaluation capabilities and determine the target network element. Since the third network element can register an evaluation capability range in the network storage function network element, the third network element is avoided from frequently updating the evaluation capability to the network storage function network element, thereby reducing communication overhead.

[0043] In one possible implementation, the first network element sends a fourth discovery request to the network storage function network element. The fourth discovery request includes first indication information and third indication information. The first indication information indicates the required evaluation type, and the third indication information indicates the required evaluation capability range, or the third indication information indicates the required evaluation capability and corresponding error range. The first network element receives a fourth discovery response from the network storage function network element. The fourth discovery response includes a fourth candidate network element, and the evaluation capability range or evaluation capability of the fourth candidate network element is within the required evaluation capability range, or the evaluation capability range or evaluation capability of the fourth candidate network element is within the error range of the required evaluation capability.

[0044] In one possible implementation, the first network element sends a third request to one or more fourth candidate network elements, the third request including a third desired evaluation capability. The first network element receives a third response from the one or more fourth candidate network elements, the third response including second information indicating the result of the one or more fourth candidate network elements meeting the third desired evaluation capability. The first network element determines the target network element from the one or more fourth candidate network elements based on the second information.

[0045] In one possible implementation, the first network element sends a third request to one or more fourth candidate network elements, the third request including actual assessment capability report indication information. The first network element receives a third response from one or more fourth candidate network elements, the third response including the actual assessment capability of the one or more fourth candidate network elements. The first network element determines the target network element from the one or more fourth candidate network elements based on the actual assessment capability.

[0046] In this embodiment, the first network element can send a fourth discovery request to the network storage function network element. This fourth discovery request can request a required evaluation capability range. In this way, the network storage function network element can determine the third network element that meets the range as the fourth candidate network element. Then, the first network element and the fourth candidate network element negotiate the simulation capabilities they support and determine the target network element. Therefore, the first network element can select the target network element from more fourth candidate network elements, which further improves the accuracy and reliability of the analysis results of the first network element.

[0047] In one possible implementation, the first network element sends a fourth request to the target network element, the fourth request including a fourth desired evaluation capability. The first network element receives a fourth response from the target network element, the fourth response including third information indicating the target network element's satisfaction with the fourth desired evaluation capability. If the target network element's evaluation capability does not meet the fourth desired evaluation capability, the first network element reselects a target network element.

[0048] In one possible implementation, the first network element sends a fourth request to the target network element, the fourth request including actual assessment capability report indication information. The first network element receives a fourth response from the target network element, the fourth response including the actual assessment capability of the target network element. If the first network element determines that the actual assessment capability of the target network element does not meet the assessment capability requirements, the first network element reselects a target network element.

[0049] In this embodiment, the first network element and the target network element can renegotiate their evaluation capabilities. If the renegotiation of the evaluation capabilities between the first network element and the target network element still fails to meet the requirements, the first network element can reselect the target network element, thereby improving the accuracy of the first network element's selection of the target network element and further improving the accuracy and reliability of the first network element's analysis results.

[0050] In one possible implementation, the first network element generates analysis results based on collected data, which includes data from one or more data sources. The first network element sends an evaluation request to the target network element, requesting an evaluation of the analysis results, and the evaluation request carries the analysis results. The first network element receives the evaluation results sent by the target network element, which include the target network element's evaluation of the analysis results. The first network element updates the analysis results based on the evaluation results.

[0051] In this embodiment, the first network element can collect data to generate analysis results, and use the evaluation function of the target network element to update the analysis results of the first network element, thereby improving the accuracy and reliability of the analysis results generated by the first network element.

[0052] Fourthly, embodiments of this application provide a method for discovering an evaluation function. This method can be executed by a network device, or by a component of the network device, such as a processor, chip, or chip system, or by a logic module or software capable of implementing all or part of the network device's functions. Taking a third network element as an example, the method provided in this fourth aspect includes: the third network element sending a registration request to a network storage function network element. The registration request is used to request the registration of the evaluation function in the network storage function network element. The registration request includes the registration information of the third network element, which is used to discover a target network element. The target network element is used to perform the evaluation task. The third network element receives a registration response from the network storage function network element, which includes the registration result of the evaluation function.

[0053] In this embodiment, the third network element can register an evaluation function with the network storage function network element, so that the first network element can discover the registered evaluation function and identify the target network element during the data analysis process, and evaluate the analysis results of the first network element based on the target network element, thereby updating the data analysis results of the network storage function network element and improving the accuracy and reliability of the analysis results generated by the first network element.

[0054] In one possible implementation, the third network element sends a first registration request to the network storage function network element. The first registration request includes the evaluation type and evaluation capability, and the third network element is used to determine the target network element.

[0055] In one possible implementation, the third network element sends a second registration request to the network storage function network element. The second registration request does not carry registration information or carries one or more of the following registration information: evaluation type and evaluation capability. The third network element is also used to determine a second candidate network element, which is used to determine the target network element.

[0056] In one possible implementation, the target network element receives a first request from a first network element, the first request including a first desired simulation capability. The target network element then sends a first response to the first network element, the first response including the target network element's constructed evaluation capabilities.

[0057] In one possible implementation, the third network element sends a third registration request to the network storage function network element. The registration information in the third registration request includes the evaluation type and evaluation capability range. The third network element is also used to determine the third candidate network element.

[0058] In one possible implementation, the third candidate network element receives a second request from the first network element, the second request including: a report indication information of a second expected simulation capability or actual evaluation capability. The third candidate network element sends a second response to the first network element, the second response including: the third candidate network element's satisfaction with the second expected evaluation capability or the third candidate network element's actual evaluation capability, and the third candidate network element is used to determine the target network element.

[0059] In one possible implementation, the third network element sends a fourth registration request to the network storage function network element. The registration information in the fourth registration request includes the evaluation type, the evaluation capability range or evaluation capability, and the third network element is also used to determine the fourth candidate network element.

[0060] In one possible implementation, the fourth candidate network element receives a third request from the first network element, the third request including: a report indicating the third expected simulation capability or actual evaluation capability. The fourth candidate network element sends a third response to the first network element, the third response including: the fourth candidate network element's fulfillment of the third expected evaluation capability or the fourth candidate network element's actual evaluation capability, and the fourth candidate network element is used to determine the target network element.

[0061] In one possible implementation, the target network element receives an evaluation request sent by the first network element. The evaluation request includes analysis results and is used to request the target network element to evaluate the analysis results. The target network element evaluates the analysis results to obtain an evaluation result, which is used to update the analysis results generated by the first network element. The target network element then sends the evaluation result back to the first network element.

[0062] Fifthly, embodiments of this application provide a method for discovering an evaluation function. The method includes: a network data analysis function network element sending a registration request to a network storage function network element. The registration request requests registration of the evaluation function. Registration information in the registration request includes an evaluation capability indication, which instructs the network data analysis function network element to support the evaluation capability. The network data analysis function network element receives a registration response from the network storage function network element, the registration response including a registration result for the evaluation function.

[0063] The evaluation function in this application embodiment can be the evaluation function module of a network element. The first network element can discover the target evaluation function module based on the registration information and trigger the target evaluation function module to evaluate the analysis results of the first network element and determine the updated analysis results, thereby improving the accuracy and reliability of the analysis results of the first network element.

[0064] In one possible implementation, the network data analysis function network element receives an evaluation request sent by a first network element. The evaluation request includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, analysis results to be verified, and verification target; the analysis results. The evaluation request is used to request the target evaluation function module to evaluate the analysis results. The network data analysis function network element evaluates the analysis results to obtain an evaluation result. The evaluation result is used to update the analysis results generated by the first network element. The evaluation result includes one or more of the following: verification results, impact of the analysis results, simulation system evaluation results, and auxiliary information for updating the analysis results. The network data analysis function network element sends the evaluation result to the first network element.

[0065] Sixthly, embodiments of this application provide a discovery apparatus for an evaluation function, comprising: a receiving unit, a sending unit, and a processing unit. The receiving unit receives a discovery request sent by a first network element, the discovery request requesting the discovery of network elements that support the evaluation requirements of the first network element. The sending unit sends a discovery response to the first network element, the discovery response including information about candidate network elements that support the evaluation requirements of the first network element, the candidate network elements being used to determine a target network element, and the target network element being used to perform the evaluation task.

[0066] In one possible implementation, the receiving unit is further configured to receive a registration request sent by a third network element. The registration request includes one or more of the following: assessment type and assessment capability information. The assessment capability information includes one or more of the following: assessment capability and assessment capability range.

[0067] In one possible implementation, the registration request includes an evaluation type and an evaluation capability. The receiving unit is specifically configured to receive a first discovery request sent by a first network element. The first discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The processing unit is configured to determine a first candidate network element based on the first discovery request. The evaluation type supported by the first candidate network element meets the required evaluation type, and the evaluation capability supported by the first candidate network element meets the required evaluation capability. The sending unit is specifically configured to send a first discovery response to the first network element. The first discovery response includes information about the first candidate network element, which is used to determine the target network element.

[0068] In one possible implementation, the registration request includes an evaluation type and an evaluation capability range. The receiving unit is specifically configured to receive a third discovery request sent by a first network element. The third discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The processing unit is further configured to determine a third candidate network element based on the third discovery request. The evaluation types supported by the third candidate network element include the required evaluation type, and the evaluation capability range supported by the third candidate network element includes the required evaluation capability. The sending unit is specifically configured to send a third discovery response to the first network element. The third discovery response includes information about the third candidate network element, which is used to determine the target network element.

[0069] In one possible implementation, the registration request includes assessment type and assessment capability information. The receiving unit is specifically configured to receive a fourth discovery request sent by a first network element. The fourth discovery request includes first indication information and third indication information. The first indication information indicates the required assessment type, and the third indication information indicates the required assessment capability range. The processing unit is further configured to determine a fourth candidate network element based on the fourth discovery request. The fourth candidate network element supports assessment types including the required assessment type, and its supported assessment capability range or capabilities are within the required assessment capability range. The sending unit is specifically configured to send a fourth discovery response to the first network element. The fourth discovery response includes a fourth candidate network element, which is used to determine the target network element.

[0070] In one possible implementation, the receiving unit is further configured to receive a registration information update request sent by a third network element. The registration information update request includes one or more of the following: the updated supported evaluation type, the updated supported evaluation capability, and the updated supported evaluation capability range.

[0071] In a seventh aspect, embodiments of this application provide an apparatus for discovering evaluation capabilities. The apparatus includes a receiving unit, a sending unit, and a processing unit. The receiving unit receives a discovery request sent by a first network element. The discovery request includes a first evaluation capability indication, and the discovery request is used to discover a network data analysis function network element that supports the evaluation capability. The sending unit sends a discovery response to the first network element. The discovery response includes information about the network data analysis function network element, and the network data analysis function network element supports the evaluation capability.

[0072] In one possible implementation, the transceiver unit is further configured to receive a registration request sent by the network data analysis function network element. The registration request includes a second evaluation capability indication, which is used to indicate that the network data analysis function network element supports the evaluation capability.

[0073] Eighthly, embodiments of this application provide an apparatus for discovering an evaluation function. The apparatus includes a receiving unit, a sending unit, and a processing unit. The sending unit sends a discovery request to a network storage function network element, the discovery request requesting the discovery of network elements that support the evaluation requirements of a first network element. The receiving unit receives a discovery response sent by the network storage function network element, the discovery response including information about candidate network elements that support the evaluation requirements of the first network element. The candidate network elements are used to determine a target network element, and the target network element is used to perform the evaluation task.

[0074] In one possible implementation, the sending unit is specifically configured to send a first discovery request to the network storage function network element. The first discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The receiving unit is specifically configured to receive a first discovery response sent by the network storage function network element. The first discovery response includes information about a first candidate network element, the evaluation type of the first candidate network element supports the required evaluation type, and the evaluation capability of the first candidate network element supports the required evaluation capability.

[0075] In one possible implementation, the processing unit is used to determine a target network element from one or more first candidate network elements, and the target network element is used to perform the evaluation task.

[0076] In one possible implementation, the sending unit is specifically configured to send a second discovery request to the network storage function network element. The receiving unit is specifically configured to receive a second discovery response sent by the network storage function network element, the second discovery response including a second candidate network element, the second candidate network element being used to determine the target network element.

[0077] In one possible implementation, the processing unit is further configured to determine a target network element from one or more second candidate network elements, and the sending unit is further configured to send a first request to the target network element, the first request including a first desired evaluation capability. The receiving unit is further configured to receive a first response sent by the target network element, the first response including the constructed evaluation capability of the target network element.

[0078] In one possible implementation, the sending unit is specifically configured to send a third discovery request to the network storage function network element. The third discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The receiving unit is further configured to receive a third discovery response from the network storage function network element. The third discovery response includes a third candidate network element, the evaluation type of the third candidate network element supports the required evaluation type, and the evaluation capability range of the third candidate network element includes the required evaluation capability.

[0079] In one possible implementation, the sending unit is further configured to send a second request to the third candidate network element, the second request including a second desired evaluation capability. The receiving unit is further configured to receive a second response sent by the third candidate network element, the second response including first information indicating whether the third candidate network element meets the second desired capability. The processing unit is further configured to determine a target network element from one or more third candidate network elements based on the first information.

[0080] In one possible implementation, the sending unit is further configured to send a second request to the third candidate network element, the second request including actual assessment capability report indication information. The receiving unit is further configured to receive a second response sent by the third candidate network element, the second response including the actual assessment capability of the third candidate network element. The processing unit is further configured to determine a target network element from one or more second candidate network elements based on the actual assessment capability.

[0081] In one possible implementation, the sending unit is specifically configured to send a fourth discovery request to the network storage function network element. The fourth discovery request includes first indication information and third indication information. The first indication information indicates the required evaluation type, and the third indication information indicates the required evaluation capability range. The receiving unit is further configured to receive a fourth discovery response from the network storage function network element. The fourth discovery response includes a fourth candidate network element and the evaluation capability range or evaluation capability of the fourth candidate network element, which is within the required evaluation capability range.

[0082] In one possible implementation, the sending unit is further configured to send a third request to the fourth candidate network element, the third request including a third desired evaluation capability. The receiving unit is further configured to receive a third response sent by the fourth candidate network element, the third response including second information indicating the result of the fourth candidate network element satisfying the third desired evaluation capability. The processing unit is further configured to determine a target network element from one or more fourth candidate network elements based on the second information.

[0083] In one possible implementation, the sending unit is further configured to send a third request to the third candidate network element, the third request including actual assessment capability report indication information. The receiving unit is further configured to receive a third response sent by the third candidate network element, the third response including the actual assessment capability of one or more fourth candidate network elements. The processing unit is further configured to determine the target network element from the one or more fourth candidate network elements based on the actual assessment capability.

[0084] In one possible implementation, the sending unit is further configured to send a fourth request to the target network element, the fourth request including a fourth desired evaluation capability. The receiving unit is further configured to receive a fourth response sent by the target network element, the fourth response including third information indicating the result of the target network element satisfying the fourth desired evaluation capability. The processing unit is further configured to, if the evaluation capability of the target network element does not satisfy the fourth desired evaluation capability, reselect a target network element.

[0085] In one possible implementation, the sending unit is further configured to send a fourth request to the target network element, the fourth request including actual assessment capability report indication information. The receiving unit is further configured to receive a fourth response sent by the target network element, the fourth response including the actual assessment capability of the target network element. The processing unit is further configured to, when the first network element determines that the actual assessment capability of the target network element does not meet the assessment capability requirements, reselect a target network element.

[0086] In one possible implementation, the sending unit is further configured to send an evaluation request to the target network element. The evaluation request requests the evaluation of the analysis results and includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, output results to be verified, and verification target. The receiving unit is further configured to receive the evaluation results sent by the target network element. The evaluation results include one or more of the following: verification results, output result impact, simulation system evaluation results, and output result update auxiliary information. The processing unit is further configured to update the analysis results based on the evaluation results.

[0087] Ninthly, embodiments of this application provide an apparatus for discovering an evaluation function. The apparatus includes a receiving unit, a sending unit, and a processing unit. The sending unit sends a registration request to a network storage function element. The registration request requests the registration of the evaluation function within the network storage function element. The registration request includes registration information of a third network element, which is used to discover a target network element. The target network element is used to perform the evaluation task. The receiving unit receives a registration response from the network storage function element, which includes the registration result of the evaluation function.

[0088] In one possible implementation, the sending unit is specifically used to send a first registration request to the network storage function network element. The first registration request includes an evaluation type and an evaluation capability. The third network element is used to determine a first candidate network element, and the first candidate network element is used to determine a target network element.

[0089] In one possible implementation, the sending unit is specifically used to send a second registration request to the network storage function network element. The second registration request does not carry registration information or carries one or more of the following registration information: evaluation type and evaluation capability. The third network element is also used to determine a second candidate network element, which is used to determine the target network element.

[0090] In one possible implementation, the receiving unit is further configured to receive a first request sent by the first network element, the first request including a first desired simulation capability. The sending unit is further configured to send a first response to the first network element, the first response including the constructed evaluation capability of the target network element.

[0091] In one possible implementation, the sending unit is further configured to send a third registration request to the network storage function network element. The registration information in the third registration request includes the evaluation type and the evaluation capability range. The third network element is further configured to determine a third candidate network element.

[0092] In one possible implementation, the receiving unit is further configured to receive a second request sent by the first network element, the second request including: a second expected simulation capability or actual evaluation capability report indication information. The sending unit is further configured to send a second response to the first network element, the second response including: a third candidate network element's satisfaction result with the second expected evaluation capability or the actual evaluation capability of the third candidate network element, the third candidate network element being used to determine the target network element.

[0093] In one possible implementation, the sending unit is specifically used to send a fourth registration request to the network storage function network element. The fourth registration request includes evaluation type and evaluation capability information. The evaluation capability information includes evaluation capability range or evaluation capability. The third network element is also used to determine the fourth candidate network element.

[0094] In one possible implementation, the receiving unit is further configured to receive a third request sent by the first network element, the third request including: a third expected simulation capability or actual evaluation capability report indication information. The sending unit is further configured to send a third response to the first network element, the third response including: the result of the fourth candidate network element satisfying the third expected evaluation capability or the actual evaluation capability of the fourth candidate network element, the third candidate network element being used to determine the target network element.

[0095] In one possible implementation, the receiving unit is further configured to receive an evaluation request sent by the first network element. The evaluation request includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, output result to be verified, and verification target. The evaluation request is used to request the target network element to evaluate the analysis results. The processing unit is further configured to evaluate the output results to obtain evaluation results. The evaluation results are used to update the analysis results generated by the first network element. The evaluation results include one or more of the following: verification results, impact of output results, simulation system evaluation results, and auxiliary information for updating the output results. The sending unit is further configured to send the evaluation results to the first network element.

[0096] Tenthly, embodiments of this application provide an apparatus for discovering an evaluation function, comprising: a receiving unit, a sending unit, and a processing unit. The sending unit is configured to send a registration request to a network storage function element, the registration request requesting registration of the evaluation function, the registration information in the registration request including an evaluation capability indication, the evaluation capability indication indicating that the network data analysis function element supports the evaluation capability. The receiving unit is configured to receive a registration response sent by the network storage function element, the registration response including a registration result for the evaluation function.

[0097] In one possible implementation, the receiving unit is further configured to receive an evaluation request sent by the first network element. The evaluation request includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, analysis results to be verified, and verification target; the analysis results. The evaluation request is used to request the target evaluation function module to evaluate the analysis results. The processing unit is configured to evaluate the analysis results and obtain an evaluation result. The evaluation result is used to update the analysis results generated by the first network element. The evaluation result includes one or more of the following: verification results, impact of the analysis results, simulation system evaluation results, and auxiliary information for updating the analysis results. The sending unit is further configured to send the evaluation result to the first network element.

[0098] In one aspect, embodiments of this application provide a network device, the network device including a processor coupled to a memory, the processor being used to store instructions, and when the instructions are executed by the processor, causing the network device to perform the methods described in the first to fifth aspects, or any possible implementation of the first to fifth aspects.

[0099] In a twelfth aspect, embodiments of this application provide a communication system, which includes a first network device, a second network device, and a third network device. The first network device is used to execute the methods described in the first to second aspects, or any possible implementations of the first to second aspects. The second network device is used to execute the methods described in the third aspect, or any possible implementations of the third aspect. The third network device is used to execute the methods described in the fourth to fifth aspects, or any possible implementations of the fourth to fifth aspects.

[0100] In a thirteenth aspect, embodiments of this application provide a computer-readable storage medium having instructions stored thereon, which, when executed, cause a computer to perform the method described in the first to second aspects, or any possible implementation of the first to second aspects; or cause a computer to perform the method described in the third aspect, or any possible implementation of the third aspect; or cause a computer to perform the method described in the fourth to fifth aspects, or any possible implementation of the fourth to fifth aspects.

[0101] In a fourteenth aspect, embodiments of this application provide a computer program product, which includes instructions that, when executed, cause a computer to implement the method described in the first to second aspects, or any possible implementation of the first to second aspects; or cause a computer to implement the method described in the third aspect, or any possible implementation of the third aspect; or cause a computer to implement the method described in the fourth to fifth aspects, or any possible implementation of the fourth to fifth aspects.

[0102] It is understood that the beneficial effects achieved by any of the above-mentioned evaluation functions, such as discovery devices, network equipment, communication systems, computer-readable media, or computer program products, can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0103] Figure 1a A schematic diagram of the system architecture of a network data analysis system provided in this application embodiment;

[0104] Figure 1b A schematic diagram of the system architecture of another network data analysis system provided in this application embodiment;

[0105] Figure 1c A schematic diagram of the system architecture of another network data analysis system provided in this application embodiment;

[0106] Figure 2 A flowchart illustrating a method for discovering an evaluation function provided in an embodiment of this application;

[0107] Figure 3 A flowchart illustrating another method for discovering an evaluation function provided in an embodiment of this application;

[0108] Figure 4 A flowchart illustrating another method for discovering an evaluation function provided in an embodiment of this application;

[0109] Figure 5 A flowchart illustrating another method for discovering an evaluation function provided in an embodiment of this application;

[0110] Figure 6 A flowchart illustrating another method for discovering an evaluation function provided in an embodiment of this application;

[0111] Figure 7 A flowchart illustrating another method for discovering an evaluation function provided in an embodiment of this application;

[0112] Figure 8 A flowchart illustrating another method for discovering an evaluation function provided in an embodiment of this application;

[0113] Figure 9 A schematic diagram of an evaluation function discovery device provided in an embodiment of this application;

[0114] Figure 10 This application provides a schematic diagram of the structure of a network device according to an embodiment of the present application.

[0115] Figure 11 This is a schematic diagram of the structure of a communication system provided in an embodiment of this application. Detailed Implementation

[0116] This application provides a method and apparatus for discovering evaluation functions, which can improve the accuracy of the inference results output by the Network Data Divided Function (NWDAF) network element.

[0117] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0118] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0119] First, some of the terms used in the embodiments of this application are introduced to facilitate understanding of the technical solutions by those skilled in the art.

[0120] Network data analytics function (NWDAF) network elements are used to process and analyze large amounts of network data collected from various sources within the network system to generate data analysis results. The data analysis results generated by NWDAF network elements can assist operators in policy formulation and network resource adjustments, thereby improving the user's service experience.

[0121] The AI ​​performance simulation function (APSF) network element provides a network simulation environment. APSF network elements can evaluate the accuracy of NWDAF network element inference results and assist in inference fine-tuning and decision optimization. An APSF network element can be a standalone network element or a network element combined with an NWDAF network element.

[0122] The Data Collection Coordination Function (DCCF) network element is used to coordinate data collection and reduce redundant data acquisition. For example, during data collection, the DCCF element maintains a record list of data collection requests to track which data sources are currently being collected. When a new data collection request is received, the DCCF checks whether the data source requested by the new request matches the requested data sources in the active list, thereby avoiding duplicate data subscriptions and collections.

[0123] The Message Framework Adaptor (MFAF) network element is used to adapt to the message framework to support data transmission between the DCCF and consumer network elements. The MFAF network element also formats the data collected by the DCCF and adapts it to the message framework so that consumer network elements can receive and process this data.

[0124] The analytics data repository function (ADRF) network element is used to store and retrieve historical data collected by the network, historical analysis results generated by NWDAF, and NWDAF machine learning (ML) models. The ADRF network element allows each network element to directly or indirectly store and retrieve historical data, analysis results, and ML models. ADRF can also subscribe to a data notification service from the DCCF to automatically store and process data that meets certain criteria upon receipt.

[0125] To make the technical solution of this application clearer and easier to understand, the system architecture of this application will be described below with reference to the accompanying drawings.

[0126] Please see Figure 1a , Figure 1a This is a schematic diagram of the system architecture of a network data analysis system provided in an embodiment of this application. Figure 1a In the example shown, the data analysis system 10 mainly includes user equipment 101, access network 102, and core network 103. Specifically, the core network elements 103 involved in this embodiment mainly include NWDAF network element 1031 and APSF network element 1032. The specific functions of each part of the data analysis system 10 are described below.

[0127] User equipment (UE) 101 includes devices that provide voice and data connectivity to users. For example, UE 101 may include a handheld device with wireless connectivity or a processing device connected to a wireless modem. UE 101 can communicate with core network equipment via access network equipment and exchange voice and data through the access network equipment.

[0128] User equipment 101 can also be referred to as a terminal, wireless terminal equipment, mobile terminal (MT) equipment, subscriber unit, subscriber station, mobile station (MS), mobile station, remote station, access point (AP), remote terminal equipment, access terminal equipment, user terminal equipment, user agent, and user device, etc., without any specific limitation.

[0129] In addition, user equipment 101 can be a mobile phone, tablet, computer with wireless transceiver capabilities, virtual reality (VR) device, augmented reality (AR) device, extended reality (XR) service terminal, cloud gaming (CG) service terminal, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc.

[0130] Access network 102 can be any device with wireless transceiver capabilities. Access network 102 can connect user equipment 101 to the wireless network. Access network device 102 can also be called a base station or a radio access network (RAN) node.

[0131] Access network 102 includes, but is not limited to, next-generation Node B (gNodeB), transmission reception point (TRP), evolved Node B (eNB), radio network controller (RNC), Node B (NB), base station controller (BSC), base transceiver station (BTS), home base station (e.g., home evolved Node B, or home Node B, HNB), base band unit (BBU), or wireless fidelity (Wi-Fi) access point (AP) in 5G communication systems.

[0132] In another network architecture, the access network 102 may include centralized unit (CU) nodes, distributed unit (DU) nodes, or RAN equipment including CU nodes and DU nodes. In some implementations, the access network equipment 102 may also include satellites, aircraft, drones, and ground station equipment connected to satellites, aircraft, and drones.

[0133] Core network 103 refers to the devices in the core network (CN) that provide service support for user equipment 101 and access network 102. Core network 103 is used to implement three main functions: registration, connection, and session management. Core network 103 includes multiple network elements. This application embodiment mainly involves NWDAF network element 1031, APSF network element 1032, and NRF network element storage function 1033. The following describes NWDAF network element 1031, APSF network element 1032, and NRF network element 1033 in detail.

[0134] NWDAF element 1031 is used to implement network data collection, training, analysis, and inference functions. Specifically, NWDAF element 1031 can collect data from different data sources and analyze the collected data based on a built-in model to provide output results. These output results can optimize network performance and provide decision support. For example, the network can select service quality (QoS) parameters, assist in traffic routing, and assist in selecting background data transmission strategies based on the output results. The different data sources mentioned above include other network elements of the core network 103, user equipment 101, third-party service servers, and network management systems.

[0135] The NWDAF network element 1031 includes a model training logical function (MTLF) and an analytics logical function (AnLF). The MTLF provides a server for training machine learning models based on customer requests, including collecting data from data sources, preprocessing data, selecting algorithms for model training, and evaluating and optimizing model performance. The AnLF performs inference analysis on network data based on the model trained by the MTLF and generates output results. The MTLF and AnLF work together to achieve the functions of the NWDAF network element 1031.

[0136] The APSF network element 1032 is used for network performance evaluation based on AI models and outputs evaluation results. The evaluation results output by the APSF network element 1032 include verification results, the impact of the output results, evaluation information from the simulation system, and auxiliary information for optimizing the output results.

[0137] The APSF network element 1032 can also collect data from the existing network, enabling the simulation environment to synchronize with the real network. For example, the APSF network element 1032 can collect network function load information, network function context, network function interaction information, and service information. Network function load information includes resource utilization (e.g., CPU and memory utilization) of the network function; network function context includes environmental, conditional, or status information related to the network function; network function interaction information includes ongoing requests or requests; and service information includes planned service data transmission.

[0138] APSF network element 1032 can also perform simulations of the output results of NWDAF network element 1031 to obtain verification results, such as the service average opinion score. APSF network element 1032 can also evaluate the effectiveness of the output results of NWDAF network element 1031 to obtain simulation impacts, such as the impact of using the output results of NWDAF network element 1031 on network functional load.

[0139] APSF network element 1032 can also perform slice evaluation of the simulation system itself. For example, APSF network element 1032 can output simulation accuracy and simulation events during the evaluation process. APSF network element 1032 can also provide simulation effect feedback to MTLF of NWDAF network element 1031, and report simulation data to NWDAF network element 1031 through the message frame adapter function of MFAF network element.

[0140] NRF element 1033 provides network element registration and discovery functions. For example, NRF element 1033 can store registration information of APSF element 1032, including the evaluation types and capabilities supported by APSF element 1032. NRF element 1033 can also receive discovery requests from NWDAF element 1031 and determine one or more APSF element 1032 that meet the requirements based on the discovery request and the registration information stored in NRF element 1033. NRF element 1033 can also send a discovery response to NWDAF element 1031, which can indicate the one or more APSF element 1032 that meet the requirements.

[0141] Please see Figure 1b , Figure 1b This is a schematic diagram of the system architecture of a network system according to an embodiment of this application. Figure 1b In the network system architecture shown, Figure 1a The core network 103 of the network system described herein is, for example, a 5G core network. In addition to NWDAF network element 1031 and APSF network element 1032, the 5G core network may also include one or more of the following network elements, which are described in detail below:

[0142] The Network Exposure Function (NEF) element manages the network data exposed to the outside world. External applications can access data within the core network through the NEF element. The NEF element also provides functions such as enabling external applications to customize their Quality of Service (QoS), subscribing to mobility state events, and distributing AF requests.

[0143] Network repository function (NRF) elements are used for network function (NF) registration, management, and status monitoring, enabling automated NF management. Each NF must register with the NRF element upon startup to provide services; registration information includes NF type, address, and service list.

[0144] Application function (AF) network elements include various services at the application layer. AF network elements can be internal applications of the operator, such as VoLTE AF, or third-party AFs, such as video servers and game servers.

[0145] Policy control function (PCF) network elements are used to implement flow-based billing, QoS control, policy enforcement, and other functions. PCF network elements can also formulate and enforce policies based on factors such as users, services, and time.

[0146] The Unified Data Repository (UDR) network element is a core component for 5G network data storage and management, providing storage capabilities for contracted data, policy data, and capability-related data.

[0147] User data management (UDM) network elements are used to store and manage user subscription data, authentication information, and authorization data. UDM network elements are the core network elements for user data management in 5G networks.

[0148] Operations, administration, and maintenance (OAM) network elements are used to monitor, manage, and maintain the normal operation of the network. OAM network element functions include performance monitoring, fault management, and configuration management.

[0149] The Access and Mobility Management Function (AMF) network element handles user equipment access requests, mobility management, location updates, and other functions. The AMF network element connects to the Radio Access Network (RAN) via the N2 interface. The AMF network element ensures that user equipment can access the network and maintains its mobility and connectivity status.

[0150] The Session Management Function (SMF) network element is used to manage and control sessions between user equipment. The SMF network element enables communication between user equipment by controlling the establishment, maintenance, and release of sessions.

[0151] User plane function (UPF) network elements are used for routing and forwarding user data packets, data interaction with the external data network (DN), and QoS processing of the user plane. UPF network elements connect to the radio access network (RAN) via the N3 interface, to the SMF network elements via the N4 interface, and to the DN network elements via the N6 interface.

[0152] Data networks (DNs) include carrier services, the internet, and third-party services. DN network elements provide network services and application resources, enabling users to access and use various applications through the network.

[0153] Please see Figure 1c , Figure 1c This is a schematic diagram of the system architecture of another network data analysis system provided in an embodiment of this application. Figure 1c In the example shown, APSF is the core module that provides the evaluation function in the embodiment of this application. APSF can be an independent network element or a functional instance integrated into other network elements, and there is no specific limitation.

[0154] exist Figure 1c In the example shown, when performing data analysis, the NWDAF network element can collect data from different data sources or historical data blocks through the Data Collection Coordination Function (DCCF) and generate analysis results based on the collected data. The APSF can receive the analysis results sent by the NWDAF network element and evaluate them. The APSF can also provide feedback on the evaluation results to the NWDAF network element, and provide feedback on the evaluation data to the NWDAF network element through the Message Framework Adapter Function (MFAF). After generating the evaluation results, the APSF can send the evaluation results to user equipment, access network equipment, and customer network elements in the core network.

[0155] based on Figure 1aThe network data analysis system 10 shown in this application also provides a method for discovering evaluation functions. The method for discovering evaluation functions provided in this application will be described below with reference to embodiments.

[0156] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for discovering an evaluation function provided in an embodiment of this application. It should be noted that this application... Figure 2 The embodiments shown are general examples. Figures 3 to 8 The illustrated embodiments provide a detailed description of the discovery method for the evaluation function in the embodiments of this application. The following first introduces... Figure 2 The illustrated embodiment, in Figure 2 In the example shown, the method includes the following steps:

[0157] 201. The third network element sends a registration request to the second network element. The registration request is used to request the registration of the evaluation function in the second network element.

[0158] The third network element sends a registration request to the second network element. The registration request is used to request the registration of the evaluation function in the second network element. The registration request includes the registration information of the third network element. The registration information is used by the first network element to discover the target network element. The target network element can evaluate the analysis results of the first network element and feed back the evaluation results to the first network element.

[0159] It should be noted that in this application embodiment, the first network element includes network elements with network analysis functions such as the NWDAF network element, the second network element includes network elements with network storage functions such as the NRF network element, and the third network element includes network elements with evaluation functions such as the APSF network element. In the example of step 201, the APSF network element 1032 sends a registration request to the NRF network element 1033 to register the evaluation function in the NRF network element 1033. This evaluation function includes, for example, simulation type and simulation verification capability. The NWDAF network element 1031 can discover the target APSF from the APSF network element 1032 registered by the NRF network element 1033, and the target APSF evaluates the analysis results of the NWDAF network element 1031.

[0160] In this embodiment of the application, the registration request may or may not carry specific registration information. When the registration request carries registration information, the registration information that may be carried in the registration request may include one or more of the following: assessment type, assessment capability, and assessment capability range.

[0161] It should be noted that, due to the different content carried in the registration request in the embodiments of this application, the corresponding methods for the first network element to discover the target network element are also different. Figures 3 to 8 The examples shown will describe several scenarios of registration requests in detail, but will not be described in detail here.

[0162] 202. The second network element sends a registration response to the third network element. The registration response is used to indicate whether the third network element has successfully registered.

[0163] The second network element sends a registration response to the third network element. The registration response includes the registration result of the evaluation function, which may be either successful or unsuccessful. For example, APSF network element 1032 sends a registration response to NRF network element 1033, indicating that the evaluation function of APSF network element 1032 has been successfully registered.

[0164] 203. The first network element sends a discovery request to the second network element. The discovery request is used to request the discovery of network elements in the second network element that support the evaluation requirements of the first network element.

[0165] The second network element receives a discovery request sent by the first network element. The discovery request is used to request the discovery of network elements in the second network element that support the evaluation requirements of the first network element. The second network element is used to store the registration information of the evaluation function. For example, NWDAF network element 1031 sends a discovery request to NRF network element 1033. This discovery request is used to request the discovery of a target APSF that meets the requirements.

[0166] In this application embodiment, the discovery request may or may not carry indication information. When the discovery request carries indication information, the indication information may include one or more of the following: first indication information, second indication information, and third indication information, wherein the first indication information is used to indicate the required assessment type, the second indication information indicates the required assessment capability, and the third indication information is used to indicate the range of assessment capabilities.

[0167] It should be noted that, according to the embodiments of this application, the method for determining the target network element differs depending on the content carried in the request. Figures 3 to 8 The embodiments shown will describe several scenarios of discovery requests in detail, which will not be elaborated here.

[0168] 204. The second network element sends a discovery response to the first network element. The discovery response is used to indicate the information of the candidate network element. The candidate network element supports the evaluation requirements of the first network element and is used to perform the evaluation task.

[0169] The second network element sends a discovery response to the first network element. The discovery response is used to indicate the information of the candidate network element. The candidate network element supports the evaluation requirements of the first network element. The candidate network element is used to determine the target network element. The target network element is used to perform the evaluation task.

[0170] After receiving a candidate network element, the first network element can directly select the target network element from the candidate network elements, or the first network element and the third network element can negotiate and evaluate the capabilities before determining the target network element from the candidate network elements. There is no specific limitation.

[0171] 205. The first network element and the third network element negotiate to determine the target network element.

[0172] Step 205 is optional. If the first network element can directly determine the target network element from the candidate network elements in step 204, then step 205 is not required. If the first network element cannot directly determine the target network element in step 204, then the first network element and the candidate network elements further negotiate to determine the target network element.

[0173] This application Figures 3 to 8 The embodiments shown detail several schemes for the first network element to determine the target network element, which will not be described in detail here.

[0174] The embodiments of this application are described in detail below. Figures 3 to 8 The example shown.

[0175] Please see Figure 3 , Figure 3 This is a flowchart illustrating another method for discovering evaluation functions provided in an embodiment of this application. Figure 3 In the method for discovering the evaluation function shown, the first network element is the NWDAF network element, the second network element is the NRF network element, and the third network element is the APSF network element. Figure 3 In the example shown, the APSF network element has registered its assessment type and assessment capabilities with the NRF network element. The NWDAF network element can directly discover the target APSF within the NRF network element without needing to negotiate assessment capabilities. Details are provided below:

[0176] exist Figure 3 In the illustrated embodiments, the method for discovering the evaluation function provided in this application includes the following steps:

[0177] 301. The third network element sends a first registration request to the second network element. The first registration request includes assessment type and assessment capability information.

[0178] The third network element sends a first registration request to the second network element. The registration information in the first registration request includes one or more of the following: assessment type and assessment capability information. The assessment capability information includes assessment capability or assessment capability range. The assessment type indicates the assessment type supported by the third network element. The assessment capability indicates the specific assessment capability value supported by the third network element. The assessment capability range indicates the range of assessment capability values ​​supported by the third network element. The assessment type and assessment capability information are described in detail below:

[0179] The assessment type can indicate one or more of the following: parameter type, scenario type, and service type. Parameter type includes parameter types that the third-party network element supports for simulation verification. For example, a parameter type of QoS verification indicates support for QoS parameter simulation verification. Another example is backoff timer verification, which indicates support for backoff timer parameter simulation verification. Scenario type includes scenario types that the third-party network element supports for simulation verification, such as shopping malls, parks, schools, and suburbs. Service type includes service types that the third-party network element supports for simulation verification, such as extended reality services and federated learning services.

[0180] The evaluation capability includes one or more of the following: supported simulation filter, supported simulation time interval, supported simulation granularity, supported simulation response time, and supported simulation reliability.

[0181] The supported simulation filtering information can represent the supported simulation conditions, such as the area of ​​interest (AoI), network slice selection aid information (S-NSSAI), data network name (DNN), and radio access technology type. When the simulation filtering information supported by APSF is a supported AoI, it means that APSF can provide simulation results within the area corresponding to that AoI.

[0182] When the simulation filtering information supported by APSF is network slice selection assistance information S-NSSAI, it means that APSF can provide simulation results for a specified network slice. For example, if the network slice A indicated by the network slice selection assistance information S-NSSAI supported by APSF is a network slice that provides high-speed data transmission services, then APSF provides the simulation results for that network slice that provides high-speed data transmission services.

[0183] When APSF supports simulation filtering information for a network data name, it means that APSF can provide simulation results for the specified network data name. For example, if APSF supports a data network name DNN1, which is an Internet communication network, it means that APSF supports performance simulation for that Internet communication network.

[0184] When the simulation filtering information supported by APSF is the radio access technology type, it means that APSF can provide simulation results for the specified radio access technology type. For example, if the radio access technology type supported by APSF is Global System for Mobile Communications (GSM), it means that APSF supports performance simulation for the GSM radio access technology type.

[0185] The supported simulation time period refers to the time period during which APSF can perform simulations. For example, the supported simulation time period is 18:00-20:00 every day.

[0186] Supported simulation granularities include, for example, slice granularity, network function (NF) granularity, user equipment (UE) granularity, protocol data unit (PDU) session granularity, quality of service flow (QoS flow) granularity, location granularity, time granularity, or a combination of the above granularities. Location granularity includes, for example, tracking area (TA), cell, and fine granularity location, while time granularity includes, for example, minutes or seconds.

[0187] When APSF supports slice-level simulation, it means that APSF supports providing slice-level simulation verification. For example, network slices include low-latency slices and high-bandwidth slices, and APSF can provide slice-level simulation verification for each type of slice separately.

[0188] When APSF supports simulation at the network function (NF) granularity, it means that APSF supports simulation verification of a specified network function. For example, APSF supports simulation verification at the network function granularity for User Plane Function (UPF) and Session Management Function (SMF), respectively.

[0189] When APSF supports UE-level simulation, it means that APSF supports simulation verification for a specified UE. For example, since different UEs have different communication characteristics, APSF supports UE-level simulation verification for different UEs, such as smartphones and IoT devices.

[0190] When APSF supports simulation at the Protocol Data Unit (PDU) session granularity, it means that APSF supports simulation verification of a specified PDU session. Since different applications have PDU sessions with different characteristics, for example, APSF supports performance simulation at the PDU session granularity for PDU sessions in high-definition video applications and PDU sessions in real-time control applications, respectively.

[0191] When APSF supports simulation at the QoS flow granularity, it means that APSF supports simulation verification of a specified QoS flow. Since different services correspond to different QoS flows, for example, APSF supports performance simulation at the QoS flow granularity for gaming services and video services, respectively.

[0192] When APSF supports simulation at a specific location granularity, it means that APSF supports location-granularity simulation verification for different locations. For example, APSF supports cell-level simulation verification, or APSF supports TA-level simulation verification.

[0193] When APSF supports simulation at a specific time granularity, it means that APSF supports simulation verification at different time granularities. For example, APSF supports simulation verification at the minute granularity, or APSF supports simulation verification at the hour granularity.

[0194] Supported simulation response time indicates the time requirement for APSF from receiving a simulation verification request to providing simulation verification results. For example, supported simulation response time can be expressed as maximum response time, minimum response time, average response time, etc.

[0195] For example, if the maximum response time supported by APSF is 5 minutes, it means that APSF will provide the simulation verification result within 5 minutes at the latest after receiving the simulation verification request.

[0196] For example, if the minimum response time supported by APSF is 1 minute, it means that APSF will provide the simulation verification result within 1 minute at the fastest after receiving the simulation verification request.

[0197] For example, if the average response time supported by APSF is 3 minutes, it means that APSF will provide simulation verification results on average within 3 minutes after receiving the simulation verification request.

[0198] Supported simulation reliability is used to measure the reliability of a simulation system or simulation results. For example, supported simulation reliability can be expressed as the confidence level, minimum confidence level, maximum confidence level, average confidence level, etc., of the simulation results that can be provided. For instance, when the supported simulation reliability of APSF indicates a confidence level of 90%, it means that the simulation results provided by APSF are 90% likely to be correct.

[0199] For example, when the simulation reliability indicator supported by APSF has a minimum confidence level of 80%, it means that the probability of the simulation results provided by APSF being correct is greater than or equal to 80%.

[0200] For example, when the simulation reliability indicator supported by APSF has a maximum confidence level of 98%, it means that the probability of the simulation results provided by APSF being correct is less than or equal to 98%.

[0201] For example, when the simulation reliability indicator supported by APSF has an average confidence level of 95%, it means that the average probability of the simulation results provided by APSF being correct is 95%.

[0202] In one possible implementation, the third network element is an AI performance simulation function (APSF) network element. The aforementioned evaluation type can be a simulation ID, and the aforementioned evaluation capability information can be simulation capability information. The simulation capability information can be at the simulation ID level, meaning that each simulation ID has corresponding simulation capability information. Different simulation IDs can correspond to the same simulation capability or different simulation capabilities; the specifics are not limited.

[0203] For example, in one instance of step 301, the APSF network element sends a first registration request to the NRF network element. This first registration request is used to register the APSF network element's simulation verification capabilities. The first registration request includes a simulation identifier and simulation capability information. The simulation identifier indicates one or more of the following: the parameter types supported by APSF for simulation verification, the scenario types supported by APSF for simulation verification, and the service types supported by APSF for simulation verification. The simulation capability information includes one or more of the following: supported simulation filtering information, supported simulation time periods, supported simulation granularity, supported simulation response time, and supported simulation capabilities.

[0204] 302. The second network element sends a first registration response to the third network element. The first registration response includes the registration result.

[0205] After receiving the first registration request, the second network element generates a first registration response and sends the first registration response to the third network element. The first registration response includes the registration result of the evaluation function, which may be either successful or unsuccessful.

[0206] For example, in one instance of step 302, the NRF network element sends a first registration response to the APSF network element. The first registration response indicates the registration result of the APSF network element's simulation verification capability. The registration result includes registration success and registration failure.

[0207] 303. The customer network element sends a subscription request to the first network element. The subscription request is used to subscribe to the analysis results of the data analysis.

[0208] The first network element receives a subscription request from the client network element. This subscription request is used to request subscription to the analysis results generated by the first network element based on the collected data. Optionally, the subscription request may carry an analytics ID, which can be used to identify the type of analysis results that the client network element needs to subscribe to. This subscription analysis result is the analysis result of the first network element. For example, if the analytics ID is "Service Experience," it means that the requested result is a service experience analysis result.

[0209] Among them, the aforementioned customer network element can also be called consumer network element. The customer network element can be a network element in user equipment 101, access network 102 or core network 103. For example, the customer network element can be a PCF network element or NF network element, etc., without specific limitations.

[0210] 304. The first network element generates analysis results based on the collected data, which includes data from one or more data sources.

[0211] The first network element collects data from one or more data sources and generates analysis results based on the collected data. Specifically, the first network element inputs the collected data into a pre-trained built-in analysis model, which then performs analysis and processing to obtain the results. For example, the analysis logic function AnLF of the NWDAF network element performs model inference on the collected data based on the model provided by the model training logic function MTLF to obtain the analysis results.

[0212] For example, the data collected by the first network element includes network performance data, user data, and service experience data. Among them, network performance data includes reference signal received power (RSRP), reference signal received quality (RSRQ), bit rate, and latency, while user data includes user equipment location information, and service experience data includes mean opinions score (MOS).

[0213] It should be noted that the data source for the data collected in this application embodiment can be a network element in the core network, or a user equipment 101, access network 102, third-party service server and gateway system, etc., and there is no specific limitation.

[0214] Correspondingly, the analysis results of the first network element include network performance analysis results, user experience analysis results, prediction and decision-making suggestions, etc. Network performance analysis results include throughput analysis results, user experience analysis results include recommended QoS parameters, and prediction and decision-making suggestions include traffic prediction results and network load prediction results.

[0215] 305. The first network element triggers an evaluation of the analysis results.

[0216] After the first network element generates the analysis results, an evaluation of the analysis results is triggered. For example, the confidence level of the analysis results generated by the first network element is lower than a threshold; or the first network element is configured to perform simulation verification for specific tasks (e.g., perform simulation verification for all recommended QoS parameters, or perform simulation verification for all recommendation-related outputs).

[0217] 306. The first network element sends a first discovery request to the second network element. The first discovery request is used to request the discovery of the first candidate network element.

[0218] The first network element sends a first discovery request to the second network element. The first discovery request includes a first instruction information and a second instruction information. The first instruction information is used to indicate the required assessment type, and the second instruction information is used to indicate the required assessment capability.

[0219] After receiving the first discovery request, the second network element determines the first candidate network element based on the first discovery request. The evaluation type of the first candidate network element supports the required evaluation type, and the evaluation capability information of the first candidate network element supports the required evaluation capability information.

[0220] Specifically, since the evaluation type and evaluation capability information of the third network element are registered in the first registration request in step 301, the second network element selects the third network element whose evaluation type and evaluation capability information both meet the requirements of the first discovery request from the registered third network elements based on the evaluation type and evaluation capability information required in the first discovery request, and determines the qualified third network element as the first candidate network element. The qualified third network element is the third network element whose registered evaluation type meets the first indication information and whose registered evaluation capability information meets the second indication information.

[0221] For example, the NRF network element receives a first discovery request sent by the NWDAF network element. The evaluation type indicated in the first discovery request is type A, and the evaluation capability information indicated in the first discovery request is capability information A. Here, type A is, for example, the QoS parameter type, and capability information A is, for example, the simulation reliability supported by APSF.

[0222] Based on the first discovery request, the NRF network element determines eligible APSF network elements from the already registered APSF network elements and designates the eligible APSFs as first candidate APSF network elements. Eligible APSF network elements are those whose registered assessment types and capabilities match the assessment types and capabilities required in the first discovery request. For example, the NRF network element discovers three APSF registrations: APSF1 supports assessment types A and B, and its supported assessment capabilities include capability information A and capability information B; APSF2 supports assessment types A, and its supported assessment capabilities include capability information B and capability information C; APSF3 supports assessment types A and B, and its supported assessment capabilities include capability information A and capability information B. The NRF network element then designates APSF1 and APSF3 as first candidate network elements based on the assessment type A and assessment capability information A indicated in the first discovery request.

[0223] 307. The second network element sends a first discovery response to the first network element, the first discovery response including the first candidate network element.

[0224] The second network element sends a first discovery response to the first network element. The first discovery response includes information about the first candidate network element. This information may include, for example, the network element type (NF type), the network element instance identifier (NF instanceID), and the fully qualified domain name (FQDN) or IP address of the network element instance.

[0225] 308. The first network element determines the target network element based on the first candidate network element.

[0226] After receiving the first discovery response, the first network element determines the target network element from one or more first candidate network elements, and the target network element is used to perform the evaluation task.

[0227] 309. The first network element requests the target network element to evaluate the analysis results and obtains the evaluation results.

[0228] The first network element requests the target network element to evaluate the analysis results and obtain the evaluation results. Specifically, the first network element sends an evaluation request to the target network element, such as an APSF network element. The APSF network element can provide performance simulation verification capabilities based on artificial intelligence (AI) and generate evaluation results corresponding to the analysis results of the first network element.

[0229] In one possible implementation, the evaluation request sent by the first network element to the target network element includes one or more of the following: evaluation type, evaluation capability, assistance info reporting indication information, results to be verified, and verification goals.

[0230] The following sections describe the contents of the evaluation request:

[0231] The evaluation type refers to the requested evaluation type, including simulation verification type, for example, the evaluation type is QoS parameter simulation verification.

[0232] Evaluation capability refers to the evaluation capability of the requested target network element. For example, evaluation capability includes simulation filter information, simulation time interval, simulation granularity, simulation response time, and simulation reliability.

[0233] The auxiliary information reporting instruction can indicate the content of the requested auxiliary information, such as the accuracy of analysis results, model accuracy, optimized analysis results, and model fine-tuning data.

[0234] The analysis result to be verified indicates the analysis result that is requested to be verified. For example, when the simulation verification type indicated by the requested evaluation type is QoS Verification, the analysis result to be verified is the recommended QoS parameters.

[0235] The verification target indicates the requested verification objective. For example, when the simulation verification type indicated by the evaluation type is QoSVerification, the verification target is the mean opinion score (MOS). The verification target can also indicate the value of the verification objective to be achieved, such as a MOS greater than 4.

[0236] In one possible implementation, the evaluation results generated by the target network element include one or more of the following: verification results, simulated impacts, evaluation results of the simulation system, and auxiliary information.

[0237] The following sections describe the contents of the evaluation results:

[0238] The verification result is the verification result corresponding to the analysis result to be verified, or information on whether the analysis result to be verified meets the verification objective. For example, when the analysis result to be verified is a recommended service quality parameter, and the verification objective is MOS>4, the verification result can be the service mean opinion score, or information indicating whether the verification objective has been achieved (i.e., whether the service mean opinion score meets the indication that MOS>4).

[0239] The impact of analysis results refers to the effect of the analysis results of the first network element on the network. For example, the PCF requests the QoS parameters of application A in area A and time period [t1,t2] from the NWDAF. The NWDAF generates analysis results, i.e., candidate QoS parameters. The PCF sets the QoS parameters according to the analysis results and executes them. The possible results after execution, such as changes in the load of the UPF and the degree of network congestion, are the impact of the analysis results.

[0240] The evaluation results of the simulation system include assessments of the target network element's confidence in the network simulation results, simulation time, and computational cost. For example, when the analysis result is the QoS parameters of application A in region A and time period [t1, t2], and the simulation result of APSF is the service experience predicted using these QoS parameters (e.g., MOS = 4), the evaluation results of the simulation system could be, for example, the confidence in the simulation results (e.g., the probability that the predicted result MOS = 4 matches the actual result is 90%), the simulation time (the time from the start to the end of the simulation, e.g., 10 minutes), or the computational cost (e.g., the computational cost required for this simulation is 1 million FLOPS (Floating-point Operations Per Second)).

[0241] For example, if the analysis result is that the average load rate of base station A is predicted to be 70% and the load change rate is 10% in the future time period [t1,t2], and the simulation result of APSF is that the average load rate of base station A is predicted to be 72% and the load change rate is 8% in the future time period [t1,t2], the evaluation result of the simulation system can be, for example, the confidence level of the simulation result, such as the probability that the predicted average load rate of 72% matches the actual result is 90%, the simulation time is, for example, 20 minutes, and the computing power cost of the simulation result is, for example, 1 million floating-point operations per second (FLOPS).

[0242] Supporting information includes one or more of the following: accuracy of analysis results / model, optimized analysis results, and model fine-tuning data.

[0243] The accuracy of the analysis result / model refers to the accuracy of the analysis result / model of the target network element in the simulation system. For example, if the analysis result of the NWDAF network element is the recommended QoS, the error between the actual service mean opinion score (MOS) and the simulated service mean opinion score (SMS) that can be achieved after using the recommended QoS in the APSF verification is used as the accuracy of the analysis result, or the model accuracy of the NWDAF network element.

[0244] The optimized analysis results can be those generated by APSF that meet the verification objectives. For example, if the verification objective in the evaluation request is a MOS greater than 4, the optimized analysis results could be optimized QoS parameters that satisfy a MOS greater than 4. If APSF verification fails to achieve the target MOS after using the recommended QoS, APSF can independently adjust the QoS parameter values ​​and verify whether the adjusted QoS parameters can achieve the target MOS. Based on this, it determines a set of QoS parameters that can achieve the target MOS, and then directly feeds back this set of QoS parameters as the updated analysis results to the first network element.

[0245] The model fine-tuning data can be model fine-tuning data generated and recommended by APSF, used to assist the first network element in model fine-tuning. For example, when APSF verifies that a customer network element cannot achieve the verification target using recommended QoS parameters, APSF generates model fine-tuning data. After synchronizing with real network parameters, APSF can set different QoS parameters and collect network data under different QoS parameter settings. This network data includes data such as reference signal received power (RSRP), reference signal received quality (RSRQ), QoS flow bit rate, QoS flow packet delay, and packet transmission / retransmission, as well as corresponding service experience values. APSF generates model training fine-tuning data under the current network state and feeds it back to the first network element to assist it in fine-tuning the model based on this data.

[0246] In one possible implementation, during the process of generating evaluation results, after receiving the evaluation request sent by the first network element, the target network element collects network data to synchronize the simulation system parameters with the real network. Then, the target network element performs simulation verification based on the evaluation request, generates evaluation results, and sends the evaluation results to the first network element.

[0247] For example, a target network element can collect network element information, network function interaction information, and service information for each network element. Network element information includes things like network function load (NF load) and network function context (NF context); network function interaction information includes things like ongoing request or subscription operations; and service information includes things like planned service data transmission. It should be noted that the target network element can collect network data periodically or based on demand triggers; there are no specific limitations.

[0248] In one example of step 309, the NWDAF network element sends an evaluation request to the target APSF network element. This evaluation request requests the target APSF network element to evaluate the analysis results of the NWDAF network element and generate evaluation results. The evaluation request carries one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, analysis results to be verified, and verification target. The evaluation results include one or more of the following: verification results, impact of analysis results, simulation system evaluation results, and auxiliary information for updating analysis results. Subsequently, the target APSF network element sends an evaluation response to the NWDAF network element, which includes the aforementioned evaluation results.

[0249] 310. The first network element updates the analysis results based on the evaluation results.

[0250] After receiving the evaluation results, the first network element updates its analysis results based on those results. Specifically, the first network element can retrain its built-in model based on the evaluation results and output updated analysis results based on the trained model. Alternatively, the first network element can fine-tune its built-in model based on the model fine-tuning data in the evaluation results and output updated analysis results based on the fine-tuned model. It is understandable that the first network element can also directly use the optimized analysis results from the evaluation results as updated analysis results; there are no specific limitations on this.

[0251] For example, the evaluation result received by the first network element includes auxiliary information. When the auxiliary information is the accuracy of the analysis result, if the first network element determines that the accuracy of the analysis result does not meet the accuracy requirements, the first network element retrains or updates the model and performs inference based on the retrained or updated model to obtain the updated analysis result.

[0252] For example, when the auxiliary information is an optimized analysis result, the first network element can directly use this optimized analysis result as its own analysis result. For another example, when the auxiliary information is model fine-tuning data, the first network element performs model fine-tuning based on the model fine-tuning data and performs inference based on the fine-tuned model.

[0253] In one example of step 310, after the NWDAF network element receives the evaluation result sent by the target APSF, it updates the analysis result based on the evaluation result. The process of the NWDAF network element updating the analysis result includes, for example, re-collecting data to train the model of the NWDAF network element, fine-tuning the model of the NWDAF network element, and using the simulation analysis result generated by the APSF as the analysis result of the NWDAF network element.

[0254] 311. The first network element sends a subscription response to the customer network element. The subscription response includes the updated analysis results.

[0255] The first network element sends a subscription response to the client network element, which includes the updated analysis results. For example, the NWDAF network element sends a subscription response to the client network element, which includes the updated recommended QoS parameters.

[0256] from Figure 3 As can be seen in the embodiments shown, Figure 3 The illustrated embodiment can leverage the evaluation capabilities provided by the APSF network element to enable the NWDAF network element to discover a target APSF that supports specific evaluation capabilities, and to perform evaluation and verification of the analysis results with the help of the APSF network element, thereby improving the accuracy and reliability of the analysis results.

[0257] In this embodiment of the application, the third network element can register the evaluation type and evaluation capability in the second network element, so that the first network element can directly discover the target network element based on the registered evaluation type and evaluation capability, and evaluate the analysis results of the first network element based on the target network element, thereby improving the accuracy and reliability of the analysis results of the first network element.

[0258] Please see Figure 4 , Figure 4 This is a flowchart illustrating another method for discovering evaluation functions provided in an embodiment of this application. Figure 4 In the method for discovering the evaluation function shown, the first network element is the NWDAF network element, the second network element is the NRF network element, and the third network element is the APSF network element. Figure 4 In the example shown, APSF network elements can register assessment types and capabilities with NRF network elements, or they can choose not to register specific assessment types and capabilities. When an NWDAF network element discovers a target APSF in an NRF network element, it first discovers a second candidate network element, then determines the target network element from the second candidate network element, and the first network element specifies the assessment capabilities that the target network element needs to achieve. The target network element then constructs the assessment capabilities. Details are as follows:

[0259] exist Figure 4 As shown in the example, the method for discovering the evaluation function in this application embodiment includes the following steps:

[0260] 401. The third network element sends a second registration request to the second network element. The second registration request does not carry registration information or carries one or more of the following registration information: assessment type and assessment capability.

[0261] A third network element sends a second registration request to a second network element. This second registration request may not carry registration information, or it may carry one or more of the following registration information: assessment type and assessment capabilities. For example, an APSF network element sends a second registration request to an NRF network element. This second registration request is used to register the APSF network element's assessment function with the NRF network element. When the second registration request does not carry registration information, meaning the APSF network element has not registered a specific assessment type and assessment capabilities, the APSF network element can construct assessment capabilities in real time during the discovery and negotiation process of the NWDAF network element. This application... Figure 4 The specific content of the assessment type and the specific content of the assessment capability in the illustrated embodiments are the same as those in the examples. Figure 3 The content of the assessment type and the assessment capability are the same in the embodiments described above, and will not be repeated here.

[0262] 402. The second network element sends a second registration response to the third network element. The second registration response includes the registration result, i.e., an indication of whether the registration was successful.

[0263] 403. The customer network element sends a subscription request to the first network element. The subscription request is used to subscribe to the analysis results of the data analysis.

[0264] 404. The first network element generates analysis results based on the collected data, which includes data from one or more data sources.

[0265] 405. The first network element prepares to trigger the target network element in the third network element to evaluate the analysis results.

[0266] The method performed by the network element in steps 402 to 405 of this embodiment is the same as described above. Figure 3 The methods performed by the network element in steps 302 and 305 of the illustrated embodiment are similar and will not be described again here.

[0267] 406. The first network element sends a second discovery request to the second network element. The first discovery request is used to request the discovery of the second candidate network element.

[0268] The first network element sends a second discovery request to the second network element. The second discovery request may not carry any indication information, or it may carry one or more of the following indication information: first indication information and second indication information, wherein the first indication information is used to indicate the required assessment type, and the second indication information is used to indicate the required assessment capability.

[0269] After receiving the second discovery request, the second network element determines the second candidate network element based on the second discovery request. Specifically, when the second discovery request does not carry indication information, any third network element that has registered the evaluation function in the second network element can be determined as the second candidate network element. That is, the second candidate network element includes one or more network elements that have registered the evaluation function in the second network element.

[0270] For example, after receiving a second discovery request from an NWDAF network element, the NRF network element determines eligible APSF network elements from among the registered APSF network elements based on this second discovery request. For instance, the NRF network element discovers four APSF registrations: APSF1 supports assessment types A and B, and supports assessment capability information A and B; APSF2 supports assessment type A, and supports assessment capability information B and C; APSF3 supports assessment types A and B, and supports assessment capability information A and B; and APSF4 has not registered specific assessment capability information. In this case, if the second discovery request does not carry indication information (i.e., it does not indicate the required assessment type and capability), the NRF network element can determine all registered APSF network elements as second candidate network elements; that is, the NRF network element can determine APSF1 through APSF4 as second candidate network elements.

[0271] When the second discovery request includes the required assessment type and the required assessment capability, the second network element selects a second candidate network element based on the required assessment type and the required assessment capability. Since the second registration request in step 401 may or may not carry registration information, if the third network element does not carry registration information during registration, any third network element that has not registered specific assessment capability information can be identified as a second candidate network element.

[0272] When a third network element registers with registration information, that is, when the third network element registers for an assessment type and assessment capability, the second network element determines the second candidate network element according to the second discovery request. The third network element that meets the requirements of the second discovery request is then identified as the second candidate network element. In other words, the assessment type of the second candidate network element supports the required assessment type, and the assessment capability of the second candidate network element supports the required assessment capability.

[0273] For example, after receiving a second discovery request from an NWDAF network element, the NRF network element determines eligible APSF network elements from the registered APSF network elements based on the second discovery request, and designates these eligible APSFs as second candidate APSF network elements. These eligible second candidate APSF network elements are those whose registered assessment type matches the assessment type required in the second discovery request, and whose registered assessment capabilities match the assessment capabilities required in the second discovery request. In the above scenario where the NRF network element discovers four APSF registration information, if the assessment type indicated by the second discovery request is type A, and the indicated assessment capability information is capability information A, the NRF network element can designate APSF1, APSF3, and APSF4 as second candidate network elements.

[0274] 407. The second network element sends a second discovery response to the first network element, wherein the first discovery response is used to indicate one or more second candidate network elements.

[0275] After determining the second candidate network element based on the second discovery request, the second network element sends a second discovery response to the first network element. The second discovery response includes information about one or more second candidate network elements, which are used to determine the target network element. The information of the second candidate network element may include, for example, network element type (NF type), network element instance ID (NF instanceID), fully qualified domain name (FQDN) or IP address of the network element instance.

[0276] 408. The first network element determines the target network element from the second candidate network elements.

[0277] The first network element determines the target network element from one or more second candidate network elements. Specifically, the first network element can randomly determine the target network element from one or more second candidate network elements, or it can select the target network element from one or more second candidate network elements based on preset rules. The preset rules include selecting the target network element based on information from the second candidate network elements, and the specific selection is not limited.

[0278] 409. The first network element sends a first request to the target network element, the first request including the first expected assessment capability.

[0279] The first network element sends a first request to the target network element. The first request includes a first desired evaluation capability, which indicates the first network element's requirement for the target network element's evaluation capabilities. After receiving the first request, the target network element can construct a simulation evaluation capability and optionally feed back the constructed evaluation capability to the first network element.

[0280] It should be noted that in the embodiments of this application, the target network element can build the evaluation capability in real time. That is, the target network element does not necessarily build the evaluation capability after receiving the first request sent by the first network element, or it can build the evaluation capability before receiving the first request. The specifics are not limited.

[0281] In addition, when the target network element constructs the evaluation capability after receiving the first request, the first network element can instruct the target network element to construct the evaluation capability through the message of the first request itself, or through the parameters in the first request (i.e. the first expected evaluation capability), without any specific limitation.

[0282] 410. The target network element sends a first response to the first network element. The first response includes the established assessment capabilities.

[0283] The target network element receives a first response sent by the first network element. The first response includes the target network element's established evaluation capabilities. Optionally, the first response may not include the target network element's established evaluation capabilities; this is not specifically limited.

[0284] 411. The first network element requests the target network element to evaluate the analysis results and obtains the evaluation results.

[0285] 412. The first network element updates the analysis results based on the evaluation results.

[0286] 413. The first network element sends a subscription response to the customer network element. The subscription response includes the updated analysis results.

[0287] The method performed by the network element in steps 411 to 413 of this embodiment is the same as described above. Figure 3 The methods performed by the network element in steps 309 and 310 of the illustrated embodiment are similar and will not be described again here.

[0288] from Figure 4 As can be seen from the illustrated embodiments, APSF network elements can construct corresponding evaluation capabilities based on the first request, thereby improving the real-time performance and flexibility of APSF network elements in constructing evaluation capabilities. That is, in the embodiments of this application, the third network element may not register specific evaluation types and evaluation capabilities with the second network element during the evaluation function registration stage, while the third network element can construct evaluation capabilities in real time and negotiate the supported evaluation capabilities with the first network element. Figure 4 The illustrated embodiment improves the accuracy and reliability of the first network element analysis results while also enhancing the flexibility of the third network element construction and evaluation capabilities.

[0289] Please see Figure 5 , Figure 5 This is a flowchart illustrating another method for discovering evaluation functions provided in an embodiment of this application. Figure 5 In the method for discovering the evaluation function shown, the first network element is an NWDAF network element, the second network element is an NRF network element, and the third network element is an APSF network element. Figure 5 In the example shown, the APSF network element can register the assessment type and assessment capability range in the NRF network element. When the NWDAF network element discovers the target APSF in the NRF network element, it can first find a third candidate network element that meets the requirements based on the assessment capability range. Then, the first network element and the third candidate network element negotiate the actual assessment capability of the third candidate network element. Finally, the first network element selects the target network element from the third candidate network element. The following is a detailed introduction:

[0290] exist Figure 5 As shown in the example, the method for discovering the evaluation function in this application embodiment includes the following steps:

[0291] 501. The third network element sends a third registration request to the second network element. The registration information in the third registration request includes the assessment type and assessment capability range.

[0292] A third network element sends a third registration request to a second network element. The registration information in the third registration request includes the assessment type and the scope of assessment capabilities. For example, an APSF network element sends a third registration request to an NRF network element. The third registration request is used to register the assessment function of the APSF network element with the NRF network element. The third registration request includes the assessment types and the scope of assessment capabilities supported by the APSF network element.

[0293] This application Figure 5 The specific content of the evaluation type in the illustrated embodiment is the same as Figure 3 The evaluation types described in the embodiments are consistent and will not be repeated here.

[0294] This application Figure 5 The evaluation capability range in the illustrated embodiment refers to the range of evaluation capabilities. In this embodiment, the evaluation capability range includes one or more of the following: the range of supported simulation filtering information, the range of supported simulation time periods, the range of supported simulation granularity, the range of supported simulation response time, and the range of supported simulation reliability.

[0295] The supported simulation filtering information range can represent the range of supported simulation conditions, such as the range of Region of Interest (AoI), the range of Network Slice Selection Auxiliary Information (S-NSSAI), the range of Data Network Name (DNN), and the range of Wireless Access Technology (WAIT) types. When the range of simulation filtering information supported by APSF is the range of supported AoIs, it means that APSF can provide simulation results for multiple regions within that AoI range.

[0296] When the simulation filtering information supported by APSF is the network slice selection assistance information S-NSSAI range, it means that APSF can provide simulation results for the specified network slice range. For example, if the network slice selection assistance information S-NSSAI range supported by APSF includes network slices A to C, and network slices A to C are all network slices that provide high-speed data transmission services, then APSF will provide simulation results for network slices A to C.

[0297] When APSF supports simulation filtering information that specifies a range of network data names, it means that APSF can provide simulation results for the specified range of network data names. For example, if APSF supports a range of network names including network name DNN1 and network name DNN2, where network name DNN1 is, for example, an Internet communication network and network data name is, for example, a voice communication network, then APSF supports performance simulation for both network names DNN1 and DNN2.

[0298] When the simulation filtering information supported by APSF is a range of radio access technology types, it means that APSF can provide simulation results for the specified range of radio access technology types. For example, if the range of radio access technology types supported by APSF includes Global System for Mobile Communications (GMS) and Code Division Multiple Access (CDMA), it means that APSF supports performance simulation for GMS and CDMA types within that range of radio access technology types.

[0299] The supported simulation time period can be any of the multiple time periods that APSF can simulate, such as 18:00-20:00 from Monday to Friday.

[0300] Supported simulation granularity ranges include, for example, slice granularity range, NF granularity range, UE granularity range, PDU session granularity range, QoS flow granularity range, location granularity range, time granularity range, or combinations of the above granularity ranges. Among them, the location granularity range includes, for example, the tracking area (TA) range, the cell range, and the fine-grained location range, and the time granularity range includes, for example, the minute range or the second range.

[0301] When the simulation granularity range supported by APSF is the slice granularity range, for example, if the slice granularity range supported by APSF is 1 to 3, it means that APSF supports simulation verification at 1 to 3 network slice granularities.

[0302] When the simulation granularity range supported by APSF is the network function (NF) granularity range, for example, if the network function (NF) granularity range supported by APSF is 1 to 3, it means that APSF supports simulation verification at the granularity of 1 to 3 network function modules.

[0303] When the simulation granularity range supported by APSF is the user equipment (UE) granularity range, for example, when the UE granularity range supported by APSF is 1 to 3, it means that APSF supports simulation verification for providing 1 to 3 UE granularities.

[0304] When the simulation granularity range supported by APSF is the PDU session granularity range, for example, when the protocol data unit (PDU) session granularity range supported by APSF is 1 to 3, it means that APSF supports simulation verification of 1 to 3 PDU session granularities.

[0305] When the simulation granularity range supported by APSF is the QoS flow granularity range, for example, if the QoS flow granularity range supported by APSF is 1 to 3, it means that APSF supports simulation verification of 1 to 3 QoS flow granularities.

[0306] When the simulation granularity range supported by APSF is the location granularity range, for example, when APSF supports a location granularity range of 1 to 3 locations, it means that APSF supports simulation verification with a location granularity of 1 to 3 locations.

[0307] When the simulation granularity range supported by APSF is the time granularity range, and the time granularity range supported by APSF is 1 to 3 hours, it means that APSF supports simulation verification with a granularity of 1 to 3 hours.

[0308] The supported simulation response time range indicates the time required for APSF to provide simulation verification results from receiving the simulation verification request. For example, the supported simulation response time range can be expressed as a time range. For instance, if the APSF's supported simulation response time range indicates a response time range of 1 to 5 minutes, it means that APSF will provide simulation verification results within 1 to 5 minutes after receiving the simulation verification request.

[0309] The supported simulation reliability range is used to measure the reliability of a simulation system or simulation results. For example, the supported simulation reliability range can be expressed as the confidence range of the simulation results that can be provided. For instance, when the simulation reliability supported by APSF indicates that the confidence level of the simulation results is in the range of 90% to 95%, it means that the probability range of the simulation results provided by APSF being correct is 90% to 95%.

[0310] 502. The second network element sends a third registration response to the third network element, which includes the registration result.

[0311] 504. The customer network element sends a subscription request to the first network element. The subscription request is used to subscribe to the analysis results of the data analysis.

[0312] 504. The first network element generates analysis results based on the collected data, which includes data from one or more data sources.

[0313] 505. The first network element prepares to trigger the target network element in the third network element to evaluate the analysis results.

[0314] The method performed by the network element in steps 502 to 505 of this embodiment is the same as described above. Figure 3 The methods performed by the network element in steps 302 and 305 of the illustrated embodiment are similar and will not be described again here.

[0315] 506. The first network element sends a third discovery request to the second network element. The third discovery request is used to request the discovery of a third candidate network element.

[0316] The first network element sends a third discovery request to the second network element. The third discovery request includes a first instruction information and a second instruction information. The first instruction information is used to indicate the required assessment type, and the second instruction information is used to indicate the required assessment capability.

[0317] The second network element determines a third candidate network element based on the third discovery request. The evaluation type of the third candidate network element supports the required evaluation type, and the evaluation capability range of the third candidate network element includes the required evaluation capabilities. Specifically, since the third network element registered the evaluation type and evaluation capability range with the second network element in step 501, the second network element can select a third candidate network element based on the required evaluation type and required evaluation capabilities in the third discovery request. That is, the evaluation type of the selected third candidate network element supports the required evaluation type in the second discovery request, and the evaluation capability range of the selected third candidate network element includes the required evaluation capabilities in the second discovery request.

[0318] For example, after receiving a third discovery request from an NWDAF network element, the NRF network element determines the APSF network element that meets the requirements from the already registered APSF network elements based on the third discovery request, and identifies the APSF network element that meets the requirements as the third candidate APSF network element. The third candidate APSF network element that meets the requirements means that the registered evaluation type meets the evaluation type required in the third discovery request, and the registered evaluation capability range includes the APSF network element with the evaluation capability required in the third discovery request.

[0319] Specifically, for example, if an NRF network element discovers three APSF registration records, where APSF1 supports assessment types A and B, and its supported assessment capability ranges include capability ranges A and B; APSF2 supports assessment types A, and its supported assessment capability ranges include capability ranges B and C; and APSF3 supports assessment types A and B, and its supported assessment capability ranges include capability ranges A and B. In this case, if a third discovery request carries indication information indicating that the required assessment type is type A, the required assessment capability is b, and assessment capability b is within capability range B, then the NRF network element can identify APSF1 through APSF3 as third candidate network elements.

[0320] 507. The second network element sends a third discovery response to the first network element. The third discovery response is used to indicate one or more third candidate network elements.

[0321] The second network element sends a third discovery response to the first network element. The third discovery response includes information about a third candidate network element, which is used to determine the target network element. The information of the third candidate network element is similar to that of the second candidate network element, and will not be repeated here.

[0322] 508. The first network element sends a second request to the third candidate network element. The second request includes instructions on the second expected assessment capability or actual assessment capability report.

[0323] The first network element sends a second request to the third candidate network element. The second request is used to negotiate the actual assessment capability of the third candidate network element. The second request includes a second expected assessment capability or actual assessment capability report indication information.

[0324] It should be noted that when there are multiple third candidate network elements, the first network element sends a second request to each of the multiple third candidate network elements.

[0325] 509. The third candidate network element sends a second response to the first network element. The second response includes the result of the third candidate network element meeting the second expected capability or the actual evaluation capability of the third candidate network element.

[0326] After receiving the first request, the third candidate network element sends a second response to the first network element. The second response includes the first information or the actual evaluation capability of the third candidate network element. The first information indicates the result of one or more third candidate network elements satisfying the second expected capability. The result of the third candidate network element satisfying the second expected capability includes satisfaction and non-satisfaction.

[0327] Specifically, when the first request includes a second expected assessment capability, the second response sent by the third candidate network element to the first network element includes the first information. When the first request includes actual assessment capability report indication information, the second response sent by the third candidate network element to the first network element includes the actual assessment capability of the third candidate network element.

[0328] 510. The first network element determines the target network element from the third candidate network elements.

[0329] After receiving the first response, the first network element determines the target network element from the third candidate network elements. Specifically, when the second response includes the first information, the first network element selects the target network element from the third candidate network elements whose actual assessment capability meets the second expected assessment capability. When the second response includes the actual assessment capability of the third candidate network element, the first network element selects the target network element based on the actual assessment capability reported by the third candidate network element.

[0330] 511. The first network element requests the target network element to evaluate the analysis results and obtains the evaluation results.

[0331] 512. The first network element updates the analysis results based on the evaluation results.

[0332] 513. The first network element sends a subscription response to the customer network element. The subscription response includes the updated analysis results.

[0333] The method performed by the network element in steps 511 to 513 of this embodiment is the same as described above. Figure 3 The methods performed by the network element in steps 309 and 310 of the illustrated embodiment are similar and will not be described again here.

[0334] from Figure 5 As can be seen in the embodiments shown, Figure 5 In the illustrated embodiment, the third network element can register an evaluation capability range during the evaluation function registration. The first network element can determine multiple third candidate network elements based on the evaluation capability range registered by the third network element. Then, the first network element and the third candidate network elements negotiate the supported evaluation capabilities and determine the target network element. Since the third network element can register an evaluation capability range with the second network element, this avoids the third network element frequently updating its evaluation capabilities to the second network element. Figure 5 The illustrated embodiment improves the accuracy and reliability of the first network element analysis results while reducing communication overhead.

[0335] Please see Figure 6 , Figure 6 This is a flowchart illustrating another method for discovering evaluation functions provided in an embodiment of this application. Figure 6 In the method for discovering the evaluation function shown, the first network element is an NWDAF network element, the second network element is an NRF network element, and the third network element is an APSF network element. Figure 6 In the example shown, the APSF network element can register an assessment type, assessment capability, or assessment capability range in the NRF network element. When the NWDAF network element discovers a target APSF in the NRF network element, it first discovers a fourth candidate network element whose assessment capability or range is close to the required assessment capability range in the discovery request. Then, the first network element and the fourth candidate network element negotiate the actual assessment capability of the fourth candidate network element. Finally, the first network element selects the target network element from the fourth candidate network element. Details are as follows:

[0336] exist Figure 6 As shown in the example, the method for discovering the evaluation function in this application embodiment includes the following steps:

[0337] 601. The third network element sends a fourth registration request to the second network element. The registration information in the fourth registration request includes the assessment type, assessment type and assessment capability range.

[0338] The third network element sends a fourth registration request to the second network element. The registration information in the fourth registration request includes the assessment type, and may also include the assessment type or assessment capability range. For example, the APSF network element sends a fourth registration request to the NRF network element. The fourth registration request is used to register the APSF network element's assessment function with the NRF network element. The fourth registration request includes the assessment types supported by the APSF network element, as well as the assessment capabilities or assessment capability range supported by the APSF network element.

[0339] This application Figure 6 The specific content of the assessment type and the specific content of the assessment capability in the illustrated embodiments are the same as those in the examples. Figure 3 The content of the assessment type and the specific content of the assessment capability are consistent in the embodiments described above, and will not be repeated here. This application Figure 6 The specific content of the assessment capability range in the illustrated embodiment is related to... Figure 5 The content regarding the assessment capability range is consistent in the embodiments described above, and will not be repeated here.

[0340] 602. The second network element sends a fourth registration response to the third network element. The fourth registration response includes the registration result.

[0341] 603. The customer network element sends a subscription request to the first network element. The subscription request is used to subscribe to the analysis results of the data analysis.

[0342] 604. The first network element generates analysis results based on the collected data, which includes data from one or more data sources.

[0343] 605. The first network element prepares to trigger the target network element in the third network element to evaluate the analysis results.

[0344] The method performed by the network element in steps 602 to 606 of this embodiment is the same as described above. Figure 3 The methods performed by the network element in steps 302 and 306 of the illustrated embodiment are similar and will not be described again here.

[0345] 606. The first network element sends a fourth discovery request to the second network element. The fourth discovery request is used to request the discovery of a fourth candidate network element.

[0346] The first network element sends a fourth discovery request to the second network element. The fourth discovery request includes a first indication information and a third indication information. The first indication information is used to indicate the required assessment type, and the third indication information is also used to indicate the required assessment capability range. The assessment capability range can also be represented by the required assessment capability and the corresponding error range.

[0347] After receiving the fourth discovery request, the second network element determines the fourth candidate network element based on the fourth discovery request. The evaluation type of the fourth candidate network element supports the required evaluation type, and the evaluation capability range or evaluation capability corresponding to the fourth candidate network element is within the required evaluation capability range.

[0348] Specifically, since the third network element registered the evaluation type and the evaluation capability range or evaluation capability in the second network element in step 601, the second network element can select the third network element whose evaluation capability range or evaluation capability is within the required evaluation capability range as the fourth candidate network element during the process of determining the fourth candidate network element based on the fourth discovery request.

[0349] For example, after receiving the fourth discovery request sent by the NWDAF network element, the NRF network element determines the APSF network element that meets the requirements from the registered APSF network elements based on the fourth discovery request, and determines the APSF network element that meets the requirements as the fourth candidate APSF network element. The fourth candidate APSF network element that meets the requirements is the APSF network element whose registered evaluation type meets the evaluation type required in the fourth discovery request, and whose registered evaluation capability range or registered evaluation capability is within the required evaluation capability range in the fourth discovery request.

[0350] Specifically, for example, when an NRF network element discovers three APSF registration information, APSF1 supports assessment types A and B, and its assessment capability ranges include capability range A and capability range B, where capability range A is within capability range A' and capability range B is within capability range B'. APSF2 supports assessment type A, and its assessment capabilities include capability b and capability c, where capability b is within capability range B' and capability c is within capability range C'. APSF3 supports assessment types A and B, and its assessment capability ranges include capability range A and capability range C.

[0351] At this point, if the fourth discovery request carries indication information indicating that the required assessment type is type A and the required assessment capability range is B', since the assessment types registered by APSF1 and APSF2 both meet the assessment types required by the fourth discovery request, and the registered assessment capabilities or assessment capability ranges are all within the assessment capability range required by the fourth discovery request, while the capability range C registered by APSF3 is not within the required assessment capability range of B', the NRF network element will determine APSF1 and APSF2 as the fourth candidate network element.

[0352] 607. The second network element sends a fourth discovery response to the first network element. The fourth discovery response is used to indicate one or more fourth candidate network elements.

[0353] The second network element sends a fourth discovery response to the first network element. The fourth discovery response includes information about a fourth candidate network element, which is used to determine the target network element. The information of the fourth candidate network element is similar to that of the second candidate network element in the above embodiment, and will not be repeated here.

[0354] 608. The first network element sends a third request to the fourth candidate network element. The third request includes instructions on the third expected assessment capability or actual assessment capability report.

[0355] The first network element sends a third request to the fourth candidate network element. The third request is used to negotiate the actual assessment capability of the fourth candidate network element. The third request includes information indicating the third expected assessment capability or the actual assessment capability report.

[0356] It should be noted that when there are multiple fourth candidate network elements, the first network element sends a third request to each of the multiple fourth candidate network elements.

[0357] 609. The fourth candidate network element sends a third response to the first network element. The third response includes the result of the third candidate network element meeting the third expected capability or the actual evaluation capability of the fourth candidate network element.

[0358] After receiving the third request, the fourth candidate network element sends a third response to the first network element. The third response includes the second information or the actual evaluation capability of the third candidate network element. The second information indicates the result of one or more fourth candidate network elements satisfying the third expected capability. The result of the fourth candidate network element satisfying the third expected capability includes satisfaction and non-satisfaction.

[0359] Specifically, when the third request includes a third expected assessment capability, the third response sent by the fourth candidate network element to the first network element includes the second information. When the third request includes actual assessment capability report indication information, the third response sent by the fourth candidate network element to the first network element includes the actual assessment capability of the fourth candidate network element.

[0360] 610. The first network element determines the target network element from the fourth candidate network element.

[0361] After receiving the third response, the first network element determines the target network element from the fourth candidate network elements. Specifically, when the third response includes the second information, the first network element selects the target network element from the fourth candidate network elements whose actual assessment capability meets the third expected assessment capability. When the third response includes the actual assessment capability of the fourth candidate network element, the first network element selects the target network element based on the actual assessment capability reported by the fourth candidate network element.

[0362] 611. The first network element requests the target network element to evaluate the analysis results and obtains the evaluation results.

[0363] 612. The first network element updates the analysis results based on the evaluation results.

[0364] 613. The first network element sends a subscription response to the customer network element. The subscription response includes the updated analysis results.

[0365] The method performed by the network element in steps 611 to 613 of this embodiment is the same as described above. Figure 3 The methods performed by the network element in steps 309 and 310 of the illustrated embodiment are similar and will not be described again here.

[0366] from Figure 6 As can be seen in the embodiments shown, Figure 6 In the illustrated embodiment, the fourth discovery request sent by the first network element to the second network element can request a required evaluation capability range. This allows the second network element to identify a third network element that fits this range as a fourth candidate network element. Subsequently, the first network element and the fourth candidate network element negotiate the supported simulation capabilities and determine the target network element. Therefore, in... Figure 6 In the illustrated embodiment, the first network element can select the target network element from among more fourth candidate network elements, further improving the accuracy and reliability of the analysis results of the first network element.

[0367] Please see Figure 7 , Figure 7 This is a flowchart illustrating another method for discovering evaluation functions provided in an embodiment of this application. Figure 6 In the method for discovering the evaluation function shown, the first network element is an NWDAF network element, the second network element is an NRF network element, and the third network element is an APSF network element. Figure 7 In the example shown, when the first network element determines that the evaluation capability of the target network element does not meet the requirements, the first network element can renegotiate the evaluation capability and reselect the target network element. Meanwhile, in... Figure 7 In the example shown, the third network element can update registration information. Details are as follows:

[0368] exist Figure 7 As shown in the example, the method for discovering the evaluation function in this application embodiment includes the following steps:

[0369] 701 to 710. Send a subscription response, which includes the updated analytics results.

[0370] This application Figure 7 In the illustrated embodiment, steps 701 to 710 are the same as those described above. Figures 3 to 6 The steps in the illustrated embodiments are similar and will not be repeated here.

[0371] 711. The first network element's ability to determine the target network element does not meet the requirements.

[0372] If the first network element determines that the evaluation capability of the target network element does not meet the requirements, then the first network element can renegotiate the evaluation capability of the target network element.

[0373] For example, if the evaluation result of the target network element is a verification result, and the error between the service experience predicted by the target network element after using the recommended QoS parameters and the actual service experience exceeds a threshold, then the first network element determines that the evaluation capability of the target network element does not meet the requirements. As another example, if the evaluation result of the target network element is the evaluation result of a simulation system, and the actual simulation response time and simulation result accuracy of the target network element do not meet the requirements, then the first network element determines that the evaluation capability of the target network element does not meet the requirements.

[0374] 712. The first network element sends a fourth request to the candidate network element. The fourth request includes actual assessment capability report instruction information.

[0375] During the process of renegotiating the assessment capabilities of the target network element, the first network element sends a fourth request to the target network element. The fourth request includes instructions on reporting the expected assessment capabilities or the actual assessment capabilities. The fourth request is used to instruct the target network element on the expected assessment capabilities of the first network element, or to instruct the target network element to report the updated performance assessment capabilities.

[0376] 713. The target network element sends a fourth response to the first network element. The fourth response includes the actual assessment capability of the target network element.

[0377] After receiving the fourth request, the target network element sends a fourth response to the first network element. The fourth response includes the third information or the target network element's updated actual assessment capability. The third information indicates the result of the target network element's updated assessment capability satisfying the fourth expected assessment capability. Specifically, when the fourth request includes the fourth expected assessment capability, the fourth response sent by the target network element includes the third information; when the fourth request includes the fourth expected assessment capability actual assessment capability reporting indication information, the fourth response sent by the target network element includes the target network element's updated actual assessment capability.

[0378] For example, the APSF determines the fastest achievable response time or the highest achievable accuracy based on its own computing power, load, system status, and other factors. The first network element determines whether to continue selecting the APSF to perform simulation based on this information. At the same time, the APSF can also update the supported capabilities in the NRF.

[0379] 714. If the third information or the updated evaluation capability does not meet the requirements, the first network element shall reselect the target network element.

[0380] After receiving the fourth response, if the third information or the updated evaluation capability in the fourth response does not meet the requirements, the first network element will reselect the target network element.

[0381] If the first network element determines that the actual evaluation capability of the target network element does not meet the evaluation capability requirements, the first network element will reselect the target network element.

[0382] 715. The third network element sends a registration information update request to the second network element.

[0383] In this embodiment, the third network element sends a registration information update request to the second network element. The registration information update request is used to request an update to the registration information of the third network element in the second network element. The registration information update request includes the updated assessment type and the updated assessment capability information, whereby the updated assessment capability information includes the updated assessment capability or the updated assessment capability range.

[0384] It should be noted that the execution sequence of step 715... Figure 7 The execution timing of other steps in the illustrated embodiments is not limited.

[0385] from Figure 7 As can be seen in the illustrated embodiment, when the first network element discovers that the evaluation capability of the target network element does not meet the requirements, the first network element and the target network element can renegotiate the evaluation capability. If the renegotiation of the evaluation capability between the first network element and the target network element still fails to meet the requirements, the first network element can reselect the target network element, thereby improving the accuracy of the first network element's selection of the target network element and further improving the accuracy and reliability of the first network element's analysis results.

[0386] Please see Figure 8 , Figure 8 This is a flowchart illustrating another method for discovering evaluation functions provided in an embodiment of this application. Figure 8 In the method for discovering the evaluation function shown, the first network element is an NWDAF network element, the second network element is an NRF network element, and the third network element is an NWDAF network element containing an evaluation function module, which is an APSF module. Figure 8 In the example shown, the APSF module has registered its assessment type and assessment capabilities with the NRF network element, allowing the NWDAF network element to directly discover the target APSF module within the NRF network element. Details are provided below:

[0387] exist Figure 8 In the illustrated embodiments, the method for discovering the evaluation function provided in this application includes the following steps:

[0388] 801. The evaluation function module sends a registration request to the second network element. The registration request includes the first evaluation capability indication.

[0389] The evaluation function module sends a registration request to the second network element. This evaluation function module can be the evaluation function module within the Network Data Analysis Function (NWDAF) network element. Therefore, it can also be understood as the NWDAF network element sending a registration request to the second network element, or an NWDAF network element containing an evaluation function module sending a registration request to the second network element. The registration request includes a first evaluation capability indication. The registration information in the registration request includes the first evaluation capability indication, which indicates that the NWDAF network element possesses evaluation capabilities, or in other words, that the first evaluation capability indication indicates that the NWDAF network element possesses AI performance simulation capabilities. The evaluation capability indication can also be called evaluation capability indication information, evaluation capability indication identifier, or AI performance simulation capability indication. For example, if the first evaluation capability indication is an AI performance simulation indication, and the NWDAF network element sends a registration request to the NRF network element, the registration request includes an AI performance simulation identifier, which indicates that the NWDAF network element supports AI performance simulation capabilities. The registration request also includes an analysis identifier or a recommendation identifier.

[0390] Optionally, the registration information in the registration request may also include one or more of the following: assessment type and assessment capability information, wherein the assessment type indicates the assessment types supported by the assessment function module, the assessment capability information indicates the assessment capabilities supported by the assessment function module, and the assessment capability information includes assessment capabilities or assessment capability range.

[0391] 802. The second network element sends a registration response to the evaluation function module. The registration response includes the registration result.

[0392] After the second network element generates a registration response, it sends a registration response to the evaluation function module. The registration response is used to indicate the registration result of the evaluation function, which includes whether the registration was successful or failed.

[0393] For example, in one example of step 802, the NRF network element sends a registration response to the APSF module of the NWDAF network element. The registration response indicates the registration result of the APSF network element's simulation verification capability. The registration result includes registration success and registration failure.

[0394] 803. The customer network element sends a subscription request to the first network element. The subscription request is used to subscribe to the analysis results of the data analysis.

[0395] 804. The first network element generates analysis results based on the collected data, which includes data from one or more data sources.

[0396] 805. The first network element is ready to trigger the target evaluation function module in the evaluation function module to evaluate the analysis results.

[0397] The methods performed by the evaluation function module and other network elements in steps 803 to 805 of this application embodiment are the same as those described above. Figure 3 In the illustrated embodiment, the methods performed by the third network element and other network elements in steps 303 and 305 are similar and will not be described again here.

[0398] 806. The first network element sends a discovery request to the second network element. The discovery request is used to request the discovery of the target NWDAF network element, or in other words, to request the discovery of an NWDAF network element that contains an evaluation function module.

[0399] The first network element sends a discovery request to the second network element. The discovery request includes a second assessment capability indication, which indicates a request to discover NWDAF network elements that support assessment functions. Optionally, the discovery request includes the required assessment type and the required assessment capability information. The discovery request also includes an analysis identifier or a recommendation identifier.

[0400] After receiving the discovery request, the second network element determines the target NWDAF network element based on the discovery request. Specifically, since the registration request in step 801 registered an evaluation function module to support the evaluation function, when the registration request registers the evaluation types and evaluation capabilities supported by the evaluation function module, the second network element selects an evaluation function module from the registered evaluation function modules whose evaluation type and evaluation capability information both meet the requirements of the discovery request, based on the evaluation type and evaluation capability information required in the discovery request, and determines the evaluation function module that meets the requirements as the target evaluation function module.

[0401] 807. The second network element sends a discovery response to the first network element, and the discovery response includes candidate NWDAF network elements.

[0402] The second network element sends a discovery response to the first network element. This response includes information about one or more candidate NWDAF network elements. The candidate NWDAF network element information includes its ID or IP address, etc. The candidate NWDAF network element contains an evaluation function module, or in other words, it supports evaluation capabilities. The evaluation types supported by the candidate NWDAF network element include those required by the first network element, and the evaluation capabilities supported by the candidate NWDAF network element include those required by the first network element.

[0403] 808. The first network element determines the target network element based on the candidate evaluation function module.

[0404] After receiving the discovery response, the first network element determines the target evaluation function module from one or more candidate evaluation function modules. The target evaluation function module is used to perform the evaluation task.

[0405] This application Figure 8In the illustrated embodiment, steps 806 to 808, where the second network elements determine candidate evaluation function modules based on the evaluation type and required evaluation capability information in the discovery request, and determine the target evaluation function module from the candidate evaluation function modules, are consistent with the aforementioned steps. Figures 3 to 7 The steps for the first network element to determine the candidate network element and the target network element in the illustrated embodiment are similar and will not be repeated here.

[0406] 809. The first network element request target evaluation function module evaluates the analysis results and obtains the evaluation results.

[0407] 810. The first network element updates the analysis results based on the evaluation results.

[0408] 811. The first network element sends a subscription response to the customer network element. The subscription response includes the updated analysis results.

[0409] The method performed by the evaluation function module and other network elements in steps 808 to 810 of this embodiment is the same as described above. Figure 3 The methods performed by the third network element and other network elements in steps 309 and 310 of the illustrated embodiment are similar and will not be described again here.

[0410] It should be noted that, in the above Figures 4 to 7 In the illustrated embodiment, the third network element can also be a network element containing an evaluation function module, such as an NWDAF network element containing APSF. That is, the evaluation function module can register its evaluation function in the second network element, and the first network element can discover candidate evaluation function modules and target evaluation function modules in the second network element, and negotiate to determine the evaluation capability of the target function module. The evaluation module registration and discovery process is the same as described above. Figures 4 to 7 The registration and discovery process for the evaluation function modules in the illustrated embodiment is similar and will not be described in detail here.

[0411] from Figure 8 As can be seen from the embodiments shown, the evaluation function in this application embodiment can be a non-independent network element, but rather an evaluation function module of a network element. The first network element can discover the target evaluation function module based on the registration information and trigger the target evaluation function module to evaluate the analysis results of the first network element, determine the updated analysis results, thereby improving the accuracy and reliability of the analysis results of the first network element.

[0412] Based on the above method embodiments, this application also provides an evaluation function discovery device, which is described in detail below.

[0413] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a detection device for an evaluation function provided in an embodiment of this application. Figure 9In the example shown, the evaluation function discovery device 900 is used to implement the various steps performed by the first network element, the second network element, the third network element or the evaluation function module in the above embodiments. The evaluation function discovery device 900 includes a receiving unit 901, a sending unit 902 and a processing unit 903.

[0414] In one possible embodiment of the evaluation function discovery device 900, the evaluation function discovery device 900 is used to implement the steps performed by the second network element in the above method embodiment.

[0415] The receiving unit 901 is used to receive a discovery request sent by a first network element. The discovery request is used to request the discovery of network elements that support the evaluation requirements of the first network element. The sending unit 902 is used to send a discovery response to the first network element. The discovery response includes information about candidate network elements. The candidate network elements support the evaluation requirements of the first network element. The candidate network elements are used to determine the target network element. The target network element is used to perform the evaluation task.

[0416] In one possible implementation, the receiving unit 901 is further configured to receive a registration request sent by a third network element. The registration request includes one or more of the following: assessment type and assessment capability information. The assessment capability information includes one or more of the following: assessment capability and assessment capability range.

[0417] In one possible implementation, the registration request includes an evaluation type and an evaluation capability. The receiving unit 901 is specifically configured to receive a first discovery request sent by a first network element. The first discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The processing unit 903 is configured to determine a first candidate network element based on the first discovery request. The evaluation type supported by the first candidate network element meets the required evaluation type, and the evaluation capability supported by the first candidate network element meets the required evaluation capability. The sending unit 902 is specifically configured to send a first discovery response to the first network element. The first discovery response includes information about the first candidate network element, which is used to determine the target network element.

[0418] In one possible implementation, the registration request includes an evaluation type and an evaluation capability range. The receiving unit 901 is specifically configured to receive a third discovery request sent by a first network element. The third discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The processing unit 903 is further configured to determine a third candidate network element based on the third discovery request. The evaluation types supported by the third candidate network element include the required evaluation type, and the evaluation capability range supported by the third candidate network element includes the required evaluation capability. The sending unit 902 is specifically configured to send a third discovery response to the first network element. The third discovery response includes information about the third candidate network element, which is used to determine the target network element.

[0419] In one possible implementation, the registration request includes assessment type and assessment capability information. The receiving unit 901 is specifically configured to receive a fourth discovery request sent by the first network element. The fourth discovery request includes first indication information and third indication information. The first indication information indicates the required assessment type, and the third indication information indicates the required assessment capability range. The processing unit 903 is further configured to determine a fourth candidate network element based on the fourth discovery request. The assessment type supported by the fourth candidate network element includes the required assessment type, and the assessment capability range or assessment capability supported by the fourth candidate network element is within the required assessment capability range. The sending unit 902 is specifically configured to send a fourth discovery response to the first network element. The fourth discovery response includes a fourth candidate network element, which is used to determine the target network element.

[0420] In one possible implementation, the receiving unit 901 is further configured to receive a registration information update request sent by a third network element. The registration information update request includes one or more of the following: the updated supported evaluation type, the updated supported evaluation capability, and the updated supported evaluation capability range.

[0421] In one possible embodiment of the evaluation function discovery device 900, the evaluation function discovery device 900 is used to implement the steps performed by the second network element in the above method embodiment.

[0422] The receiving unit 901 is used to receive a discovery request sent by a first network element. The discovery request includes a first assessment capability indication and is used to discover network data analysis function network elements that support assessment capabilities. The sending unit 902 is used to send a discovery response to the first network element. The discovery response includes information about the network data analysis function network element, which supports assessment capabilities.

[0423] In one possible implementation, the transceiver unit is further configured to receive a registration request sent by the network data analysis function network element. The registration request includes a second evaluation capability indication, which is used to indicate that the network data analysis function network element supports the evaluation capability.

[0424] In one possible embodiment of the evaluation function discovery device 900, the evaluation function discovery device 900 is used to implement the steps performed by the first network element in the above method embodiment.

[0425] The sending unit 902 is used to send a discovery request to the network storage function network element. The discovery request is used to request the discovery of network elements that support the evaluation requirements of the first network element. The receiving unit 901 is used to receive a discovery response sent by the network storage function network element. The discovery response includes information about candidate network elements. The candidate network elements support the evaluation requirements of the first network element. The candidate network elements are used to determine the target network element. The target network element is used to perform the evaluation task.

[0426] In one possible implementation, the sending unit 902 is specifically configured to send a first discovery request to the network storage function network element. The first discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The receiving unit 901 is specifically configured to receive a first discovery response sent by the network storage function network element. The first discovery response includes information about a first candidate network element, the evaluation type of the first candidate network element supports the required evaluation type, and the evaluation capability of the first candidate network element supports the required evaluation capability.

[0427] In one possible implementation, the processing unit 903 is used to determine a target network element from one or more first candidate network elements, and the target network element is used to perform the evaluation task.

[0428] In one possible implementation, the sending unit 902 is specifically configured to send a second discovery request to the network storage function network element. The receiving unit 901 is specifically configured to receive a second discovery response sent by the network storage function network element, the second discovery response including a second candidate network element, the second candidate network element being used to determine the target network element.

[0429] In one possible implementation, the processing unit 903 is further configured to determine a target network element from one or more second candidate network elements, and the sending unit 902 is further configured to send a first request to the target network element, the first request including a first desired evaluation capability. The receiving unit 901 is further configured to receive a first response sent by the target network element, the first response including the constructed evaluation capability of the target network element.

[0430] In one possible implementation, the sending unit 902 is specifically configured to send a third discovery request to the network storage function network element. The third discovery request includes first indication information and second indication information. The first indication information indicates the required evaluation type, and the second indication information indicates the required evaluation capability. The receiving unit 901 is further configured to receive a third discovery response sent by the network storage function network element. The third discovery response includes a third candidate network element, the evaluation type of the third candidate network element supports the required evaluation type, and the evaluation capability range of the third candidate network element includes the required evaluation capability.

[0431] In one possible implementation, the sending unit 902 is further configured to send a second request to the third candidate network element, the second request including a second desired evaluation capability. The receiving unit 901 is further configured to receive a second response sent by the third candidate network element, the second response including first information indicating whether the third candidate network element meets the second desired capability. The processing unit 903 is further configured to determine a target network element from one or more third candidate network elements based on the first information.

[0432] In one possible implementation, the sending unit 902 is further configured to send a second request to the third candidate network element, the second request including actual assessment capability report indication information. The receiving unit 901 is further configured to receive a second response sent by the third candidate network element, the second response including the actual assessment capability of the third candidate network element. The processing unit 903 is further configured to determine a target network element from one or more second candidate network elements based on the actual assessment capability.

[0433] In one possible implementation, the sending unit 902 is specifically configured to send a fourth discovery request to the network storage function network element. The fourth discovery request includes first indication information and third indication information. The first indication information indicates the required evaluation type, and the third indication information indicates the required evaluation capability range. The receiving unit 901 is further configured to receive a fourth discovery response from the network storage function network element. The fourth discovery response includes a fourth candidate network element and the evaluation capability range or evaluation capability of the fourth candidate network element, which is within the required evaluation capability range.

[0434] In one possible implementation, the sending unit 902 is further configured to send a third request to the fourth candidate network element, the third request including a third desired evaluation capability. The receiving unit 901 is further configured to receive a third response sent by the fourth candidate network element, the third response including second information indicating the result of the fourth candidate network element satisfying the third desired evaluation capability. The processing unit 903 is further configured to determine a target network element from one or more fourth candidate network elements based on the second information.

[0435] In one possible implementation, the sending unit 902 is further configured to send a third request to the third candidate network element, the third request including actual assessment capability report indication information. The receiving unit 901 is further configured to receive a third response sent by the third candidate network element, the third response including the actual assessment capability of one or more fourth candidate network elements. The processing unit 903 is further configured to determine a target network element from one or more fourth candidate network elements based on the actual assessment capability.

[0436] In one possible implementation, the sending unit 902 is further configured to send a fourth request to the target network element, the fourth request including a fourth desired evaluation capability. The receiving unit 901 is further configured to receive a fourth response sent by the target network element, the fourth response including third information, the third information being used to indicate the result of the target network element satisfying the fourth desired evaluation capability. The processing unit 903 is further configured to, if the evaluation capability of the target network element does not satisfy the fourth desired evaluation capability, then the first network element reselects a target network element.

[0437] In one possible implementation, the sending unit 902 is further configured to send a fourth request to the target network element, the fourth request including actual assessment capability report indication information. The receiving unit 901 is further configured to receive a fourth response sent by the target network element, the fourth response including the actual assessment capability of the target network element. The processing unit 903 is further configured to, when the first network element determines that the actual assessment capability of the target network element does not meet the assessment capability requirements, reselect a target network element.

[0438] In one possible implementation, the sending unit 902 is further configured to send an evaluation request to the target network element. The evaluation request requests the evaluation of the analysis results and includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, output results to be verified, and verification target. The receiving unit 901 is further configured to receive the evaluation results sent by the target network element. The evaluation results include one or more of the following: verification results, output result impact, simulation system evaluation results, and output result update auxiliary information. The processing unit 903 is further configured to update the analysis results based on the evaluation results.

[0439] In one possible embodiment of the evaluation function discovery device 900, the evaluation function discovery device 900 is used to implement the steps performed by the third network element in the above method embodiment.

[0440] The sending unit 902 is used to send a registration request to the network storage function network element. The registration request requests the registration of the evaluation function in the network storage function network element. The registration request includes the registration information of the third network element, which is used to discover the target network element. The target network element is used to perform the evaluation task. The receiving unit 901 is used to receive the registration response sent by the network storage function network element. The registration response includes the registration result of the evaluation function.

[0441] In one possible implementation, the sending unit 902 is specifically used to send a first registration request to the network storage function network element. The first registration request includes an evaluation type and an evaluation capability. The third network element is used to determine a first candidate network element, and the first candidate network element is used to determine a target network element.

[0442] In one possible implementation, the sending unit 902 is specifically used to send a second registration request to the network storage function network element. The second registration request does not carry registration information or carries one or more of the following registration information: evaluation type and evaluation capability. The third network element is also used to determine a second candidate network element, which is used to determine the target network element.

[0443] In one possible implementation, the receiving unit 901 is further configured to receive a first request sent by the first network element, the first request including a first desired simulation capability. The sending unit 902 is further configured to send a first response to the first network element, the first response including the constructed evaluation capability of the target network element.

[0444] In one possible implementation, the sending unit 902 is further configured to send a third registration request to the network storage function network element. The registration information in the third registration request includes the evaluation type and the evaluation capability range. The third network element is further configured to determine a third candidate network element.

[0445] In one possible implementation, the receiving unit 901 is further configured to receive a second request sent by the first network element, the second request including: a second expected simulation capability or actual evaluation capability report indication information. The sending unit 902 is further configured to send a second response to the first network element, the second response including: a third candidate network element's satisfaction result of the second expected evaluation capability or the actual evaluation capability of the third candidate network element, the third candidate network element being used to determine the target network element.

[0446] In one possible implementation, the sending unit 902 is specifically used to send a fourth registration request to the network storage function network element. The fourth registration request includes evaluation type and evaluation capability information. The evaluation capability information includes evaluation capability range or evaluation capability. The third network element is also used to determine the fourth candidate network element.

[0447] In one possible implementation, the receiving unit 901 is further configured to receive a third request sent by the first network element, the third request including: a third expected simulation capability or actual evaluation capability report indication information. The sending unit 902 is further configured to send a third response to the first network element, the third response including: the result of the fourth candidate network element satisfying the third expected evaluation capability or the actual evaluation capability of the fourth candidate network element, the third candidate network element being used to determine the target network element.

[0448] In one possible implementation, the receiving unit 901 is further configured to receive an evaluation request sent by the first network element. The evaluation request includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, output result to be verified, and verification target. The evaluation request is used to request the target network element to evaluate the analysis results. The processing unit 903 is further configured to evaluate the output results to obtain an evaluation result. The evaluation result is used to update the analysis results generated by the first network element. The evaluation result includes one or more of the following: verification result, impact of output result, simulation system evaluation result, and auxiliary information for updating the output result. The sending unit 902 is further configured to send the evaluation result to the first network element.

[0449] In one possible embodiment of the evaluation function discovery device 900, the evaluation function discovery device 900 is used to implement the steps performed by the evaluation function module in the above method embodiments.

[0450] The sending unit 902 is used to send a registration request to the network storage function network element. The registration request is used to request the registration of the evaluation function. The registration information in the registration request includes an evaluation capability indication, which is used to indicate that the network data analysis function network element supports the evaluation capability. The receiving unit 901 is used to receive a registration response sent by the network storage function network element. The registration response includes the registration result of the evaluation function.

[0451] In one possible implementation, the receiving unit 901 is further configured to receive an evaluation request sent by the first network element. The evaluation request includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, analysis results to be verified, and verification target, and analysis results. The evaluation request is used to request the target evaluation function module to evaluate the analysis results. The processing unit 903 is configured to evaluate the analysis results and obtain evaluation results. The evaluation results are used to update the analysis results generated by the first network element. The evaluation results include one or more of the following: verification results, impact of analysis results, simulation system evaluation results, and auxiliary information for updating analysis results. The sending unit 902 is further configured to send the evaluation results to the first network element.

[0452] It is understandable that the receiving unit 901, transmitting unit 902, and processing unit 903 in the discovery device 900 for evaluating functions can function as modules with... Figure 1a The various modules in the network data analysis system 10 are mapped to each other, thereby realizing the functions of each module in the network data analysis system 10.

[0453] It should be understood that the division of units in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, all units in the device can be implemented entirely through software calls from processing elements; all units can be implemented entirely in hardware; or some units can be implemented through software calls from processing elements, and others in hardware. For example, each unit can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as a program in memory, called and executed by a processing element of the device. Moreover, these units can be fully or partially integrated together, or implemented independently. The processing element mentioned here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above units can be implemented through integrated logic circuits in the processor element or through software calls from processing elements.

[0454] It is worth noting that, for the sake of simplicity, the above method embodiments are described as a series of actions. However, those skilled in the art should know that this application is not limited to the order of the described actions. Furthermore, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0455] Other reasonable combinations of steps that can be conceived by those skilled in the art based on the above description also fall within the scope of protection of this application. Furthermore, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0456] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a network device provided in an embodiment of this application. Figure 10 As shown, the network device 1000 includes a processor 1001, a memory 1002, a communication interface 1003, and a bus 1004. The processor 1001, memory 1002, and communication interface 1003 are coupled via the bus (not shown in the figure). The memory 1002 stores instructions. When the instructions in the memory 1002 are executed, the network device 1000 performs the methods executed by the first network element, the second network element, the third network element, or the evaluation function module in the above method embodiments.

[0457] Network device 1000 may be one or more integrated circuits configured to implement the above methods, such as: one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. Furthermore, when the units in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units may be integrated together and implemented as a system-on-a-chip (SOC).

[0458] The processor 1001 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0459] The memory 1002 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0460] The memory 1002 stores executable program code, and the processor 1001 executes the executable program code to implement the functions of the aforementioned units or modules, thereby realizing the above-mentioned method for discovering the performance evaluation function. That is, the memory 1002 stores instructions for executing the above-mentioned method for discovering the performance evaluation function.

[0461] The communication interface 1003 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the network device 1000 and other devices or communication networks.

[0462] In addition to the data bus, the 1004 bus can also include a power bus, a control bus, and a status signal bus. The bus can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The bus can be divided into address bus, data bus, and control bus.

[0463] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a communication system provided in an embodiment of this application. Figure 11 In the example shown, the communication system 1100 includes a first network device 1101, a second network device 1102, and a third network device 1103. The first network device 1101 is used to execute the method executed by the second network element in the above method embodiment, the second network device 1102 is used to execute the method executed by the first network element in the above method embodiment, and the third network device 1103 is used to execute the method executed by the third network element or the evaluation function module.

[0464] In another embodiment of this application, a computer-readable storage medium is also provided, which stores computer-executable instructions. When the processor of the device executes the computer-executable instructions, the device executes the method performed by the discovery device for the evaluation function in the above method embodiment.

[0465] In another embodiment of this application, a computer program product is also provided, which includes computer-executable instructions stored in a computer-readable storage medium. When the processor of the device executes the computer-executable instructions, the device performs the method executed by the discovery device for evaluating the function in the above method embodiments.

[0466] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0467] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0468] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0469] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0470] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for discovering assessment functions, characterized in that, include: The network storage function network element receives a discovery request sent by the first network element, the discovery request being used to request the discovery of network elements that support the evaluation requirements of the first network element; The network storage function network element sends a discovery response to the first network element. The discovery response includes information about candidate network elements. The candidate network elements support the evaluation requirements of the first network element. The candidate network elements are used to determine the target network element. The target network element is used to perform the evaluation task.

2. The method according to claim 1, characterized in that, The method further includes: The network storage function network element receives a registration request sent by a third network element. The registration request includes one or more of the following: assessment type and assessment capability information. The assessment capability information includes one or more of the following: assessment capability and assessment capability range.

3. The method according to claim 2, characterized in that, The registration request includes the assessment type and assessment capability, and the network storage function network element receives the discovery request sent by the first network element, including: The network storage function network element receives a first discovery request sent by the first network element. The first discovery request includes first indication information and second indication information. The first indication information is used to indicate the required assessment type, and the second indication information is used to indicate the required assessment capability. The method further includes: The network storage function network element determines the first candidate network element based on the first discovery request. The evaluation type supported by the first candidate network element meets the required evaluation type, and the evaluation capability supported by the first candidate network element meets the required evaluation capability. The network storage function network element sends a discovery response to the first network element, including: The network storage function network element sends a first discovery response to the first network element. The first discovery response includes information about the first candidate network element, which is used to determine the target network element.

4. The method according to claim 2, characterized in that, The registration request includes the assessment type and assessment capability range, and the network storage function network element receives the discovery request sent by the first network element, including: The network storage function network element receives a third discovery request sent by the first network element. The third discovery request includes first indication information and second indication information. The first indication information is used to indicate the required assessment type, and the second indication information is used to indicate the required assessment capability. The method further includes: The network storage function network element determines a third candidate network element based on the third discovery request. The evaluation types supported by the third candidate network element include the required evaluation types, and the evaluation capability range supported by the third candidate network element includes the required evaluation capabilities. The network storage function network element sends a discovery response to the first network element, including: The network storage function network element sends a third discovery response to the first network element. The third discovery response includes information about the third candidate network element, which is used to determine the target network element.

5. The method according to claim 2, characterized in that, The registration request includes assessment type and assessment capability information, and the network storage function network element receives the discovery request sent by the first network element, including: The network storage function network element receives a fourth discovery request sent by the first network element. The fourth discovery request includes a first indication information and a third indication information. The first indication information is used to indicate the required assessment type, and the third indication information is used to indicate the required assessment capability range. The method further includes: The network storage function network element determines a fourth candidate network element based on the fourth discovery request. The evaluation types supported by the fourth candidate network element include the required evaluation types, and the evaluation capability range or evaluation capability supported by the fourth candidate network element is within the required evaluation capability range. The network storage function network element sends a discovery response to the first network element, including: The network storage function network element sends a fourth discovery response to the first network element. The fourth discovery response includes the fourth candidate network element, which is used to determine the target network element.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The network storage function network element receives a registration information update request sent by a third network element. The registration information update request includes one or more of the following: the updated supported evaluation type, the updated supported evaluation capability, and the updated supported evaluation capability range.

7. A method for discovering assessment functions, characterized in that, include: The network storage function network element receives a discovery request sent by the first network element. The discovery request includes a first evaluation capability indication and is used to discover network data analysis function network elements that support evaluation capabilities. The network storage function network element sends a discovery response to the first network element. The discovery response includes information from the network data analysis function network element, which supports evaluation capabilities.

8. The method according to claim 7, characterized in that, The method further includes: The network storage function network element receives a registration request sent by the network data analysis function network element. The registration request includes a second evaluation capability indication, which is used to indicate that the network data analysis function network element supports evaluation capabilities.

9. A method for discovering assessment functions, characterized in that, include: The first network element sends a discovery request to the network storage function network element, the discovery request being used to request the discovery of network elements that support the evaluation requirements of the first network element; The first network element receives a discovery response sent by the network storage function network element. The discovery response includes information about candidate network elements, which support the evaluation requirements of the first network element. The candidate network elements are used to determine the target network element, and the target network element is used to perform the evaluation task.

10. The method according to claim 9, characterized in that, The first network element sends a discovery request to the network storage function network element, including: The first network element sends a first discovery request to the network storage function network element. The first discovery request includes first indication information and second indication information. The first indication information is used to indicate the required assessment type, and the second indication information is used to indicate the required assessment capability. The first network element receives a first discovery response sent by the network storage function network element. The first discovery response includes information about a first candidate network element, the evaluation type of the first candidate network element supports the required evaluation type, and the evaluation capability of the first candidate network element supports the required evaluation capability.

11. The method according to claim 10, characterized in that, The method further includes: The first network element determines a target network element from one or more first candidate network elements, and the target network element is used to perform the evaluation task.

12. The method according to claim 9 or 10, characterized in that, The first network element sends a discovery request to the network storage function network element, including: The first network element sends a second discovery request to the network storage function network element; The first network element receives a second discovery response sent by the network storage function network element. The second discovery response includes a second candidate network element, which is used to determine the target network element.

13. The method according to claim 12, characterized in that, The method further includes: The first network element determines the target network element from one or more second candidate network elements; The first network element sends a first request to the target network element, the first request including a first expected assessment capability; The first network element receives a first response sent by the target network element, the first response including the constructed evaluation capabilities of the target network element.

14. The method according to claim 9, characterized in that, The first network element sends a discovery request to the network storage function network element, including: The first network element sends a third discovery request to the network storage function network element. The third discovery request includes first indication information and second indication information. The first indication information is used to indicate the required assessment type, and the second indication information is used to indicate the required assessment capability. The first network element receives a third discovery response sent by the network storage function network element. The third discovery response includes a third candidate network element. The evaluation type of the third candidate network element supports the required evaluation type, and the evaluation capability range of the third candidate network element includes the required evaluation capability.

15. The method according to claim 13, characterized in that, The method further includes; The first network element sends a second request to the third candidate network element, the second request including a second desired evaluation capability; The first network element receives a second response sent by the third candidate network element. The second response includes first information, which is used to indicate whether the third candidate network element meets the second expected capability. The first network element determines the target network element from one or more of the third candidate network elements based on the first information.

16. The method according to claim 13, characterized in that, The method further includes; The first network element sends a second request to the third candidate network element, the second request including actual assessment capability report indication information; The first network element receives a second response sent by the third candidate network element, the second response including the actual evaluation capability of the third candidate network element; The first network element determines the target network element from one or more second candidate network elements based on the actual evaluation capability.

17. The method according to claim 9, characterized in that, The first network element sends a discovery request to the network storage function network element, including: The first network element sends a fourth discovery request to the network storage function network element. The fourth discovery request includes a first indication information and a third indication information. The first indication information is used to indicate the required assessment type, and the third indication information is used to indicate the required assessment capability range. The first network element receives a fourth discovery response sent by the network storage function network element. The fourth discovery response includes the fourth candidate network element, and the evaluation capability range or evaluation capability of the fourth candidate network element is within the required evaluation capability range.

18. The method according to claim 17, characterized in that, The method further includes; The first network element sends a third request to the fourth candidate network element, the third request including a third expected evaluation capability; The first network element receives a third response sent by the fourth candidate network element. The third response includes second information, which is used to indicate the result of the fourth candidate network element meeting the third expected evaluation capability. The first network element determines the target network element from one or more of the fourth candidate network elements based on the second information.

19. The method according to claim 17, characterized in that, The method further includes; The first network element sends a third request to the third candidate network element, the third request including actual assessment capability report indication information; The first network element receives a third response sent by the third candidate network element, the third response including the actual evaluation capabilities of one or more of the fourth candidate network elements; The first network element determines the target network element from one or more of the fourth candidate network elements based on the actual evaluation capability.

20. The method according to claims 9 to 19, characterized in that, The method further includes: The first network element sends a fourth request to the target network element, the fourth request including a fourth desired assessment capability; The first network element receives a fourth response sent by the target network element, the fourth response including third information, the third information being used to indicate the result of the target network element meeting the fourth expected evaluation capability; If the evaluation capability of the target network element does not meet the fourth expected evaluation capability, then the first network element reselects the target network element.

21. The method according to claims 9 to 19, characterized in that, The method further includes: The first network element sends a fourth request to the target network element, the fourth request including actual assessment capability report indication information; The first network element receives a fourth response sent by the target network element, the fourth response including the actual evaluation capability of the target network element; When the first network element determines that the actual evaluation capability of the target network element does not meet the evaluation capability requirements, the first network element reselects the target network element.

22. The method according to any one of claims 9 to 21, characterized in that, The method further includes: The first network element sends an evaluation request to the target network element. The evaluation request is used to request an evaluation of the analysis results. The evaluation request includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, analysis results to be verified, and verification target. The first network element receives the evaluation results sent by the target network element, and the evaluation results include one or more of the following: verification results, impact of analysis results, simulation system evaluation results, and auxiliary information for updating analysis results; The first network element updates the analysis results based on the evaluation results.

23. A method for discovering assessment functions, characterized in that, include: The network data analysis function network element sends a registration request to the network storage function network element. The registration request is used to request the registration of the evaluation function. The registration information in the registration request includes an evaluation capability indication, which is used to instruct the network data analysis function network element to support the evaluation capability. The network data analysis function network element receives a registration response sent by the network storage function network element, and the registration response includes the registration result of the evaluation function.

24. The method according to claim 23, characterized in that, The method further includes: The network data analysis function network element receives an evaluation request sent by the first network element. The evaluation request includes one or more of the following: simulation identifier, simulation capability, auxiliary information reporting instruction, analysis result to be verified and verification target, and analysis result. The evaluation request is used to request the target evaluation function module to evaluate the analysis result. The network data analysis function network element evaluates the analysis results to obtain evaluation results. The evaluation results are used to update the analysis results generated by the first network element. The evaluation results include one or more of the following: verification results, impact of analysis results, simulation system evaluation results, and auxiliary information for updating analysis results. The network data analysis function network element sends the evaluation result to the first network element.

25. A network device, characterized in that, The device includes a processor coupled to a memory, the processor storing instructions that, when executed by the processor, cause the network device to perform the method of any one of claims 1 to 6, or cause the network device to perform the method of any one of claims 7 to 8; or cause the network device to perform the method of any one of claims 9 to 22; or cause the network device to perform the method of any one of claims 23 to 24.

26. A communication system, characterized in that, The device includes a first network device, a second network device, and a third network device. The first network device is used to perform the method of any one of claims 1 to 6, or to perform the method of any one of claims 7 to 8. The second network device is used to perform the method of any one of claims 9 to 22. The third network device is used to perform the method of any one of claims 23 to 24.

27. A computer-readable storage medium having instructions stored thereon, characterized in that, When the instructions are executed, they cause the computer to perform the method of any one of claims 1 to 6, or to perform the method of any one of claims 7 to 8; or to perform the method of any one of claims 9 to 22; or to perform the method of any one of claims 23 to 24.

28. A computer program product, the computer program product comprising instructions, characterized in that, When the instructions are executed, they cause the computer to implement the method of any one of claims 1 to 6, or to implement the method of any one of claims 7 to 8; or to implement the method of any one of claims 9 to 22; or to implement the method of any one of claims 23 to 24.

Images