Analysis performance monitoring method and device

Abstract

The invention discloses an analysis performance monitoring method and device, and the method comprises the steps that a data analysis consumer sends a second request to a network element management system, and the second request is used for requesting the analysis performance of an analysis function based on analysis granularity monitoring; the network element management system sends a first request to a first function network element, the first request is used for triggering monitoring of analysis performance of a first analysis function, and the first analysis function is used for executing data analysis; the first function network element outputs an analysis performance result of the first analysis function according to the first request; and the data analysis consumer obtains an analysis performance result of the first analysis function. According to the method provided by the embodiment of the invention, the analysis performance of the analysis function can be evaluated, and the data analysis execution capability of the analysis function can be determined according to the analysis performance of the analysis function, so that the capability of identifying and solving network problems is improved in a targeted manner, and the network service quality is guaranteed.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an analytical performance monitoring method and apparatus. Background Technology

[0002] Management data analytics service (MDAS) processes and analyzes network and service-related data. This includes analyzing network performance metrics (PM), key performance indicators (KPIs), quality of experience (QoE) reports, alarms, configuration data, network analytics data, and service experience data to provide useful analytical outputs. For example, it can predict faults; analyze network energy consumption and provide recommended energy-saving solutions; analyze network latency issues and identify the root causes of excessive latency, along with suggested solutions. MDAS can also identify problems affecting network and service performance and help proactively identify potential issues that may lead to potential faults and / or performance degradation.

[0003] The function used to perform MDAS can be called the management data analytics function (MDAF), which performs performance metrics on the network function (NF) by acquiring data from the network function (NF).

[0004] In some cases, monitoring the ability of MDAF to process and analyze data to ensure the timeliness of analysis results can quickly help identify and resolve network problems. However, there is currently no way to measure the performance of MDAF. Summary of the Invention

[0005] This application provides an analysis performance monitoring method and apparatus that can evaluate the analysis performance of analysis functions, help determine the data analysis execution capability of analysis functions based on their analysis performance, and thereby improve the ability to identify and solve network problems in a targeted manner, ensuring network service quality.

[0006] In a first aspect, this application provides an analysis performance monitoring method applied to a first functional network element. The method includes: receiving a first request, the first request being used to trigger monitoring of the analysis performance of a first analysis function, the first analysis function being used to perform data analysis; and sending the analysis performance results of the first analysis function according to the first request.

[0007] In this embodiment, the data analysis consumer performs performance evaluation on the data analysis process of the analysis function based on the analysis granularity request, thereby achieving performance evaluation of the first analysis function. This allows the data consumer to obtain analysis performance results based on arbitrary analysis granularity. This can help the data analysis consumer improve its ability to identify and resolve network problems in a targeted manner based on the analysis granularity, ensuring network service quality.

[0008] In one feasible implementation, receiving a first request includes: receiving a first request from a network element management system; and sending the analysis performance result of a first analysis function according to the first request, including: sending the analysis performance result of the first analysis function to the network element management system according to the first request; or sending the analysis performance result of the first analysis function to a data analysis consumer according to the first request.

[0009] In this embodiment, the interaction between the data analysis consumer and the first functional network element is realized through data relay in the network element management system, completing the transmission of the analysis performance results of the first analysis function, which reduces the complexity of the transmission of analysis performance results. Furthermore, the first functional network element directly sends the analysis performance results of the first analysis function to the data analysis consumer, ensuring the timeliness of the transmission of analysis performance results.

[0010] In one feasible implementation, the first request further includes a first analysis type, which is used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

[0011] In one feasible implementation, the first analysis type includes at least one of the following: fault analysis, performance analysis, energy saving analysis, coverage correlation analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource correlation analysis, data correlation analysis, user throughput analysis, and software upgrade verification.

[0012] In this embodiment, the first request carries a first analysis type in order to trigger the analysis performance monitoring of the first analysis type of the first analysis function. This enables the analysis performance of the first analysis function to be obtained at a more accurate granularity, thereby providing a more precise reference for optimizing network services.

[0013] In one feasible implementation, after receiving the first request, the method further includes: creating a target agent for monitoring the analytical performance of the first analytical function.

[0014] In one feasible implementation, the analysis performance result of the first analysis function includes at least one of the following: the time taken for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time.

[0015] In this embodiment, the analysis performance results include the time taken to complete the data analysis, which can measure the analysis efficiency of the first analysis function and thus determine whether it is necessary to switch the analysis function. The analysis performance results include the number of timely deliveries or the timely delivery rate of the data analysis results, or the number of delayed deliveries or the delayed delivery rate, which can measure the analysis efficiency of the first analysis function and the network quality, and determine whether it is necessary to switch the analysis function or whether it is necessary to optimize the network quality.

[0016] In one possible implementation, the first request carries the monitoring time.

[0017] In one feasible implementation, the method further includes: obtaining time information of a first analysis function indication, the time information being related to the analysis performance of the first analysis function; and sending the analysis performance result of the first analysis function according to a first request, including: sending the analysis performance result of the first analysis function according to the time information and the first request.

[0018] In one feasible implementation, the time information includes a first time when the first analysis function receives the request for the first data analysis, and a second time when the result of the first data analysis is output; and / or the time information includes a third time when the first analysis function delivers the result of the second data analysis, and the delivery time of the second data analysis, wherein the delivery time of the second data analysis is a preset latest time for delivering the result of the second data analysis.

[0019] Secondly, this application provides an analysis performance monitoring method, which includes: being applied to a data analysis consumer; the method includes: sending a second request to a network element management system, the second request being used to request analysis performance monitoring of analysis functions based on analysis granularity, the analysis granularity including at least one of the following: network element, slice, subnet, service, or region; and receiving analysis performance results from the analysis functions of the network element management system.

[0020] In one feasible implementation, after sending the second request, the method further includes: receiving an identifier of a first analysis function from the network element management system, the first analysis function corresponding to an analysis granularity; and sending a data analysis request to the first analysis function according to the identifier of the first analysis function, the data analysis request being used to request the first analysis function to perform data analysis.

[0021] In one feasible implementation, the second request further includes a first analysis type, which is used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

[0022] In one feasible implementation, the first analysis type includes at least one of the following: fault analysis, performance analysis, energy saving analysis, coverage correlation analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource correlation analysis, data correlation analysis, user throughput analysis, and software upgrade verification.

[0023] In one feasible implementation, the analysis performance result of the first analysis function includes at least one of the following: the time taken for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time.

[0024] Thirdly, this application provides an analysis performance monitoring method, which includes: being applied to a network element management system; the method includes: receiving a second request from a data analysis consumer, the second request being used to request monitoring of the analysis performance of an analysis function based on analysis granularity, the analysis granularity including at least one of the following: network element, slice, subnet, service, or region; according to the second request, sending a first request to a first functional network element, the first request being used to trigger monitoring of the analysis performance of a first analysis function, the first analysis function being used to perform data analysis, the first analysis function corresponding to the analysis granularity.

[0025] In one feasible implementation, after receiving the second request from the data analytics consumer, the method further includes sending an identifier of the first analytics function to the data analytics consumer.

[0026] In one feasible implementation, the second request further includes a first analysis type, which is used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

[0027] In one feasible implementation, the first analysis type includes at least one of the following: fault analysis, performance analysis, energy saving analysis, coverage correlation analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource correlation analysis, data correlation analysis, user throughput analysis, and software upgrade verification.

[0028] In one feasible implementation, the method further includes: receiving the analysis performance result of the first analysis function from the first functional network element; and sending the analysis performance result of the first analysis function to the data analysis consumer.

[0029] In one feasible implementation, the analysis performance result of the first analysis function includes at least one of the following: the time taken for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time.

[0030] In one possible implementation, the first request carries the monitoring time.

[0031] Fourthly, a communication device is provided, the communication device including units or modules for performing any of the possible methods in the first, second or third aspects described above.

[0032] Fifthly, embodiments of this application provide a communication device, which includes at least one processor and a memory; wherein the memory is used to store computer programs or instructions; and at least one processor is used to execute the computer programs or instructions in the memory, such that the methods that may be implemented in any of the first to third aspects described above are executed.

[0033] Sixthly, embodiments of this application provide a communication system, which includes a first functional network element, a data analysis consumer, and a network element management system. The first functional network element is used to perform the method described in any one of the first aspects above, the data analysis consumer is used to perform the method described in any one of the second aspects above, and the network element management system is used to perform the method described in any one of the third aspects above.

[0034] In a seventh aspect, embodiments of this application provide a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which, when executed, cause the computer to perform the method described in any of the above methods.

[0035] Eighthly, embodiments of this application provide a computer program product, the computer program product including: computer program code, which, when executed by a computer, causes the computer to perform the method described in any of the above methods.

[0036] Ninthly, embodiments of this application provide a chip coupled to a memory for reading and executing program instructions in the memory, so that the device in which the chip is located implements the method described in any of the above methods. Attached Figure Description

[0037] Figure 1A This application provides a schematic diagram of the network architecture of a communication system.

[0038] Figure 1B This application provides another network architecture for a communication system.

[0039] Figure 2A A flowchart of an MDAS provided for an embodiment of the application.

[0040] Figure 2B This is an information model class diagram of MDA provided in an embodiment of this application.

[0041] Figure 3This is a flowchart of an analytical performance monitoring method provided in an embodiment of this application.

[0042] Figure 4A This is a flowchart of another analytical performance monitoring method provided in this embodiment.

[0043] Figure 4B An information model class diagram provided for an embodiment of this application.

[0044] Figure 5A This is a flowchart of another analytical performance monitoring method provided in this embodiment.

[0045] Figure 5B Another information model class diagram provided for embodiments of this application.

[0046] Figure 6 A flowchart of another analytical performance monitoring method provided in the embodiments of this application.

[0047] Figure 7 This is a schematic diagram of the structure of a communication device provided in an embodiment of this application.

[0048] Figure 8 This is a schematic diagram of another communication device provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. The terms "system" and "network" in the embodiments of this application can be used interchangeably. Unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship; for example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be one or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish between network elements and similar items with essentially the same function. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0050] References to "one embodiment" or "some embodiments" in the embodiments described in this application mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0051] The following detailed embodiments further illustrate the objectives, technical solutions, and beneficial effects of this application. It should be understood that the following are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the technical solutions of this application should be included within the scope of protection of this application.

[0052] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0053] The following describes the scenarios involved in the embodiments of this application.

[0054] The technical solution provided in this application can be applied to various communication systems, such as 5G mobile communication systems, future evolution systems, or multiple communication convergence systems, as well as existing communication systems. The application scenarios of the technical solution provided in this application can include various scenarios, such as machine-to-machine (M2M), macro-micro communication, enhanced mobile broadband (eMBB), ultra-reliable and low-latency communication (uRLLC), and massive machine-type communication (mMTC). These scenarios may include, but are not limited to, communication scenarios between terminal devices, communication scenarios between network devices, and communication scenarios between network devices and terminal devices. Network devices include access network devices and core network devices. The following descriptions all use the scenario of communication between network devices and terminal devices as examples.

[0055] See also Figure 1A , Figure 1A This application provides a network architecture for a communication system, which may include: an MDA management service (MnS) consumer and an MDA MnS producer. The MDA MnS consumer requests and retrieves MDA results, while the MDA MnS producer acquires network or service-related data, generates MDA results, and provides them to the MDA MnS consumer. The MDA MnS producer may include a CN domain MDA MnS producer and a RAN domain MDA MnS producer, both of which can obtain service data from the CN domain and the radio access network (RAN) domain, respectively, for MF data analysis. For example, a CN domain MDA MnS provider can obtain the input data required for data analysis from network data analytics function (NWDAF) network elements or other 5th generation core network (5GC) network functions (NF) network elements, while a RAN domain MDA MnS provider can obtain the input data required for data analysis from equipment such as base stations (gNB).

[0056] Or you may refer to Figure 1B , Figure 1B Another network architecture for a communication system provided in this application embodiment may include: an operations support system (OSS), a management data analytics function (MDAF), core network (CN) elements, and a network function virtualization (NFV) management and orchestration (MANO) system.

[0057] MDAF can act as an MDAMS provider, typically residing within the vendor's management domain. The operator's OSS system can act as an MDAMS consumer, sending specific analysis requests, such as fault analysis requests, to MDAF through the services provided by MDAF.

[0058] MDAF can collect PM, FM, and CM related information from network elements (mainly CN network elements, which can be physical network functions (PNFs) or virtual network functions (VNFs) in order to perform MDA.

[0059] When a network element is deployed in the form of a VNF, MDAF can also obtain virtual resource usage data information of the network element through the NFV MANO system, such as the utilization rate of the network element's virtual CPU, storage, and network.

[0060] MADF provides analytical outputs based on the processing and analysis of collected data, including statistics or forecasts, root cause analysis, and optimization suggestions for network and service operations, and generates analytical reports to be sent to OSS.

[0061] The RAN domain includes AN devices. An AN device is a device in a mobile communication system that connects a terminal device to a wireless network. As a node in the radio access network, an AN device can also be referred to as an access network element, base station, RAN node (or device, or network element), access point (AP), network device, small tower, etc. The RAN devices in this application embodiment include, but are not limited to: next-generation base stations (g node B, gNB) in 5G, evolved node B (eNB), radio network controller (RNC), node B (NB), base station controller (BSC), base transceiver station (BTS), home base station (e.g., home-evolved node B, or home node B, HNB), baseband unit (BBU), wireless fidelity (WiFi) access point, world interoperability for microwave access (WiMAX) base station, transmitting and receiving point (TRP), transmitting point (TP), or mobile switching center, etc. In systems employing different radio access technologies, the names of devices with base station functions may vary. For example, in 5G communication systems, they are called RAN or gNB (5G NodeB); in LTE systems, they are called evolved NodeB (eNB or eNodeB); and in 3G communication systems, they are called Node B, etc. In some deployments of AN devices, they can include centralized units (CU) and distributed units (DU). In other deployments, CUs can be further divided into CU-control plane (CP) and CU-user plane (UP). In still other deployments, AN devices can be radio units (RO). In yet another deployment, AN devices can be in an open radio access network (ORAN) architecture, etc.For example, when the AN device is an ORAN architecture, the AN device in this application embodiment can be an access network element in ORAN, or a module of an access network element, etc. In the ORAN system, CU can also be called open (O)-CU, DU can also be called O-DU, CU-CP can also be called O-CU-CP, CU-UP can also be called O-CU-UP, and RU can also be called O-RU.

[0062] Network elements in the CN domain can be divided into two categories: user plane function network elements (also referred to as user plane network elements) and control plane function network elements (also referred to as control plane network elements). Control plane function network elements include access management network elements, network access network elements, session management network elements, data management network elements, policy control network elements, and network slice-specific and SNPN authentication and authorization function (NSSAAF) network elements in 5G communication systems.

[0063] User plane network elements are responsible for forwarding and receiving user data in terminal devices. They can receive user data from the data network and transmit it to the terminal device through the access network equipment; user plane network elements can also receive user data from the terminal device through the access network equipment and forward it to the data network. The transmission resources and scheduling functions that provide services to the terminal device in the user plane network element are managed and controlled by the SMF network element. In 5G communication systems, this user plane network element can be a user plane function (UPF) network element. In future communication systems, the user plane network element can still be a UPF network element, or it can have other names; this application embodiment does not limit this.

[0064] The access management network element is a control plane network element provided by the operator's network, responsible for access control and mobility management of terminal devices accessing the operator's network. This includes functions such as mobility state management, allocation of temporary user identities, authentication, and user management. In 5G communication systems, this access management network element can be an access and mobility management function (AMF) network element. In future communication systems, the access management network element may still be an AMF network element, or it may have other names; this application does not limit this.

[0065] The open network element is primarily responsible for supporting secure interaction between the 3GPP network and third-party applications. It can securely expose network capabilities and events to third parties to enhance or improve application service quality. It can also ensure the secure acquisition of relevant data from third parties by the 3GPP network to enhance intelligent network decision-making. Simultaneously, this network element supports recovering structured data from or storing structured data in a unified database. In 5G communication systems, this open network element can be a Network Exposure Function (NEF) element. In future communication systems, the open network element can still be a NEF element, or it can have other names; this application embodiment does not limit this.

[0066] The session management network element is primarily responsible for session management in mobile networks, such as session establishment, modification, and release. Specific functions include assigning IP addresses to users and selecting user plane network elements that provide packet forwarding capabilities. In 5G communication systems, this session management network element can be a session management function (SMF) network element. In future communication systems, the session management network element may still be an SMF network element, or it may have other names; this application does not limit the specific name.

[0067] The data management network element is used for generating authentication credentials, processing user identifiers (such as storing and managing permanent user identities), and managing access control and subscription information. In 5G communication systems, this data management network element can be a unified data management (UDM) network element. In future communication systems, unified data management can still be a UDM network element, or it can have other names; this application embodiment does not limit this.

[0068] The policy control network element primarily supports providing a unified policy framework to control network behavior, providing policy rules to the control layer network functions, and is also responsible for acquiring user subscription information related to policy decisions. In 4G communication systems, this policy control network element can be a policy and charging rules function (PCRF) network element. In 5G communication systems, this policy control network element can be a policy control function (PCF) network element. In future communication systems, the policy control network element can still be a PCF network element, or it can have other names; this application embodiment does not limit this.

[0069] It is understood that, for ease of description, the term "network element" may be omitted in the following text. For example, in the embodiments of this application, the NWDAF network element and NWDAF have the same meaning; the word "network element" is omitted for ease of description, and the rest are similar. Furthermore, it should be noted that the embodiments of this application do not limit the names of each network element in the communication system. For example, in communication systems of different standards, each network element may have other names; and for example, when multiple network elements are integrated into the same physical device, the physical device may also have other names.

[0070] The prior art involved in the embodiments of this application is described below.

[0071] 1. Network element performance measurement process

[0072] See also Figure 2A , Figure 2A A flowchart for measuring network element performance is provided for the application embodiment, such as Figure 2A As shown, the process for measuring network element performance may include the following steps:

[0073] (1) The network element performance measurement service consumer requests the network element performance measurement service provider to create a measurement task for one or more NFs to perform (service) performance measurement on the NFs.

[0074] (2) The network element performance measurement service provider checks whether it is necessary to collect new measurement types from the NF to be measured.

[0075] (3) If a new metric type needs to be collected: a. The network element performance metric service provider requests the NF to collect performance-related data. b. The network element performance metric service provider receives a request confirmation from the NF (which includes performance-related data).

[0076] (4) The network element performance measurement service provider obtains the performance measurement results of the NF based on the received performance-related data and returns the measurement results (response) to the network element performance measurement service consumer.

[0077] The NF being measured can be gNB, AMF, SMF, UPF, PCF, UDM, NWDAF, etc.

[0078] For example, when the NF being measured is NWDAF, the specific performance metrics can be the duration of NWDAF providing analytics services, the number of service requests received, the number of analytics events delivered late, etc.

[0079] 2. MDA Information Model Class Diagram

[0080] The information model class diagram of MDA is used to represent the model structure that exists in the MDAS process, as well as the relationships between classes.

[0081] Figure 2B An information model class diagram of MDA provided in this application embodiment, such as Figure 2B As shown, the MDA model structure includes two classes: MDA Entity and MDAF (MDAFunction). The MDA Entity acts as a proxy class to represent a subnet, managed network element, or managed function. An MDA Entity consists of one or more MDAFs, indicating that an MDAF can exist within a subnet or a managed function. An MDAF consists of one or more MDA Requests and MDA Reports, with the MDA Requests and MDA Reports being related.

[0082] Please refer to Table 1 for the time-related parameters of MDAS provided in the embodiments of this application:

[0083] Table 1

[0084]

[0085] As shown in Table 1, the analytics window refers to the time period of the task being analyzed. For example, in a fault analysis scenario, this parameter indicates which specific time period the MDAF will analyze for faults (e.g., 7:00-9:00). However, this time parameter can only be used to determine which time period's MDA report to obtain, and cannot determine the service performance of MDAS.

[0086] In summary, current specifications only measure the performance of NFs in the CN or RAN domains, lacking a performance measurement method for the management functions of MDAS (e.g., MDAF). In the management and operations field, end-customer value needs to be considered, and analysis timeliness is one aspect. Customers may want to know the duration of MDAF analysis tasks and whether analysis reports are produced in a timely manner to ensure timely task completion. However, the existing time-related parameters in current specifications cannot measure the timeliness of MDAF analysis reports at different granularities.

[0087] Example 1: Based on this, please refer to Figure 3 , Figure 3 A flowchart of an analytical performance monitoring method provided in this application embodiment is shown below. Figure 3 As shown, the method includes the following steps:

[0088] 101. The data analysis consumer sends a second request to the network element management system. This second request requests performance monitoring based on the analysis granularity, where the analysis granularity includes at least one of the following: network element, slice, subnet, service, or region. Correspondingly, the network element management system receives the second request.

[0089] The data analysis consumer in this embodiment can also be referred to as a management data analysis consumer, MDA consumer, or MDA MnS consumer, etc. The element management system (EMS) is a system used to manage network elements in the network element network (CN). The analysis function can also be referred to as management data analysis function (MDAF), data analysis function, etc., and is used to perform data analysis.

[0090] The data analytics consumer sends a second request to the EMS to request performance monitoring of analytics functions at the granularity of network element, slice, subnet, service, or region. Monitoring analytics performance at the network element level means monitoring the performance of analytics functions on each individual network element, or monitoring the performance of analytics functions on a specific (or type of) network element. Correspondingly, monitoring analytics performance at the slice / subnet / service / region level means monitoring the performance of analytics functions on each slice / subnet / service / region, or monitoring the performance of analytics functions on a specific (or type of) slice / subnet / service / region.

[0091] 102. Based on the second request, the network element management system sends a first request to the first functional network element. The first request is used to trigger the monitoring of the analysis performance of the first analysis function. The first analysis function is used to perform data analysis, and the first analysis function corresponds to the analysis granularity. Correspondingly, the first functional network element receives the first request.

[0092] The primary function network element is one that can monitor the performance of the primary analysis function. In other words, the primary function network element can create a target agent and monitor the performance of the primary analysis function through the target agent. The primary function network element can also be called the primary function, the primary management function (network element), etc.

[0093] After receiving the second request, EMS can send the first request to the first functional network element. The first request triggers the monitoring of the analysis performance of the first analysis function. At this time, the first analysis function is a specific analysis function determined by EMS based on the analysis granularity sent by the data analysis consumer. In other words, the first analysis function is the analysis function corresponding to the analysis granularity in the second request.

[0094] For example, assuming the analysis granularity in the first request is business-level, then the first analysis function is an analysis function that performs data analysis on a specific business function. If multiple analysis functions perform data analysis on a specific business function, then the first analysis function can also include multiple analysis functions.

[0095] Optionally, the first request may also include a first analysis type, which is used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

[0096] Optionally, the first analysis type includes at least one of the following: fault analysis, performance analysis, energy saving analysis, coverage correlation analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource correlation analysis, data correlation analysis, user throughput analysis, and software upgrade verification.

[0097] An analytics function can perform data analysis for multiple analytics types. For example, for a specific business, the first analytics function can perform both fault analysis and energy-saving analysis. Fault analysis is used to predict and prevent potential faults or to quickly restore business operations after a fault has occurred, while energy-saving analysis is used to identify energy efficiency issues and provide energy-saving recommendations. The first request is used to request performance monitoring of the first analytics function for its first analytics type, including performance monitoring of all or some analytics types of the first analytics function.

[0098] 103. The first functional network element sends the analysis performance results of the first analysis function to the network element management system according to the first request. Correspondingly, the network element management system receives the analysis performance results of the first analysis function.

[0099] After receiving the first request, the first functional network element monitors the analysis performance of the first analysis function, obtains the analysis performance results of the first analysis function, and sends them to the EMS.

[0100] Optionally, the analysis performance results of the first analysis function include at least one of the following: the time taken for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time.

[0101] Optionally, the time it takes for the first analysis function to complete the first data analysis refers to the difference between the first time the first analysis function receives the data analysis request and the second time it outputs the result of the first data analysis.

[0102] Optionally, timely completion of data analysis refers to the third time after which the results of the data analysis are delivered, less than (or equal to) the scheduled delivery time of the data analysis. Delayed completion of data analysis refers to the third time after which the results of the data analysis are delivered, greater than (or equal to) the scheduled delivery time of the data analysis. The scheduled delivery time of the data analysis refers to the latest time preset for delivering the results of the data analysis. The proportion of timely completion of data analysis refers to the percentage of data analyses (number) completed timely by the first analysis function out of all data analyses (number) performed by the first analysis function (within the monitoring time). Correspondingly, the proportion of delayed completion of data analysis refers to the percentage of data analyses (time) completed timely by the first analysis function out of all data analyses performed by the first analysis function (within the monitoring time). All data analyses performed by the first analysis function within the monitoring time are the sum of timely completion of data analyses and delayed completion of data analyses.

[0103] The results of data analysis can refer to the data analysis report generated by the first analysis function based on the data analysis request.

[0104] As can be seen, in this embodiment, the analysis performance results include the time taken to complete the data analysis, which can measure the analysis efficiency of the first analysis function and thus determine whether it is necessary to switch the analysis function. The analysis performance results include the number of timely deliveries or the timely delivery rate of the data analysis results, or the number of delayed deliveries or the delayed delivery rate, which can measure the analysis efficiency of the first analysis function and the network quality, and determine whether it is necessary to switch the analysis function or whether it is necessary to optimize the network quality.

[0105] In addition, the time it takes for the first analysis function to complete the first data analysis can be simply referred to as the analysis time of the first data analysis. The number or percentage of data analyses that the first analysis function completes on time within the monitoring period can be simply referred to as the timely delivery quantity or timely delivery rate; the number or percentage of data analyses that the first analysis function delays in completing within the monitoring period can be simply referred to as the delayed delivery quantity or delayed delivery rate.

[0106] Among them, the number of timely deliveries / timely delivery rate and the number of delayed deliveries / delayed delivery rate are both related to the monitoring time. Optionally, the monitoring time can be carried in the first request.

[0107] For example, the monitoring time is 7:00-9:00, indicating that the first functional network element monitors the analytical performance of the first analysis function during this time period. The first request may indicate the start point (7:00) and duration of the monitoring time, or the end point (9:00) and duration of the monitoring time, etc. Alternatively, the monitoring duration can be determined according to preset information or protocol specifications, and the first request may indicate the start or end point of the monitoring time, etc. This embodiment does not limit the method of indicating the monitoring time.

[0108] Optionally, the method further includes the following steps:

[0109] 1031. The first analysis function indicates time information to the first function network element.

[0110] Among them, the time information is the time information related to the analytical performance of the first analytical function.

[0111] 1032 (replacing step 103): The first functional network element sends the analysis performance results of the first analysis function to the network element management system based on the time information and the first request.

[0112] The time information sent by the first analysis function includes at least one of the following:

[0113] (1) The time information includes the first time when the first analysis function receives the request for the first data analysis and the second time when the result of the first data analysis is output.

[0114] This time information can be used to determine the analysis duration of the first data analysis.

[0115] (2) The third time when the first analysis function delivers the results of the second data analysis, and the delivery time of the second data analysis, wherein the delivery time of the second data analysis is the preset latest time for delivering the results of the second data analysis.

[0116] This time information can be used to determine the quantity or rate of on-time deliveries, as well as the quantity or rate of delayed deliveries.

[0117] It should be noted that the time information can be sent by the first analysis function through a dedicated message or an existing message (display indication). Alternatively, the time information can also be determined based on the relevant data interaction between the first analysis function and the first functional network element (implicit indication). For example, the second time when the first analysis function outputs the result of the first data analysis could be the time when the first functional network element receives the first data analysis report sent by the first analysis function.

[0118] 104. The network element management system sends the analysis performance results of the first analysis function to the data analysis consumer.

[0119] After receiving the analysis performance results from the first analysis function, EMS sends them to the analysis consumer.

[0120] Optionally, after the data analysis consumer sends the second request (101), the method further includes:

[0121] 1021. The consumer sends a data analysis request to at least one analysis function, which includes a first analysis function (this step may be before, after, or simultaneously with step 102).

[0122] 1022. The first analysis function performs data analysis based on the data analysis request and outputs the results of the data analysis.

[0123] A data analysis request, also known as a Management Data Analysis Request (MDA request), is used to request (manage) data analysis. A data analysis consumer can send a data analysis request to any analysis function, including the first analysis function. The first analysis function performs data analysis based on the received data analysis request, completes the analysis, and outputs the results, or in other words, a (management) data analysis report (MDA report), to the data analysis consumer.

[0124] Optionally, after the network element management system receives the second request from the data analysis consumer, the method further includes: the network element management system sending an identifier of a first analysis function to the data analysis consumer; and the data analysis consumer sending a data analysis request to the first analysis function.

[0125] For example, after the EMS identifies the first analysis function, it sends the identifier (ID) of the first analysis function to the data analysis consumer, causing the data analysis consumer to send a data analysis request to the specific first analysis function. The first analysis function performs data analysis based on the received data analysis request and outputs a data analysis report.

[0126] The data analysis request may include identification information such as the address or number of the data analysis consumer. The first analysis function will feed back the data analysis report corresponding to each data analysis request to the corresponding data analysis consumer according to the identification information.

[0127] Furthermore, if the data analysis request includes the identification information of the data analysis consumer, the data analysis consumer can also instruct that the analysis performance results corresponding to the data analysis request should be fed back to the data analysis consumer. In this way, after completing the data analysis, the first functional network element can directly feed back the analysis performance results to the data analysis consumer without needing to go through EMS forwarding.

[0128] As can be seen, in this embodiment, the data analysis consumer performs performance evaluation on the data analysis process of the analysis function based on the analysis granularity request, thereby achieving performance evaluation of the first analysis function. This allows the data consumer to obtain analysis performance results based on arbitrary analysis granularity. This can help the data analysis consumer to specifically improve its ability to identify and resolve network problems based on the analysis granularity, ensuring network service quality.

[0129] Furthermore, by using the network element management system as a data relay to facilitate interaction between the data analysis consumer and the first functional network element, the transmission of the analysis performance results of the first analysis function can be completed, thus reducing the complexity of the analysis performance results transmission. Meanwhile, the first functional network element directly sends the analysis performance results of the first analysis function to the data analysis consumer, ensuring the timeliness of the transmission of analysis performance results.

[0130] Example 2: The above examples describe the overall interaction flow of various network elements during the execution of the performance monitoring analysis method. The first functional network element may be located in different positions, which may lead to differences in the specific interaction flow within the performance monitoring analysis method. This example describes the case where the first functional network element is located in a second analysis function, which is different from the first analysis function.

[0131] See also Figure 4A , Figure 4A A flowchart of another analytical performance monitoring method provided in this embodiment is shown below. Figure 4A As shown, the method includes the following steps:

[0132] 201. The data analysis consumer sends a second request, which requests the analysis performance based on the analysis granularity monitoring function. Correspondingly, the EMS receives the second request.

[0133] The description of step 201 can be found in the description of step 101 in the aforementioned embodiment 1, and will not be repeated here.

[0134] 202. EMS sends a first request, which triggers monitoring of the analytical performance of the first analytical function. The first analytical function is the analytical function corresponding to the analytical granularity. Correspondingly, the second analytical function receives the first request.

[0135] In this embodiment, the first functional network element is a network element other than the first analysis function. For example, the first functional network element can be a second analysis function (or perform data analysis), or other functional network elements. When the first functional network element is the second analysis function, the first analysis function and the second analysis function can reside in the same analysis function entity (e.g., the same MDA entity), or they can reside in two different analysis function entities. This embodiment is described with the first functional network element being the second analysis function.

[0136] After receiving the first request, the second analysis function triggers the initiation of monitoring the analysis performance of the first analysis function. Alternatively, after receiving the first request, the second analysis function triggers the creation of a target agent, which then monitors the analysis performance of the first analysis function. Assuming the first request includes a monitoring time, the second analysis function will initiate monitoring of the analysis performance of the first analysis function according to the monitoring time. A description of the monitoring time can be found in step 103 of the aforementioned embodiment, and will not be repeated here.

[0137] 203. Data Analysis: The consumer sends a data analysis request. Correspondingly, the first analysis function receives the data analysis request.

[0138] 204. The first analysis function performs data analysis based on the data analysis request and obtains the data analysis results.

[0139] The results of data analysis can be presented in the form of a data analysis report. Hereafter, we will use "data analysis report" instead of "data analysis results".

[0140] The descriptions of steps 203 and 204 can be found in the descriptions of steps 1021 and 1022 in the aforementioned embodiment one, and will not be repeated here.

[0141] 205. The first analysis function outputs the results of the data analysis. Correspondingly, the second analysis function receives the results of the data analysis.

[0142] In this embodiment, time information related to the analytical performance of the first analysis function is indicated by a data analysis report. Therefore, after the first analysis function completes data analysis and generates a data analysis report, it can output the data analysis report to the second analysis function, so that the second analysis function can assess the analytical performance of the first analysis function based on the time information.

[0143] Optionally, in step 203, the address information of the second analysis function can be carried in the data analysis request sent by the data analysis consumer, so that the first analysis function can send the data analysis report to the second analysis function.

[0144] Data analysis reports can indicate time information in the following ways:

[0145] (1) The first analysis function in the data analysis report receives the data analysis request at the first moment.

[0146] After the second analysis function obtains the first time contained in the data analysis report, it uses this first time as the start time of the data analysis of the first analysis function (e.g., the first data analysis). Then, it uses the time when it receives the data analysis report as the end time of the first data analysis. The difference between the end time and the start time is the analysis time taken for the first analysis function to complete the first data analysis.

[0147] (2) The data analysis report carries the first analysis function and outputs the results of the data analysis at the second time.

[0148] After completing the data analysis based on the received data analysis request, the first analysis function outputs a data analysis report. This report may carry a second time (in the form of a timestamp or other format) indicating when the report was generated or issued. This second time can serve as the end time of the data analysis performed by the first analysis function (e.g., the first data analysis).

[0149] Optionally, the moment the first analysis function receives the data analysis request can also be included in the data analysis report as the start time of the data analysis for the first analysis function.

[0150] Optionally, the second analysis function receives data analysis requests from data analysis consumers and uses the time when the data analysis request is received as the start time of the data analysis.

[0151] Specifically, the path for the data analysis consumer to send a data analysis request to the first analysis function can be: data analysis consumer → first analysis function; or it can be: data analysis consumer → EMS → second analysis function (first function network element) → first analysis function. In this case, the EMS can also send the data analysis request to both the second analysis function and the first analysis function at the same time. In addition, the EMS will inform the first analysis function to perform data analysis for the data analysis request, while the second analysis function will monitor the analysis performance of the first analysis function for the data analysis request.

[0152] (3) Include the delivery time of data analysis in the results of data analysis.

[0153] The data analysis request received by the first analysis function may carry the delivery time set by the data analysis consumer for that data analysis (e.g., the second data analysis). The first analysis function performs data analysis on the data analysis request and outputs a data analysis report, which also carries the delivery time of the second data analysis. After receiving the data analysis report (of the second data analysis) from the first analysis function, the second analysis function retrieves the delivery time from it and can then determine the latest time to deliver the data analysis report of the second data analysis.

[0154] Additionally, it is necessary to obtain the (actual) time when the first analysis function delivers the data analysis report for the second data analysis. The time when the first analysis function delivers the data analysis report can be determined in the following ways:

[0155] 1) The time it takes for the first analysis function to deliver data analysis reports to data analysis consumers.

[0156] The first analysis function receives a data analysis request from a data analysis consumer, completes the data analysis corresponding to the request, and then sends the data analysis report back to the data analysis consumer.

[0157] Assuming that the time when the first analysis function delivers the data analysis report is the same time when the first analysis function delivers the data analysis report to the data analysis consumer, this time information can be carried in the data analysis report sent by the first analysis function to the second analysis function, just as the second time mentioned above, or it can be carried in other special messages.

[0158] 2) The time it takes for the first analysis function to deliver the data analysis report to the second analysis function.

[0159] Assuming that the time when the first analysis function delivers the data analysis report is the same time when the first analysis function delivers the data analysis report to the second analysis function, then the second analysis function can determine the time when the first analysis function delivers the data analysis report based on the time when it receives the data analysis report.

[0160] 3) The time it takes for the second analysis function to deliver data analysis reports to data analysis consumers.

[0161] In one possible scenario, after the first analysis function completes the data analysis, the corresponding data analysis report is sent to the second analysis function, which then delivers it to the data analysis consumer. In this case, the time it takes for the first analysis function to deliver the data analysis report can be considered the time it takes for the second analysis function to deliver the data analysis report to the data analysis consumer.

[0162] The second analysis function obtains the analysis performance results of the first analysis function based on time information, including the analysis time taken for the first analysis function to complete the first data analysis. It may also include the number of timely deliveries, the timely delivery rate, the number of delayed deliveries, and the delayed delivery rate of the first analysis function within the monitoring time. The process of determining the number of timely deliveries, the timely delivery rate, the number of delayed deliveries, and the delayed delivery rate based on time information includes:

[0163] Let A be the time when the data analysis report from the first analysis function is delivered to the second data analysis function, and B be the delivery time of the second data analysis. If A is earlier than (or no later than) B, the second data analysis is considered timely delivery; if A is later than (or no earlier than) B, the second data analysis is considered delayed delivery. Assuming that within the monitoring period, the number of data analyses delivered timely by the first analysis function is the first quantity, and the total number of data analyses delivered is the total quantity, then the timely delivery rate of the first analysis function is: first quantity / (divided by) total quantity * 100%. Similarly, assuming the number of data analyses delivered delayed by the first analysis function is the second quantity, then the delayed delivery rate of the second analysis function is: second quantity / total quantity * 100%.

[0164] In this embodiment, time information related to data analysis performed by the first analysis function is included in the data analysis report, or the time information is determined based on the time when the second analysis function obtains the data analysis report and the time when the second analysis function forwards the data analysis report. This ensures the accuracy of the time information obtained by the second analysis function and improves the accuracy of the analysis results.

[0165] 206. The first analysis function outputs the results of the data analysis. Correspondingly, the data analysis results are received by the consumer.

[0166] After the first analysis function generates the data analysis results, it can also output a data analysis report to the data analysis consumer so that the consumer can obtain the results of the data analysis.

[0167] Optionally, since the second analysis function also receives the data analysis report, the data analysis consumer can subscribe to the data analysis results from the first analysis function via the second analysis function, and then the second analysis function can forward the received data analysis report to the data analysis consumer. That is, step 206 can also be replaced with:

[0168] 206a. Data analysis consumers subscribe to the results of data analysis through the second analysis function.

[0169] 206b. The second analysis function receives the data analysis results from the first analysis function and forwards the data analysis results from the first analysis function to the data analysis consumer.

[0170] To address this situation, the process of establishing a connection between the data analysis consumer and the second analysis function, and using the second analysis function as an intermediary node between the data analysis consumer and the first analysis function, can be as follows:

[0171] (1) After receiving the second request, EMS identifies the first analysis function and the second analysis function, and sends the first analysis function and the second analysis function as associated objects to the data analysis consumer. The data analysis consumer sends a data analysis request to the first analysis function and subscribes to the data analysis report corresponding to the data analysis request from the second analysis function. It also instructs the consumer to send the data analysis report of the data analysis request to the second analysis function.

[0172] (2) The data analysis consumer sends a data analysis request to the first analysis function and subscribes to the data analysis report corresponding to the data analysis request (based on the identifier, number, etc. of the data analysis request) from all analysis functions. After obtaining the corresponding data analysis report, the second analysis function sends it to the data analysis consumer.

[0173] If there is only one secondary analytics function used for analytics performance, then the data analytics consumer subscribes to the data analytics reports from that single secondary analytics function.

[0174] Alternatively, it may include other ways of establishing a connection between data analysis consumers and the second analysis function, which is not limited in this application.

[0175] 207. The second analysis function sends the analysis performance results of the first analysis function. Correspondingly, the EMS receives the analysis performance results of the first analysis function.

[0176] 208. EMS sends the analysis performance results of the first analysis function. Correspondingly, the data analysis consumer receives the analysis performance results of the first analysis function.

[0177] Optionally, steps 207 and 208 can be replaced by step 207' (not shown in the figure): the second analysis function sends the analysis performance results of the first analysis function. Correspondingly, the data analysis consumer receives the analysis performance results of the first analysis function.

[0178] For example, the performance results of the first analysis function can be forwarded to the data analysis consumer by the EMS, or they can be sent directly to the data analysis consumer by the second analysis function. The method by which the second analysis function establishes a connection with the data analysis consumer has been described above and will not be repeated here.

[0179] After receiving the analysis performance results of the first analysis function, the data analysis consumer can determine the quality of the first analysis function based on these results. If the analysis time of the first analysis function is too long, the data analysis report of the first analysis function can be considered unusable; or if the analysis timeliness of the first analysis function is low, the current network quality can be considered poor, and it is necessary to switch analysis functions, etc. This embodiment does not limit how the data analysis consumer optimizes the selection of analysis functions or optimizes network quality.

[0180] To summarize the above description, for the case where the first functional network element is located in the second analysis function, the corresponding information model class diagram can be found in [reference needed]. Figure 4B ,like Figure 4B As shown, the second analysis function (MDAFunciton 2), or the target agent (AnalyticMonitoringAgent) created in the second analysis function, resides within the same analysis function entity (MDA entity) as the first analysis function (MDAFunciton 1), and the second analysis function is associated with the data analysis report (MDAReport). The parameters related to the second analysis function are shown in Table 2 below:

[0181] Table 2

[0182]

[0183] As shown in Table 2, the relevant parameters for the second analysis function include the following:

[0184] Analysis Type: Indicates the type of data analysis performed by the first analysis function. The second analysis function monitors the analysis performance of this analysis type from the first analysis function.

[0185] Analysis granularity (including subnet / slice / service / region / network element ID, etc.): This indicates that the second analysis function performs analysis performance monitoring on the first analysis function based on the analysis granularity.

[0186] Analysis performance monitoring metrics: (data analysis) analysis duration, on-time delivery rate / delayed delivery rate of data analysis reports, number of delayed data analysis reports delivered / number of on-time data analysis reports delivered.

[0187] Monitoring time: refers to the time during which the second analysis function monitors the performance of the first analysis function. This includes the monitoring start time and the monitoring end time.

[0188] Domain name of the data analysis report: If the second analysis function receives the data analysis report from the first analysis function and establishes an association with the domain name of the data analysis report, then the second analysis function can read the time information in the data analysis report and obtain the analysis performance results of the first analysis function.

[0189] As can be seen, in this embodiment, the data analysis consumer performs performance evaluation on the data analysis process of the analysis function based on the analysis granularity request, thereby evaluating the analysis performance of the first analysis function. This allows the data consumer to obtain analysis performance results based on arbitrary analysis granularity. This can help the data analysis consumer improve its ability to identify and resolve network problems in a targeted manner based on the analysis granularity, ensuring network service quality. Furthermore, since the first functional network element is located in a second analysis function that is different from the first analysis function, the process of monitoring analysis performance is executed by the second analysis function, which can reduce the impact on the data analysis of the first analysis function and ensure the efficiency of data analysis.

[0190] Example 3: This example describes the case where the first functional network element is located in the first analysis function.

[0191] See also Figure 5A , Figure 5A A flowchart of another analytical performance monitoring method provided in this embodiment is shown below. Figure 5A As shown, the method includes the following steps:

[0192] 301. The data analysis consumer sends a second request, which requests the performance of the analysis function based on the analysis granularity, wherein the analysis granularity includes at least one of the following: network element, slice, subnet, service, or region. Correspondingly, the network element management system receives the second request.

[0193] The description of step 301 can be found in the description of step 101 in the aforementioned embodiment 1, and will not be repeated here.

[0194] 302. Based on the second request, the network element management system sends a first request to the first functional network element (first analysis function). The first request is used to trigger monitoring of the analysis performance of the first analysis function, which is used to perform data analysis. The first analysis function corresponds to the analysis granularity. Correspondingly, the first functional network element receives the first request.

[0195] In this embodiment, the first functional network element is the first analysis function, which includes two possibilities: 1) Each analysis function can both perform data analysis and monitor its own analysis performance, or it can create a target agent to monitor its own analysis performance. 2) The first analysis function, determined according to the analysis granularity, happens to have the ability to monitor its own analysis performance.

[0196] Since the first functional network element overlaps with the first analysis function, the first analysis function and the first functional network element will be described in the following description, that is, the first functional network element will no longer be described separately.

[0197] 303. Data Analysis: The consumer sends a data analysis request. Correspondingly, the first analysis function receives the data analysis request.

[0198] The methods and processes for data analysis consumers to send data analysis requests and for the first analysis function to receive data analysis requests can be found in the relevant description of step 1021 above, and will not be repeated here.

[0199] 304. The first analysis function performs data analysis based on the data analysis request and obtains the results of the data analysis.

[0200] 305. The first analysis function sends the results of data analysis to the data analysis consumer.

[0201] In this embodiment, after obtaining the data analysis results, the first analysis function can directly send them to the data analysis consumer. For example, the data analysis request sent by the data analysis consumer includes the consumer's identification information, and the first analysis function can then provide the data analysis report to the consumer based on this identification information.

[0202] 306. The first analysis function obtains the analysis performance results of the first analysis function.

[0203] In this embodiment, the analysis performance is monitored by the first analysis function itself. As described above, the analysis performance results may include at least one of the following: the time it takes for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time. The time it takes for the first analysis function to complete the first data analysis may also be referred to as the analysis duration of the first data analysis; the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time may also be referred to as the timely delivery quantity or timely delivery rate; and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time may also be referred to as the delayed delivery quantity or delayed delivery rate.

[0204] The analysis time for the first data analysis refers to the difference between the first time the first analysis function receives the data analysis request and the second time the first analysis function outputs the results of the first data analysis. In this embodiment, the second time when the first analysis function outputs the results of the first analysis data can be either the time when the first analysis function generates the data analysis report or the time when the first analysis function sends the data analysis report to the data analysis consumer.

[0205] The timely delivery quantity, timely delivery rate, delayed delivery, and delayed delivery rate are related to the third time of delivery of the results of the first data analysis, as well as the preset delivery time of the results of the first data analysis (the latest time to deliver the results of the first data analysis) (see step 103 in the aforementioned embodiment for a detailed description). In this embodiment, the third time of delivery of the data analysis results may be the time when the first analysis function sends the data analysis report to the data analysis consumer. The delivery time of the data analysis may be carried in the data analysis request corresponding to the data analysis.

[0206] 307. The first analysis function sends the analysis performance results of the first analysis function, and correspondingly, the EMS receives the analysis performance results of the first analysis function.

[0207] 308. EMS sends the analysis performance results of the first analysis function. Correspondingly, the data analysis consumer receives the analysis performance results of the first analysis function.

[0208] In this embodiment, the first analysis function requested for monitoring by the data analysis consumer is determined by the EMS. Therefore, the EMS can also obtain the analysis performance results of the first analysis function and forward them to the data analysis consumer.

[0209] Optionally, the first analysis function can directly send its analysis performance results to the data analysis consumer. Specifically, after determining the first analysis function based on the analysis granularity, the EMS can send the identifier of the first analysis function to the data analysis consumer. When sending a data analysis request to the first analysis function, the data analysis consumer can include its own identifier in the request, instructing the first analysis function to provide its analysis performance results back to the consumer. The data analysis consumer determines the performance of the first analysis function to optimize its selection or network quality, etc., which is not limited in this embodiment.

[0210] There is no strict order between steps 305 and steps 306-307.

[0211] To summarize the above description, for the case where the first functional network element is located in the first analysis function, the corresponding information model class diagram can be found in [reference needed]. Figure 5B ,like Figure 5B As shown, the first functional network element (Funciton1), or the target agent (AnalyticMonitoringAgent) created in the first analysis function, is located in the same analysis function entity (MDA entity) as the first analysis function (MDAFunciton1), and the first functional network element is associated with the data analysis report (MDAReport) and the data analysis request (MDARequest). The parameters related to the first functional network element are shown in Table 3 below:

[0212] Table 3

[0213]

[0214] As shown in Table 3 above, the parameters related to the first functional network element in this embodiment, compared with Table 2 in Embodiment 2, include the addition of the domain name for data analysis requests.

[0215] As can be seen, in this embodiment, the data analysis consumer performs performance evaluation on the data analysis process of the analysis function based on the analysis granularity request, thereby achieving performance evaluation of the first analysis function. This allows the data consumer to obtain analysis performance results based on arbitrary analysis granularity. This can help the data analysis consumer improve its ability to identify and resolve network problems in a targeted manner based on the analysis granularity, ensuring network service quality. Furthermore, since the first functional network element is set within the first analysis function, the first analysis function both performs data analysis and performs performance evaluation on its own data analysis process. This reduces the number of data interactions through interfaces, improves the efficiency of generating analysis performance results, and ensures the timeliness of the first analysis function's analysis performance.

[0216] The above-described Examples 2 and 3 both address the scenario where a data analysis consumer requests performance monitoring for an unknown (based on the granularity of analysis) analytical function. The following describes the scenario where a data analysis consumer requests performance monitoring for a known first analytical function.

[0217] Example 4: See below Figure 6 , Figure 6 A flowchart of another analytical performance monitoring method provided in the embodiments of this application is shown below. Figure 6 As shown, the method includes the following steps:

[0218] 401. The data analysis consumer sends a first request, which requests monitoring of the analysis performance of the first analysis function. Correspondingly, the second analysis function receives the first request.

[0219] In this embodiment, since the data analysis consumer is already aware that the analysis performance of the first analysis function needs to be monitored, the data analysis consumer can directly send a message to the second analysis function requesting monitoring of the analysis performance of the first analysis function.

[0220] The first request may include the monitoring time. A description of the monitoring time can be found in the relevant description in step 103 of the aforementioned embodiment, and will not be repeated here.

[0221] 402. Data Analysis: The consumer sends a data analysis request. Correspondingly, the first analysis function receives the data analysis request.

[0222] 403. The first analysis function performs data analysis based on the data analysis request and obtains the results of the data analysis.

[0223] 404. The first analysis function sends the results of the data analysis. Correspondingly, the data analysis consumer receives the results of the data analysis.

[0224] The process by which consumers request the first analysis function to perform data analysis and obtain the results from the first analysis function does not require the participation of the second analysis function, and will not be elaborated here.

[0225] 405. The second analysis function obtains time information from the first analysis function.

[0226] Consistent with the foregoing description, the analysis performance results may include at least one of the following: the time taken for the first analysis function to complete the first data analysis, or the analysis time of the first data analysis; the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, or the number or rate of timely delivery; the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time, or the number or rate of delayed delivery.

[0227] The analysis time of the first data analysis refers to the difference between the first time the first analysis function receives the data analysis request and the second time the first analysis function outputs the result of the first data analysis.

[0228] The on-time delivery quantity, on-time delivery rate, delayed delivery, and delayed delivery rate are related to the third time of delivery of the results of the first data analysis, as well as the preset delivery time of the results of the first data analysis.

[0229] In this embodiment, the first analysis function can record a first time, a second time, and a third time, and then send these as time information to the second analysis function. The delivery time of the first data analysis can be carried in the data analysis request sent by the data analysis consumer. The first analysis function reads the data analysis request and obtains the delivery time of the first data analysis, and then forwards it as time information to the second analysis function.

[0230] Before the first analysis function sends time information to the second analysis function, the second analysis function can also send a time information request to the first analysis function to request time information. Alternatively, the second analysis function can also subscribe to time information from the first analysis function.

[0231] 406. The second analysis function performs performance analysis based on time information to obtain the analysis performance results of the first analysis function.

[0232] The second analysis function performs performance analysis based on the time information obtained within the monitoring period to obtain the analysis performance results. The specific process for obtaining the analysis performance results can be found in the relevant description of step 205 in the aforementioned embodiment two, and will not be repeated here.

[0233] 407. The second analysis function sends the analysis performance results of the first analysis function. Correspondingly, the data analysis consumer receives the analysis performance results of the first analysis function.

[0234] After obtaining the analysis performance results of the first analysis function, the second analysis function can send them to the data analysis consumer so that the data analysis consumer can determine the quality of the first analysis function's analysis performance and thus optimize the selection of analysis functions or optimize network quality, etc. This embodiment does not limit this.

[0235] As can be seen, in this embodiment of the application, when a data analysis consumer requests a second analysis function to perform an analysis performance evaluation on a specific first analysis function, it can quickly and specifically obtain time information during the analysis of the network element of the first function, so that the second analysis function can quickly obtain the analysis performance results of the first analysis function, thus ensuring the timeliness of obtaining the analysis performance results of the first analysis function.

[0236] Please see Figure 7 , Figure 7 This is a schematic diagram of a communication device provided in an embodiment of this application. This communication device can be used to execute any of the methods described in the foregoing embodiments.

[0237] like Figure 7 As shown, the communication device includes a processing module 1501 and a transceiver module 1502. The processing module 1501 may be one or more processors, and the transceiver module 1502 may be a transceiver or a communication interface. This communication device can be used to implement the functions of the first functional network element, the data analysis consumer, and the network element management system involved in any of the above method embodiments. These network elements or network functions may be network components in hardware devices, software functions running on dedicated hardware, or virtualized functions instantiated on a platform (e.g., a cloud platform). Optionally, the communication device may also include a storage module 1503 for storing the program code and data of the communication device.

[0238] In a first instance, the communication device can function as a first device or a chip within a first device, and execute the steps performed by the first functional network element in embodiments one through four of the above method. The transceiver module 1502 supports communication with data analysis consumers or network element management systems, etc. The processing module 1501 can be used to support the execution of actions performed by the first functional network element in the above method embodiments, excluding sending and receiving.

[0239] Specifically, the transceiver module 1502 is used to receive a first request, which is used to trigger the monitoring of the analysis performance of the first analysis function, and the first analysis function is used to perform data analysis; the processing module 1501, according to the first request, sends the analysis performance results of the first analysis function in conjunction with the transceiver module 1502.

[0240] In one feasible implementation, receiving a first request includes: receiving a first request from a network element management system; and sending the analysis performance results of a first analysis function according to the first request includes: sending the analysis performance results of the first analysis function to the network element management system according to the first request.

[0241] In one feasible implementation, the first request further includes a first analysis type, which is used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

[0242] In one feasible implementation, the first analysis type includes at least one of the following: fault analysis, performance analysis, energy saving analysis, coverage correlation analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource correlation analysis, data correlation analysis, user throughput analysis, and software upgrade verification.

[0243] In one feasible implementation, after receiving the first request, the processing module 1501 is further configured to: create a target agent, which is used to monitor the analysis performance of the first analysis function.

[0244] In one feasible implementation, the analysis performance result of the first analysis function includes at least one of the following: the time taken for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time.

[0245] In one possible implementation, the first request carries the monitoring time.

[0246] In one feasible implementation, the transceiver module 1502 is further configured to: obtain time information of the first analysis function indication, the time information being related to the analysis performance of the first analysis function; and send the analysis performance result of the first analysis function according to the first request, including: sending the analysis performance result of the first analysis function according to the time information and the first request.

[0247] In one feasible implementation, the time information includes a first time when the first analysis function receives the request for the first data analysis, and a second time when the result of the first data analysis is output; and / or the time information includes a third time when the first analysis function delivers the result of the second data analysis, and the delivery time of the second data analysis, wherein the delivery time of the second data analysis is a preset latest time for delivering the result of the second data analysis.

[0248] In a second example, the communication device can function as a second device or a chip within a second device, and execute the steps performed by the data analysis consumer in embodiments one through four of the above method. The transceiver module 1502 supports communication with the first functional network element or network element management system, etc. The processing module 1501 can be used to support the execution of actions other than sending and receiving performed by the data analysis consumer in the above method embodiments.

[0249] Specifically, the transceiver module 1502 is used to send a second request to the network element management system. The second request is used to request the analysis performance of the analysis function based on the analysis granularity. The analysis granularity includes at least one of the following: network element, slice, subnet, service, or region. The transceiver module 1502 is also used to receive the analysis performance results of the analysis function from the network element management system.

[0250] In one feasible implementation, after sending the second request, the transceiver module 1502 is further configured to: receive the identifier of the first analysis function from the network element management system, the first analysis function corresponding to the analysis granularity; and the processing module 1501 is configured to send a data analysis request to the first analysis function according to the identifier of the first analysis function, the data analysis request being used to request the first analysis function to perform data analysis.

[0251] In one feasible implementation, the second request further includes a first analysis type, which is used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

[0252] In one feasible implementation, the first analysis type includes at least one of the following: fault analysis, performance analysis, energy saving analysis, coverage correlation analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource correlation analysis, data correlation analysis, user throughput analysis, and software upgrade verification.

[0253] In one feasible implementation, the analysis performance result of the first analysis function includes at least one of the following: the time taken for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time.

[0254] In a third example, the communication device can function as a third device or a chip within a third device, and execute the steps performed by the network element management system in embodiments one through four of the above methods. The transceiver module 1502 supports communication with the first functional network element or the data analysis consumer. The processing module 1501 can be used to support the execution of actions other than sending and receiving performed by the network element management system in the above method embodiments.

[0255] Specifically, the transceiver module 1502 is used to: receive a second request from a data analysis consumer, the second request being used to request monitoring the analysis performance of the analysis function based on the analysis granularity, the analysis granularity including at least one of the following: network element, slice, subnet, service, or region; and the processing module 1501 is used to send a first request to the first functional network element according to the second request, in conjunction with the transceiver module 1502, the first request being used to trigger monitoring of the analysis performance of the first analysis function, the first analysis function being used to perform data analysis, and the first analysis function corresponding to the analysis granularity.

[0256] In one feasible implementation, after receiving the second request from the data analysis consumer, the transceiver module 1502 is further configured to: send the identifier of the first analysis function to the data analysis consumer.

[0257] In one feasible implementation, the second request further includes a first analysis type, which is used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

[0258] In one feasible implementation, the first analysis type includes at least one of the following: fault analysis, performance analysis, energy saving analysis, coverage correlation analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource correlation analysis, data correlation analysis, user throughput analysis, and software upgrade verification.

[0259] In one feasible implementation, the method further includes: receiving the analysis performance result of the first analysis function from the first functional network element; and sending the analysis performance result of the first analysis function to the data analysis consumer.

[0260] In one feasible implementation, the analysis performance result of the first analysis function includes at least one of the following: the time taken for the first analysis function to complete the first data analysis, the number or percentage of data analyses completed by the first analysis function in a timely manner within the monitoring time, and the number or percentage of data analyses completed by the first analysis function with delay within the monitoring time.

[0261] In one possible implementation, the first request carries the monitoring time.

[0262] The processing module 1501 may be a processor that can execute computer execution instructions stored in the storage module to cause the chip to perform the methods involved in any of the above embodiments.

[0263] Furthermore, a processor may include a controller, an arithmetic logic unit (ALU), and registers. For example, the controller is primarily responsible for instruction decoding and issuing control signals for the operations corresponding to the instructions. The ALU is primarily responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations, and can also perform address operations and translations. Registers are primarily responsible for storing register operands and intermediate operation results temporarily stored during instruction execution. In specific implementations, the processor's hardware architecture can be an ASIC architecture, a microprocessor without interlocked piped stages architecture (MIPS), an advanced reduced instruction set machine (RISC) machine (ARM) architecture, or a network processor (NP) architecture, etc. The processor can be single-core or multi-core.

[0264] The storage module can be an internal storage module of the chip, such as a register or cache. Alternatively, the storage module can be an external storage module, such as ROM or other types of static storage devices that can store static information and instructions, such as RAM.

[0265] It should be noted that the functions of the processor and interface can be implemented through hardware design, software design, or a combination of both; no restrictions are imposed here.

[0266] like Figure 8 The diagram shows a structural schematic of another communication device provided in this application embodiment. The communication device 1100 includes a processor 1101. Optionally, the communication device 1100 may further include an interface circuit 1102, with the processor 1101 and the interface circuit 1102 coupled to each other. It is understood that the interface circuit 1102 can be a transceiver or an input / output interface. Optionally, the communication device 1100 may further include a memory 1103 (shown as dashed lines in the figure), which stores instructions executed by the processor 1101, or input data required by the processor 1101 to execute instructions, or data generated after the processor 1101 executes instructions.

[0267] For specific implementation details of the processor 1101, interface circuit 1102, and memory 1103 described above, please refer to the relevant descriptions in Embodiments 1 to 4.

[0268] When the aforementioned communication device is a chip applied in the first device, the chip implements the function of the first functional network element in the above method embodiment. The chip receives information from other modules (such as radio frequency modules or antennas) in the first device, which is sent to the first analysis function by the data analysis consumer or the network element management system; or, the chip sends information to other modules (such as radio frequency modules or antennas) in the first device, which is sent to the data analysis consumer or the network element management system by the first functional network element.

[0269] When the aforementioned communication device is a chip applied to the second device, the chip implements the data analysis consumer function in the above method embodiments. The chip receives information from other modules (such as an RF module or antenna) in the second device, which is sent to the data analysis consumer by the first functional network element or network element management system; or, the chip sends information to other modules (such as an RF module or antenna) in the second device, which is sent to the first functional network element or network element management system by the data analysis consumer.

[0270] When the aforementioned communication device is a chip applied to a third device, the chip implements the functions of the network element management system in the above method embodiments. The chip receives information from other modules (such as radio frequency modules or antennas) in the second device, which is sent to the network element management system by the first functional network element or data analysis consumer; or, the chip sends information to other modules (such as radio frequency modules or antennas) in the second device, which is sent to the first functional network element or data analysis consumer by the network element management system.

[0271] Furthermore, it should be noted that the aforementioned transceiver unit and / or processing unit can be implemented through virtual modules. For example, the processing unit can be implemented through software functional units or virtual devices, and the transceiver unit can be implemented through software functions or virtual devices. Alternatively, the processing unit or transceiver unit can also be implemented through physical devices. For example, if the device is implemented using a chip / chip circuit, the transceiver unit can be an input / output circuit and / or a communication interface, performing input operations (corresponding to the aforementioned receiving operation) and output operations (corresponding to the aforementioned sending operation); the processing unit is an integrated processor, microprocessor, or integrated circuit.

[0272] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0273] This application provides a communication device, which includes at least one processor and a memory; wherein the memory is used to store computer programs or instructions; and at least one processor is used to execute the computer programs or instructions in the memory, such that the methods corresponding to each device or network element in any of the above methods are executed.

[0274] This application provides a communication system, which includes a first device corresponding to a first functional network element, a second device corresponding to a data analysis consumer, and a third device corresponding to a network element management system.

[0275] This application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which, when executed, cause the computer to perform the method described in any of the above methods.

[0276] This application provides a computer program product, which includes computer program code. When the computer program code is run, it causes the computer to perform the method described in any of the above methods.

[0277] This application provides a chip coupled to a memory for reading and executing program instructions in the memory, so that the device containing the chip implements the method described in any of the above methods.

[0278] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a particular embodiment can be found in the relevant descriptions of other embodiments. It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0279] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0280] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0281] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An analytical performance monitoring method, characterized by, The method applied to a first functional network element comprises: receiving a first request, the first request being used to trigger monitoring of analysis performance of a first analysis function, the first analysis function being used to perform data analysis; sending an analysis performance result of the first analysis function according to the first request.

2. The method of claim 1, wherein, The receiving of the first request comprises: receiving the first request from a network element management system; The sending of the analysis performance result of the first analysis function according to the first request comprises: sending the analysis performance result of the first analysis function to the network element management system according to the first request; or sending the analysis performance result of the first analysis function to a data analysis consumer according to the first request.

3. The method according to claim 1 or 2, characterized in that, The first request further comprises a first analysis type, the first request being used to trigger analysis performance monitoring of the first analysis type of the first analysis function.

4. The method of claim 3, wherein, The first analysis type comprises at least one of fault analysis, performance analysis, energy saving analysis, coverage related analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource related analysis, data correlation analysis, user throughput rate analysis, and software upgrade verification.

5. The method according to any one of claims 1 to 4, characterized in that, After the receiving of the first request, the method further comprises: creating a target agent, the target agent being used to monitor the analysis performance of the first analysis function.

6. The method according to any one of claims 1 to 5, characterized in that, The analysis performance result of the first analysis function comprises at least one of a time length for the first analysis function to complete first data analysis, a number or a proportion of data analysis completed in time by the first analysis function within a monitoring time, and a number or a proportion of data analysis delayed by the first analysis function within the monitoring time.

7. The method of claim 6, wherein, The monitoring time is carried in the first request.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: obtaining time information indicated by the first analysis function, the time information being related to the analysis performance of the first analysis function; The sending of the analysis performance result of the first analysis function according to the first request comprises: sending the analysis performance result of the first analysis function according to the time information and the first request.

9. The method of claim 8, wherein, The time information comprises a first time at which the first analysis function receives a request for first data analysis, and a second time at which a result of the first data analysis is outputted; And / or The time information comprises a third time at which the first analysis function delivers a result of second data analysis, and a delivery time of the second data analysis, the delivery time of the second data analysis being a preset latest time for delivering the result of the second data analysis.

10. An analytical performance monitoring method characterized by, The method applied to a data analysis consumer comprises: sending a second request to a network element management system, the second request being used to request monitoring of analysis performance of an analysis function based on an analysis granularity, the analysis granularity comprising at least one of a network element, a slice, a sub-network, a service, or a region; receiving an analysis performance result of the analysis function from the network element management system.

11. The method of claim 10, wherein, After the sending of the second request, the method further comprises: receiving an identification of a first analysis function from the network element management system, the first analysis function corresponding to the analysis granularity; According to the identifier of the first analytics function, a data analysis request is sent to the first analytics function, the data analysis request being used to request the first analytics function to perform data analysis.

12. The method according to claim 10 or 11, characterized in that, The second request further comprises a first analysis type, and the first request is used to trigger analysis performance monitoring of the first analysis type of the first analytics function.

13. The method of claim 12, wherein, The first analysis type comprises at least one of fault analysis, performance analysis, energy saving analysis, coverage related analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource related analysis, data correlation analysis, user throughput rate analysis, and software upgrade verification.

14. The method according to any one of claims 11-13, characterized in that, The analysis performance result of the first analytics function comprises at least one of a time length for the first analytics function to complete first data analysis, a number or a proportion of data analysis completed in time by the first analytics function within a monitoring time, and a number or a proportion of data analysis completed with delay by the first analytics function within the monitoring time.

15. An analytical performance monitoring method characterized by, The method is applied to a network element management system, and the method comprises: receiving a second request from a data analysis consumer, the second request being used to request monitoring of analysis performance of an analytics function based on an analysis granularity, the analysis granularity comprising at least one of a network element, a slice, a sub-network, a service, or a region; according to the second request, sending a first request to a first function network element, the first request being used to trigger monitoring of analysis performance of a first analytics function, the first analytics function being used to perform data analysis, and the first analytics function corresponding to the analysis granularity.

16. The method of claim 15, wherein, After the receiving of the second request from the data analysis consumer, the method further comprises: sending an identifier of the first analytics function to the data analysis consumer.

17. The method according to claim 15 or 16, characterized in that, The second request further comprises a first analysis type, and the first request is used to trigger analysis performance monitoring of the first analysis type of the first analytics function.

18. The method of claim 17, wherein, The first analysis type comprises at least one of fault analysis, performance analysis, energy saving analysis, coverage related analysis, service level specification analysis, mobility management analysis, maintenance management analysis, resource related analysis, data correlation analysis, user throughput rate analysis, and software upgrade verification.

19. The method according to any one of claims 15-18, characterized in that, The method further comprises: receiving an analysis performance result of the first analytics function from the first function network element; sending the analysis performance result of the first analytics function to the data analysis consumer.

20. The method of claim 19, wherein, The analysis performance result of the first analytics function comprises at least one of a time length for the first analytics function to complete first data analysis, a number or a proportion of data analysis completed in time by the first analytics function within a monitoring time, and a number or a proportion of data analysis completed with delay by the first analytics function within the monitoring time.

21. The method of claim 20, wherein, The first request carries the monitoring time.

22. A communications device, characterized by The apparatus comprises units or modules for implementing the method according to any one of claims 1 to 21.

23. A communication apparatus, comprising: at least one processor and a memory. The memory is configured to store computer programs or instructions; the at least one processor is configured to execute the computer programs or instructions in the memory, so that the method in any one of claims 1-21 is executed.

24. A communication system, comprising: a first functional network element, a data analytics consumer, and a network element management system; wherein the first functional network element is configured to execute the method in any one of claims 1-9, the data analytics consumer is configured to execute the method in any one of claims 10-14, and the network element management system is configured to execute the method in any one of claims 15-21.

25. A chip system, characterized by The chip system comprises at least one processor, a memory, and an interface circuit, the memory, the interface circuit, and the at least one processor are interconnected by a line, and the at least one memory stores instructions; when the instructions are executed by the processor, the method in any one of claims 1-21 is implemented.

26. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program is executed, the method in any one of claims 1-21 is implemented.

27. A computer program product, characterised in that, The computer program product comprises instructions, and when the instructions are executed, the method in any one of claims 1-21 is implemented.

Images