Network performance management method, storage medium, electronic device, and computer program product

Abstract

Embodiments of the present application provide a network performance management method, a storage medium, an electronic device and a computer program product, wherein the method comprises: sending an inference request to a second network element; receiving an inference report corresponding to an inference category sent by the second network element, wherein the inference report comprises at least one of the following: a set of recommended measures, managed object information, and a network performance indicator; and managing network performance according to the managed object information, the network performance indicator and a preset policy. Through the embodiments of the present application, the first network element can receive the managed object information and the network performance indicator generated by the model inference, and provide a flexible network performance management mode based on the managed object information, the network performance indicator and the preset policy, thereby achieving the effect of guaranteeing network performance and solving the problem that it is difficult to guarantee network performance when multiple models are deployed in the related art.

Description

Technical Field

[0001] This application relates to the field of communications, and more specifically, to a network performance management method, storage medium, electronic device, and computer program product. Background Technology

[0002] With the rapid development of artificial intelligence and machine learning technologies, numerous models are widely deployed in various network environments to perform complex tasks such as fault analysis, network performance optimization, and network energy saving. However, the simultaneous deployment of multiple models can cause mutual interference, leading to performance degradation when executing model inference results, and consequently, an overall degrade in network performance. For example, network energy saving and Service Level Agreement (SLA) guarantees often conflict.

[0003] In summary, there is still no good solution to the problem of ensuring network performance when deploying multiple models in existing technologies. Summary of the Invention

[0004] This application provides a network performance management method, storage medium, electronic device, and computer program product to at least solve the problem of difficulty in guaranteeing network performance when deploying multiple models in related technologies.

[0005] According to one embodiment of this application, a network performance management method is provided, comprising: sending an inference request to a second network element; wherein the inference request includes an inference category; receiving an inference report corresponding to the inference category sent by the second network element, wherein the inference report includes at least one of the following: a set of suggested measures, managed object information, and network performance indicators; managing network performance based on the managed object information, network performance indicators, and preset policies; wherein the set of suggested measures, managed object information, and network performance indicators are generated by the second network element through model inference corresponding to the inference category.

[0006] According to another embodiment of this application, a network performance management method is also provided, comprising: receiving an inference request sent by a first network element; wherein the inference request includes an inference category; sending an inference report corresponding to the inference category, including a set of suggested measures, managed object information, and network performance indicators, to the first network element, so that the first network element manages network performance according to the managed object information, network performance indicators, and preset strategies; wherein the set of suggested measures, managed object information, and network performance indicators are generated by a second network element through a model corresponding to the inference category.

[0007] According to yet another embodiment of this application, a computer-readable storage medium is also provided, which stores a computer program, wherein the computer program is executed by a processor to perform the steps in any of the above method embodiments.

[0008] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0009] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0010] In this application embodiment, a network performance management method, storage medium, electronic device, and computer program product are designed. Since the first network element can receive the managed object information and network performance indicators generated by model inference, and provide a flexible network performance management method based on the managed object information and network performance indicators combined with preset strategies, the network performance is guaranteed, thus solving the problem of difficulty in guaranteeing network performance when deploying multiple models in related technologies. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the network architecture of the network performance management method according to an embodiment of this application;

[0012] Figure 2 This is a hardware structure block diagram of a computer terminal for a network performance management method according to an embodiment of this application;

[0013] Figure 3 This is a flowchart of a network performance management method for a first network element according to an embodiment of this application;

[0014] Figure 4 This is a flowchart of a network performance management method for a second network element according to an embodiment of this application;

[0015] Figure 5 This is a flowchart (a) illustrating the implementation of a network performance remediation strategy in one embodiment of this application;

[0016] Figure 6 This is a flowchart (II) illustrating the implementation of a network performance remediation strategy in one embodiment of this application;

[0017] Figure 7 This is a flowchart (a) illustrating the implementation of a strategy to prevent network performance degradation in one embodiment of this application;

[0018] Figure 8 This is a flowchart (II) illustrating the implementation of a strategy to prevent network performance degradation in one embodiment of this application. Detailed Implementation

[0019] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar management objects and are not necessarily used to describe a specific order or sequence.

[0021] Figure 1 This is a schematic diagram of the network architecture of the network performance management method according to an embodiment of this application. The network architecture includes: a Business Support System (BSS), a Cross Domain Management Function (CD-MnF), a Domain Management Function (Domain-MnF), and Network Elements (NEs). The CD-MnF manages one or more Domain Management Functions. The Domain Management Functions can manage one or more Network Elements.

[0022] Business support systems are geared towards telecommunications services, providing functions and management services such as billing, settlement, accounting, customer service, sales, network monitoring, telecommunications service lifecycle management, and service intent translation. These systems can be either operator systems or systems specific to vertical industries.

[0023] A cross-domain management function unit, also known as a network management function (NMF), can be a network management entity such as a network management system (NMS), a network management service producer (MnS Producer), a network management service consumer (MnS Consumer), or a network function management service consumer (NFMS_C). The cross-domain management function unit provides one or more of the following management functions or services: network lifecycle management, network deployment, network fault management, network performance management, network configuration management, network assurance, network optimization, and translation of network intents from communication service providers (Intent-CSPs). The network referred to in the above management functions or services can include one or more network elements or subnetworks, or it can be a network slice. In other words, network management functional units can be Network Slice Management Function (NSMF), Management Data Analytical Function (MDAF), Self-Organization Network function (SONFunction), Intent-Driven Management Service (Intent DrivenMnS), or Close Control Loop Management Service.

[0024] The domain management function unit, also known as the network subnet management function unit (NSMF) or network element management function unit, can be a wireless automation engine (MBB automationengine, MAE), a network element management system (EMS), a network function management service provider (NFMS_P), a network slice subnet management function (NSSMF), a domain management data analysis function (Domain MDAF), a self-organization network function (SON Function), a domain intent management function unit, or a close control loop management service, an MnS producer, an MnS consumer, or other network element management entities. Domain management function units can be classified in the following ways, including but not limited to: Classified by network type, they can be divided into: Radio Access Network (RAN) Domain Management Function (RAN domain MnF), Core Network Domain Management Function (CN domain MnF), and Transport Network Domain Management Function (TN domain MnF), etc. It should be noted that a domain management function unit can also be a domain network management system, capable of managing one or more of the access network, core network, or transport network. Classified by administrative region, they can be divided into: domain management function units for a specific region, such as the domain management function unit for city A, the domain management function unit for city B, etc.The domain management functional unit provides one or more of the following functions or management services: lifecycle management of subnetworks or network elements, deployment of subnetworks or network elements, fault management of subnetworks or network elements, performance management of subnetworks or network elements, assurance of subnetworks or network elements, optimization functions of subnetworks or network elements, and translation of subnetwork or network element intents (Intent from Network Operator, Intent-NOP). The aforementioned subnetworks include one or more network elements. A subnetwork can also include smaller subnetworks, that is, one or more subnetworks can form a larger subnetwork. This subnetwork can also be a network slice subnetwork.

[0025] A network element is an entity that provides network services, including core network elements, radio access network elements, or transport network elements. Specifically, core network elements may include, but are not limited to, Access and Mobility Management Function (AMF) entities, Session Management Function (SMF) entities, Policy Control Function (PCF) entities, Network Data Analysis Function (NWDAF) entities, Network Repository Function (NRF) entities, and gateways. Wireless access network elements may include, but are not limited to: various base stations (e.g., Generation Node B (gNB), Evolved Node B (eNB), Central Unit Control Panel (CUCP), Central Unit (CU), Distributed Unit (DU), Central Unit User Panel (CUUP), etc.). In this embodiment, network function (NF) is also referred to as network element (NE). A network element can provide one or more of the following management functions or services: network element lifecycle management, deployment, fault management, performance management, assurance, optimization functions, and translation of network element intent, etc.

[0026] The methods and embodiments provided in this application can run on network elements in the above-described network architecture. For example, network elements include, but are not limited to, mobile terminals, computer terminals, or similar computing devices. Taking running on a computer terminal as an example... Figure 2 This is a hardware structure block diagram of a computer terminal for a network performance management method according to an embodiment of this application, as shown below. Figure 2As shown, a hardware board may include one or more ( Figure 2 Only one is shown in the diagram. A processor 201 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 202 for storing data are also shown. The computer terminal 200 may further include transmission devices for communication functions and input / output devices. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, the computer terminal may also include components that are more complex than those described above. Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown.

[0027] The memory 202 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the network performance management method in this embodiment. The processor 201 executes various functional applications and data processing by running the computer program stored in the memory 202, thus implementing the aforementioned method. The memory 202 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 202 may further include memory remotely located relative to the processor 201, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The transmission device is used to receive or send data via a network. Specific examples of the network mentioned above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network elements via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0029] This embodiment provides a network performance management method. Figure 3 This is a flowchart of a network performance management method for a first network element according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps:

[0030] Step S302: Send an inference request to the second network element; wherein the inference request includes an inference category;

[0031] Step S304: Receive an inference report sent by the second network element corresponding to the inference category, wherein the inference report includes at least one of the following: a set of suggested measures, information on the managed object, and network performance indicators;

[0032] Step S306: Manage network performance based on the managed object information, the network performance indicators, and the preset policy.

[0033] In some embodiments, the set of suggested measures, the managed object information, and the network performance indicators are generated by the second network element through model inference corresponding to the inference category.

[0034] In some embodiments, the first network element may be a network management / network entity in the system; for example, the first network element is a consumer. The second network element may be an entity in the system that provides network management / network services. Both may include at least one of the following: User Equipment (UE), gNB, eNB, core network element, Operation, Administration, Maintenance (OAM), and cross-domain OAM. This application does not impose any limitations on this.

[0035] In some embodiments, the inference request may include at least one inference category, which may include at least one of the following: fault analysis, network performance optimization, network energy saving, SLA assurance, etc. This application does not impose any limitations on this.

[0036] In some embodiments, the second network element may use one or more models to perform inference for a reasoning category. Each model will generate a set of corresponding suggested measures, managed object information, and network performance indicators for the task of that reasoning category.

[0037] In some embodiments, the second network element may independently determine at least one model to generate a reasoning result based on a reasoning request that includes a reasoning category. This application does not impose any limitations on this.

[0038] In some embodiments, the model performing inference can be a machine learning model, and this application does not limit the type of model.

[0039] Through the embodiments of this application, since the first network element can receive the managed object information and network performance indicators generated by model inference, and provide a flexible network performance management method based on the managed object information and network performance indicators combined with preset strategies, the effect of ensuring network performance is achieved, thus solving the problem of difficulty in ensuring network performance when deploying multiple models in related technologies.

[0040] In some embodiments, step S306 may include the following steps:

[0041] Step S3062: In response to the preset strategy being a network performance remediation strategy, the suggested measures in the suggested measures set are implemented according to the suggested measures set and the managed object information, and the target object causing the deterioration of the target network performance index is adjusted if the target network performance index deteriorates.

[0042] Step S3064: In response to the preset strategy being a strategy to prevent network performance degradation, an evaluation is performed based on the managed object information and / or the network performance indicators, and a determination is made based on the evaluation results as to whether to implement the recommended measures.

[0043] In some embodiments, network performance management in multi-model deployment scenarios can be achieved in two ways. First, remediation of network performance degradation caused by implementing recommended measures, focusing on how to adjust the target objects causing network performance degradation to restore network performance; second, prevention of network performance degradation, focusing on how to prevent performance degradation caused by multi-model deployment to ensure network performance.

[0044] In some embodiments, a network performance metric that deteriorates due to the implementation of the recommended measures is referred to as a target network performance metric. Different adjustment measures may be taken depending on the target object that causes the deterioration of the target network performance metric. The target network performance metric may be one or more.

[0045] In some embodiments, the inference request further includes: a request to provide feedback on the candidate model corresponding to the inference category and its candidate model information. Prior to step S304, the following steps may also be included:

[0046] Step S3032: Receive the candidate model and its candidate model information from the second network element;

[0047] Step S3034: Determine the model to be used for inference based on the candidate model and its information, and feed it back to the second network element so that the second network element can perform inference based on the model.

[0048] In some embodiments, a first network element can obtain candidate models and their information from a second network element by including a request in the inference request that provides feedback on candidate models corresponding to the inference category and their information, or by querying available candidate models and their information. This allows the first network element to select and determine the model to perform the inference. It is understood that each candidate model can perform inference and obtain inference results (including a set of suggested measures, managed object information, and network performance indicators). Here, a candidate model can be understood as a candidate inference scheme, which may include one or more models, each with corresponding candidate model information.

[0049] In some embodiments, candidate model information may include at least one of the following: managed object information related to the candidate model, and network performance metrics related to the candidate model. Managed object information related to the candidate model may include at least one of the following: the type of affected network entity, the identifier of the affected network entity, the affected region, the affected time information, and the affected scope. Network performance metrics related to the candidate model may include at least one of the following: performance metrics (PM) and key performance indicators (KPIs), wherein PM includes, but is not limited to, data transmission rate, packet loss rate, and latency, and KPIs include, but are not limited to, service availability, response time, and throughput. This application does not impose any limitations on this.

[0050] In some embodiments, a candidate model corresponding to a reasoning category may include one or more; however, the requirements involved in the runtime of each candidate model may differ, and therefore the managed object information and network performance metrics associated with the candidate model may differ. Furthermore, the managed object information and network performance metrics associated with the candidate model can be considered as enhanced attributes of the candidate model, and each candidate model can be configured to possess such enhanced attributes so that each candidate model has the aforementioned candidate model information.

[0051] In some embodiments, the managed object information related to the candidate model can be divided into: managed object information directly affected by the inference results of implementing the candidate model, and managed object information indirectly affected by the inference results of implementing the candidate model, based on the estimated scope of its impact on the managed object. Network performance indicators related to the candidate model can be divided into network performance indicators directly affected by the inference results of implementing the candidate model, and performance indicators indirectly affected by the inference results of implementing the candidate model, based on the estimated specific impact on network performance. This application does not impose any limitations on this.

[0052] For example, a candidate model A corresponding to energy saving, whose inference results only affect the base station, directly affects the managed object information including the base station, indirectly affects the managed object information including UPF, directly affects the network performance indicators including base station energy consumption, and indirectly affects the performance indicators including base station throughput.

[0053] In some embodiments, the following steps may be included before step S306:

[0054] Step S305A: Determine the association information associated with the set of suggested measures based on the reasoning report, wherein the reasoning report further includes the association information and / or association relationships;

[0055] Step S305B: Set an association identifier for the set of suggested measures as the association information.

[0056] In some embodiments, the association information provides an identifier for the associated entity, which may include at least one of the following associated entities: the association identifier, the model identifier for generating the suggested measures through indexing, the inference function identifier, the inference request identifier, the inference report identifier, the inference category identifier, and the suggested measures set identifier.

[0057] In some embodiments, an association refers to the ability to further index other related entities in the association information based on one related entity. For example, indexing from the inference report identifier to the inference function identifier, or indexing from the inference report identifier to the model identifier that generated the proposed measure.

[0058] In some embodiments, setting an association identifier can directly or indirectly index other associated entities in the association information.

[0059] In some embodiments, the second network element can determine the association information associated with the set of suggested measures in various ways. For example, the first network element provides the set of suggested measures, association information, and association relationships through an inference report, enabling the second network element to directly determine the association information associated with the set of suggested measures based on the inference report. For example, the first network element provides the set of suggested measures and association information through an inference report, the second network element establishes association relationships, and determines the association information associated with the set of suggested measures based on the established association relationships. For example, the first network element provides the set of suggested measures and association relationships, and then the second network element determines the association information associated with the set of suggested measures. For example, the second network element sets an association identifier for the set of suggested measures as the association information associated with the set of suggested measures.

[0060] In some embodiments, the managed object information includes: managed object information implementing the recommended measures, and managed object information affected by the implementation of the recommended measures; the network performance indicators include: network performance indicators that gain from implementing the recommended measures, and network performance indicators affected by implementing the recommended measures.

[0061] In some embodiments, in a multi-model deployment scenario, implementing newly generated model-based recommendations may impact the entire network architecture, including but not limited to information on managed objects affected by the recommendations and network performance metrics affected by the recommendations. The information on managed objects affected by the recommendations is used to estimate the affected entities, regions, or times, while the network performance metrics affected by the recommendations are used to estimate the specific impact on network performance. This includes whether the recommendations have a positive effect on network performance metrics or a negative effect. Therefore, they can be categorized into network performance metrics that gain from the recommendations and network performance metrics that are negatively affected by the recommendations.

[0062] In some embodiments, the information on managed objects affected by the implementation of the recommended measures may include at least one of the following: the identifier of the network entity affected by the implementation of the recommended measures, the affected area, and the affected time information. The network performance indicators may include at least one of the following: PM and KPI, wherein PM includes, but is not limited to, data transmission rate, packet loss rate, and latency, and KPI includes, but is not limited to, service availability, response time, and throughput. This application does not impose any limitations on this.

[0063] In some embodiments, the managed object information and the network performance indicators can help assess the potential impact of implementing the recommended measures, and then, based on the managed object information and the network performance indicators, combined with preset strategies, to efficiently and dynamically manage network performance.

[0064] In some embodiments, step S3062, which involves implementing the recommended measures in the recommended measures set based on the recommended measures set and the managed object information, and adjusting the target object causing the deterioration of the target network performance indicator when the target network performance indicator in the network performance indicators deteriorates, may include the following steps:

[0065] Step S3062-2: Configure the managed objects indicated by the managed object information implementing the recommended measures according to the set of recommended measures;

[0066] Step S3062-4: A network performance monitoring instance containing the associated information is created by a third network element for the managed object indicated by the managed object information, so as to monitor the network performance indicators. The third network element is used to create and monitor the network performance indicators.

[0067] Step S3062-6: Determine the target object when the target network performance indicators are monitored to deteriorate;

[0068] Step S3062-8: Adjust the target object according to preset rules.

[0069] In one exemplary embodiment, the third network element can be a performance monitoring producer entity.

[0070] In some embodiments, the set of recommended measures may include at least one of the following: a set ID of recommended measures, a managed object implementing the recommended measures, a recommended measure action for the managed object implementing the recommended measures, and configuration parameters matching the recommended measure action. The configuration parameters include, but are not limited to, specific values ​​or ranges. There may be one or more managed objects implementing the recommended measures, and there may be one or more recommended measure actions for each managed object. This application does not impose any limitations on this.

[0071] In some embodiments, implementing the recommended measures means that the first network element issues configuration parameters matching the action of the recommended measures to the managed object implementing the recommended measures, so that the managed object implementing the recommended measures can be configured according to the configuration parameters.

[0072] In some embodiments, in a multi-model deployment scenario, implementing the proposed measures for generating new models may affect the entire network system. Network performance indicators are a direct feedback of network performance. Network performance monitoring instances created by third network elements can continuously monitor changes in network performance indicators, ensuring real-time control of the network status, enabling first network elements to promptly detect signs of network performance degradation and take timely adjustment measures.

[0073] In some embodiments, step S3062-6 may include the following steps:

[0074] Steps S3062-62: In response to the deterioration of the target network performance index, index the target object based on the association information associated with the target network performance index and the set of all currently running suggested measures. The target object includes at least one of the following: conflict model, conflict reasoning category, conflict reasoning function, and conflict suggested measures set.

[0075] In some embodiments, the association between target network performance indicators and target objects is established through the associated information, so that when network performance indicators deteriorate, the target objects causing the deterioration can be quickly located and identified based on the associated information, and targeted adjustments can be made.

[0076] In an exemplary embodiment, after implementing the recommended measures from the set of recommended measures corresponding to the SLA guarantee model, it is necessary to monitor the throughput metric of base station A (the network performance metric that gains from implementing the recommended measures); after implementing the recommended measures from the set of recommended measures corresponding to the coverage optimization model, it is necessary to monitor the network coverage metric (the network performance metric that gains from implementing the recommended measures); after implementing the recommended measures from the set of recommended measures corresponding to the energy-saving model, it is necessary to monitor both the throughput metric (the network performance metric affected by implementing the recommended measures) and the energy consumption metric (the network performance metric that gains from implementing the recommended measures) of base station A. Since the set of recommended measures corresponding to the energy-saving model has already been implemented, all currently running sets of recommended measures include the set of recommended measures corresponding to the SLA guarantee model, the set of recommended measures corresponding to the coverage optimization model, and the set of recommended measures corresponding to the energy-saving model. When the throughput metric is abnormal, the first network element can index to the SLA guarantee model and the energy-saving model based on the throughput metric and the associated information, thereby identifying the two conflicting models that caused the throughput metric to deteriorate.

[0077] In another exemplary embodiment, after implementing the recommended measures in the recommended measures set corresponding to the SLA guarantee model, it is necessary to monitor the throughput indicator of base station A (the network performance indicator that gains due to the implementation of the recommended measures); after adding the recommended measures in the recommended measures set corresponding to the energy-saving model, it is necessary to monitor two indicators of base station A: throughput indicator (the network performance indicator affected by the implementation of the recommended measures) and energy consumption indicator (the network performance indicator that gains due to the implementation of the recommended measures). When the throughput indicator decreases, the first network element can index to the recommended measures set corresponding to the SLA guarantee model and the recommended measures set corresponding to the energy-saving model based on the throughput indicator and the associated information, thereby identifying the two conflicting recommended measures sets that cause the throughput indicator to decrease.

[0078] In some embodiments, when the target object is the set of conflict-proposed measures, after indexing the association information associated with the target object based on the target network performance metrics and the set of all currently running proposed measures in steps S3062-62, the following steps may also be included:

[0079] Step S3062-62A: Determine the target set of recommended measures that needs to be adjusted according to the priority of the set of recommended measures, wherein the priority of the recommended measures comes from the reasoning report or is pre-configured by the first network element;

[0080] Step S3062-62B: Determine the target recommended measures set based on the preset local policy and / or the target network performance indicators.

[0081] In some implementations, the second network element provides the first network element with a priority set of suggested measures through an inference report. The priority set of suggested measures is the priority of all suggested measures currently running by the first network element. This allows the second network element to determine the priority of each suggested measure set in the conflicting suggested measure set based on the priority set of suggested measures, and to identify the suggested measure set with the lower priority as the target suggested measure set that needs to be adjusted, and then take adjustment measures.

[0082] In some embodiments, the first network element may pre-configure the priority of the proposed measures set, and, upon determining the conflicting proposed measures set, determine the priority of each proposed measures set in the conflicting proposed measures set according to the pre-configured priority, identifying the proposed measures set with lower priority as the target proposed measures set that needs adjustment, and then taking adjustment measures. This application does not limit the pre-configuration method.

[0083] In some embodiments, the first network element indirectly determines the priority of each set of suggested measures in the conflict suggested measures set through a preset local policy.

[0084] In some embodiments, step S3062-62B may further include the following steps:

[0085] Step S3062-62B-2: Calculate the target value of each set of suggested measures in the conflict suggestion set according to the preset local policy and / or the target network performance index;

[0086] Step S3062-62B-4: Determine the priority of the set of conflict-related proposed measures based on the target value;

[0087] Step S3062-62B-6: Determine the set of target recommended measures according to the priority.

[0088] In an exemplary embodiment, the first network element indirectly determines priority based on a utility function. Specifically, after implementing the recommended measures corresponding to the SLA guarantee model, it is necessary to monitor the throughput indicator of base station A (the network performance indicator that gains from implementing the recommended measures); after adding the recommended measures corresponding to the energy-saving model, it is necessary to monitor two indicators of base station A: throughput indicator (the network performance indicator affected by implementing the recommended measures) and energy consumption indicator (the network performance indicator that gains from implementing the recommended measures). When the throughput indicator decreases, the first network element can index the recommended measure set corresponding to the SLA guarantee model and the recommended measure set corresponding to the energy-saving model based on the throughput indicator and the association information associated with all currently running recommended measure sets. The conflicting recommended measure set includes the recommended measure set corresponding to the originally implemented SLA guarantee model and the recommended measure set corresponding to the energy-saving model. The throughput indicator corresponding to the SLA guarantee model is calculated using a utility function, and the throughput indicator and / or energy consumption indicator corresponding to the energy-saving model are calculated using a utility function. Priority is determined by comparing the values ​​calculated by the utility function. Specifically, the smaller the calculated value, the lower / higher the priority; the larger / smaller the calculated value, the higher the priority. This application does not impose any restrictions on the relationship between the size of the target value and its priority.

[0089] In some embodiments, the first network element determines its priority based on the inference report before the configuration is issued. In other embodiments, the first network element obtains its priority after a conflict occurs, based on the target network performance indicators.

[0090] In some embodiments, step S3062-8, adjusting the target object according to a preset rule, includes at least one of the following:

[0091] Revoke all currently running recommendations in the target set of recommended measures;

[0092] Revoke at least one currently running recommendation from the target set of recommended measures;

[0093] Request the second network element to deactivate the conflict model;

[0094] The second network element is requested to re-execute the inference of the conflict model;

[0095] The second network element is requested to retrain the conflict model.

[0096] In some embodiments, revoking at least one running recommendation in the target recommendation set can be done by revoking the configuration of a managed object that partially implements the recommendation. For example, canceling the entry into power-saving mode for partially conflicting base stations.

[0097] In some embodiments, the evaluation based on the managed object information and / or the network performance metrics in step S3064 may include the following steps:

[0098] Step S3064-2A: Evaluate whether the managed object information overlaps with the managed object information related to the proposed measures already requested to be implemented by the first network element and / or whether the network performance indicators overlap with the network performance indicators related to the proposed measures already requested to be implemented by the first network element, and obtain the evaluation result; or,

[0099] Step S3064-2B: Evaluate whether the managed object information and the network performance indicators related to the proposed measures that have been requested to be implemented by multiple first network elements overlap, and obtain the evaluation result.

[0100] In an exemplary embodiment, the managed object information related to the newly added energy-saving recommendation measures includes base station A and base station B. The originally implemented SLA guarantee recommendation measures included base station A and base station C. (It should be noted that since the recommendation measures corresponding to the energy-saving model have not been implemented, the recommendation measures already requested for operation only involve the recommendation measures corresponding to the SLA guarantee model, i.e., the SLA guarantee recommendation measures). Because the managed object information related to both recommendation measures includes base station A, there is overlap.

[0101] In another exemplary embodiment, the network performance metrics related to the newly added energy-saving recommendations include throughput metrics (network performance metrics affected by the implementation of the recommendations) and energy consumption metrics (network performance metrics that gain from the implementation of the recommendations); the network performance metrics related to the SLA guarantee recommendations in this implementation include throughput metrics (network performance metrics that gain from the implementation of the recommendations). Since the network performance metrics related to both recommendations include throughput metrics and are contradictory, there is overlap.

[0102] In another exemplary embodiment, the newly added managed object information and network performance indicators related to the energy-saving recommendations include the throughput indicator of base station A (network performance indicators affected by the implementation of the recommendations) and energy consumption (network performance indicators that gain from the implementation of the recommendations); the original network performance indicators related to the implementation of SLA guarantee recommendations included the throughput of base station A (network performance indicators that gain from the implementation of the recommendations). Since both sets of recommended measures include the throughput indicator of base station A and are contradictory, there is overlap. It is understood that the three exemplary embodiments described above use different granularities for determining whether there is overlap.

[0103] In some embodiments, different first network elements implement different sets of inference-based recommendations. For example, in a network, consumer 1 implements SLA guarantee recommendations, consumer 2 implements coverage optimization recommendations, and consumer 3 needs to implement energy-saving recommendations. Consumer 3 then obtains from a fourth network element the managed object information and / or network performance indicators related to the SLA guarantee recommendations requested by consumer 1 and the coverage optimization recommendations requested by consumer 2. This information is then evaluated against the managed object information and / or network performance indicators related to consumer 3's energy-saving recommendations to determine if there is overlap and obtain the evaluation result. The evaluation process is similar to the three exemplary embodiments described above and will not be repeated here.

[0104] In some embodiments, determining whether to implement the recommended measures based on the evaluation results in step S3064 may include the following steps:

[0105] Step S3064-4A: In response to the overlapping evaluation results, the implementation of the recommended measures is abandoned;

[0106] Step S3064-4B: In response to the assessment result being non-overlapping, the recommended measures are implemented.

[0107] In some embodiments, if the assessment results are overlapping, implementing both recommended measures simultaneously would result in a conflict. To prevent this conflict, the recommended measures are abandoned, or the conflicting recommended measures are abandoned, such as not implementing the recommended measures for the conflicting regulated entities. If the assessment results are not overlapping, implementing both recommended measures simultaneously may not result in a conflict, and therefore the recommended measures can be implemented.

[0108] In some embodiments, prior to step S3064-2B, the following steps may also be included:

[0109] Step S3064-1B: Obtain from the fourth network element the managed object information and / or related network performance indicators related to the proposed measures that the plurality of first network elements have requested to run; wherein, the fourth network element is used to record the proposed measures that the plurality of first network elements have requested to run, and the managed object information and / or related network performance indicators related to the proposed measures that the plurality of first network elements have requested to run.

[0110] In some embodiments, the fourth network element can be an AI inference history producer entity.

[0111] In some embodiments, if the evaluation result is non-overlapping, after the first network element implements the recommended measures, it sends the implemented recommended measures, the managed object information and / or network performance indicators to the fourth network element so that the fourth network element can store and record the results and respond to the record.

[0112] Through the embodiments of this application, since the first network element can receive the managed object information and network performance indicators generated by model inference, and provide a flexible network performance management method based on the managed object information and network performance indicators combined with preset strategies, the effect of ensuring network performance is achieved, thus solving the problem of difficulty in ensuring network performance when deploying multiple models in related technologies.

[0113] This embodiment provides a network performance management method. Figure 4 This is a flowchart of a network performance management method for a second network element according to an embodiment of this application, such as... Figure 4 As shown, the process includes the following steps:

[0114] Step S402: Receive an inference request sent by the first network element; wherein the inference request includes an inference category;

[0115] Step S404: Send a reasoning report corresponding to the reasoning category, including a set of suggested measures, managed object information, and network performance indicators, to the first network element, so that the first network element manages network performance based on the managed object information, the network performance indicators, and preset strategies.

[0116] In some embodiments, the set of suggested measures, the managed object information, and the network performance indicators are generated by the second network element through model inference corresponding to the inference category.

[0117] In some embodiments, the set of suggested measures, the managed object information, and the network performance indicators are generated by the second network element through model inference corresponding to the inference category.

[0118] In some embodiments, the first network element may be a network management entity in the system; for example, the first network element is a consumer. The second network element may be an entity providing network services in the system. Both may include at least one of the following: UE, gNB, eNB, core network element, OAM, cross-domain OAM. This application does not impose any limitations on this.

[0119] In some embodiments, the inference request may include at least one inference category, which may include at least one of the following: fault analysis, network performance optimization, network power saving, and SLA assurance. This application does not impose any limitations on this.

[0120] In some embodiments, the second network element may use one or more models to perform inference for a reasoning category. Each model will generate a set of corresponding suggested measures, managed object information, and network performance indicators for the task of that reasoning category.

[0121] In some embodiments, the second network element may independently determine at least one model to generate a reasoning result based on a reasoning request that includes a reasoning category. This application does not impose any limitations on this.

[0122] Through the embodiments of this application, the second network element provides the first network element with at least one managed object information and network performance indicators generated by model inference, so that the first network element can provide a flexible network performance management method based on the managed object information and network performance indicators combined with preset strategies, thereby achieving the effect of ensuring network performance and solving the problem of difficulty in ensuring network performance when deploying multiple models in related technologies.

[0123] In some embodiments, the preset strategy may include: a network performance remediation strategy and a network performance degradation prevention strategy. That is, network performance management in multi-model deployment scenarios can be achieved from two aspects. First, remediation of network performance degradation caused by implementing recommended measures focuses on how to adjust the target object causing the network performance degradation to restore network performance; second, prevention of network performance degradation focuses on how to prevent performance degradation caused by multi-model deployment to ensure network performance.

[0124] In some embodiments, the inference request further includes: a request to provide feedback on the candidate model corresponding to the inference category and its candidate model information; prior to step S404, the request further includes:

[0125] Step S4042: Send the candidate model and its candidate model information to the first network element;

[0126] Step S4044: Perform inference based on the model determined by the first network element.

[0127] In some embodiments, the second network element can provide the first network element with candidate models and their information by having the first network element include a request in the inference request that returns a candidate model corresponding to the inference category, or a request to query available candidate models and their information. This allows the first network element to select and determine the model to perform the inference. It is understood that each candidate model can perform inference and obtain inference results (including a set of suggested measures, managed object information, and network performance indicators). Here, a candidate model can be understood as a candidate inference scheme, which may include one or more models, each with corresponding candidate model information.

[0128] In some embodiments, candidate model information may include at least one of the following: managed object information related to the candidate model, and network performance metrics related to the candidate model. Managed object information related to the candidate model may include at least one of the following: the type of affected network entity, the identifier of the affected network entity, the affected region, the affected time information, and the affected scope. Network performance metrics related to the candidate model may include at least one of the following: performance metrics (PM) and key performance indicators (KPIs), wherein PM includes, but is not limited to, data transmission rate, packet loss rate, and latency, and KPIs include, but are not limited to, service availability, response time, and throughput. This application does not impose any limitations on this.

[0129] In some embodiments, a candidate model corresponding to a reasoning category may include one or more; however, the requirements involved in the runtime of each candidate model may differ, and therefore the managed object information and network performance metrics associated with the candidate model may differ. Furthermore, the managed object information and network performance metrics associated with the candidate model can be considered as enhanced attributes of the candidate model, and each candidate model can be configured to possess such enhanced attributes so that each candidate model has the aforementioned candidate model information.

[0130] In some embodiments, the managed object information related to the candidate model can be divided into: managed object information directly affected by the inference results of implementing the candidate model, and managed object information indirectly affected by the inference results of implementing the candidate model, based on the estimated scope of its impact on the managed object. Network performance indicators related to the candidate model can be divided into network performance indicators directly affected by the inference results of implementing the candidate model, and performance indicators indirectly affected by the inference results of implementing the candidate model, based on the estimated specific impact on network performance. This application does not impose any limitations on this.

[0131] For example, a candidate model A corresponding to energy saving, whose inference results only affect the base station, directly affects the managed object information including the base station, indirectly affects the managed object information including UPF, directly affects the network performance indicators including base station energy consumption, and indirectly affects the performance indicators including base station throughput.

[0132] In some embodiments, the associated information includes at least one of the following: an association identifier, a model identifier for generating the suggested measures through indexing, an inference function identifier, an inference request identifier, an inference report identifier, an inference category identifier, and an identifier for a set of suggested measures.

[0133] In some embodiments, the association information provides an identifier for the associated entity, which may include at least one of the following associated entities: the association identifier, the model identifier for generating the suggested measures through indexing, the inference function identifier, the inference request identifier, the inference report identifier, the inference category identifier, and the suggested measures set identifier.

[0134] In some embodiments, an association refers to further indexing to other related entities in the association information based on one related entity. For example, indexing to the inference function identifier based on the inference report identifier.

[0135] In some embodiments, setting an association identifier can directly or indirectly index other associated entities in the association information.

[0136] In some embodiments, the managed object information includes: managed object information implementing the recommended measures, and managed object information affected by the implementation of the recommended measures; the network performance indicators include: network performance indicators that gain from implementing the recommended measures, and network performance indicators affected by implementing the recommended measures.

[0137] In some embodiments, in a multi-model deployment scenario, implementing newly generated model-based recommendations may impact the entire network architecture, including but not limited to information on managed objects affected by the recommendations and network performance metrics affected by the recommendations. The information on managed objects affected by the recommendations is used to estimate the affected entities, regions, or times, while the network performance metrics affected by the recommendations are used to estimate the specific impact on network performance. This includes whether the recommendations have a positive effect on network performance metrics or a negative effect. Therefore, they can be categorized into network performance metrics that gain from the recommendations and network performance metrics that are negatively affected by the recommendations.

[0138] In some embodiments, the information on managed objects affected by the implementation of the recommended measures may include at least one of the following: the identifier of the network entity affected by the implementation of the recommended measures, the affected area, and the affected time information. The network performance indicators may include at least one of the following: PM and KPI, wherein PM includes, but is not limited to, data transmission rate, packet loss rate, and latency, and KPI includes, but is not limited to, service availability, response time, and throughput. This application does not impose any limitations on this.

[0139] In some embodiments, the managed object information and the network performance indicators can help assess the potential impact of implementing the recommended measures, and then, based on the managed object information and the network performance indicators, combined with preset strategies, to efficiently and dynamically manage network performance.

[0140] In some embodiments, the set of recommended measures may include at least one of the following: a set ID of recommended measures, a managed object implementing the recommended measures, a recommended measure action for the managed object implementing the recommended measures, and configuration parameters matching the recommended measure action. The configuration parameters include, but are not limited to, specific values ​​or ranges. There may be one or more managed objects implementing the recommended measures, and there may be one or more recommended measure actions for each managed object. This application does not impose any limitations on this.

[0141] Through the embodiments of this application, the second network element provides the first network element with at least one managed object information and network performance indicators generated by model inference, so that the first network element can provide a flexible network performance management method based on the managed object information and network performance indicators combined with preset strategies, thereby achieving the effect of ensuring network performance and solving the problem of difficulty in ensuring network performance when deploying multiple models in related technologies.

[0142] Since this application involves multiple model scenarios, for ease of description, inference categories are used for differentiation, without imposing any restrictions.

[0143] Figure 5 This is a flowchart (I) illustrating the implementation of a network performance remediation strategy in one embodiment of this application, as follows: Figure 5 As shown, the process includes the following steps:

[0144] (Multi-model deployment scenario: The network already has an SLA-guaranteed model that is running, and the AIML inference producer (gNB) has inference capabilities and an energy-efficient model).

[0145] Step S502: The consumer (OAM, the first network element in the above method embodiment) sends an inference request to the AIML inference producer (gNB) (the second network element in the above method embodiment) to request base station energy-saving inference;

[0146] Step S504: The AIML inference producer (gNB) sends an inference report to the consumer. The inference report includes: a set of energy-saving recommendations, including at least one recommendation, such as suggesting the NF / gNB enter an energy-saving state; energy-saving related information, such as inference function identifier, model identifier, and recommendation set identifier; relationships, further indexed to other related entities based on the inference report identifier, such as model identifier and inference function identifier; energy-saving network performance indicators, including key network performance gain indicators (energy consumption in this case), and / or other negatively affected indicators (such as throughput); and information on energy-saving managed objects, including information on managed objects affected by the implementation of the recommendations, including but not limited to network entity identifiers (such as base station or network element identifiers), affected areas, and time information.

[0147] Step S506: The consumer associates the set of energy-saving suggested measures with the energy-saving association information. Simultaneously, based on the priority set of suggested measures configured by the consumer, the consumer implements the set of energy-saving suggested measures associated with the energy-saving association information. That is, configuration is sent to the managed entity (gNB, NF) indicated by the managed object information implementing the energy-saving suggested measures. For example, configurations for entering energy-saving state can be sent to multiple base stations, with each base station having different configurations for entering energy-saving state.

[0148] In step S508, the consumer performs network performance monitoring based on the inference report, monitoring the energy-saving managed objects and energy-saving network performance indicators indicated by the energy-saving managed object information. Specifically, this can involve requesting the performance monitoring producer (the third network element in the above method embodiment) to create a performance monitoring instance, which must include energy-saving association information indicating the trigger source or cause of performance degradation. More specifically, for example, if the original SLA model inference result requires monitoring the throughput indicator (key gain indicator) of base station A, when adding an energy-saving inference result, both the throughput (affected indicator) and energy consumption (key gain indicator) of base station A need to be monitored.

[0149] In step S510, the consumer detects a deterioration in the energy-saving network performance indicators and identifies the set of conflicting recommended measures that led to this deterioration based on associated information. When the throughput indicator is abnormal, the consumer can index the set of recommended measures corresponding to the SLA model and the set of recommended measures corresponding to the energy-saving model based on the network performance monitoring results and their associated information, thereby identifying the two models that caused the network performance degradation. When the consumer has not set priorities, it sets the priorities of the conflicting recommended measures set according to the utility function and determines the target recommended measures set with lower priority based on the priorities.

[0150] Step S512, the consumer takes adjustment measures in response to the target set of suggested measures, specifically including the following measures:

[0151] Completely revoke the recommended measures associated with the action, i.e., the recommended measures in step S504;

[0152] The proposed measure is to partially revoke the association, that is, to revoke the configuration changes made to some managed objects in step S504. For example, to cancel the power-saving state for some base stations;

[0153] Deactivate;

[0154] Re-execute the reasoning;

[0155] Model retraining.

[0156] In this embodiment, for the purpose of distinction, the base station used to provide inference is referred to as the AIML inference producer (gNB), and the managed object that implements the proposed measures is referred to as the managed entity (gNB, NF).

[0157] In this embodiment, the energy-saving suggestion set includes at least one suggestion measure, such as suggesting that the managed entity NF or gNB enter an energy-saving state; the energy-saving association information includes at least one of the following: association identifier, model identifier for indexing and generating energy-saving suggestion measures (which may be the identifier of one, multiple, or a group of models), inference function identifier, inference request identifier, inference report identifier, inference category identifier, and suggestion set identifier; network performance indicators include network performance indicators that gain from implementing energy-saving suggestion measures and network performance indicators that are affected by implementing energy-saving suggestion measures, wherein the network performance indicators that gain from implementing energy-saving suggestion measures are performance indicators that will bring gains from implementing suggestion measures, such as PM and KPI, and the network performance indicators that are affected by implementing energy-saving suggestion measures are network performance indicators that may have a negative impact from implementing energy-saving suggestion measures, such as PM and KPI; the energy-saving managed object information includes managed object information that implements energy-saving suggestion measures and managed object information that is affected by implementing energy-saving suggestion measures, wherein the managed object information that is affected by implementing energy-saving suggestion measures includes, but is not limited to, network entity identifiers (such as base station or network element identifiers), affected area, and time information.

[0158] In this embodiment, after implementing SLA guarantee recommendations, it is necessary to monitor the throughput metric of base station A (the network performance metric that gains from implementing SLA guarantee recommendations); after implementing energy-saving recommendations, it is necessary to monitor two metrics of base station A: throughput (the network performance metric affected by implementing energy-saving recommendations) and energy consumption (the network performance metric that gains from implementing energy-saving recommendations). When the throughput metric decreases, the first network element can index to the SLA guarantee recommendation set and the energy-saving recommendation set based on the throughput metric and the associated energy-saving information, thereby identifying the two recommendation sets that cause the throughput metric to decrease.

[0159] In this embodiment, the declining network performance metric is determined to be the throughput metric. Therefore, the priorities of the SLA guarantee recommendation set and the energy-saving recommendation set are set based on the utility function and the throughput metric. The throughput metric of the SLA guarantee recommendations is calculated using the utility function, and the throughput metric of the energy-saving recommendations is also calculated using the utility function. The priorities are determined by comparing the calculated values. Specifically, the smaller the calculated value, the lower the priority; the larger the calculated value, the higher the priority.

[0160] Figure 6 This is a flowchart (II) illustrating the implementation of a network performance remediation strategy in one embodiment of this application, as shown below. Figure 6 As shown, the process includes the following steps:

[0161] (Multi-model deployment scenario: The network already has an SLA guarantee model in operation, and the AIM inference producer (gNB) has inference capabilities and a base station energy-saving model).

[0162] Step S602: The consumer (the first network element in the above method embodiment) sends an inference request to the AIML inference producer (gNB) (the second network element in the above method embodiment) to request energy-saving inference. The inference request also includes a request to provide feedback on the candidate model and its information.

[0163] Step S604: The AIML inference producer (gNB) sends energy-saving candidate models and their corresponding candidate model information to the consumer;

[0164] Step S606: The consumer determines the energy-saving model based on the energy-saving candidate model and the energy-saving candidate model information, and feeds it back to the AIML inference producer (gNB);

[0165] In step S608, the AIML inference producer (gNB) generates an energy-saving inference report based on the energy-saving model determined by the consumer and sends it to the consumer. The energy-saving inference report includes: energy-saving related information, a set of energy-saving recommended measures, energy-saving network performance indicators, and information on energy-saving managed objects.

[0166] Step S610: Consumers establish association relationships and determine energy-saving association information associated with energy-saving recommendations based on the association relationships;

[0167] Step S612: Implement the set of energy-saving recommendations associated with the energy-saving information, that is, issue configurations to the managed entities (gNB, NF) indicated by the managed object information implementing the energy-saving recommendations. For example, configurations for entering energy-saving state can be issued to multiple base stations, with each base station having a different configuration for entering energy-saving state;

[0168] Step S614: The consumer monitors network performance based on the inference report, monitoring the managed objects indicated by the energy-saving managed object information and the energy-saving network performance indicators. Specifically, this may involve requesting the performance monitoring producer (the third network element in the above method embodiment) to create a performance monitoring instance, which must include associated information indicating the trigger source or cause of performance degradation.

[0169] Step S616: The consumer detects the deterioration of the energy-saving network performance index, identifies the set of conflicting proposed measures that caused the deterioration of the energy-saving network performance index based on the associated information, sets the priority of the set of conflicting proposed measures according to the utility function, and determines the set of target proposed measures with lower priority based on the priority.

[0170] Step S618: Consumers take adjustment measures in response to the target set of recommended measures, specifically including the following measures:

[0171] Completely revoke the recommended measures associated with the action, i.e., the recommended measures in step S608;

[0172] The proposed measure is to partially revoke the association, that is, to revoke the configuration changes made to some managed objects in step S608. For example, to cancel the energy-saving state for some base stations;

[0173] Deactivate;

[0174] Re-execute the reasoning;

[0175] Model retraining.

[0176] In this embodiment, for the purpose of distinction, the base station used to provide inference is referred to as the AIML inference producer (gNB), and the managed object indicated by the managed object information for implementing the proposed measures is referred to as the managed entity (gNB, NF).

[0177] In this embodiment, the energy-saving candidate model information includes: managed object information related to the candidate model and network performance indicators related to the candidate model.

[0178] In this embodiment, the energy-saving suggestion set includes at least one suggestion measure, such as suggesting that the managed entity NF or gNB enter an energy-saving state; the energy-saving association information includes at least one of the following: association identifier, model identifier for indexing and generating energy-saving suggestion measures (which may be the identifier of one, multiple, or a group of models), inference function identifier, inference request identifier, inference report identifier, inference category identifier, and suggestion set identifier; the energy-saving network performance indicators include network performance indicators that gain from implementing energy-saving suggestion measures and network performance indicators that are affected by implementing energy-saving suggestion measures, wherein the network performance indicators that gain from implementing energy-saving suggestion measures are performance indicators that will bring gains from implementing energy-saving suggestion measures, such as PM and KPI, and the network performance indicators that are affected by implementing energy-saving suggestion measures are network performance indicators that may have a negative impact from implementing energy-saving suggestion measures, such as PM and KPI; the energy-saving managed object information includes managed object information that implements energy-saving suggestion measures and managed object information that is affected by implementing energy-saving suggestion measures, wherein the managed object information that is affected by implementing energy-saving suggestion measures includes, but is not limited to, network entity identifiers (such as base station or network element identifiers), affected area, and time information.

[0179] Figure 7 This is a flowchart (I) illustrating the implementation of a strategy to prevent network performance degradation in one embodiment of this application. Figure 7 As shown, the process includes the following steps:

[0180] (Multi-model deployment scenario: The network already has an SLA guarantee model in operation, and the AIML inference producer (gNB) has inference capabilities and a base station energy-saving model).

[0181] In step S702, the consumer (the first network element in the above method embodiment) sends an inference request to the AIML inference producer (gNB) (the second network element in the above method embodiment) to request base station energy-saving inference;

[0182] In step S704, the AIML inference producer (gNB) sends a first energy-saving inference report to the consumer, which includes: a first set of energy-saving recommended measures, a first set of energy-saving network performance indicators, and information on the first energy-saving managed object.

[0183] Step S706: The consumer assesses whether there is any overlap between the first energy-saving network performance indicators and / or the first energy-saving managed object information and the SLA-guaranteed network performance indicators and / or the SLA-guaranteed managed object information, and evaluates the results.

[0184] Step S708: If the evaluation result shows that there is overlap, the recommended measures in the first set of energy-saving recommended measures shall be abandoned, or the recommended measures in the set of recommended measures for the conflicting managed objects shall be abandoned.

[0185] Step S710: The consumer resends the inference request to the AIML inference producer (gNB) to request energy-saving analysis. The inference request also includes a request for feedback of candidate models and their information.

[0186] Step S712: The AIML inference producer (gNB) sends energy-saving candidate models and their corresponding candidate model information to the consumer;

[0187] Step S714: The consumer selects and determines the energy-saving model based on the energy-saving candidate model and its information, and feeds back to the AIML inference producer (gNB).

[0188] In step S716, the AIML inference producer (gNB) generates a second energy-saving inference report through the energy-saving model, and the consumer receives the second energy-saving inference report sent by the AIML inference producer (gNB). The second energy-saving inference report includes: a second set of energy-saving recommended measures; second energy-saving network performance indicators; and second energy-saving managed object information.

[0189] Step S718: The consumer assesses whether there is any overlap between the second energy-saving network performance indicators and / or the second energy-saving managed object information and the SLA guarantee network performance indicators and / or the SLA guarantee managed object information, and obtains the assessment results.

[0190] Step S720: If the evaluation result is non-overlapping, implement the recommended measures in the second set of energy-saving recommendations, that is, issue configurations to the managed entities (gNB, NF) indicated by the managed object information implementing energy-saving recommendations in the second set of recommended measures according to the second set of recommended measures.

[0191] In this embodiment, the energy-saving candidate model information includes: managed object information related to the candidate model and network performance indicators related to the candidate model.

[0192] In this embodiment, the first set of energy-saving recommendations is generated by the AIML inference producer (gNB) using at least one energy-saving model for energy-saving inference, while the at least one energy-saving model used to generate the second set of energy-saving recommendations is determined by the consumer based on the energy-saving candidate model and energy-saving candidate model information sent by the AIML inference producer. It can also be understood that the two have different ways of selecting the energy-saving model for generating the set of recommendations. Under the two selection methods, the energy-saving model for generating the set of recommendations may be the same or different.

[0193] Figure 8 This is a flowchart (II) illustrating the implementation of a strategy to prevent network performance degradation in one embodiment of this application. Figure 8 As shown, the process includes the following steps:

[0194] (Multi-model deployment scenario: In the network, the SLA guarantee recommendations and the energy-saving recommendations are implemented by different consumers. Consumer 1 (the first network element in the above method embodiment) implements the SLA guarantee recommendations, while consumer 2 (the first network element in the above method embodiment) implements the energy-saving recommendations.)

[0195] Step S802, Consumer 2 sends an inference request to AIML inference producer (gNB) (the second network element in the above method embodiment) to request base station energy-saving inference;

[0196] In step S804, the AIML inference producer (gNB) sends a first energy-saving inference report to consumer 2. The first energy-saving inference report includes: consumer identifier (consumer 2), a set of first energy-saving suggested measures corresponding to consumer identifier (consumer 2), first energy-saving network performance indicators, and information on the first energy-saving managed object.

[0197] Step S806, the AI ​​inference history producer (the fourth network element in the above method embodiment) sends the inference history of consumer 1 to consumer 2, wherein the inference history includes: consumer identifier (consumer 1), SLA guarantee recommendation measures that consumer 1 has requested to be implemented, SLA guarantee managed object information, and SLA guarantee network performance indicators.

[0198] Step S808: Consumer 2 evaluates whether there is any overlap between the first energy-saving network performance indicators and / or the first energy-saving managed object information and the SLA guarantee network performance indicators and / or the SLA guarantee managed object information and obtains the evaluation results;

[0199] Step S810: In response to the assessment result indicating overlap, the recommended measures in the first set of energy-saving recommendations are abandoned.

[0200] In step S812, consumer 2 resends the inference request to the AIML inference producer (gNB) to request base station energy-saving inference. The inference request also includes a request to provide feedback on candidate models and their information.

[0201] Step S814: The AIML inference producer (gNB) sends energy-saving candidate models and their information to consumer 2;

[0202] In step S816, consumer 2 determines the energy-saving model based on the candidate model and the candidate model information, and feeds it back to the AIML inference producer (gNB);

[0203] In step S818, the AIML inference producer (gNB) generates a second energy-saving inference report through the energy-saving model determined by consumer 2, and sends the second energy-saving inference report to consumer 2. The second energy-saving inference report includes: consumer identifier (consumer 2), a set of second energy-saving suggested measures corresponding to consumer identifier (consumer 2), second energy-saving network performance indicators, and information on the second energy-saving managed object.

[0204] Step S820: The consumer assesses whether there is any overlap between the second energy-saving network performance indicators and / or the second energy-saving managed object information and the consumer 1's SLA guarantee network performance indicators and / or SLA guarantee managed object information and obtains the assessment results.

[0205] Step S822: If the evaluation result is non-overlapping, implement the recommended measures in the second set of energy-saving recommendations, that is, issue configurations to the managed entities (gNB, NF) indicated by the managed object information in the second managed object information according to the second set of energy-saving recommendations.

[0206] In step S824, consumer 2 sends a second energy-saving inference report to the AI ​​inference history producer or sends implemented second energy-saving recommendations, second energy-saving managed object information, and second energy-saving network performance indicators.

[0207] Through the embodiments of this application, since the first network element can receive managed object information and network performance indicators generated by at least one model inference, and provide a flexible network performance management method based on the managed object information and network performance indicators combined with preset strategies, the effect of ensuring network performance is achieved, thus solving the problem of difficulty in ensuring network performance when deploying multiple models in related technologies.

[0208] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to perform the steps in any of the above method embodiments.

[0209] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0210] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0211] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0212] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods described in various embodiments of this application.

[0213] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0214] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular hardware and software combination.

[0215] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A network performance management method, applied to a first network element, characterized in that, include: Send an inference request to the second network element, wherein the inference request includes an inference category; Receive an inference report sent by the second network element corresponding to the inference category, wherein the inference report includes at least one of the following: a set of recommended measures, information on the managed object, and network performance indicators; Network performance is managed based on the managed object information, the network performance indicators, and preset policies; The suggested measures set, the managed object information, and the network performance indicators are generated by the second network element through model reasoning corresponding to the reasoning category.

2. The method according to claim 1, characterized in that, The method of managing network performance based on the managed object information, the network performance indicators, and preset policies includes: In response to the preset strategy being a network performance remediation strategy, the suggested measures in the suggested measures set are implemented according to the suggested measures set and the managed object information, and the target object causing the deterioration of the target network performance index is adjusted if the target network performance index deteriorates. In response to the preset strategy being a strategy to prevent network performance degradation, an evaluation is performed based on the managed object information and / or the network performance indicators, and a determination is made based on the evaluation results as to whether to implement the recommended measures.

3. The method according to claim 2, characterized in that, The inference request further includes: a request to provide feedback on the candidate model corresponding to the inference category and its candidate model information; prior to receiving the inference report corresponding to the inference category sent by the second network element, it further includes: Receive the candidate model and its candidate model information from the second network element; The model for performing inference is determined based on the candidate model and its information and fed back to the second network element so that the second network element performs inference based on the model.

4. The method according to claim 3, characterized in that, Before managing network performance based on the managed object information, the network performance indicators, and the preset policy, the method further includes: The inference report determines the association information related to the set of recommended measures, wherein the inference report further includes the association information and / or association relationships; or, An association identifier is set for the set of suggested measures as the association information.

5. The method according to claim 4, characterized in that, The information on the managed objects includes: information on the managed objects implementing the recommended measures, and information on the managed objects affected by the implementation of the recommended measures; The network performance metrics include: network performance metrics that gain from implementing the recommended measures, and network performance metrics that are affected by implementing the recommended measures.

6. The method according to claim 5, characterized in that, The step of implementing the recommended measures in the recommended measures set based on the recommended measures set and the managed object information, and adjusting the target object causing the deterioration of the target network performance index when the target network performance index deteriorates, includes: Configure the managed objects indicated by the managed object information implementing the recommended measures according to the set of recommended measures; A third network element creates a network performance monitoring instance containing the associated information for the managed object indicated by the managed object information, in order to monitor the network performance indicators. The third network element is used to create and monitor the network performance indicators. The target object is identified when the target network performance indicators are monitored to deteriorate. The target object is adjusted according to preset rules.

7. The method according to claim 6, characterized in that, The step of determining the target object when the target network performance indicators are monitored to deteriorate includes: In response to the deterioration of the target network performance metric, the target object is indexed based on the association information associated with the target network performance metric and the set of all currently running recommended measures. The target object includes at least one of the following: conflict model, conflict inference category, conflict inference function, and conflict recommended measures set.

8. The method according to claim 7, characterized in that, When the target object is the set of conflict-recommended measures, after indexing the target object with the association information associated with the target network performance metrics and all currently running sets of recommended measures, the method further includes: The set of target recommended measures that need to be adjusted is determined based on the priority of the recommended measures set, wherein the priority of the recommended measures comes from the inference report or is pre-configured by the first network element; or, Calculate the target value of each set of conflict suggestion measures in the set of conflict suggestion measures according to the preset local policy and / or the target network performance index; The priority of the set of conflict-related proposed measures is determined based on the target value; The set of target recommendations is determined based on the stated priority.

9. The method according to claim 8, characterized in that, The adjustment of the target object according to preset rules includes at least one of the following: Revoke all currently running recommendations in the target set of recommended measures; Revoke at least one currently running recommendation from the target set of recommended measures; Request the second network element to deactivate the conflict model; The second network element is requested to re-execute the inference of the conflict model; The second network element is requested to retrain the conflict model.

10. The method according to claim 4, characterized in that, The associated information includes at least one of the following: the associated identifier, the model identifier for generating the suggested measures through indexing, the inference function identifier, the inference request identifier, the inference report identifier, the inference category identifier, and the suggested measures set identifier.

11. The method according to claim 2, characterized in that, The evaluation based on the managed object information and / or the network performance metrics includes: The evaluation results are obtained by assessing whether the managed object information overlaps with the managed object information related to the proposed measures already requested to be implemented by the first network element, and / or whether the network performance indicators overlap with the network performance indicators related to the proposed measures already requested to be implemented by the first network element; or, alternatively... The evaluation results are obtained by assessing whether the managed object information and the network performance indicators related to the proposed measures that have been requested to be implemented by multiple first network elements overlap.

12. The method according to claim 11, characterized in that, The step of determining whether to implement the recommended measures based on the assessment results includes: In response to the overlapping assessment results, the recommended measures are abandoned; or, If the assessment results show no overlap, the recommended measures will be implemented.

13. The method according to claim 11, characterized in that, Before assessing whether the managed object information and / or the network performance indicators related to the proposed measures requested by multiple first network elements overlap and obtaining the assessment results, the process further includes: The fourth network element obtains information on managed objects and / or related network performance indicators related to the proposed measures that have been requested to be implemented by the plurality of first network elements; wherein, the fourth network element is used to record the proposed measures that have been requested to be implemented by the plurality of first network elements, and the information on managed objects and / or related network performance indicators related to the proposed measures that have been requested to be implemented by the plurality of first network elements.

14. A network performance management method, applied to a second network element, characterized in that, include: Receive an inference request sent by a first network element; wherein the inference request includes an inference category; Send a reasoning report, which corresponds to the reasoning category, to the first network element, including a set of suggested measures, managed object information, and network performance indicators, so that the first network element can manage network performance based on the managed object information, the network performance indicators, and preset strategies; The suggested measures set, the managed object information, and the network performance indicators are generated by the second network element through model reasoning corresponding to the reasoning category.

15. The method according to claim 14, characterized in that, The inference request further includes: a request to provide feedback on the candidate model and its information corresponding to the inference category; before sending the inference report corresponding to the inference category, including a set of suggested measures, managed object information, and network performance indicators, to the first network element, it also includes: Send the candidate model and its candidate model information to the first network element; Inference is performed based on the model determined by the first network element.

16. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 15.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 15.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 15.

Images