Network management method and apparatus

Abstract

Embodiments of the present application relate to the technical field of communications, and provide a network management method and apparatus. The network management method is applied to a first node for information prediction in a communication network, and comprises: acquiring reference data matching a scenario of network management and current measured information, wherein the reference data is used for indicating the prediction quality of at least one prediction model matching the scenario; processing the current measured information by means of each prediction model among the at least one prediction model to obtain current predicted information corresponding to the prediction model; and sending the current predicted information corresponding to each prediction model and a quality index of the current predicted information to a second node in the communication network, such that on the basis of the current predicted information of which the quality index satisfies a quality condition among the current predicted information corresponding to each prediction model, the second node performs a network management operation for the scenario. According to the embodiments of the present application, quality evaluation can be carried out on the predicted information, thereby improving the network performance and user experience.

Description

Network management method and device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 27, 2023, with application number 202311270785.X and invention name “Network Management Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of communication technology, and in particular to a network management method and apparatus. Background Art

[0003] Artificial intelligence (AI) is a technology that simulates the human brain to perform complex calculations and is widely used for data analysis. For example, the Release 17 of the 3rd Generation Partnership Project (3GPP) proposed applying AI to the new radio (NR, also known as the new wireless network, which is the 5G wireless network) to analyze measurement information to obtain prediction information (such as prediction information on resource status and user equipment (UE) trajectory prediction information). The prediction information can be used to improve network performance and user experience. The measurement information may include information measured from the communication network that can be used to obtain prediction information, such as load information, user location, and other information.

[0004] In related technologies, measurement information used to obtain prediction information is usually collected by nodes in a communication network (such as NR), such as 5G base stations (gNBs). The collected measurement information is then processed based on AI technology (such as a pre-trained machine learning model) to obtain prediction information. Network management operations such as network energy saving, load balancing, and mobility optimization are then performed based on the prediction information.

[0005] However, the prediction information obtained through AI technology is likely to be inaccurate. This can easily lead to inaccurate network management operations based on the prediction information, resulting in a failure to improve network performance and user experience.

[0006] Summary of the Invention

[0007] To solve the above technical problems, the present application provides a network management method in which nodes in a communication network can determine quality indicators of different prediction information to determine more accurate prediction information for more accurate network management operations, thereby improving network performance and user experience.

[0008] In a first aspect, an embodiment of the present application provides a network management method, which is applied to a first node performing information prediction in a communication network, the method comprising: obtaining reference data and current measurement information that match a scenario for network management, wherein the reference data is used to indicate the prediction quality of at least one prediction model that matches the scenario; processing the current measurement information through each prediction model of at least one prediction model respectively to obtain current prediction information corresponding to the prediction model; determining a quality indicator of the current prediction information corresponding to each of the at least one prediction models based on the reference data; and performing network management operations for the scenario according to the current prediction information whose quality indicator meets the quality conditions.

[0009] In an embodiment of the present application, a first node in a communication network can obtain reference data that matches a scenario managed by the network. The reference data is used to indicate the prediction quality of at least one prediction model that matches the scenario. Based on the reference data, the first node can then determine the quality indicators of the current prediction information corresponding to each of the at least one prediction models. In other words, the quality (such as accuracy and / or precision) of the prediction information currently obtained by each prediction model is evaluated, and the evaluation results are sent to a second node in the communication network. Based on this, the second node performs network management operations based on the current prediction information whose quality indicators meet the quality conditions, thereby avoiding network operations based on inaccurate prediction information, thereby improving network performance and user experience.

[0010] According to the first aspect, the reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

[0011] In the embodiment of the present application, by presetting a test set, a quality indicator of historical prediction information, historical prediction information, and one of the prediction auxiliary information, the prediction information can be evaluated before the actual information corresponding to the predicted network information is generated, thereby ensuring network management based on more accurate prediction information and improving network performance and user experience.

[0012] According to the first aspect, or any implementation method of the first aspect above, the reference information includes a preset test set; based on the reference data, the quality indicators of the current prediction information corresponding to at least one prediction model are determined, including: processing the test measurement information through each prediction model respectively to obtain the test prediction information corresponding to the prediction model; for each prediction model, according to the test prediction information corresponding to the prediction model, the test prediction information corresponding to the test measurement information, and the actual information corresponding to the test prediction information, the quality evaluation of the test prediction information corresponding to the prediction model is performed to obtain the quality indicator of the current prediction information corresponding to the prediction model.

[0013] In the embodiment of the present application, the quality indicators of the preset test set are used to approximately represent the quality indicators of the current prediction information, so that the prediction information can be evaluated before the actual information corresponding to the predicted network information is generated, thereby ensuring network management based on more accurate prediction information and improving network performance and user experience.

[0014] According to the first aspect, or any implementation method of the first aspect above, the reference information includes a preset test set; obtaining reference data that matches the network management scenario includes one of the following steps: reading the preset test set stored by the first node; receiving the preset test set sent by the third node, wherein the third node is a node in the communication network that is different from the first node and the second node; receiving the preset test set sent by the second node.

[0015] In an embodiment of the present application, the first node may obtain a preset test set from the third node or the second node, thereby ensuring that the quality of the current prediction information can be subsequently evaluated based on the preset test set.

[0016] According to the first aspect, or any implementation method of the first aspect above, the reference information includes a preset test set; obtaining reference data that matches the network management scenario includes: receiving an original test set sent by a third node, wherein the third node is a node in the communication network that is different from the first node and the second node, and the original test set includes multiple test sets, each test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; receiving evaluation requirement information sent by the second node; determining the test set in the original test set that meets the evaluation requirement information as the preset test set.

[0017] In an embodiment of the present application, when the third node sends the original test set, the first node can determine a preset test set that meets the evaluation requirements of the second node from the original test set based on the evaluation requirement information sent by the second node, thereby ensuring that the subsequent quality evaluation of the prediction information based on the preset test set is more in line with the requirements of the second node.

[0018] According to the first aspect, or any implementation method of the first aspect above, the reference information includes quality indicators of historical prediction information; based on the reference data, the quality indicators of the current prediction information corresponding to at least one prediction model are determined, including: for each prediction model, the quality indicators of the historical prediction information corresponding to the prediction model are determined as the quality indicators of the current prediction information corresponding to the prediction model.

[0019] In the embodiment of the present application, the quality indicators of historical prediction information are used to approximate the quality indicators of current prediction information, thereby enabling evaluation of the prediction information before the actual information corresponding to the predicted network information is generated, thereby ensuring network management based on more accurate prediction information and improving network performance and user experience.

[0020] According to the first aspect, or any implementation method of the first aspect above, the reference information includes a quality indicator of historical prediction information; obtaining reference data that matches the scenario managed by the network includes: obtaining the quality indicator of historical prediction information in a fourth node, wherein the fourth node is a node in the communication network that historically sends prediction information for the scenario to the first node; or, when the first node historically predicts information for the scenario, reading the quality indicator of the historical prediction information stored by the first node itself.

[0021] In an embodiment of the present application, when historical prediction information exists in the fourth node, the quality indicator of the historical network information can be obtained from the fourth node. When historical prediction information exists in the first node itself, the quality indicator of the historical network information stored in the first node can be directly read by the first node, thereby ensuring that the quality indicator of the current prediction information can be determined subsequently through the quality indicator of the historical prediction information.

[0022] According to the first aspect, or any implementation method of the first aspect above, the reference information includes historical prediction information; based on the reference data, the quality indicators of the current prediction information corresponding to at least one prediction model are determined, including: for each prediction model, the historical prediction information corresponding to the prediction model or the current prediction information corresponding to the prediction model is input into a pre-trained evaluation model to obtain the quality indicator of the current prediction information corresponding to the prediction model; wherein the evaluation model is a model trained using historical network information and the quality indicator of the historical network information.

[0023] In the embodiment of the present application, the quality index of the current prediction information is obtained through a pre-trained evaluation model, so that the prediction information can be evaluated before the actual information corresponding to the predicted network information is generated, thereby ensuring network management based on more accurate prediction information and improving network performance and user experience.

[0024] According to the first aspect, or any implementation method of the first aspect above, the reference information includes historical prediction information; obtaining reference data that matches the scenario managed by the network includes: obtaining historical prediction information of the neighboring nodes of the first node and the first node that matches the scenario managed by the network, and quality indicators of the historical prediction information.

[0025] In an embodiment of the present application, the first node can obtain historical prediction information for training the evaluation model from itself and neighboring nodes, that is, nodes adjacent to the first node. In this way, the historical prediction information can be obtained relatively more efficiently, thereby improving the evaluation efficiency of the prediction information.

[0026] According to the first aspect, or any implementation of the first aspect above, after determining the quality indicators of the current prediction information corresponding to at least one prediction model based on the reference data, the method also includes: when sending the current prediction information corresponding to each prediction model to the second node in the communication network, sending a retention indication of the current prediction information corresponding to each prediction model to the second node.

[0027] In an embodiment of the present application, by instructing the second node to retain the received prediction information, it can be ensured that the first node can obtain historical prediction information when performing quality evaluation of the prediction information, thereby ensuring that network management operations are performed based on more accurate prediction information, thereby improving network performance and user experience.

[0028] According to the first aspect, or any implementation of the first aspect above, before obtaining reference data that matches the scenario managed by the network, the method also includes: receiving a prediction evaluation indication sent by the second node, the prediction evaluation indication being used to indicate the second node's demand information for the quality indicator of the prediction information; and obtaining reference data that matches the scenario managed by the network according to the prediction evaluation indication.

[0029] In this embodiment of the present application, the prediction evaluation instruction can indicate the second node's requirements for the quality indicators of the prediction information. In this way, the reference data that matches the network management scenario obtained by the first node based on the prediction evaluation instruction can better meet the second node's quality evaluation requirements for the prediction information. For example, the quality evaluation requirements include whether to perform quality evaluation, the accuracy and / or precision conditions of the prediction information, and so on.

[0030] According to the first aspect, or any implementation of the first aspect above, the reference data includes prediction auxiliary information; obtaining the reference data matching the network management scenario includes: obtaining the prediction auxiliary information based on at least one prediction model; and determining, based on the reference data, a quality indicator of current prediction information corresponding to each of the at least one prediction model, including: for each prediction model, determining attribute information and / or an operating environment of the prediction model indicated by the prediction auxiliary information as the quality indicator of the current prediction information corresponding to the prediction model.

[0031] In an embodiment of the present application, the first node implicitly represents the quality indicator of the current prediction information through prediction auxiliary information that matches the network management scenario, thereby ensuring that subsequent network management can be performed based on more accurate current prediction information, thereby improving network performance and user experience.

[0032] In a second aspect, an embodiment of the present application provides a network management method, which is applied to a second node that uses prediction information in a communication network, the method comprising: receiving current prediction information and a quality indicator of the current prediction information corresponding to each prediction model in at least one prediction model sent by a first node that performs information prediction in the communication network; performing network management operations for a scenario based on the current prediction information corresponding to each prediction model whose quality indicator meets the quality conditions; wherein the quality indicator of the current prediction information corresponding to each prediction model is determined by the first node based on reference data matching the scenario of network management, the reference data is used to indicate the prediction quality of at least one prediction model matching the scenario, and the prediction model is used to perform information prediction.

[0033] According to the second aspect, the reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

[0034] According to the second aspect, or any implementation of the second aspect above, the reference data includes a preset test set; before receiving the current prediction information and the quality indicator of the current prediction information corresponding to each prediction model in at least one prediction model sent by the first node performing information prediction in the communication network, the method also includes: sending the preset test set to the first node; or sending evaluation requirement information to the first node.

[0035] According to the second aspect, or any implementation of the second aspect above, the reference information includes a quality indicator of historical prediction information or historical prediction information; the method also includes: in the case of receiving current prediction information corresponding to each prediction model in at least one prediction model sent by a first node for information prediction in a communication network, also receiving a retention indication of the current prediction information corresponding to each prediction model sent by the first node.

[0036] According to the second aspect, or any implementation method of the second aspect above, the reference information includes quality indicators of historical prediction information or historical prediction information; before receiving current prediction information and quality indicators of current prediction information corresponding to each prediction model in at least one prediction model sent by a first node performing information prediction in a communication network, the method also includes: sending a prediction evaluation indication to the first node, and the prediction evaluation indication is used to indicate the second node's requirement information for the quality indicator of the prediction information.

[0037] According to the second aspect, or any implementation method of the above second aspect, the reference data includes prediction auxiliary information; based on the current prediction information corresponding to each prediction model, the quality indicators of which meet the quality conditions, network management operations for the scenario are performed, including: determining the prediction requirements for the scenario; for each prediction model, based on the degree of matching between the prediction auxiliary information corresponding to the prediction model and the prediction requirements, determining the current prediction information corresponding to each prediction model, the quality indicators of which meet the quality conditions; based on the current prediction information, the quality indicators meet the quality conditions, network management operations for the scenario are performed.

[0038] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0039] In a third aspect, an embodiment of the present application provides a network management method, which is applied to a communication network, wherein the communication network includes a first node for performing information prediction and a second node for using the prediction information. The method includes: a first node for obtaining reference data and current measurement information that match a scenario for network management, wherein the reference data is used to indicate the prediction quality of at least one prediction model that matches the scenario; processing the current measurement information through each prediction model in at least one prediction model respectively to obtain current prediction information corresponding to the prediction model; determining the quality indicator of the current prediction information corresponding to at least one prediction model based on the reference data; sending the current prediction information and the quality indicator of the current prediction information corresponding to each prediction model to the second node in the communication network; and a second node for performing network management operations for the scenario based on the current prediction information whose quality indicator in the current prediction information corresponding to each prediction model meets the quality conditions.

[0040] According to the third aspect, the reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

[0041] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0042] In a fourth aspect, an embodiment of the present application provides a network management device, which is applied to a first node in a communication network for information prediction, the device comprising: an information acquisition module, for acquiring reference data and current measurement information matching a scenario of network management, wherein the reference data is used to indicate the prediction quality of at least one prediction model matching the scenario; processing the current measurement information through each prediction model of at least one prediction model respectively to obtain current prediction information corresponding to the prediction model; a quality evaluation module, for determining, based on the reference data, the quality index of the current prediction information corresponding to each of the at least one prediction models; an information sending module, for sending the current prediction information and the quality index of the current prediction information corresponding to each prediction model to a second node in the communication network, wherein the second node is used to perform network management operations for the scenario based on the current prediction information whose quality index in the current prediction information corresponding to each prediction model meets the quality conditions.

[0043] According to the fourth aspect, the reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

[0044] According to the fourth aspect, or any implementation method of the above fourth aspect, the reference information includes a preset test set; the quality evaluation module is specifically used to: process the test measurement information through each prediction model respectively to obtain the test prediction information corresponding to the prediction model; for each prediction model, according to the test prediction information corresponding to the prediction model, the test prediction information corresponding to the test measurement information and the actual information corresponding to the test prediction information, the test prediction information corresponding to the prediction model is quality evaluated to obtain the quality index of the current prediction information corresponding to the prediction model.

[0045] According to the fourth aspect, or any implementation method of the above fourth aspect, the reference information includes a preset test set; an information acquisition module is specifically used to perform one of the following steps: reading the preset test set stored by the first node; receiving the preset test set sent by the third node, wherein the third node is a node in the communication network that is different from the first node and the second node; receiving the preset test set sent by the second node.

[0046] According to the fourth aspect, or any implementation method of the above fourth aspect, the reference information includes a preset test set; the information acquisition module is specifically used to: receive the original test set sent by the third node, wherein the third node is a node in the communication network that is different from the first node and the second node, and the original test set includes multiple test sets, each test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; receive evaluation requirement information sent by the second node; and determine the test set in the original test set that meets the evaluation requirement information as the preset test set.

[0047] According to the fourth aspect, or any implementation method of the fourth aspect above, the reference information includes quality indicators of historical prediction information; the quality evaluation module is specifically used to: for each prediction model, determine the quality indicators of the historical prediction information corresponding to the prediction model as the quality indicators of the current prediction information corresponding to the prediction model.

[0048] According to the fourth aspect, or any implementation method of the above fourth aspect, the reference information includes the quality indicator of the historical prediction information; the information acquisition module is specifically used to: obtain the quality indicator of the historical prediction information in the fourth node, wherein the fourth node is the node in the communication network that historically sends the prediction information of the scenario by the first node; or, when the first node historically predicts information for the scenario, read the quality indicator of the historical prediction information stored by the first node itself.

[0049] According to the fourth aspect, or any implementation method of the fourth aspect above, the reference information includes historical prediction information; the quality evaluation module is specifically used to: for each prediction model, input the historical prediction information corresponding to the prediction model or the current prediction information corresponding to the prediction model into a pre-trained evaluation model to obtain the quality index of the current prediction information corresponding to the prediction model; wherein the evaluation model is a model trained using historical network information and the quality index of the historical network information.

[0050] According to the fourth aspect, or any implementation method of the above fourth aspect, the reference information includes historical prediction information; the information acquisition module is specifically used to: obtain the historical prediction information of the neighboring nodes of the first node and the first node that matches the network management scenario, as well as the quality indicators of the historical prediction information.

[0051] According to the fourth aspect, or any implementation method of the above fourth aspect, the information sending module is also used to: when sending the current prediction information corresponding to each prediction model to the second node in the communication network, send a retention indication of the current prediction information corresponding to each prediction model to the second node.

[0052] According to the fourth aspect, or any implementation method of the above fourth aspect, the information acquisition module is specifically used to: receive a prediction evaluation indication sent by the second node, the prediction evaluation indication is used to indicate the second node's demand information for the quality indicator of the prediction information; according to the prediction evaluation indication, obtain reference data that matches the network management scenario.

[0053] According to the fourth aspect, or any implementation method of the above fourth aspect, the reference data includes prediction auxiliary information; obtaining reference data that matches the network management scenario includes: obtaining prediction auxiliary information based on at least one prediction model; a quality evaluation module is specifically used to: for each prediction model, determine the attribute information and / or operating environment of the prediction model indicated by the prediction auxiliary information as the quality indicator of the current prediction information corresponding to the prediction model.

[0054] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0055] In a fifth aspect, an embodiment of the present application provides a network management device, which is applied to a second node that uses prediction information in a communication network, and the device includes: an information receiving module, which is used to receive current prediction information and a quality indicator of the current prediction information corresponding to each prediction model in at least one prediction model sent by a first node that performs information prediction in the communication network; an information application module, which is used to perform network management operations for a scenario based on the current prediction information corresponding to each prediction model whose quality indicator meets the quality conditions; wherein the quality indicator of the current prediction information corresponding to each prediction model is determined by the first node based on reference data matching the scenario of network management, the reference data is used to indicate the prediction quality of at least one prediction model matching the scenario, and the prediction model is used to perform information prediction.

[0056] According to the fifth aspect, the reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

[0057] According to the fifth aspect, or any implementation method of the above fifth aspect, the reference data includes a preset test set; the device also includes: an information sending module, which is used to: send the preset test set to the first node before the information receiving module receives the current prediction information and the quality indicator of the current prediction information corresponding to each prediction model in at least one prediction model sent by the first node performing information prediction in the communication network; or send evaluation requirement information to the first node.

[0058] According to the fifth aspect, or any implementation method of the above fifth aspect, the reference information includes a quality indicator of historical prediction information or historical prediction information; the information receiving module is also used to: when receiving the current prediction information corresponding to each prediction model in at least one prediction model sent by the first node for information prediction in the communication network, also receive the retention indication of the current prediction information corresponding to each prediction model sent by the first node.

[0059] According to the fifth aspect, or any implementation method of the above fifth aspect, the reference information includes the quality indicator of historical prediction information or historical prediction information; the device also includes: an information sending module, which is used to: before the information receiving module receives the current prediction information and the quality indicator of the current prediction information corresponding to each prediction model in at least one prediction model sent by the first node performing information prediction in the communication network, send a prediction evaluation indication to the first node, and the prediction evaluation indication is used to indicate the second node's demand information for the quality indicator of the prediction information.

[0060] According to the fifth aspect, or any implementation method of the above fifth aspect, the reference data includes prediction auxiliary information; the information application module is specifically used to: determine the prediction requirements for the scenario; for each prediction model, according to the degree of matching between the prediction auxiliary information corresponding to the prediction model and the prediction requirements, determine the current prediction information corresponding to each prediction model whose quality indicators meet the quality conditions; based on the current prediction information whose quality indicators meet the quality conditions, perform network management operations for the scenario.

[0061] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0062] In a sixth aspect, the present application provides a communication network, comprising a first node for performing information prediction and a second node for using the prediction information; the first node is used to obtain reference data and current measurement information that match a scenario managed by the network, wherein the reference data is used to indicate the prediction quality of at least one prediction model that matches the scenario; the current measurement information is processed by each prediction model in at least one prediction model to obtain current prediction information corresponding to the prediction model; based on the reference data, the quality indicator of the current prediction information corresponding to at least one prediction model is determined; the current prediction information corresponding to each prediction model and the quality indicator of the current prediction information are sent to the second node in the communication network; the second node is used to perform network management operations for the scenario based on the current prediction information whose quality indicator in the current prediction information corresponding to each prediction model meets the quality conditions.

[0063] According to the sixth aspect, the reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

[0064] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0065] In the seventh aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory; the processor and the memory are connected; the memory is used to store one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement a method as in any one of the first to third aspects and any one of the implementation methods of the first to third aspects.

[0066] In an eighth aspect, an embodiment of the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in any possible implementation of the first to third aspects or the first to third aspects.

[0067] In a ninth aspect, an embodiment of the present application provides a computer program comprising instructions for executing the method of the first to third aspects or any possible implementation of the first to third aspects.

[0068] In a tenth aspect, an embodiment of the present application provides a chip comprising a processing circuit and a transceiver pin. The transceiver pin and the processing circuit communicate with each other via an internal connection path, and the processing circuit executes the method of aspects 1 to 3 or any possible implementation of aspects 1 to 3 to control the receive pin to receive a signal and control the transmit pin to send a signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0070] Figure 1 is a block diagram of artificial intelligence applications in the new air interface;

[0071] FIG2 is a structural block diagram of a network device provided in an embodiment of the present application;

[0072] FIG3a is a structural block diagram of a network management device according to an embodiment of the present application;

[0073] FIG3b is a structural block diagram of a network management device according to an embodiment of the present application;

[0074] FIG3c is a structural block diagram of a communication network provided in an embodiment of the present application;

[0075] FIG4 is a flow chart of a network management method according to an embodiment of the present application;

[0076] FIG5a is one of the exemplary diagrams of a data collection process provided by an embodiment of the present application;

[0077] FIG5 b is one of the example diagrams of a data collection process provided by an embodiment of the present application;

[0078] FIG5c is one of the exemplary diagrams of a data collection process provided by an embodiment of the present application;

[0079] FIG6 is a flow chart of a network management method according to an embodiment of the present application;

[0080] FIG7 is a flow chart of a network management method according to an embodiment of the present application;

[0081] FIG8 is a flow chart of a network management method according to an embodiment of the present application;

[0082] FIG9 is one of the flow charts of a network management method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0084] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0085] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0086] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0087] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0088] To facilitate understanding of the embodiments of the present application, some background technologies involved in the embodiments of the present application are first introduced:

[0089] For example, Figure 1 is a block diagram of artificial intelligence applications in the new radio interface (NR). As shown in Figure 1, 3GPP's Release 17 approved the SI (Study Item), proposing the application of artificial intelligence (AI) to NR to improve network performance and user experience through AI-based data collection and analysis. Data collection includes storing data from nodes, such as gNBs (5G base stations), gNB-CUs (5G base station centralized units), gNB-DUs (5G base station distributed units), UEs (user equipment), or other management entities, as a database for AI model training and data analysis and inference. The model training host uses training data to train a model and produce a relatively optimal AI model (e.g., a neural network model, support vector machine (SVM) model, etc.). The model inference host uses the AI ​​model to analyze and infer the data collected, thereby achieving reasonable AI-based predictions of network operations and making policy adjustments to the network. Relevant policy adjustments can be planned by the execution (actor) node, that is, the execution entity, and sent to multiple network entities for operation, that is, the application of relevant policies to achieve network management. The specific performance generated by the network applying the relevant policies can be input into the database again for storage, that is, feedback. In one example, the model training node can also further perform model training based on the model performance feedback sent by the model prediction node to obtain a better AI model, and deploy / update the model to ensure that the model prediction node can use the better AI model.

[0090] However, the prediction information obtained through AI technology is likely to be inaccurate. This can easily lead to inaccurate network management operations based on the prediction information, resulting in a failure to improve network performance and user experience.

[0091] An embodiment of the present application provides a network management method to solve the above-mentioned problem. In this method, the nodes of the communication network can obtain reference data that matches the scenario of network management. The reference data is used to indicate the prediction quality of at least one prediction model that matches the scenario. Then, the nodes of the communication network can determine the quality index of the current prediction information corresponding to the at least one prediction model based on the reference data, that is, evaluate the quality (such as accuracy and / or precision) of the prediction information currently obtained by each prediction model. Based on this, the nodes of the communication network perform network management operations based on the current prediction information whose quality index meets the quality conditions, which can avoid network operations based on inaccurate prediction information, thereby improving network performance and user experience.

[0092] Before describing the technical solution of the embodiment of the present application, the operating platform of the network management method of the embodiment of the present application is first described in conjunction with the accompanying drawings. The embodiment of the present application can be applied to nodes of a communication network. Among them, the communication network can be, for example, a 4G system, a 5G system, an NTN system (non-terrestrial network, non-terrestrial network) or any other communication network involving AI application scenarios. In the specific implementation of the embodiment of the present application, the node can be, for example, a terminal device, a network device or other device.

[0093] The terminal device is a device with wireless transceiver capabilities. The terminal device can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; can also be deployed on water (such as ships); and can also be deployed in the air (such as airplanes, balloons, and satellites). The terminal device can be a mobile phone, a tablet computer, a computer with wireless transceiver capabilities, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical care, a wireless terminal device in smart grids, a wireless terminal in transportation safety, a wireless terminal device in smart cities, a wireless terminal in smart homes, and user equipment (UE).

[0094] A network device is a device in a wireless network, such as a radio access network (RAN) node that connects a terminal device to the wireless network. Examples of RAN nodes include gNBs, transmission reception points (TRPs), evolved node Bs (eNBs), radio network controllers (RNCs), node Bs (NBs), base station controllers (BSCs), base transceiver stations (BTSs), home base stations (e.g., home evolved node Bs or HNBs), base band units (BBUs), wireless fidelity (Wi-Fi) access points (APs), and integrated access and backhaul (IABs).

[0095] For example, Figure 2 is a structural block diagram of a network device provided in an embodiment of the present application. As shown in Figure 2, in a network structure, the network device may be a centralized unit (central unit, CU) or a distributed unit (distributed unit, DU), or the network device may also be a device including a CU and a DU. CU and DU can be understood as units obtained by dividing the base station from a logical functional perspective. CU and DU can be understood as a division of the base station from a logical functional perspective. Among them, CU and DU can be physically separated or deployed together, and the embodiment of the present application does not make specific limitations on this. In one example, a CU can be connected to a DU, or multiple DUs can share a CU to save costs and facilitate network expansion.

[0096] Among them, the division of CU and DU can be divided according to the protocol stack. One possible way is to deploy the radio resource control (RRC), service data adaptation protocol stack (SDAP) and packet data convergence protocol (PDCP) layer in the CU, and the remaining radio link control (RLC) layer, media access control (MAC) layer and physical layer in the DU. The embodiment of the present application does not limit the above-mentioned protocol stack division method, and any protocol stack division method that can be used for network devices composed of CU and DU can be used in this application. In one example, the CU and DU can be connected through the F1 interface (an open ground interface). The CU represents the gNB and is connected to the core network through the Ng interface.

[0097] Still referring to Figure 2 , the network device in the embodiments of the present application may also refer to a centralized unit control plane (CU-CP) node or a centralized unit user plane (CU-UP) node, or the network device may be both a CU-CP and a CU-UP. The CU-CP is responsible for control plane functions, primarily including RRC and PDCP-C. PDCP-C is primarily responsible for encryption and decryption, integrity protection, and data transmission of control plane data. The CU-UP is responsible for user plane functions, primarily including SDAP and PDCP-U. SDAP is primarily responsible for processing core network data and mapping flows to bearers. PDCP-U is primarily responsible for encryption and decryption, integrity protection, header compression, sequence number maintenance, and data transmission on the data plane. The CU-CP and CU-UP are connected via the E1 interface. The CU-CP represents the gNB and connects to the core network via the Ng interface. It connects to the DU via F1-C (control plane). The CU-UP connects to the DU via F1-U (user plane). Of course, another possible implementation is to also include PDCP-C in the CU-UP.

[0098] The access network device mentioned in the embodiments of the present application may be a device including at least one of a CU and a DU, or a device including a control plane CU node (CU-CP node) and a user plane CU node (CU-UP node) and a DU node.

[0099] Wireless access network equipment and terminal devices can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; they can also be deployed on water; and they can also be deployed in the air on aircraft, balloons, and satellites. The embodiments of this application do not limit the specific application scenarios of wireless access network equipment and terminal devices.

[0100] The embodiments of the present application can be applied to downlink signal transmission, uplink signal transmission, and device-to-device (D2D) signal transmission. For downlink signal transmission, the transmitting device is a wireless access network device, and the corresponding receiving device is a terminal device. For uplink signal transmission, the transmitting device is a terminal device, and the corresponding receiving device is a wireless access network device. For D2D signal transmission, the transmitting device is a terminal device, and the corresponding receiving device is also a terminal device. The embodiments of the present application do not limit the direction of signal transmission.

[0101] The wireless access network device and the terminal device, as well as the terminal device and the terminal device, can communicate through the licensed spectrum (licensed spectrum), can also communicate through the unlicensed spectrum (unlicensed spectrum), or can communicate through the licensed spectrum and the unlicensed spectrum at the same time. The wireless access network device and the terminal device, as well as the terminal device and the terminal device, can communicate through the spectrum below 6 gigahertz (GHz), that is, 6G, or can communicate through the spectrum above 6G, or can use the spectrum below 6G and the spectrum above 6G at the same time. The embodiments of the present application do not limit the spectrum resources used between the wireless access network device and the terminal device.

[0102] It should be understood that the network device shown in Figure 2 is only an example of a network device, and the network device may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in Figure 2 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0103] For example, Figure 3a is a block diagram of a network management device provided in an embodiment of the present application. As shown in Figure 3a, the device is applied to a first node in a communication network for information prediction and may include: a collection module 301, an evaluation module 302, and an application module 303.

[0104] An information acquisition module 301 is configured to acquire reference data matching a scenario managed by the network and current measurement information, wherein the reference data indicates the prediction quality of at least one prediction model matching the scenario; and to process the current measurement information using each of the at least one prediction model to obtain current prediction information corresponding to the prediction model.

[0105] A quality evaluation module 302 is configured to determine, based on reference data, a quality indicator of current prediction information corresponding to each of at least one prediction model;

[0106] The information sending module 303 is used to send the current prediction information corresponding to each prediction model and the quality indicator of the current prediction information to the second node in the communication network, wherein the second node is used to perform network management operations for the scenario based on the current prediction information corresponding to each prediction model whose quality indicator meets the quality conditions.

[0107] For example, Figure 3b is one of the structural block diagrams of a network management device provided in an embodiment of the present application. As shown in Figure 3b, the network management device is used for a second node using prediction information in a communication network, and the device includes:

[0108] An information receiving module 304 is configured to receive current prediction information and a quality indicator of the current prediction information corresponding to each prediction model in at least one prediction model sent by a first node performing information prediction in a communication network;

[0109] An information application module 305 is configured to perform scenario-specific network management operations based on the current prediction information corresponding to each prediction model, the quality indicators of which meet the quality conditions;

[0110] Among them, the quality indicator of the current prediction information corresponding to each prediction model is determined by the first node based on reference data matching the scenario managed by the network. The reference data is used to indicate the prediction quality of at least one prediction model matching the scenario, and the prediction model is used to perform information prediction.

[0111] It should be understood that the apparatus shown in Figures 3a and 3b is merely an example of a network management apparatus, and that the network management apparatus may have more or fewer components than shown, may combine two or more components, or may have a different configuration of components. The various components shown in Figures 3a and 3b may be implemented in hardware, including one or more signal processing and / or application specific integrated circuits, software, or a combination of hardware and software.

[0112] For example, Figure 3c is a block diagram of a communication network provided in an embodiment of the present application. As shown in Figure 3c, the communication network includes a first node 306 for performing information prediction and a second node 307 for using the prediction information;

[0113] The first node 306 is configured to obtain reference data and current measurement information that match a scenario managed by the network, wherein the reference data is used to indicate a prediction quality of at least one prediction model that matches the scenario; process the current measurement information using each of the at least one prediction model to obtain current prediction information corresponding to the prediction model; determine a quality indicator of the current prediction information corresponding to each of the at least one prediction model based on the reference data; and send the current prediction information and the quality indicator of the current prediction information corresponding to each prediction model to a second node in the communication network;

[0114] The second node 307 is configured to perform a scenario-specific network management operation according to the current prediction information corresponding to each prediction model, the quality indicator of which meets the quality condition.

[0115] In a specific application, the number of the first node 306 and the second node 307 may be one or more. The first node 306 and the second node 307 may be, for example, terminals or network devices, etc., and the embodiment of the present application does not impose any specific limitation on this.

[0116] It should be understood that the network shown in FIG3 c is only one example of a communication network, and that a communication network may have more or fewer components than shown, may combine two or more components, or may have a different configuration of components. The various components shown in FIG3 c may be implemented in hardware, including one or more signal processing and / or application specific integrated circuits, software, or a combination of hardware and software.

[0117] For ease of understanding, the network management method provided in the embodiment of the present application is described in detail below with reference to Figures 4 to 9.

[0118] For example, Figure 4 is a flowchart of a network management method provided by an embodiment of the present application. As shown in Figure 4, a network management method provided by an embodiment of the present application is applied to a node of a communication network. The method may include the following steps:

[0119] S401: Node 1 obtains reference data matching a scenario managed by a network and current measurement information, wherein the reference data is used to indicate prediction quality of at least one prediction model matching the scenario.

[0120] The first node and the second node can be specifically described in the embodiment of Figure 2 above, and will not be repeated here. In an optional embodiment, the reference data may include one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model. For the purpose of reasonable layout, this will be described in detail in conjunction with Figures 6 to 9.

[0121] For ease of understanding and description, the application of AI on the RAN side is still used as an example for explanation. AI can have a variety of application scenarios in communication networks, that is, network management scenarios. Taking RAN3 as an example, the scenario can include scenarios such as network energy saving, load balancing, and mobility optimization. The current measurement information that matches the network management scenario is the measurement information used to obtain the current prediction information, and the current prediction information is used to perform network management for the scenario. In other words, the measurement information is the input of the prediction model. Among them, the current time is the time for quality evaluation and the time requested by the second node in the embodiment of the present application. The following describes the process of network management in these three scenarios.

[0122] Network Energy Saving: To address the high energy consumption, CO2 emissions, and operating expenses (OPEX) caused by the rapid growth of base stations in communications networks, network energy saving can be implemented. This approach may involve different network layers, with mechanisms operating on different timescales. Cell activation / deactivation is a spatial energy saving solution that leverages hierarchical traffic offloading to reduce energy consumption across the radio access network (RAN). When expected traffic falls below a fixed threshold, a cell may be shut down, and served users may be offloaded to a new target cell. Efficient energy consumption can also be achieved through other means, such as load reduction, coverage modification, or other RAN configuration adjustments. The optimal energy saving decision depends on many factors, including the load of different RAN nodes, RAN node capabilities, KPI / QoS requirements, the number of active users and UE mobility, and cell utilization. To avoid severe network performance degradation and / or energy efficiency degradation caused by erroneous cell deactivation, machine learning (ML) techniques can be used to optimize energy saving decisions by analyzing data collected within the RAN network. ML algorithms can predict energy efficiency and load conditions for the next cycle, enabling better decisions on cell activation / deactivation to save energy. Based on the predicted load, the system, i.e., the communication network, can dynamically configure energy-saving strategies (e.g., shutdown timing and granularity, offloading actions) to maintain a balance between system performance and energy efficiency and reduce energy consumption.

[0123] Load balancing: The goal of load balancing is to evenly distribute load between cells and cell areas, shift traffic away from congested cells or congested cell areas, or offload users from one cell, cell area, carrier, or standard to improve network performance. This can be achieved by optimizing cell reselection / handover parameters (such as handover thresholds) and handover actions. This automated optimization delivers a high-quality user experience while increasing system capacity and minimizing manual intervention in network management and optimization tasks. If load balancing decisions rely on current or past cell load status, rapid changes in network traffic load and resource availability, particularly in high-mobility and high-connection scenarios, can lead to ping-pong handovers between cells, resulting in cell overload and degraded user service quality. Load balancing can make it difficult to guarantee overall network and service performance. For load balancing, UEs in congested cells can be offloaded to target cells through handover procedures or handover configuration adaptation. For example, if UEs with time-varying traffic loads are offloaded to a target cell, the target cell may become overloaded with the newly arrived traffic, making it difficult to determine whether service performance after the offloading action meets expectations. To address these issues, solutions based on predictive models can be introduced to improve load balancing performance. Based on various measurements and feedback from users and network nodes, as well as historical data, predictive model-based solutions and predictive load can improve load balancing performance, providing a higher quality user experience and increasing system capacity.

[0124] Mobility Optimization: Mobility management ensures service continuity during mobility by minimizing dropped calls, radio link failures (RLFs), unnecessary handovers, and ping-pong. In future high-frequency networks, as single-node coverage decreases, the frequency of UE handovers between nodes will become very high, especially for highly mobile UEs. Furthermore, for applications with stringent QoS requirements such as reliability and latency, QoE (Quality of Experience) is sensitive to handover performance. Therefore, mobility management should avoid handover failures and reduce latency during handovers. Furthermore, due to the randomness and uncertainty of the transmission environment, feedback-based adjustments may be less effective. Beyond the baseline mobility scenario, optimization areas for mobility include dual connectivity, CHO, and DAPS, all of which have additional aspects to address in mobility optimization. Mobility aspects of SON that can be enhanced through the use of AI / ML include reducing the likelihood of mobility-related unexpected events. Examples of such unexpected events include: intra-system handovers that are too late, too early, or intra-system handovers to the wrong cell.

[0125] Late intra-system handover: After the UE stays in a cell for a long time, a radio link failure (RLF) occurs; the UE attempts to re-establish the radio link connection in a different cell.

[0126] Too early intra-system handover: RLF occurs shortly after a successful handover from the source cell to the target cell, or the handover fails during the handover process; the UE attempts to re-establish the radio link connection in the source cell.

[0127] Intra-system handover to an incorrect cell: RLF occurs shortly after a successful handover from a source cell to a target cell, or a handover fails during the handover process; the UE attempts to reestablish the radio link connection in a cell other than the source cell and the target cell.

[0128] Successful switchover: During a successful switchover, there was a potential problem.

[0129] RAN AI can observe multiple HO (Handover) events with relevant parameters, use this information to train its ML model, and try to identify the parameter sets that lead to successful HO and the parameter sets that lead to unexpected events.

[0130] UE location / mobility / performance prediction: Predicting the UE's location is a key part of mobility optimization, as many mobility-related RRM actions (e.g., selecting a handover target cell) can benefit from the predicted UE location / mobility path. UE mobility prediction is also a key factor in optimizing early data forwarding, especially for CHO (conditional handover). When a UE is served by certain cells, UE performance prediction is a key factor in determining which is the best mobile target to maximize efficiency and performance.

[0131] Traffic Steering: By adjusting the handover trigger point, the optimal combination of Pcell / PSCell / Scell ​​(primary serving cell / primary serving cell of a primary cell group (dual connectivity) / primary secondary cell / secondary cell) is selected to serve users, achieving efficient resource processing. Providing mobility or dual connectivity-related information to the RAN node can also improve existing traffic steering.

[0132] For example, before initiating a handover, the source gNB can use UE performance feedback collected for past successful handovers and received from neighboring gNBs. Similarly, for the dual connectivity scenario, before triggering the addition of a secondary gNB or triggering an SN change, the eNB can use information (feedback) received from the gNB in ​​the past to successfully complete the SN addition or SN change procedure.

[0133] In both reported examples, the source RAN node of a mobility event or the RAN node acting as the master node (eNB for EN-DC, gNB for NR-DC) can use feedback received from another RAN node as input to AI / ML functions supporting traffic-related decisions (e.g., selection of the target cell in the case of mobility, selection of PSCell / Scell(s) in the other case) in order to optimize future decisions.

[0134] It can be understood that the above-mentioned network management scenarios are all examples. The embodiments of this application do not limit the specific application scenarios. Any scenario that obtains predictive information through artificial intelligence technology for network management can be applied to this application.

[0135] Figure 5a is one of the example diagrams of a data collection process provided by an embodiment of the present application. As shown in Figure 5a, the 3GPP RAN3 117th meeting introduced a general new process at the XN port (an open interface for interconnection between NG-RAN nodes, NG-RAN consists of multiple gNBs connected to the 5G core network) for transmitting data for artificial intelligence. The new process is called the Data Collection Procedure. This process is use case-independent, data type-independent, and non-UE-related. Among them, use case-independent means that the data for artificial intelligence (such as AI / ML (machine learning) data) transmitted by the process can be used for all AI use cases, including but not limited to the three use cases (load balancing, mobility optimization, and energy saving) exemplified in S401 above; data type-independent means that the AI / ML data transmitted by the process can include input data, output data, feedback data, etc. of the AI ​​model, such as prediction information and measurement information; non-UE-related means that the process is not associated with a specific UE, but can also be used to transmit data for specific UEs, such as per-UE performance feedback information.

[0136] The data reporting methods in the above data collection process may include: one-time reporting and periodic reporting. The process also supports partial reporting, that is, when the requested node can provide some of the requested items, the process startup does not default to failure. Still referring to Figure 5a, NG-RAN node 1 initiates the process by sending a data collection request message to NG-RAN node 2 to start AI / ML related information reporting, or to stop AI / ML related information reporting. After receiving it, NG-RAN node 2 can perform different actions based on the IE (information element, information element, abbreviated as information element) in the data collection request message, specifically:

[0137] In case the Registration Request IE is set to "start", the requested data collection report shall be started according to the parameters given in the request;

[0138] In the case where the Registration Request IE is set to "Stop", all cell data collection reports shall be stopped and reporting shall be terminated.

[0139] If NG-RAN node 2 is able to provide all requested information, it shall initiate a data collection report based on the request of NG-RAN node 1 and respond with a data collection response message, i.e. a feedback data collection response message, which may contain the collected data.

[0140] If the NG-RAN node 2 is able to provide some but not all of the requested information (i.e. the procedure supports partial reporting), it shall initiate data collection reporting for the accessed requested information and include the node measurement initiation result or the per-cell measurement initiation result or both in the data collection response message.

[0141] If the Reporting Period IE is present in the Data Collection Request message, it indicates the periodic data collection and reporting period, i.e., periodic reporting. For example, Figure 5b is one of the diagrams illustrating a data collection process according to an embodiment of the present application. As shown in Figure 5b, in the periodic reporting scenario, NG-RAN node 2 can periodically send a Data Collection Update message to NG-RAN node 1 to provide feedback on the data collected during each period. If the Reporting Period IE is not present in the Data Collection Request message, NG-RAN node 2 defaults to reporting only once, i.e., a one-time report.

[0142] If the Requested Prediction Time IE in the Data Collection Request message is present, it indicates the specific point in time to which the prediction of the requested information applies. The NG-RAN node 2 shall take it into account when generating the requested prediction information.

[0143] When starting measurements, the Reporting Characteristics IE in the Data Collection Request indicates the type of object on which the NG-RAN node 2 should perform measurements or predictions. As of RAN3 Meeting 120, the data agreed to be transmitted in this process includes: predicted resource status information (predicted radio resource status, predicted number of active UEs, predicted number of RRC connections), UE performance feedback information (average UE throughput DL, average UE throughput UL, average packet delay, average packet loss). The NG-RAN node 2 should include the following in the Data Collection Update message:

[0144] If the first bit "Predicted Radio Resource Status" of the Report Feature IE included in the AI / ML Information Request message is set to "1", the Radio Resource Status IE is predicted.

[0145] If the second bit "Predicted Activated UE Number" of the Reporting Features IE included in the AI / ML Information Request message is set to "1", then the Predicted Activated UE Number IE;

[0146] If the third bit "Predicted RRC (Radio Resource Control) Connection" of the Report Feature IE included in the AI / ML Information Request message is set to "1", the RRC Connection IE is predicted.

[0147] If the fourth bit "Average UE Throughput DL" of the Reporting Features IE included in the AI / ML Information Request message is set to "1", then the "Average UE Throughput DL" IE.

[0148] If the fifth bit "Average UE Throughput UL" ​​of the Reporting Feature IE included in the AI / ML Information Request message is set to "1", then the UE Average Throughput UL IE.

[0149] If the sixth bit "Average Packet Delay" of the Reporting Characteristics IE included in the AI / ML Information Request message is set to "1", the Average Packet Delay IE.

[0150] If the seventh bit "Average Packet Loss" of the Report Characteristics IE included in the AI / ML Information Request message is set to "1", the Average Packet Loss IE.

[0151] Figure 5c is one of the example diagrams of a data collection process provided by an embodiment of the present application. As shown in Figure 5c, if the requested data collection report cannot be initiated, NG-RAN node 2 shall send a data collection failure message with an appropriate cause value, i.e., a feedback data collection failure message.

[0152] It can be understood that the current measurement information may be transmitted in the manner described in the embodiments of FIG. 5 a to FIG. 5 c .

[0153] S402 , node 1 processes current measurement information through each prediction model of at least one prediction model to obtain current prediction information corresponding to the prediction model.

[0154] When a node in a communication network obtains current measurement information, it can process the current measurement information through each prediction model in at least one prediction model to obtain current prediction information corresponding to the prediction model. In one example, the node is a first node, and the first node itself can execute S402. In one example, the node is a second node, and S402 can specifically include: the second node sends the current measurement information to the first node through the process shown in Figures 5a to 5b above, and instructs the first node to perform information prediction, and then the first node processes the current measurement information through each prediction model in at least one prediction model to obtain current prediction information corresponding to the prediction model.

[0155] In specific applications, the current prediction information of the same scenario can be obtained through at least one prediction model. These prediction models can be different types of neural network models, and / or models obtained by training the same type of neural network model with different samples. The embodiments of the present application are not limited to this. The specific current prediction information for different scenarios may be different. For example, the prediction information involved in network energy saving, load balancing, and mobility optimization in the scenario shown in S401 above may include: resource status prediction information (such as predicted wireless resource status, predicted number of activated UEs and / or predicted number of RRC connections), UE trajectory prediction information (predicted cell identity and corresponding residence time), etc.

[0156] S403 , node 1 determines, based on the reference data, a quality indicator of current prediction information corresponding to at least one prediction model.

[0157] In an optional embodiment, the above-mentioned reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information and prediction auxiliary information; wherein, the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

[0158] It is understandable that for different reference data, the nodes of the communication network may determine the quality indicators in different ways. To facilitate understanding and reasonable layout, detailed descriptions will be given in the embodiments of Figures 6 to 9 later.

[0159] S404 , node 1 sends the current prediction information corresponding to each prediction model and the quality indicator of the current prediction information to node 0 .

[0160] When the node of the communication network is a first node, such as node1, S404 may include: the first node sends the quality indicator and the obtained current prediction information to the second node, so that the second node performs network management operations for the scenario based on the current prediction information that the quality indicator meets the quality conditions.

[0161] It is understandable that node 1 may carry the quality indicator of the current prediction information when sending the current prediction information, or may send the current prediction information and the quality indicator of the current prediction information separately. The embodiment of the present application does not limit the specific sending method of each information.

[0162] S405 , node 0 performs a scenario-specific network management operation based on the current prediction information corresponding to each prediction model and the current prediction information whose quality indicators meet the quality conditions.

[0163] For example, when the quality metric includes accuracy, the second node selects current prediction information with an accuracy greater than an accuracy threshold and performs scenario-specific network management operations, such as load balancing. If the node in the communication network is the second node, the second node itself can execute S404. For example, the above-mentioned prediction information is transmitted between gNBs. After receiving the prediction information, the gNB will combine local information to formulate a corresponding AI strategy. For example, after node0 receives load prediction information from neighboring nodes node1 / 2 / 3, it determines which neighboring nodes are relatively idle. Combining local load prediction information with information such as UE performance, it selects which UEs to handover to which cells for load balancing optimization. For another example, node0 predicts the UE's movement trajectory and sends it to the target cell in advance to perform mobility enhancement optimization.

[0164] It is understood that when there is only one prediction model, the current prediction information may be one. In this case, if the current prediction information does not meet the quality conditions, the node of the communication network may stop performing network management operations for the scenario. In one example, the node of the communication network may also feedback response information indicating that the prediction information is unqualified to the first node to obtain prediction information that meets the quality conditions, thereby improving network performance and user experience.

[0165] The above S402 to S405 will be described in detail below with reference to FIG. 6 to FIG. 9 .

[0166] Figure 6 is a flow chart of a network management method provided by an embodiment of the present application. As shown in Figure 6, the present embodiment can use the precision and / or accuracy of a specific test set (prediction information with known actual values, such as historical prediction information corresponding to the actual information) to approximate the evaluation result of the prediction information, i.e., the quality indicator. This embodiment does not limit the specific structure and form of the node; for example, it can be a node that can predict or requires prediction information, such as a gNB, UE, 5GC, OAM, etc.

[0167] In an optional example, the network management method provided in the embodiment of the present application may include:

[0168] S601, node c sends an evaluation pre-configuration message to node 1.

[0169] The third-party node node c, that is, the third node, sends an evaluation preconfiguration message, that is, a preset test set, to all nodes node 1, that is, the first node, which can generate prediction information, and preconfigures test sets (such as test data sets and data set indexes) and evaluation indicators and other data for evaluation for these nodes that can generate prediction information. If S601 is present, S602 is optional. If both S601 and S602 exist, the evaluation preconfiguration message sent by node0, that is, the second node in S602, does not need to carry a specific test data set, but only needs to carry a test set index to indicate the test data set stored in node1. At this time, it is equivalent to the second node sending evaluation requirement information (including the test set index) to the first node, so that the first node determines the test set that meets the evaluation requirement information in the original test set stored by itself as the preset test set, that is, the test data set indicated by the test set index. The third-party node node c can be other nodes in communication networks such as 5GC, OAM, NG-RAN, etc., and the embodiments of the present application do not limit this.

[0170] The above-mentioned evaluation preconfiguration message may specifically include the following information: a test set and identification information, and the identification information includes an AI model / task / function identification. Among them, the AI ​​model / task / function identification is used to indicate a specific AI model / task / function. The test set includes: a data set index and a test data set, and the test data set includes specific test data, such as the input and output of the AI ​​model, that is, the test measurement information and the test prediction information corresponding to the test measurement information, and the actual information corresponding to the test prediction information. In an optional example, the evaluation preconfiguration message may also include an evaluation indication and / or an evaluation index. Among them, the evaluation indication is used to indicate whether it is necessary to calculate the predicted evaluation result, and the implementation form can be a 1-bit indication information. If the evaluation indication is not carried, it is assumed that calculation is required, that is, quality evaluation is performed. The evaluation index is used to indicate how node1 calculates the predicted evaluation based on the test set, that is, how the quality index is calculated.

[0171] For example, the evaluation metric can be one or more of the following: mean squared error (MSE), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), accuracy, precision, recall, and a comprehensive evaluation index (F1). MSE is the ratio of the square of the deviation between the predicted and actual values ​​to the number of observations; the larger the error, the larger the value. RMSE is the square root of MSE. MAE is the average of the absolute values ​​of the deviation between the predicted and actual values. MAPE is the percentage of the average of the absolute values ​​of the deviation between the predicted and actual values ​​divided by the actual values. It is a statistical metric commonly used to measure forecast accuracy, such as in time series forecasting. Accuracy is the number of correctly classified samples divided by the total number of samples. Precision is the number of correctly predicted positive samples divided by the number of samples predicted as positive. Recall is the number of correctly predicted positive samples divided by the number of samples labeled as positive. The comprehensive evaluation index is the weighted harmonic average of Precision and Recall, that is, F1 = 2(Precision+Recall) / (Precision*Recall).

[0172] The embodiments of this application do not limit the association between the above information. Based on the evaluation pre-configuration message, a node can indicate to node 1 which AI model(s), task(s), or function(s) requires the use of which datasets and evaluation metrics to calculate and predict the evaluation results. AI models, tasks, or functions refer to predictive models, tasks, or functions based on artificial intelligence technology. For example, one possible implementation of the evaluation pre-configuration message is as follows: Dataset 1, AI model ID List 1, and evaluation metrics MSE and MAE. This evaluation pre-configuration message indicates that Dataset 1 is applicable to the AI ​​model identified by ID List 1, and the evaluation metrics can be one or more of MSE and MAE. Dataset 1 can be a test set, i.e., containing a dataset index and a test dataset, or, in S602 below, Dataset 1 can be simply a dataset index. Similarly, another evaluation pre-configuration message can be: Dataset 2, AI model ID 2, and evaluation metrics MSE and MAE. This other evaluation pre-configuration message indicates that Dataset 2 is applicable to the AI ​​model identified by ID 2, and the evaluation metrics can be one or more of MSE and MAE.

[0173] It is understood that different prediction models / tasks / functions can use different or the same data sets, i.e., test sets, and the evaluation indicators in different evaluation pre-configuration messages can be the same or different. The above evaluation pre-configuration messages are only examples. The embodiments of this application do not limit the specific content and structure of the evaluation pre-configuration messages, and can be set according to specific application scenarios.

[0174] S602, node0 sends an evaluation pre-configuration message to node1.

[0175] The node node0 receiving the prediction information sends prediction information evaluation auxiliary information to the node node1 generating the prediction information, wherein the prediction information evaluation auxiliary information is also the evaluation pre-configuration message in S602.

[0176] If S601 does not exist, that is, there is no preconfigured test set, then the above-mentioned prediction information evaluation auxiliary information may include: the following information: test set and identification information, the identification information includes the AI ​​model / task / function identification. Among them, the AI ​​model / task / function identification is used to indicate a specific AI model / task / function. The test set includes: a data set index and a test data set, and the test data set includes the input and output of the AI ​​model, that is, the test measurement information and the test prediction information corresponding to the test measurement information, and the actual information corresponding to the test prediction information. In an optional example, the evaluation preconfiguration message may also include an evaluation indication and / or an evaluation index. Among them, the evaluation indication is used to indicate whether the prediction evaluation result needs to be calculated, and the implementation form can be a 1-bit indication information. If the evaluation indication is not carried, it is assumed that calculation is required, that is, quality evaluation is performed. The evaluation index is used to indicate how node1 calculates the prediction evaluation based on the test set, that is, the calculation method of the quality index. The specific description of the evaluation index is shown in S601. The embodiment of the present application does not limit the association relationship between the above-mentioned information.

[0177] If S601 exists, that is, the test set is preconfigured, there is no need to transmit the complete test data set. The above-mentioned prediction information evaluation auxiliary information does not need to carry a specific test data set, but only needs to carry a test set index to indicate the test data set stored in node1. Information other than the test set index in the prediction information evaluation auxiliary information can be found in the description of S601 or the description of the prediction information evaluation auxiliary information when S601 does not exist, and will not be repeated here. The embodiment of the present application does not limit the association relationship of the above-mentioned information.

[0178] In an optional implementation, the process of sending the evaluation auxiliary information of the prediction information may include one of the following (1) and (2):

[0179] (1) The data collection request message (see the description of the embodiment of Figures 5a to 5c above for details) carries auxiliary information for prediction information evaluation, that is, specifying a test set and evaluation metrics when requesting prediction information. Different test sets and the same / different evaluation metrics can be specified when requesting different prediction information, or the same test set and evaluation metrics can be specified when requesting all prediction information.

[0180] (2) Use a new process to transmit the prediction information evaluation auxiliary information. That is, a new process is added that is different from the process of the data collection request message to transmit the prediction information evaluation auxiliary information.

[0181] S603, node 1 sends prediction information and evaluation results to node 0.

[0182] In one example, node 1 may carry the evaluation result when sending prediction information to node 0.

[0183] Node 1 locally calculates the prediction information to be sent in S603, i.e., the evaluation result of the current prediction information, based on the evaluation preconfiguration message, and sends it to node 0. Optionally, at least one of the following data may also be sent in S603: AI model / task / function identifier, dataset index, and evaluation indicator identifier.

[0184] In an embodiment of the present application, the node that generates prediction information calculates the evaluation result of the prediction information by obtaining a specific test set. The node that requests the prediction information can obtain the corresponding evaluation index while obtaining the prediction information, and then evaluate the credibility / precision / accuracy of the prediction information, which helps to improve the accuracy of AI decisions related to the prediction information and further improve the accuracy of AI business.

[0185] Figure 7 is a flow chart of a network management method provided by an embodiment of the present application. As shown in Figure 7, the present embodiment can use the precision and / or accuracy of historical prediction information to approximate the evaluation results of the prediction information. This embodiment does not restrict the node; it can be a gNB, UE, 5GC, OAM, or other node that can predict or requires prediction information. Node i in this embodiment generally refers to any node that can have actual information / evaluation results of historical prediction information.

[0186] In an optional example, if the actual information / evaluation result of the historical prediction information comes from node i, that is, the fourth node, for example, node 1 has previously sent the UE trajectory prediction to node i in the past, after the UE switches to node i, node i will obtain the actual trajectory of the UE, or calculate the evaluation result of the UE trajectory prediction, then the following S702-704 needs to be executed.

[0187] In an optional example, if the actual information / evaluation result of the historical prediction information comes from node 1, which is the fourth node. For example, node 1 has previously predicted the local load, then when the actual moment corresponding to the predicted historical prediction information is reached, node 1 will obtain the local historical actual load, or calculate the evaluation result of the load prediction, that is, node i is node 1, then there is no need to execute the following S703-704.

[0188] Specifically:

[0189] In an optional example, the above method may include:

[0190] In step S701, node0 sends a prediction evaluation indication message to node1 to indicate whether a prediction evaluation result is required when requesting prediction information. The prediction evaluation indication message can be a 1-bit indication message or a specific evaluation result indicator. Optionally, node0 can also indicate the minimum accuracy of the prediction information. If the requirement is not met, node1 does not need to send the prediction information, but instead indicates to node0 that the prediction information request failed and the reason (for example, the accuracy of the prediction information does not meet the requested minimum accuracy). It is understandable that, in one example, S701 may not be executed.

[0191] The method of sending the prediction evaluation indication information is similar to the method of sending the evaluation auxiliary information of the prediction information in the embodiment of Figure 6 above. The difference lies in the specific information sent. For the same parts, please refer to the method of sending the evaluation auxiliary information of the prediction information in the embodiment of Figure 6 above, which will not be repeated here.

[0192] In an optional example, the above method may include S702, node 1 sends prediction information and a prediction information retention indication to node i. Exemplarily, the prediction information retention indication may include: a time period for retaining the prediction information, or a time period for retaining the prediction evaluation result and an evaluation index indication (indicating which evaluation index to select). Different retention time periods / evaluation indexes may be specified when requesting different prediction information, or the same retention time period / evaluation index may be specified when requesting all prediction information. Any format and / or content of indication information for indicating the retention of the sent prediction information can be used in this application, and the embodiments of this application are not limited to this.

[0193] S703: Node 1 sends a historical prediction evaluation request to node i. The historical prediction evaluation request is equivalent to a historical prediction evaluation request message, which is used by node 1 to request the evaluation results of the prediction information previously sent by node i. The historical prediction evaluation request message may include at least one of the following: AI model / task / function identifier, UE identifier list, cell identifier list, and node identifier list. Specifically:

[0194] AI model / task / function identifiers are used to indicate specific AI models / tasks / functions.

[0195] UE identification list is used to indicate which UEs' historical prediction evaluation results are requested.

[0196] The cell identifier list is used to indicate which cells' historical prediction evaluation results are requested.

[0197] The node identifier list is used to indicate which nodes' historical prediction evaluation results are requested.

[0198] A time index, which can be a time point or a time period, is used to indicate when the historical prediction evaluation results are requested.

[0199] The historical prediction information identifier list, such as resource status prediction, UE trajectory prediction, UE performance prediction, etc., is used to indicate which prediction item's historical prediction evaluation results are requested.

[0200] Evaluation indicators are used to indicate how the evaluation results of the request history prediction are calculated. They can be one or more of the following indicators: mean squared error (MSE), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), accuracy, precision, recall, and comprehensive evaluation index (F1).

[0201] Node 0 identifier. This indicates that the historical prediction evaluation result requested by node i is sent to node 0. Accordingly, the identifier may also carry a corresponding reason, for example, the historical prediction evaluation result requested is the evaluation result of the latest prediction information.

[0202] The embodiment of the present application does not limit the association relationship of the above information.

[0203] S704, node i feeds back historical prediction evaluation results to node 1.

[0204] In one example, node i can directly send the historical prediction evaluation results to node0. The historical prediction evaluation results, that is, the quality indicators of the historical prediction information, can be the actual values ​​of the historical prediction information, or the evaluation results of the historical prediction information obtained by calculation. Among them, the calculation method of the evaluation results of the historical prediction information is similar to the calculation method of the evaluation results of the current prediction information in the embodiment of Figure 6 above. The difference is that the prediction information is different. For the same parts, please refer to the description of the embodiment of Figure 6 above, which will not be repeated here. In an optional example, the historical prediction evaluation results can also specifically include at least one of the following: AI model identifier, prediction task identifier, function identifier, UE identifier, time index, and historical prediction information identifier.

[0205] The embodiment of the present application does not limit the association relationship of the above information.

[0206] S705, node 1 sends the prediction information and evaluation results to node 0, as well as the prediction information retention instruction

[0207] In one example, when node1 sends prediction information to node0, it carries the evaluation result of the above historical prediction information as the evaluation result of the current prediction information. Optionally, when node1 sends prediction information to node0, it can also carry at least one of the following: AI model / task / function identifier, historical prediction related information (UE identifier, time index, node i identifier), evaluation index identifier, and prediction information retention indication. That is, in one example, node1 may not send the prediction information retention indication.

[0208] It is understandable that S702 may be executed before or after S701.

[0209] In an embodiment of the present application, the node that generates prediction information obtains the accuracy of historical prediction information to approximately represent the evaluation result of the prediction information. The node that requests prediction information can obtain the corresponding evaluation indicators while obtaining the prediction information, and then evaluate the credibility / precision / accuracy of the prediction information, which helps to improve the accuracy of AI decisions related to the prediction information and further improve the accuracy of AI business.

[0210] Figure 8 is a flow chart of a network management method provided in an embodiment of the present application. As shown in Figure 8, this embodiment of the present application uses an AI evaluation model to infer the evaluation results of prediction information. This embodiment does not restrict the nodes; these nodes can be gNBs, UEs, 5GCs, OAMs, and other nodes that can or require prediction information. Node i in this embodiment generally refers to any node that can have actual information / evaluation results for historical prediction information.

[0211] S801, node0 sends a prediction evaluation instruction to node1.

[0212] S802, node 1 sends prediction information and a prediction information retention indication to node i.

[0213] The above S801 to S802 are the same as S701 to S702 and are not described again here. For details, please refer to the description of the embodiment of Figure 7 above.

[0214] S803, node 1 sends a historical prediction evaluation request to node i.

[0215] The above S803 is similar to the above S703, except that S803 collects historical prediction information for training to obtain an AI evaluation model, which does not need to be sent to node 0 and does not need to carry the node 0 identifier.

[0216] S804, node i feeds back the historical prediction evaluation results to node 1.

[0217] The above S804 is the same as the above S704 and will not be repeated here. For details, please refer to the description of the embodiment of Figure 7 above.

[0218] It is understandable that S802 to S804 may be executed before or after S801 .

[0219] S805, node 1 obtains the evaluation result of the prediction information through the pre-trained evaluation model.

[0220] Node 1 collects the evaluation results of neighboring nodes and local historical prediction information as AI training data, which is used to train a local AI evaluation model, i.e., an evaluation model, to obtain the evaluation results of the prediction information. In one possible implementation, after node 1 infers the prediction information through the local AI model, it can process the prediction information through the local AI evaluation model to infer the evaluation results of the prediction information. Among them, a possible implementation method of inferring the evaluation results of the prediction information through the AI ​​evaluation model is as follows:

[0221] The input of the AI ​​evaluation model: output data, i.e., prediction information, and optionally, input data, i.e., current measurement information.

[0222] Output of the AI ​​evaluation model: evaluation results.

[0223] In other words, in one example, node 1 can input current prediction information and current measurement information into the AI ​​evaluation model to obtain an evaluation result. It is understandable that, in this case, the training data used to train the AI ​​evaluation model also includes historical measurement information.

[0224] S806 , node 1 sends the prediction information and evaluation results, as well as a prediction information retention instruction to node 0 .

[0225] When node1 sends prediction information to node0, it carries the evaluation results of the historical prediction information as the evaluation results of the current prediction information. Optionally, node1 can also send prediction information to node0 with at least one of the following: an AI model / task / function identifier, an AI evaluation model identifier, an evaluation indicator identifier, and a prediction information retention indication. In other words, in one example, node1 may not send a prediction information retention indication.

[0226] In an embodiment of the present application, the node that generates prediction information infers the evaluation result of the prediction information, and the node that requests the prediction information can obtain the corresponding evaluation index while obtaining the prediction information, and then evaluate the credibility / precision / accuracy of the prediction information, which helps to improve the accuracy of AI decisions related to the prediction information and further improve the accuracy of AI business.

[0227] Figure 9 is a flow chart of a network management method provided by an embodiment of the present application. As shown in Figure 9, the embodiment of the present application can use AI auxiliary information, i.e., prediction auxiliary information, to implicitly represent the evaluation results of the prediction information. For example, by default, the prediction information from AI services similar to the AI ​​services in the scenario is more accurate. In this embodiment, there is no restriction on the node, and it can be a node such as gNB, UE, 5GC, OAM, etc. that can predict or require prediction information. Node i in this embodiment generally refers to a node that can have actual information / evaluation results of historical prediction information.

[0228] The main process is as follows:

[0229] S901, node1 sends prediction information and AI-assisted information to node0.

[0230] The node node1 that generates the prediction information carries the AI ​​auxiliary information when sending the prediction information to the node node0 that receives the prediction information. The AI ​​auxiliary information is used to implicitly indicate the evaluation result of the prediction information.

[0231] The AI ​​auxiliary information may be one or more of the following: AI business identification / usable scenarios (belonging to the operating environment / model attributes), AI model identification (belonging to the model attributes), AI function identification (belonging to the model attributes), local data characteristics (the degree of data non-IID, which is related to the generalization ability of the AI ​​model) (belonging to the operating environment), equipment vendor identification to which the model can be applied (belonging to the operating environment), model input and output parameters (belonging to the model attributes), training parameters (belonging to the model attributes) and other model parameters.

[0232] S902, node 0 obtains an evaluation result of the prediction information based on the AI ​​auxiliary information.

[0233] After receiving the AI-assisted information, Node0 evaluates and analyzes the prediction information based on the prediction requirements of the network management scenario. In S902 above, Node0 determines the quality indicators of the current prediction information corresponding to each prediction model that meet the quality conditions based on the matching degree between the prediction auxiliary information corresponding to the prediction model and the prediction requirements.

[0234] For example, AI-assisted information 1 from node1 includes: mobility optimization AI service; AI-assisted information 2 from node2 includes: load balancing AI service. Prediction information comes from node11 / node12 and includes: UE's QoE prediction. Node0 is performing UE's mobility optimization service, which means that the second node's predicted demand is the UE's mobility optimization service, and it is expected to determine the UE's switching strategy based on the evaluation result of the prediction information. Based on AI-assisted information 1 and AI-assisted information 12, nde0 determines that AI-assisted information 1 is more consistent with node0's predicted demand. Therefore, it determines that node1's prediction information is more accurate, and thus switches the UE to node1.

[0235] It is understandable that node1 and node2 are both first nodes. In one example, the above two types of AI-assisted information can also come from the same first node. The embodiment of the present application does not limit the number of first nodes and whether the AI-assisted information comes from the same node, which depends on the specific application requirements.

[0236] In an embodiment of the present application, a node requesting prediction information can obtain corresponding AI auxiliary information to approximate the evaluation results while obtaining the prediction information, and then evaluate the credibility / precision / accuracy of the prediction information, which helps to improve the accuracy of AI decisions related to the prediction information and further improve the accuracy of AI services.

[0237] It is understandable that, without the need to perform quality evaluation on the prediction information based on the current prediction information, the execution order of the above S402 and S403 can be arbitrary, that is, S402 and S403 can be executed sequentially or simultaneously, and the embodiment of the present application does not limit this.

[0238] In an embodiment of the present application, nodes in a communication network can obtain reference data that matches a scenario managed by the network. The reference data is used to indicate the prediction quality of at least one prediction model that matches the scenario. Based on the reference data, the nodes in the communication network can determine the quality indicators of the current prediction information corresponding to each of the at least one prediction model, that is, evaluate the quality (such as accuracy and / or precision) of the prediction information currently obtained by each prediction model. Based on this, the nodes in the communication network perform network management operations based on the current prediction information whose quality indicators meet the quality conditions, thereby avoiding network operations based on inaccurate prediction information, thereby improving network performance and user experience.

[0239] In addition, the network management device and communication network shown in Figures 3a to 3c of the present application respectively include hardware and / or software modules that perform the corresponding functions in order to realize the functions of the network management method in the above-mentioned embodiments of the present application. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0240] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the network management method in the above-mentioned embodiment.

[0241] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the network management method in the above-mentioned embodiment.

[0242] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0243] Any content of each embodiment of this application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0244] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0245] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A network management method, characterized in that: Applied to a first node performing information prediction in a communication network, the method comprises: Acquire reference data matching a scenario managed by the network and current measurement information, wherein the reference data is used to indicate a prediction quality of at least one prediction model matching the scenario; Processing the current measurement information through each prediction model of the at least one prediction model respectively to obtain current prediction information corresponding to the prediction model; Based on the reference data, determining a quality indicator of current prediction information respectively corresponding to the at least one prediction model; The current prediction information corresponding to each prediction model and the quality indicator of the current prediction information are sent to a second node in the communication network, wherein the second node is used to perform network management operations for the scenario according to the current prediction information corresponding to each prediction model whose quality indicator meets the quality conditions.

2. The method according to claim 1, characterized in that The reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

3. The method according to claim 2, characterized in that The reference information includes the preset test set; The determining, based on the reference data, a quality indicator of current prediction information respectively corresponding to the at least one prediction model comprises: Processing the test measurement information through each prediction model respectively to obtain test prediction information corresponding to the prediction model; For each prediction model, quality evaluation is performed on the test prediction information corresponding to the prediction model based on the test prediction information corresponding to the prediction model, the test prediction information corresponding to the test measurement information, and the actual information corresponding to the test prediction information to obtain a quality indicator of the current prediction information corresponding to the prediction model.

4. The method according to claim 2 or 3, characterized in that: The reference information includes the preset test set; The step of obtaining reference data matching the network management scenario includes one of the following steps: Reading the preset test set stored in the first node; Receiving the preset test set sent by a third node, wherein the third node is a node in the communication network different from the first node and the second node; Receive the preset test set sent by the second node.

5. The method according to claim 2 or 3, characterized in that: The reference information includes the preset test set; The obtaining of reference data matching the network management scenario includes: receiving an original test set sent by a third node, wherein the third node is a node different from the first node and the second node in the communication network, and the original test set includes a plurality of test sets, each test set including test measurement information, test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; receiving evaluation requirement information sent by the second node; A test set in the original test set that meets the evaluation requirement information is determined as the preset test set.

6. The method according to claim 2, characterized in that The reference information includes a quality indicator of the historical prediction information; The determining, based on the reference data, a quality indicator of current prediction information respectively corresponding to the at least one prediction model comprises: For each prediction model, a quality indicator corresponding to the prediction model among the quality indicators of the historical prediction information is determined as the quality indicator of the current prediction information corresponding to the prediction model.

7. The method according to claim 2 or 6, characterized in that: The reference information includes a quality indicator of the historical prediction information; The obtaining of reference data matching the network management scenario includes: Acquire a quality indicator of the historical prediction information in a fourth node, wherein the fourth node is a node in the communication network to which the first node historically sent the prediction information of the scenario; or, In the case where the first node historically predicts information for the scenario, a quality indicator of the historical prediction information stored by the first node itself is read.

8. The method according to claim 2, characterized in that: The reference information includes the historical prediction information; The determining, based on the reference data, a quality indicator of current prediction information respectively corresponding to the at least one prediction model comprises: For each prediction model, the historical prediction information corresponding to the prediction model or the current prediction information corresponding to the prediction model is Information is input into a pre-trained evaluation model to obtain a quality indicator of current prediction information corresponding to the prediction model; wherein the evaluation model is a model trained using the historical network information and the quality indicator of the historical network information.

9. The method according to claim 2 or 8, characterized in that: The reference information includes the historical prediction information; The obtaining of reference data matching the network management scenario includes: Obtain neighboring nodes of the first node and historical prediction information in the first node that matches the scenario managed by the network, as well as a quality indicator of the historical prediction information.

10. The method according to any one of claims 6 to 9, characterized in that After determining the quality index of the current prediction information respectively corresponding to the at least one prediction model based on the reference data, the method further includes: In the case of sending the current prediction information corresponding to each prediction model to the second node in the communication network, a retention indication of the current prediction information corresponding to each prediction model is sent to the second node.

11. The method according to any one of claims 6 to 10, characterized in that The obtaining of reference data matching the network management scenario includes: receiving a prediction evaluation indication sent by the second node, where the prediction evaluation indication is used to indicate requirement information of the second node for a quality indicator of prediction information; According to the prediction evaluation indication, reference data matching the scenario of network management is obtained.

12. The method according to claim 2, characterized in that: The reference data includes the prediction auxiliary information; The obtaining of reference data matching the network management scenario includes: Acquiring the prediction auxiliary information according to the at least one prediction model; The determining, based on the reference data, a quality indicator of current prediction information respectively corresponding to the at least one prediction model comprises: For each prediction model, the attribute information and / or the operating environment of the prediction model indicated by the prediction auxiliary information is determined as a quality indicator of the current prediction information corresponding to the prediction model.

13. A network management method, characterized in that: Applied to a second node using prediction information in a communication network, the method comprises: Receiving current prediction information corresponding to each prediction model in at least one prediction model sent by a first node performing information prediction in the communication network and a quality indicator of the current prediction information; Performing a network management operation for the scenario according to the current prediction information corresponding to each prediction model, in which the quality indicator meets the quality condition; Among them, the quality indicator of the current prediction information corresponding to each prediction model is determined by the first node based on reference data matching the scenario managed by the network, and the reference data is used to indicate the prediction quality of at least one prediction model matching the scenario, and the prediction model is used to perform information prediction.

14. The method according to claim 13, characterized in that The reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

15. The method according to claim 14, characterized in that The reference data includes the preset test set; Before receiving the current prediction information and the quality indicator of the current prediction information respectively corresponding to each prediction model in at least one prediction model sent by the first node performing information prediction in the communication network, the method further includes: Sending the preset test set to the first node; or, Send evaluation requirement information to the first node.

16. The method according to claim 14, characterized in that The reference information includes a quality indicator of the historical prediction information or the historical prediction information; The method further comprises: In the case of receiving current prediction information corresponding to each prediction model in at least one prediction model sent by a first node performing information prediction in the communication network, a retention indication for the current prediction information corresponding to each prediction model sent by the first node is also received.

17. The method according to claim 14 or 16, characterized in that The reference information includes a quality indicator of the historical prediction information or the historical prediction information; Before receiving the current prediction information and the quality indicator of the current prediction information respectively corresponding to each prediction model in at least one prediction model sent by the first node performing information prediction in the communication network, the method further includes: A prediction evaluation indication is sent to the first node, where the prediction evaluation indication is used to indicate the second node's requirement information for a quality indicator of prediction information.

18. The method according to claim 14, characterized in that The reference data includes the prediction auxiliary information; The performing a network management operation for the scenario according to the current prediction information corresponding to each prediction model, in which the quality indicator meets the quality condition, includes: determining the forecasted demand for the scenario; For each prediction model, according to the matching degree between the prediction auxiliary information corresponding to the prediction model and the prediction demand, determine the current prediction information corresponding to each prediction model, in which the quality indicator meets the quality condition; According to the current prediction information that the quality indicator meets the quality condition, a network management operation for the scenario is performed.

19. A network management method, characterized in that: Applied to a communication network, the communication network includes a first node that performs information prediction and a second node that uses the predicted information, the method includes: The first node is used to obtain reference data and current measurement information matching a scenario managed by the network, wherein the reference data is used to indicate the prediction quality of at least one prediction model matching the scenario; process the current measurement information through each prediction model of the at least one prediction model respectively to obtain current prediction information corresponding to the prediction model; determine the quality index of the current prediction information corresponding to each of the at least one prediction model based on the reference data; and send the current prediction information corresponding to each prediction model and the quality index of the current prediction information to the second node in the communication network; The second node is used to perform network management operations for the scenario according to the current prediction information corresponding to each prediction model, in which the quality indicator meets the quality condition.

20. The method according to claim 19, characterized in that The reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

21. A network management device, characterized in that: Applied to a first node for information prediction in a communication network, the device comprises: An information acquisition module, configured to acquire reference data and current measurement information matching a scenario managed by the network, wherein the reference data is used to indicate a prediction quality of at least one prediction model matching the scenario; and to process the current measurement information through each prediction model of the at least one prediction model to obtain current prediction information corresponding to the prediction model; A quality evaluation module, used to determine, based on the reference data, a quality indicator of current prediction information respectively corresponding to the at least one prediction model; An information sending module is used to send the current prediction information corresponding to each prediction model and the quality indicator of the current prediction information to the second node in the communication network, wherein the second node is used to perform network management operations for the scenario according to the current prediction information corresponding to each prediction model whose quality indicator meets the quality conditions.

22. The device according to claim 21, characterized in that The reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

23. The device according to claim 22, characterized in that The reference information includes the preset test set; The quality evaluation module is specifically used for: Processing the test measurement information through each prediction model respectively to obtain test prediction information corresponding to the prediction model; For each prediction model, quality evaluation is performed on the test prediction information corresponding to the prediction model based on the test prediction information corresponding to the prediction model, the test prediction information corresponding to the test measurement information, and the actual information corresponding to the test prediction information to obtain a quality indicator of the current prediction information corresponding to the prediction model.

24. The device according to claim 22 or 23, characterized in that The reference information includes the preset test set; The information acquisition module is specifically used to perform one of the following steps: Reading the preset test set stored in the first node; Receiving the preset test set sent by a third node, wherein the third node is a node in the communication network different from the first node and the second node; Receive the preset test set sent by the second node.

25. The device according to claim 22 or 23, characterized in that The reference information includes the preset test set; The information acquisition module is specifically used for: receiving an original test set sent by a third node, wherein the third node is a node different from the first node and the second node in the communication network, and the original test set includes a plurality of test sets, each test set including test measurement information, test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; receiving evaluation requirement information sent by the second node; A test set in the original test set that meets the evaluation requirement information is determined as the preset test set.

26. The device according to claim 22, characterized in that The reference information includes a quality indicator of the historical prediction information; The quality evaluation module is specifically used for: For each prediction model, a quality indicator corresponding to the prediction model among the quality indicators of the historical prediction information is determined as the quality indicator of the current prediction information corresponding to the prediction model.

27. The device according to claim 22 or 26, characterized in that The reference information includes a quality indicator of the historical prediction information; The information acquisition module is specifically used for: Acquire a quality indicator of the historical prediction information in a fourth node, wherein the fourth node is a node in the communication network to which the first node historically sent the prediction information of the scenario; or, In the case where the first node historically predicts information for the scenario, a quality indicator of the historical prediction information stored by the first node itself is read.

28. The device according to claim 22, characterized in that The reference information includes the historical prediction information; The quality evaluation module is specifically used for: For each prediction model, the historical prediction information corresponding to the prediction model or the current prediction information corresponding to the prediction model is input into a pre-trained evaluation model to obtain a quality indicator of the current prediction information corresponding to the prediction model; wherein the evaluation model is a model trained using the historical network information and the quality indicator of the historical network information.

29. The device according to claim 22 or 28, characterized in that The reference information includes the historical prediction information; The information acquisition module is specifically used for: Obtain neighboring nodes of the first node and historical prediction information in the first node that matches the scenario managed by the network, as well as a quality indicator of the historical prediction information.

30. The device according to any one of claims 26 to 29, characterized in that The information sending module is also used for: In the case of sending the current prediction information corresponding to each prediction model to the second node in the communication network, a retention indication of the current prediction information corresponding to each prediction model is sent to the second node.

31. The device according to any one of claims 26 to 30, characterized in that The information acquisition module is specifically used for: receiving a prediction evaluation indication sent by the second node, where the prediction evaluation indication is used to indicate requirement information of the second node for a quality indicator of prediction information; According to the prediction evaluation indication, reference data matching the scenario of network management is obtained.

32. The device according to claim 22, characterized in that The reference data includes the prediction auxiliary information; The obtaining of reference data matching the network management scenario includes: Acquiring the prediction auxiliary information according to the at least one prediction model; The quality evaluation module is specifically used for: For each prediction model, the attribute information and / or the operating environment of the prediction model indicated by the prediction auxiliary information is determined as a quality indicator of the current prediction information corresponding to the prediction model.

33. A network management device, characterized in that: Applied to a second node using prediction information in a communication network, the device comprises: An information receiving module, configured to receive current prediction information corresponding to each prediction model in at least one prediction model sent by a first node performing information prediction in the communication network and a quality indicator of the current prediction information; An information application module, configured to perform a network management operation for the scenario according to the current prediction information corresponding to each prediction model, in which the quality indicator meets the quality condition; Among them, the quality indicator of the current prediction information corresponding to each prediction model is determined by the first node based on reference data matching the scenario managed by the network, and the reference data is used to indicate the prediction quality of at least one prediction model matching the scenario, and the prediction model is used to perform information prediction.

34. The device according to claim 33, characterized in that The reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

35. The device according to claim 34, characterized in that The reference data includes the preset test set; The device also includes: an information sending module, which is used to: Before the information receiving module receives the current prediction information and the quality indicator of the current prediction information respectively corresponding to each prediction model in at least one prediction model sent by the first node performing information prediction in the communication network, Sending the preset test set to the first node; or, Send evaluation requirement information to the first node.

36. The device according to claim 34, characterized in that The reference information includes a quality indicator of the historical prediction information or the historical prediction information; The information receiving module is further used for: In the case of receiving current prediction information corresponding to each prediction model in at least one prediction model sent by a first node performing information prediction in the communication network, a retention indication for the current prediction information corresponding to each prediction model sent by the first node is also received.

37. The device according to claim 34 or 36, characterized in that The reference information includes a quality indicator of the historical prediction information or the historical prediction information; The device also includes: an information sending module, which is used to: Before the information receiving module receives the current prediction information and the quality indicator of the current prediction information respectively corresponding to each prediction model in at least one prediction model sent by the first node performing information prediction in the communication network, A prediction evaluation indication is sent to the first node, where the prediction evaluation indication is used to indicate the second node's requirement information for a quality indicator of prediction information.

38. The device according to claim 34, characterized in that The reference data includes the prediction auxiliary information; The information application module is specifically used for: determining the forecasted demand for the scenario; For each prediction model, according to the matching degree between the prediction auxiliary information corresponding to the prediction model and the prediction demand, determine the current prediction information corresponding to each prediction model, in which the quality indicator meets the quality condition; According to the current prediction information that the quality indicator meets the quality condition, a network management operation for the scenario is performed.

39. A communication network, characterized in that: The communication network includes a first node that performs information prediction and a second node that uses the predicted information; The first node is used to obtain reference data and current measurement information matching a scenario managed by the network, wherein the reference data is used to indicate the prediction quality of at least one prediction model matching the scenario; process the current measurement information through each prediction model of the at least one prediction model respectively to obtain current prediction information corresponding to the prediction model; determine the quality index of the current prediction information corresponding to each of the at least one prediction model based on the reference data; and send the current prediction information corresponding to each prediction model and the quality index of the current prediction information to the second node in the communication network; The second node is used to perform network management operations for the scenario according to the current prediction information corresponding to each prediction model, in which the quality indicator meets the quality condition.

40. The communication network according to claim 39, characterized in that The reference data includes one of the following: a preset test set, quality indicators of historical prediction information, historical prediction information, and prediction auxiliary information; wherein the preset test set includes test measurement information and test prediction information corresponding to the test measurement information, and actual information corresponding to the test prediction information; the prediction auxiliary information is used to indicate the attribute information and / or operating environment of each prediction model.

41. An electronic device, characterized in that: include: Processor and memory; The processor is connected to the memory; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 20.

42. A computer-readable storage medium, characterized in that: The method comprises a computer program, wherein when the computer program is run on an electronic device, the electronic device executes the method according to any one of claims 1 to 20.

43. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal comprising a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 20.

44. A computer program product, characterized in that The invention comprises a computer program, which, when executed by an electronic device, enables the electronic device to execute the method according to any one of claims 1 to 20.

Images