Data processing method in a communication network, network side device, and readable storage medium

Abstract

The present application discloses a data processing method in a communication network and a network side device, which belongs to the field of communication technology. The data processing method in a communication network of an embodiment of the present application includes: a first network element determines a first accuracy of a first model, the first accuracy being for indicating the accuracy degree of the first model used for actual inference; and when the first accuracy satisfies a predetermined condition, the first network element retrains the first model or reselects a second model.

Description

[Technical field]

[0001] The present application relates to the field of communication technology, and in particular to a data processing method and network side device in a communication network. [Background technology]

[0002] In a communication network, some network elements are usually introduced to perform intelligent model training, and then perform inference tasks based on the trained model to obtain inference results, which can assist the policy decision-making of devices inside and outside the network to improve the intelligent level of the policy decision-making of the devices.

[0003] However, in practical applications, the accuracy achieved by a model after the training stage does not represent the inference accuracy that can be achieved when the model is used for practical inference, i.e., when performing an inference task based on a model, the accuracy of the inference result may be lower than that of the model training stage, in which case, when the inference result is provided to devices inside and outside the network, it may lead to erroneous policy decisions or the execution of inappropriate operations. Summary of the Invention [Problem to be solved by the invention]

[0004] The embodiments of the present application provide a data processing method and a network side device in a communication network that can solve the problem of affecting policy decisions when the accuracy of a model during actual inference is low. [Means for solving the problem]

[0005] In a first aspect, a first network element determining a first accuracy of a first model, said first accuracy being for indicating a degree of accuracy of said first model used in actual inference; if the first accuracy satisfies a predetermined condition, the first network element retrains the first model or reselects a second model.

[0006] In a second aspect, a determination module adapted to determine a first accuracy of a first model, the first accuracy being for indicating a degree of accuracy of the first model used in actual inference; and a model training module adapted to retrain the first model or reselect a second model if the first accuracy satisfies a predetermined condition.

[0007] In the third aspect, a second network element performing an inference task based on a first model, the first model being trained by the first network element, the first network element including a model training function network element; The present invention provides a method for processing data in a communications network, the method comprising: transmitting at least one of the usage information and the first data of the first model to the first network element; and / or transmitting first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store the first data of the inference task, the seventh network element including a data storage function network element.

[0008] In the fourth aspect: a task execution module adapted to execute an inference task based on a first model, the first model being trained by a first network element, the first network element including a model training function network element; The present invention provides a data processing device in a communication network comprising: a transmission module used for transmitting at least one of the usage information and first data of the first model to the first network element and / or transmitting first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store first data of the inference task, the seventh network element including a data storage function network element.

[0009] In a fifth aspect, there is provided a network side device comprising a processor and a memory in which programs or commands executable on the processor are stored, and when the programs or commands are executed by the processor, the steps of the method according to the first aspect are realized.

[0010] In a sixth aspect, there is provided a network side device comprising a processor and a communication interface, the processor being used to determine a first accuracy of a first model, the first accuracy being for indicating a degree of accuracy of the first model to be used for actual inference, and to retrain the first model or reselect a second model if the first accuracy satisfies a predetermined condition.

[0011] In a seventh aspect, there is provided a network side device comprising a processor and a memory in which a program or command executable on the processor is stored, and when the program or command is executed by the processor, the steps of the method according to the third aspect are realized.

[0012] In an eighth aspect, a network side device is provided, comprising: a processor; and a communication interface; the processor is used for executing an inference task based on a first model, the first model being trained by a first network element, the first network element including a model training function network element; the communication interface is used for transmitting at least one of usage information of the first model and first data to the first network element, and / or transmitting first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store first data of the inference task, and the seventh network element including a data storage function network element.

[0013] In a ninth aspect, there is provided a data processing system in a communication network comprising a first network side device and a second network side device, the first network side device being capable of executing steps of the method described in the first aspect, and the second network side device being capable of executing steps of the method described in the third aspect.

[0014] In a tenth aspect, there is provided a readable storage medium having a program or command stored thereon, the program or command causing the steps of the method according to the first aspect to be realized or the steps of the method according to the third aspect to be realized when executed by a processor.

[0015] In an eleventh aspect, there is provided a chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being configured to execute a program or command to implement the method described in the first aspect or to implement the method described in the third aspect.

[0016] In a twelfth aspect, there is provided a computer program / program product stored on a storage medium, the computer program / program product being adapted to implement the steps of the method according to the first aspect or to implement the steps of the method according to the third aspect when executed by at least one processor. Effect of the Invention

[0017] In an embodiment of the present application, a first accuracy of a first model when used for actual inference is determined, and if the first accuracy does not satisfy a predetermined condition, the first model can be retrained. Therefore, if the accuracy of the first model when used for actual inference decreases, the accuracy of the first model can be adjusted by retraining the first model or reselecting a second model, thereby improving the accuracy of the first model when used for actual inference, thereby better assisting devices inside and outside the network in making correct policy decisions or taking appropriate actions. [Brief description of the drawings]

[0018] [Figure 1] 1 is a schematic diagram of a wireless communication system according to an embodiment of the present application; [Diagram 2] 1 is a schematic flow chart of a data processing method in a communication network according to an embodiment of the present application; [Diagram 3] 1 is a schematic flow chart of a data processing method in a communication network according to an embodiment of the present application; [Figure 4] 1 is a schematic flow chart of a data processing method in a communication network according to an embodiment of the present application; [Diagram 5] FIG. 2 is a structural schematic diagram of a data processing device in a communication network according to an embodiment of the present application; [Figure 6] FIG. 2 is a structural schematic diagram of a data processing device in a communication network according to an embodiment of the present application; [Figure 7] 1 is a structural schematic diagram of a communication device according to an embodiment of the present application; [Figure 8]FIG. 2 is a structural schematic diagram of a network side device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, the technical solutions in the embodiments of the present application will be clearly described with reference to the drawings in the embodiments of the present application, and it is to be understood that the described embodiments are only a part of the embodiments of the present application, and are not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without any creative efforts shall fall within the scope of protection of the present application.

[0020] The terms "first", "second", etc. in the specification and claims of the present application are not intended to describe a particular order or sequence, but are intended to distinguish between similar objects. The terms used in this manner may be interchangeable in some cases so that the embodiments of the present application may be implemented in an order other than that shown or described herein, and it should be understood that the objects distinguished by "first" and "second" generally belong to the same type, and the number of objects is not limited, for example, the first object may be one or more. In the specification and claims, "and / or" indicates at least one of the objects connected, and the symbol " / " generally indicates that the related objects before and after are in an "or" relationship.

[0021] It should be noted that the technology described in the embodiments of the present application is not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, and can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency Division Multiple Access (SC-FDMA), and other systems. In the embodiments of the present application, the terms "system" and "network" are often used interchangeably, and the technology described herein may be used in the above systems and wireless communication technologies, or in other systems and wireless communication technologies. However, in the following description, a 5th generation (5G) system is described for illustrative purposes, and 5G terminology is used in most of the following description, but these technologies are applicable to systems other than 5G systems, for example, they are also applicable to 6th generation (6G) communication systems.

[0022] 1 shows a block diagram of a wireless communication system to which an embodiment of the present application can be applied. The wireless communication system includes a terminal 11 and a network side device 12. The terminal 11 may be a mobile phone, a tablet personal computer, a laptop computer (also called a notebook computer), a personal digital assistant (PDA), a palmtop computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an in-vehicle device (Vehicle User Equipment, VUE), a pedestrian user equipment (PUE), a smart home (household devices equipped with wireless communication functions such as a refrigerator, a television, a washing machine, or furniture), a game machine, a personal computer, a mobile phone, a mobile terminal, a mobile user equipment (MID), a mobile internet device ... The network side device 12 may be a terminal side device such as a wireless computer (PC), an automated teller machine or a kiosk. The wearable device includes a smart watch, a smart bracelet, a smart headphone, a smart glass, a smart jewelry (smart bangle, smart bracelet, smart ring, smart necklace, smart anklet, smart wristband, smart wear, etc. It should be noted that in the embodiment of the present application, the specific type of the terminal 11 is not limited. The network side device 12 may include an access network device or a core network device, and the access network device 12 may be called a radio access network device, a radio access network (Radio Access Network, RAN), a radio access network function or a radio access network unit.The access network device 12 may include a base station, a Wireless Local Area Network (WLAN) access point, a WiFi node, etc. The base station may be called a Node B, an evolved Node B (eNB), an access point, a base transceiver station (BTS), a radio base station, a radio transceiver, a Basic Service Set (BSS), an Extended Service Set (ESS), a Home B node, a Home evolved B node, a Transmitting Receiving Point (TRP), or any other suitable term in the above field, and the base station is not limited to a specific technical term as long as it can achieve the same technical effect. It should be noted that in the embodiment of the present application, a base station in an NR system is merely taken as an example, but the specific type of the base station is not limited.The core network devices are: Core Network Node, Core Network Function, Mobility Management Entity (MME), Access and Mobility Management Function (AMF), Session Management Function (SMF), User Plane Function (UPF), Policy Control Function (PCF), Policy and Charging Rules Function (PCRF), Edge Application Server Discovery Function (EASDF), Unified Data Management (UDM), Unified Data Repository (UDR), Home Subscriber Server (HSS), Centralized network configuration (CNC), Network Repository Function (NRF), Network Exposure Function (NEF), Local NEF (Local NEF or L-NEF), Binding Support Function (BSF), Application Function (Application Node B), Application Node C, Application Node D, Application Node E, Application Node F ... The core network device may include, but is not limited to, at least one of the following: A-F (Auditory Function, AF) and the like. It should be noted that in the embodiment of the present application, a core network device in a New Radio (NR) system is simply described as an example, but the specific type of the core network device is not limited.

[0023] In a communication network, some network elements can be introduced to perform intelligent data analysis and generate data analysis results for some tasks, which can assist the policy decision-making of devices inside and outside the network, with the purpose of improving the intelligent level of the policy decision-making of the devices through artificial intelligence (AI) methods.

[0024] For example, a Network Data Analytics Function (NWDAF) can train an artificial intelligence / machine learning (AI / ML) model based on training data to obtain a corresponding model suitable for a certain AI task. The NWDAF performs model inference based on an AI / ML model and inference input data to obtain data analysis results (analytics) corresponding to a specific AI inference task, and the data analysis results (analytics) are also called inference results. A PCF in a network executes an intelligent Policy Control and Charging (PCC) policy based on a certain inference result, for example, formulates an intelligent user stay policy based on the analysis result of a user's business behavior, and improves the user's business experience. Alternatively, an AMF executes an intelligent mobility management operation based on a certain inference result or analytics, for example, intelligently pages a user based on the analysis result of the user's movement trajectory, and improves the paging reachability rate.

[0025] It is assumed that devices inside and outside the network must make correctly optimized policy decisions based on AI data analysis results based on correct data analysis results. If the accuracy rate of the data analysis results is low and they are provided to devices inside and outside the network as erroneous information for reference, it will ultimately lead to erroneous policy decisions or the execution of inappropriate operations, so it is necessary to ensure the accuracy of the data analysis results. However, in actual applications, due to differences in data distribution, lack of generalization ability of the model, etc., the model accuracy achieved by the model after the training stage does not represent the inference accuracy that can be achieved when this model is used for actual inference (generally, the actual inference accuracy of the model is lower than the accuracy of the model training stage). In this way, the above problems of erroneous policy decisions or the execution of inappropriate operations are likely to occur.

[0026] In order to solve the above technical problems, the embodiments of the present application provide a data processing method and a network side device in a communication network, and when the accuracy of a first model decreases when used in actual inference, the accuracy of the first model can be adjusted by retraining the first model or reselecting a second model to improve the accuracy of the first model when used in actual inference, thereby better assisting devices inside and outside the network to make correct policy decisions or take appropriate behavioral actions.

[0027] Hereinafter, with reference to the drawings, a data processing method in a communication network and a network side device provided in the embodiments of the present application will be described in detail through several embodiments and application scenarios thereof.

[0028] As shown in Fig. 2, an embodiment of the present application provides a data processing method 200 in a communication network, which can be performed by a first network element, which may be the network side equipment in the embodiment shown in Fig. 1. In other words, the method can be performed by software or hardware installed in the first network element or the network side equipment. The method includes the following steps:

[0029] S202: A first network element determines a first accuracy of the first model, the first accuracy being to indicate a degree of accuracy of the first model used for actual inference.

[0030] The first network element may be a model training function network element in a communication network, where the model training function network element has an AI / ML model training function and can be used to train an AI / ML model based on training data. Optionally, the first network element may be a model training logical function (MTLF).

[0031] The first model can be obtained by training the first network element. Specifically, the first network element can obtain training data from other network elements (e.g., network elements that can provide training data), and can obtain the first model after performing AI / ML training based on the training data. The training data includes input data and label data, and the label data corresponds to the input data and may be the ground truth of data, i.e., facts and data that actually occur.

[0032] When the first network element is obtained by training the first model, it can determine a first accuracy of the first model, and the first accuracy may be the inference accuracy (Accuracy in Use) of the first model. In this embodiment, the first accuracy may be for indicating the accuracy of the first model used in actual inference, specifically, the first accuracy may be for indicating the degree of correctness and / or the degree of error of the inference result during actual inference.

[0033] It should be noted that the first accuracy can be expressed in various forms, for example, a specific percentage value such as 90%, a categorical expression form such as high, medium, low, or normalized data such as 0.9, where the expression form of the first accuracy is not specifically limited. The first accuracy can indicate the accuracy or error of the inference result of the task by the first model in a positive or negative manner. For example, the accuracy of the inference of the first model can be indicated negatively by calculating the inference error or inference error rate of the model. There are various methods for calculating the inference error or inference error rate, for example, the mean absolute error (MAE), the mean square error (MSE), etc.

[0034] Optionally, in one embodiment, the step of the first network element determining a first accuracy of the first model comprises: a first network element acquiring first data; and the first network element determining a first accuracy based on the first data.

[0035] The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

[0036] The inference input data may be model input data when performing an inference task based on a first model, the inference result data may be model output result obtained after inferring the inference input data based on the first model, and the label data may be actual result data corresponding to the inference input data.

[0037] In this embodiment, the step of the first network element obtaining the first data can be realized in multiple ways. Optionally, in a first realization manner, the step of the first network element obtaining the first data includes: receiving usage information of the first model transmitted by a second network element; determining source information of the first data in response to the usage information; and obtaining the first data according to the source information.

[0038] In a second implementation, the step of the first network element acquiring the first data includes: The method may include receiving the first data transmitted by a second network element.

[0039] The second network element can perform an inference task based on the first model. The second network element may be a model inference function network element in a communication network, and the model inference function network element has a model inference function and can infer inference input data corresponding to the inference task based on the first model to obtain inference result data. Optionally, the second network element may be an Analytics Logical Function (AnLF) network element. Specifically, the second network element may be a network element that previously requests the first network element to obtain the first model, or a network element that previously requests the first network element to obtain model information of the first model. When receiving the first data transmitted by the second network element, the first network element can receive the first data from the network element that previously requests the first network element to obtain the first model or model information of the first model.

[0040] In a third implementation, the step of the first network element acquiring the first data includes: The method may include a step of receiving the first data transmitted by a seventh network element, the seventh network element including a data storage function network element, and specifically may be an Analytics Data Repository Function (ADRF) network element.

[0041] In the above first realization manner, the first network element can train and obtain the first model before receiving the usage information of the first model sent by the second network element, and then send the model information of the first model to the second network element. Specifically, when the second network element uses the first model to perform an inference task, it can send a model request message to the first network element, and the model request message is for requesting to obtain the first model. After receiving the model request message, if the first network element trains and obtains the first model, it can send the model information of the first model to the second network element; if the first network element does not train the first model, it can train the first model, and after training and obtaining the first model, it sends the model information of the first model to the second network element. Alternatively, the first network element can actively send the model information of the first model to the second network element after training and obtaining the first model, and when the second network element needs to perform an inference task, it can use the first model to perform the inference task without sending a model request message to the first network element. When training the first model, the first network element obtains training data from another network element (a network element capable of providing training data), and then trains the first model based on the training data. When transmitting model information of the first model to the second network element, the first network element can transmit the model information by Nnwdaf_MLModelProvision_Notify or Nnwdaf_MLModelInfo_Response.

[0042] The model information for the first model above is as follows: The first model, and a second accuracy of the first model for indicating a degree of accuracy of model output results provided by the first model during the training or testing phase.

[0043] The second accuracy may be the accuracy in training (AiT) of the first model, and may be used to indicate the accuracy of the model training stage, and specifically, the second accuracy may be used to indicate the correctness and / or error of the training result of the model training stage. Optionally, the second accuracy may be equal to the number of times the model's decision result is correct divided by the total number of decisions, i.e., second accuracy=number of times the decision result is correct / total number of decisions. Here, the correctness of the decision result may be that the decision result and the label data are consistent, and / or that the difference between the decision result and the label data is within an acceptable range. Optionally, the first network element may be configured with a validation data set for evaluating the second accuracy of the model, and the validation set includes data for model input and actual label data. The first network element can input the validation input data into the trained model to obtain output data, compare the output data with the actual label data to determine whether the decision result is correct, and finally obtain the second accuracy of the first model through a second accuracy calculation formula.

[0044] It should be noted that the second accuracy can be expressed in various forms, for example, a specific percentage value such as 90%, a categorical expression form such as high, medium, low, or normalized data such as 0.9, where the expression form of the second accuracy is not specifically limited. The second accuracy can indicate the accuracy or error degree of the first model in the model training stage positively or negatively. For example, the accuracy of the first model in the training stage can be indicated negatively by calculating the training error or training error rate of the model. There are various methods for calculating the training error or training error rate, for example, MAE, MSE, etc.

[0045] After the first network element transmits the model information of the first model to the second network element, the second network element can execute an inference task based on the first model. For a specific implementation manner of the execution of the inference task by the second network element, refer to the specific implementation manner of the corresponding step in the embodiment shown in FIG. 3, and will not be described in detail here. After the second network element executes the inference task, it can transmit its usage information of the first model to the first network element, and the first network element can receive the usage information of the first model sent by the second network element. Optionally, before receiving the usage information of the first model sent by the second network element, the first network element may transmit a first request message to the second network element, where the first request message is for requesting to obtain the usage information of the first model by the second network element. When the second network element receives the first request message, it can transmit the usage information of the first model by the second network element to the first network element. That is, the usage information of the first model by the second network element may be actively transmitted by the second network element to the first network element, or may be transmitted by the second network element to the first network element upon receiving the first request message of the first network element, but is not specifically limited thereto. Model identifier information for the first model; task identifier information for an inference task to be performed based on the first model; and Conditional information for the inference task; and subject information of the inference task.

[0046] The model identifier information (Model ID) can be used to indicate which model the second network element uses, and the model identifier information of the first model can be used to indicate the first model. The task identifier information (analytic ID) can identify an inference task type and can be used to determine a corresponding model, and the task identifier information of an inference task performed based on the first model can be used to determine the first model. The condition limiting information (analytics filter information) of the inference task can be used to limit the execution range of the inference task, such as a time range, an area range, etc. The target information (analytics target) of the inference task can be used to indicate the target of the inference task, and the target of the inference task can be, for example, a target terminal (User Equipment, UE) (i.e., a UE with a task target) or a certain network function entity (Network Function, NF) instance.

[0047] Optionally, the usage information of the first model by the second network element may include at least one of the following three: inference input data when the second network element executes an inference task based on the first model, inference result data corresponding to the inference input data, and label data corresponding to the inference input data. The inference input data can be collected by the second network element, for example, the second network element can collect and infer the inference input data when it receives a task request for an inference task by another network element (e.g., a consumer network element). Alternatively, the inference input data can be actively collected by the second network element. The inference result data can be obtained based on the first model and the inference input data when the second network element executes the inference task. The label data can be obtained by the second network element from another network element (e.g., a source device of the label data), for example, the second network element can send a data acquisition request to the other network element to request it to acquire the label data.

[0048] After receiving the usage information of the first model by the second network element, the first network element can determine source information of the first data according to the usage information. The source information of the first data can be: a third network element for providing inference input data corresponding to the inference task; and a fourth network element for providing label data corresponding to the inference task.

[0049] The third network element may include a source device of the inference input data. The third network element may be determined by the first network element, and specifically, the first network element may determine a target and a scope of the inference task according to information such as the analytics filter information and the analytics target of the inference task in the usage information, and may determine from which network element to obtain the inference input data corresponding to the inference task according to the target and scope and metadata information, and the determined network elements are the third network elements.

[0050] The fourth network element may include a source device of label data. The fourth network element may be determined by the first network element, and specifically, the first network element may determine a corresponding network element device type (NF type) capable of providing the data type according to an output data type of the first model, and determine a network element instance corresponding to the network element device type according to an object of the inference task, limited information, etc., and the network element instance may be the fourth network element. For example, the first model is a UE mobility model, the output data type of the first model is a UE location, and it may be determined that the AMF type is UE location data information according to the output data type, and when it queries from the UDM or NRF according to the object of the inference task UE1 and limited information such as an Area Of Interest (AOI) that the corresponding AMF instance is AMF1, the AMF1 is the fourth network element, and the first network element may take the AMF1 as a source of label data and obtain an actual UE location label data value from the AMF1.

[0051] Optionally, the above source information may be A first model corresponding to the inference task; The input data type of the first model; and an output data type of the first model.

[0052] The first model may be determined by the first network element according to the model identifier information and / or the task identifier information in the usage information. For example, when the usage information includes task identifier information (analytics ID) and there is a mapping relationship between the task identifier information and the model identifier information (model ID), for a certain analytics ID (e.g., analytics ID=UE mobility for predicting a user's movement trajectory), it may be determined that the corresponding model identifier information is model1 based on the mapping relationship, and the model corresponding to the model1 is the first model corresponding to the inference task.

[0053] The input data type (which can be said to be metadata information) and output data type of the first model relate to the result data for the inference task or prediction to which the first model is specifically applied. For example, when the first model is used to predict a user's movement trajectory, the input data type of the first model may include a UE ID, time, a current business state of the UE, etc., and the output data type may include a UE location (e.g., a Tracking Area (TA) / cell), etc.

[0054] After determining the source information of the first data, the first network element can obtain the first data according to the source information. The first network element obtaining the first data according to the source information includes: Sending a get input data request message to a third network element, the get input data request message being for requesting to obtain inference input data; sending a get label data request message to the fourth network element, where the get label data request message is to request to obtain the label data.

[0055] The input data acquisition request message can be used by the third network element to determine which inference input data to feed back to the first network element, and the input data acquisition request message can include: type information of the inference input data; Target information corresponding to the inference input data; and time information corresponding to the inference input data.

[0056] The type information of the inference input data, the target information corresponding to the inference input data, and the time information corresponding to the inference input data are determined by the first network element according to the input data type of the inference process, the inference target, and the time of the target of inference. That is, the first network element determines the inference input data type that needs to be acquired according to the input data type of the inference process, determines the target of the inference input data that needs to be acquired according to the inference target, and determines the time information (timestamp, time zone, etc.) of the inference input data according to the time of the target of inference. When the inference process is a statistical calculation performed for a certain past time or a prediction performed for a certain future time, the time information may be a past time or a future time.

[0057] The label data acquisition request message can be used by the fourth network element to determine which label data to feed back to the first network element, and the label data acquisition request message includes: Type information of the label data; Target information corresponding to the label data; and time information corresponding to the label data.

[0058] The type information of the label data, the target information corresponding to the label data, and the time information corresponding to the label data are respectively determined by the first network element according to the output data type of the inference process, the inference target, and the time of the inference target. That is, the first network element determines the label data type that needs to be acquired according to the output data type of the inference process, determines the target of the label data that needs to be acquired according to the inference target, and determines the time information (timestamp, time zone, etc.) of the label data according to the time of the inference target. If the inference process is a statistical calculation performed for a certain past time or a prediction performed for a certain future time, the time information may be a past time or a future time.

[0059] For example, when an MTLF (first network element) sends a label data acquisition request message to an AMF or LMF (third network element), and the label data type carried in the request information is UE location, the target information is UE ID1, and the time information is a specific time period, the request information is used to request the AMF / LMF to feed back the UE location value of UE ID1 in a specific time period.

[0060] After receiving the input data acquisition request message, the third network element can send corresponding inference input data to the first network element. In this way, the first network element can obtain the inference input data from the third network element. Similarly, after receiving the label data acquisition request message, the fourth network element can send corresponding label data to the first network element, and the first network element can obtain the label data from the fourth network element. It should be noted that when the second network element performs one or more inference processes to obtain multiple inference output results during the execution of the inference task, the first network element needs to correspondingly obtain multiple label data values ​​corresponding to the multiple inference output results from the fourth network element.

[0061] Also, what should be explained is that the above describes how the first network element obtains the inference input data and / or label data, and as for the inference result data, when the first network element obtains the inference result data, it can obtain it from the second network element, and the inference result data obtained from the second network element can be obtained after the second network element executes an inference task based on the first model. Also, since the inference result data can be obtained by inference based on the inference input data and the first model, when the first network element obtains the inference result data, the first network element may obtain the inference input data, and then perform inference based on the inference input data and the first model to obtain the inference result data. In this way, the first network element does not need to obtain the inference result data from other network elements, and therefore the data acquisition step can be simplified.

[0062] In the above second realization manner, that is, in a situation where the first network element obtains the first data by receiving the first data transmitted by the second network element, the first data obtained by the first network element may include at least one of three: inference input data, inference result data, and label data. The inference input data can be collected by the second network element, for example, when the second network element receives a task request of an inference task by another network element (for example, a consumer network element), the second network element can collect the inference input data and infer it. Alternatively, the inference input data can be actively collected by the second network element. The inference result data can be obtained by the second network element based on the first model and the inference input data when the second network element executes the inference task. The label data can be obtained by the second network element from another network element (for example, a source device of the label data), for example, the second network element can obtain the label data by sending a data acquisition request to the other network element.

[0063] Optionally, before the first network element receives the first data transmitted by the second network element, The method may further include a step of the first network element sending a second request message to the second network element, the second request message being for requesting to obtain the first data collected by the second network element.

[0064] That is, when the first network element obtains the first data, the first network element can send a second request message to the second network element, the second request message is for requesting to obtain the first data collected by the second network element, and when the second network element receives the second request message, the second network element can send the first data to the first network element.

[0065] Optionally, the second request message may be a subscribe message. Optionally, the second request information is: Identifier information for an inference task; Information on the constraints of the reasoning task; the subject information of the inference task; Identifier information for the first model; Input data type information of the first model; and output data type information of the first model.

[0066] Optionally, the second request information may further include a reason for the request, which may be, for example, that the first model needs to be retrained, or that the accuracy of the first model does not meet the accuracy requirement or is degraded, etc.

[0067] In the above third implementation manner, that is, in a situation where the first network element obtains the first data by receiving the first data sent by the seventh network element, the first data obtained by the first network element may include at least one of the following three: inference input data, inference result data, and label data. The first data can be stored in the seventh network element by the second network element, specifically, the second network element can send first instruction information to the seventh network element after performing the inference task, the first instruction information is for instructing the seventh network element to store the first data of the inference task. Optionally, the first instruction information is: Identifier information for an inference task; Information on the constraints of the reasoning task; the subject information of the inference task; inference input data corresponding to an inference task; inference result data corresponding to the inference task; and label data corresponding to the inference task.

[0068] The storage reason information may be, for example, that the second network element has completed an inference task, that the seventh network element needs to periodically store the first data, that the accuracy of the first model used when the second network element performs the inference task does not meet the accuracy requirement or is degraded, etc.

[0069] The first data stored in the seventh network element can be transmitted to the seventh network element by the second network element. The manner in which the second network element obtains the first data may be that inference input data is collected by the second network element. For example, when the second network element receives a task request for an inference task by another network element (e.g., a consumer network element), it can collect the inference input data and perform inference. Alternatively, the inference input data can be actively collected by the second network element. The inference result data can be obtained based on the first model and the inference input data when the second network element executes the inference task. The label data can be obtained by the second network element from another network element (e.g., a source device of the label data), for example, the second network element can obtain the label data by sending a data acquisition request to the other network element.

[0070] Optionally, before the first network element receives the first data transmitted by the seventh network element, The method may further include the step of: the first network element sending a third request message to the seventh network element, where the third request message is for requesting to obtain the first data.

[0071] That is, when the first network element obtains the first data, the first network element can send a third request message to the seventh network element, the third request message is for requesting to obtain the first data stored in the seventh network element, and the seventh network element can send the first data to the first network element when receiving the third request message.

[0072] Optionally, the third request message may be a subscribe message. Optionally, the third request information is: Identifier information for an inference task; Information on the constraints of the reasoning task; the subject information of the inference task; Identifier information for the first model; Input data type information of the first model; and output data type information of the first model.

[0073] Optionally, the third request information may further include a reason for the request, which may be, for example, that the first model needs to be retrained, or that the accuracy of the first model does not meet the accuracy requirement or is degraded, etc.

[0074] It should be noted that in practical application, the first network element can obtain the first data by any one or more of the above three methods, i.e., the first network element can obtain the first data from the second network element, and / or determine the source information of the first data according to the usage information of the first model by the second network element and obtain the first data according to the source information, and / or obtain the first data from the seventh network element.

[0075] After obtaining the first data, the first network element can determine a first accuracy of the first model according to the first data.

[0076] When determining the first accuracy according to the first data, optionally, when the first data includes inference input data and label data but does not include inference result data, the first network element can input the inference input data to the first model, determine the inference result data corresponding to the inference input data, and then determine the first accuracy according to the inference result data and the label data. When the first data includes inference result data and label data, the first network element can directly determine the first accuracy according to the inference result data and the label data.

[0077] When determining the first accuracy according to the inference result data and the label data, specifically, the inference result data is compared with the label data, and a ratio between the number of times the inference result is correct and the total number of times the inference is made is determined, and the ratio is the first accuracy of the first model. Here, the inference result being correct may be that the inference result data and the label data match, or that the difference between the inference result data and the label data is within an allowable range. The expression format of the first accuracy may be a percentage value (e.g., 90%), a categorical expression format (e.g., high, medium, low), or a normalized numerical value (e.g., 0.9), but is not specifically limited thereto.

[0078] S204: If the first accuracy satisfies a predetermined condition, the first network element retrains or reselects the first model.

[0079] After determining the first accuracy of the first model, the first network element can determine whether the first accuracy satisfies a predetermined condition and then determine whether it is necessary to retrain the first model or reselect the second model.

[0080] Optionally, in one embodiment, the first accuracy satisfies the predetermined condition by: the first accuracy is less than a second accuracy for indicating a degree of accuracy of a model output result provided by the first model in a training phase or a testing phase; The first accuracy is smaller than the second accuracy, and the difference between the first accuracy and the second accuracy is greater than a predetermined value; and the first accuracy is smaller than a predetermined accuracy, and the predetermined accuracy can be set according to actual needs and is not specifically limited herein.

[0081] If the first accuracy meets a predetermined condition, it can indicate that the accuracy of the first model used in actual inference does not meet the actual needs, and at this time, the first network element can retrain the first model or reselect the second model. The retraining of the first model can be correcting the first model without changing the model structure of the first model, or can be changing the model structure of the first model and training the first model with a new model structure. Optionally, the retraining of the model can have two realization manners, one is to redo the model training from one based on the training data, and the other is to fine-tune the first model based on the training data, which can converge faster and save resources. The reselection of the second model can be reselecting a new existing other model, and the reselected second model can be a model whose accuracy is higher than the first threshold or meets the performance requirements of the model.

[0082] Optionally, in one embodiment, when the first network element retrains the first model, obtaining target training data, the target training data including target input data and target label data corresponding to the target input data; and retraining the first model based on the target training data.

[0083] The target training data is different from the training data used to previously train the first model. When retraining the first model, the first model can use the new training data to perform model training to adjust the accuracy of the first model, thereby improving the accuracy of the retrained first model.

[0084] Optionally, as an embodiment, the first network element obtaining the target training data includes: Obtaining first training data for use in training a first model; determining a fifth network element and obtaining second training data from the fifth network element, the fifth network element being for providing the training data; The method may include at least one of determining a sixth network element and obtaining inference data from the sixth network element, the sixth network element being for providing the inference data.

[0085] Each of the first training data, the second training data, and the inference data includes input data and label data.

[0086] The fifth network element may include a source device of training data. Optionally, in one embodiment, the first network element determining the fifth network element includes: The method may include determining a fifth network element as a function of the second information.

[0087] The second information includes task identifier information of an inference task executed based on the first model and / or condition limiting information of the inference task. The task identifier information (analytic ID) can identify an inference task type and can be used to determine a corresponding model. The inference task condition limiting information (analytics filter information) can be used to limit the execution scope of the inference task, such as a time range, a domain range, etc.

[0088] The sixth network element may include a source device of the inference data. Optionally, in one embodiment, the first network element determining the sixth network element includes: The method may include determining a sixth network element in response to the second information.

[0089] The first network element may obtain the second model after retraining the first model or reselecting the second model. After the second model is obtained, optionally, in one embodiment, Sending model information of the retrained second model or the reselected second model to a second network element, and performing an inference task based on the second model by the second network element; The method may further include at least one of transmitting model information of the second model obtained by retraining or the reselected second model to a seventh network element and storing the model information of the second model by the seventh network element.

[0090] The second network element here may be the second network element in the current task, i.e., the second network element that transmits the usage information of the first model to the first network element after previously performing an inference task based on the first model. After the first network element transmits the model information of the second model to the second network element, the second network element can re-perform the previous inference task or perform a new inference task based on the second model. Since the second model is a retrained model, the accuracy of the inference result data is high. Optionally, the second network element may be another network element that performs an inference task based on the second model, and after the first network element transmits the model information of the second model to the other second network element, the other second network element can perform an inference task based on the second model. Since the second model is a retrained model, the accuracy of the inference result data is high.

[0091] The seventh network element includes a data storage function network element, i.e., a network element that stores model information of the second model. Optionally, the seventh network element may be an Analytics Data Repository Function (ADRF) network element. Storing the model information of the second model in the seventh network element can facilitate other network elements to find the model or data.

[0092] Model information for the second model is as follows: Model identifier information for the second model; and task identifier information for an inference task to be performed based on the second model; and Information on the scope of the second model; a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during the training phase or the testing phase; Training data for the second model; and and a second model.

[0093] The model identifier information (Model ID) of the second model is for indicating the second model. The task identifier information (analytic ID) of the inference task executed based on the second model can be used to determine the corresponding second model. The application range information of the second model can be used to limit the execution range of the inference task, such as a time range, an area range, etc. The third accuracy of the second model can be called the accuracy in training (AiT) of the second model, and is for describing the accuracy of the discrimination or decision that can be achieved after the model is trained. Specifically, the third accuracy can be used to indicate the degree of correctness and / or the degree of error of the model output result presented by the second model in the training stage or the test stage. The method of determining the third accuracy is the same as the method of determining the second accuracy of the first model described above, and the expression format of the third accuracy may be the same as the expression format of the second accuracy, but will not be described here repeatedly. The training data of the second model is training data used when training the second model, and may include input data and label data, and may specifically be target training data obtained when retraining the first model described above. The second model may include, but is not limited to, descriptive information and / or a model file for the second model, which may include elements such as a complete network structure and parameter information for generating the second model.

[0094] It should be noted that the model information sent by the first network element to the second network element and the seventh network element may be the same or different. For example, the first network element can send the second model to the second network element, and can send model identifier information of the second model, task identifier information of the inference task performed based on the second model, applicability information of the second model, a third accuracy of the second model, training data of the second model, and the second model to the seventh network element.

[0095] In an embodiment of the present application, a first accuracy of a first model when used for actual inference is determined, and if the first accuracy does not satisfy a predetermined condition, the first model can be retrained. Therefore, if the accuracy of the first model when used for actual inference decreases, the accuracy of the first model can be adjusted by retraining the first model or reselecting a second model, thereby improving the accuracy of the first model when used for actual inference, thereby better assisting devices inside and outside the network in making correct policy decisions or taking appropriate actions.

[0096] As shown in FIG. 3, an embodiment of the present application provides a data processing method 300 in a communication network, which can be performed by a second network element, and the second network element may be the network side equipment in the embodiment shown in FIG. 1. In other words, the method can be performed by software or hardware installed in the second network element or the network side equipment. It should be noted that the second network element may be independently arranged as a network element equipment different from the first network element in the embodiment shown in FIG. 2, or may be arranged together with the same network element equipment, for example, together with the NWDAF, in which case the NWDAF can provide a model inference function while providing a model training function. The method shown in FIG. 3 includes the following steps.

[0097] S302: A second network element performs an inference task based on a first model, the first model being trained by the first network element, and the first network element including a model training function network element.

[0098] The first model can be obtained by training the first network element based on training data, and the training data can be obtained by the first network element from another network element (a network element that can provide training data). In this step, the second network element can perform an inference task based on the first model trained by the first network element.

[0099] Optionally, as an embodiment, the second network element may obtain the first model from the first network element before performing the inference task based on the first model, specifically, sending a model request message to a first network element, the model request message to request to obtain the first model; and receiving model information of the first model transmitted by the first network element.

[0100] After the second network element sends a model request message to the first network element, if the first network element has trained the first model, it can send model information of the first model to the second network element, and at this time, the second network element can receive the model information of the first model sent by the first network element. If the first network element has not trained the first model, it can obtain training data from another network element (a network element that can provide training data), train the first model based on the training data, and send the model information of the first model to the second network element, and at this time, the second network element can receive the model information of the first model sent by the first network element. When the second network element receives the model information of the first model, it can receive it by Nnwdaf_MLModelProvision_Notify or Nnwdaf_MLModelInfo_Response.

[0101] Model information for the first model is as follows: The first model, and a second accuracy of the first model for indicating a degree of accuracy of model output results provided by the first model during a training phase or a testing phase.

[0102] The second accuracy of the first model may be the accuracy in training (AiT) of the first model, which can be used to indicate the accuracy of the model training stage, and specifically, the second accuracy can be used to indicate the correctness and / or error of the training result of the model training stage. Optionally, the second accuracy may be equal to the number of times the model's decision result is correct divided by the total number of decisions, i.e., second accuracy=number of times the decision result is correct / total number of decisions. Here, the correctness of the decision result may be that the decision result and the label data are consistent, and / or that the difference between the decision result and the label data is within an acceptable range. The expression format of the second accuracy may be a percentage value (e.g., 90%), a categorical expression format (e.g., high, medium, low), or a normalized numerical value (e.g., 0.9), but is not specifically limited thereto. The second accuracy can indicate the accuracy or error of the first model in the model training stage positively or negatively. For example, the accuracy of the first model in the training stage can be indicated negatively by calculating the training error or training error rate of the model. There are various methods for calculating the training error or training error rate, for example, MAE, MSE, etc.

[0103] After receiving the model information of the first model, the second network element can perform an inference task based on the first model.

[0104] Optionally, as an embodiment, the second network element performing an inference task based on the first model includes: receiving a task request for an inference task sent by an eighth network element and performing the inference task based on the first model; obtaining a task request for an inference task simulated and triggered by the second network element; and performing the inference task based on the first model.

[0105] That is, the second network element may perform an inference task based on the first model by being triggered when receiving an inference task sent by the eighth network element, and / or a validation test stage may be set by the second network element, in which the second network element itself simulates the triggering of the inference task to estimate the accuracy of the model inference. The eighth network element includes a consumer network element, and may specifically be a consumer network function entity (consumer NF), etc., and the consumer NF may be a 5G network element or an AF terminal, etc. When the eighth network element sends an inference task to the second network element, it may send it by Nnwdaf_AnalyticsSubscription_Subscribe or Nnwdaf_AnalyticsInfo_Request.

[0106] When the inference task is transmitted by the eighth network element to the second network element, if the inference task carries inference input data, the inference input data can be input to the first model, thereby obtaining inference result data. If the inference task does not carry inference input data, when the second network element executes the inference task based on the first model, Sending an obtain input data request message to a third network element, the obtain input data request message being for requesting to obtain inference input data corresponding to the inference task; receiving inference input data transmitted by a third network element; The method may include inputting the inference input data into the first model to obtain inference result data.

[0107] The third network element here may be the third network element in the embodiment shown in Figure 2, and the third network element can provide inference input data corresponding to the inference task. The input data acquisition request message can be used by the third network element to determine which inference input data to feed back to the second network element, and the input data acquisition request message may include at least one of type information of the inference input data, target information corresponding to the inference input data, and time information corresponding to the inference input data. Specifically, the corresponding content in the embodiment shown in Figure 2 can be referred to, and will not be described here repeatedly. When the second network element sends the input data acquisition request message to the third network element, it can send it by Nnwdaf_AnalyticsSubscription_Subscribe or Nnwdaf_AnalyticsInfo_Request.

[0108] After receiving the input data acquisition request message, the third network element can send corresponding inference input data to the second network element. After receiving the inference input data sent by the third network element, the second network element can input the inference input data into the first model, thereby obtaining corresponding inference result data.

[0109] For example, the second network element can perform inference calculation using model1 corresponding to analytics ID=UE mobility (analytics ID=UE mobility for identifying an inference task type and, for example, predicting a user's movement trajectory) and values ​​of input data corresponding to model1 (e.g., UE ID, time, current business state of the UE) to obtain an output value of the inference result UE location.

[0110] When an inference task is triggered by a second network element, when the second network element executes the inference task based on the first model, inference input data can be generated or determined by the second network element, and inference result data can be obtained by inputting the inference input data into the first model.

[0111] It should be noted that when the second network element executes the inference task, it can perform the inference calculation process once to obtain multiple output result values, or it can perform the inference multiple times to obtain multiple inference output result values.

[0112] Optionally, as an example, when the inference task performed by the second network element is sent by the eighth network element, the second network element may perform the inference task using the first model and then send the resulting inference result data to the eighth network element to assist the eighth network element in making a policy decision.

[0113] S304: Send at least one of the usage information of the first model and the first data to the first network element and / or send first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store the first data of the inference task, the seventh network element including a data storage function network element.

[0114] After the second network element executes the inference task, it can send at least one of the usage information of its first model and the first data to the first network element. After the first network element receives the usage information and at least one of the first data, it can decide whether to retrain the first model or reselect the second model. For a specific implementation manner, refer to the corresponding steps in the embodiment shown in FIG. 2, and will not be described here repeatedly.

[0115] The usage information of the first model by the second network element is Model identifier information for the first model; task identifier information for an inference task to be performed based on the first model; and Conditional information for the inference task; and subject information of the inference task.

[0116] The model identifier information (Model ID) of the first model can be used to indicate that the model used by the second network element is the first model. The task identifier information (analytic ID) can identify an inference task type, and the task identifier information of the inference task performed based on the first model can be used to determine the first model. The condition restriction information (analytics filter information) of the inference task can be used to restrict the execution scope of the execution of the inference task by the second network element, such as a time range, an area range, etc. The target information (analytics target) of the inference task can be used to indicate the target of the inference task performed by the second network element, and the target of the inference task can be, for example, a Target UE (i.e., a UE with a task target) or a certain NF instance. Optionally, the usage information of the first model by the second network element may include inference input data and / or inference result data when the second network element performs the inference task based on the first model.

[0117] Optionally, as an embodiment, before the second network element transmits the usage information of the first model to the first network element, The method further includes at least one of: the second network element receiving a first request message sent by the first network element, the first request message being for requesting the second network element to obtain usage information of the first model.

[0118] That is, when the second network element receives the first request message of the first network element, it can send the usage information of the first model to the first network element.

[0119] The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

[0120] The inference input data may be model input data when performing an inference task based on a first model, the inference result data may be model output result obtained after inferring the inference input data based on the first model, and the label data may be actual result data corresponding to the inference input data.

[0121] Optionally, as an embodiment, before the second network element transmits the first data to the first network element, a second network element collecting inference input data; and the second network element collecting label data corresponding to the inference input data.

[0122] That is, if the first data includes inference input data, the inference input data can be collected by the second network element. If the first data includes label data corresponding to the inference input data, the label data can also be collected by the second network element. Optionally, if the first data includes inference result data, the inference result data can be obtained after inference based on the first model and the inference input data by the second network element.

[0123] Optionally, the second network element collecting inference input data comprises: collecting inference input data when the second network element receives a task request for an inference task by an eighth network element, where the eighth network element includes a consumer network element; and the second network element actively collecting inference input data.

[0124] That is, the second network element may collect inference input data for inference when receiving a task request of an inference task, or may actively collect inference input data. The task request of the inference task can be sent to the second network element by the eighth network element, the eighth network element includes a consumer network element, specifically may be a consumer NF, etc., the consumer NF may be a 5G network element or an AF terminal, etc.

[0125] Optionally, the second network element collecting label data corresponding to the inference input data includes: The method may include the second network element sending a data retrieval request to a fourth network element, the data retrieval request for requesting to retrieve label data corresponding to the inference input data.

[0126] The fourth network element may be a source device of the label data. When the second network element collects the label data, it can send a data request to the fourth network element, and when the fourth network element receives the data acquisition request, it can send the corresponding label data to the second network element.

[0127] Optionally, before the second network element transmits the first data to the first network element, The method may further include a step of receiving a second request message sent by the first network element, the second request message being for requesting to obtain the first data collected by the second network element.

[0128] That is, when the second network element receives the second request message sent by the first network element, it can send the first data to the first network element.

[0129] Optionally, the second request message may be a subscribe message. Optionally, the second request information comprises: Identifier information for an inference task; Information on the constraints of the reasoning task; the subject information of the inference task; Identifier information for the first model; Input data type information of the first model; and output data type information of the first model.

[0130] Optionally, the second request information may further include a reason for the request, which may be, for example, that the first model needs to be retrained, or that the accuracy of the first model does not meet the accuracy requirement or is degraded, etc.

[0131] In this embodiment, after performing the inference task, the second network element may send first instruction information to the seventh network element, where the first instruction information is for instructing the seventh network element to store first data of the inference task, and the seventh network element includes a data storage function network element, and may specifically be an ADRF.

[0132] Optionally, the first indication information comprises: Identifier information for an inference task; Information on the constraints of the reasoning task; the subject information of the inference task; inference input data corresponding to an inference task; inference result data corresponding to the inference task; and label data corresponding to the inference task.

[0133] As for the storage reason information, for example, the second network element completes an inference task, and the seventh network element needs to periodically store the first data, and the accuracy of the first model used when the second network element performs the task inference does not meet the accuracy requirement or is deteriorated.

[0134] After the second network element sends the first indication information to the seventh network element, the seventh network element can store the first data according to the first indication information. The first data can be determined by the second network element in the above manner, and will not be described again here. After the seventh network element stores the first data, the first network element can obtain the first data from the seventh network element when it needs to obtain it. For a specific implementation manner, please refer to the corresponding content in the embodiment shown in Figure 2, and will not be described again here.

[0135] Optionally, as an embodiment, the second network element is The method may further include receiving model information of a second model sent by the first network element, where the second model is obtained by retraining the first model by the first network element or the second model is a model reselected by the first network element.

[0136] Specifically, after receiving the usage information and / or the first data of the first model by the second network element, the first network element may retrain the first model or reselect the second model if it determines that the first accuracy of the first model meets a predetermined condition. After retraining to obtain the second model or reselecting the second model, the first network element may transmit model information of the second model to the second network element, and at this time, the second network element may receive the model information of the second model transmitted by the first network element. Thereafter, the second network element may use the second model to re-perform an inference task previously performed or perform a new inference task. Since the second model is a retrained model, the accuracy of the inference result data is high.

[0137] Model information for the second model is as follows: Model identifier information for the second model; and task identifier information for an inference task to be performed based on the second model; and Information on the scope of the second model; a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during the training phase or the testing phase; Training data for the second model; and and a second model.

[0138] The model identifier information (Model ID) of the second model is for indicating the second model. The task identifier information (analytic ID) of the inference task executed based on the second model can be used to determine the corresponding second model. The application range information of the second model can be used to limit the execution range of the inference task, such as a time range, an area range, etc. The third accuracy of the second model can be called the accuracy in training (AiT) of the second model, and is for describing the accuracy of the discrimination or decision that can be achieved after the model is trained. Specifically, the third accuracy can be used to indicate the degree of correctness and / or the degree of error of the model output result presented by the second model in the training phase or the test phase. The method of determining the third accuracy is the same as the method of determining the second accuracy of the first model in the embodiment shown in FIG. 2 above, and the expression format of the third accuracy may be the same as the expression format of the second accuracy, but will not be described here repeatedly. The training data of the second model is training data used in training the second model, and may include input data and label data, and may specifically be target training data obtained in retraining the first model. The second model includes, but is not limited to, description information and / or a model file for the second model, and the model file may include elements such as a complete network structure and parameter information for generating the second model.

[0139] In an embodiment of the present application, a first accuracy of a first model when used for actual inference is determined, and if the first accuracy does not satisfy a predetermined condition, the first model can be retrained. Therefore, if the accuracy of the first model when used for actual inference decreases, the accuracy of the first model can be adjusted by retraining the first model to increase the accuracy of the first model when used for actual inference, thereby better assisting devices inside and outside the network in making correct policy decisions or appropriate behavioral operations.

[0140] In a possible application scenario, the data processing method in a communication network provided in the embodiment of the present application can be shown in Figure 4. The data processing method in a communication network shown in Figure 4 may include the following steps:

[0141] Step 1: A first network element obtains training data from a fifth network element.

[0142] Step 2: The first network element trains a first model based on the training data.

[0143] Step 3: The second network element sends a model acquisition request for the first model to the first network element.

[0144] Step 4: The first network element sends model information of the first model to the second network element.

[0145] It should be noted that the execution order of steps 1 to 4 above may be step 1, step 2, step 3, and step 4, or may be step 3, step 1, step 2, and step 4.

[0146] Step 5: The second network element receives the inference task of the eighth network element.

[0147] It should be noted that the execution order of steps 1 to 4 and step 5 may be steps 1 to 4 and step 5, or may be step 5 and steps 1 to 4.

[0148] Step 6: The second network element obtains the inference input data from the third network element, and performs the inference task based on the inference input data and the first model.

[0149] Step 7: The second network element obtains the label data from the fourth network element.

[0150] Step 8: The second network element sends the inference result data to the eighth network element.

[0151] It should be noted that the second network element can set a verification test stage, in which the second network element itself can simulate the triggering of an inference task to estimate the accuracy of the model inference. Specifically, after performing step 4, the second network element can simulate the triggering of an inference task and execute the inference task based on the first model, in which case, the above steps 5 to 7 are replaced with the second network element simulating the triggering of an inference task and executing the inference task based on the first model, and FIG. 3 only takes the second network element performing steps 5 to 7 as an example to describe.

[0152] Step 9: The first network element sends a first request message to the second network element, the first request message being for requesting to obtain usage information of the first model by the second network element and / or first data collected by the second network element.

[0153] Step 10: The second network element sends the usage information of the first model and / or the first data to the first network element.

[0154] The usage information for the first model is Model identifier information for the first model; task identifier information for an inference task to be performed based on the first model; and Conditional information for the inference task; and subject information of the inference task.

[0155] The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

[0156] Step 11: The first network element determines source information of the first data according to the usage information.

[0157] The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

[0158] The source information of the first data is a third network element for providing inference input data corresponding to the inference task; and a fourth network element for providing label data corresponding to the inference task.

[0159] Step 12a: The first network element sends a get input data request message to the third network element.

[0160] The input data acquisition request message is for requesting to acquire inference input data. The input data acquisition request message includes the following: type information of the inference input data; Target information corresponding to the inference input data; and time information corresponding to the inference input data.

[0161] Step 12b: The first network element sends a label data obtain request message to the fourth network element.

[0162] The label data acquisition request message is for requesting to acquire label data. The label data acquisition request message includes the following: Type information of the label data; Target information corresponding to the label data; and time information corresponding to the label data.

[0163] It should be noted that the first network element is capable of performing at least one of steps 12a and 12b above.

[0164] It should also be noted that the first network element can selectively perform step 11, step 12a, and step 12b. For example, when the second network element does not send the usage information of the first model, but only sends the first data, and the first data includes inference input data, inference result data, and label data, the first network element does not need to perform step 11, step 12a, and step 12b. When the second network element sends both the usage information of the first model and the first data, if the first data includes only label data, the first network element can perform step 11 and step 12a, and if the first data includes only inference input data, the first network element can perform step 11 and step 12b. The steps specifically performed by the first network element can be determined according to the actual situation, and are not specifically limited here, as long as it is guaranteed that the first network element can obtain the inference input data, inference result data, and label data.

[0165] In addition, when the first network element obtains the first data, it can obtain the first data from the seventh network element, and Figure 4 only describes as an example the obtaining of the usage information and / or the first data of the first model from the second network element.

[0166] Step 13: The first network element determines a first accuracy of the first model.

[0167] The first accuracy is intended to indicate the degree of accuracy of the first model used in actual inference, and may be a degree of correctness or a degree of error.

[0168] Step 14: The first network element determines whether the first accuracy satisfies a predetermined condition.

[0169] The first accuracy satisfies a predetermined condition, the first accuracy is less than a second accuracy for indicating a degree of accuracy of a model output result provided by the first model in a training phase or a testing phase; The first accuracy is smaller than the second accuracy, and the difference between the first accuracy and the second accuracy is greater than a predetermined value; the first accuracy being less than the predetermined accuracy.

[0170] If the first accuracy satisfies the predetermined condition, at least one of steps 15a to 15c can be executed. If the first accuracy does not satisfy the predetermined condition, there is no need to execute the subsequent steps, and here, at least one of steps 15a to 15c is executed as an example.

[0171] Step 15a: A first network element obtains first training data for use in training a first model.

[0172] Step 15b: The first network element obtains second training data from the fifth network element.

[0173] Step 15c: The first network element obtains inference data from the sixth network element.

[0174] Step 16: The first network element reselects the first model or retrains the first model based on the target training data to obtain the second model.

[0175] The target training data here includes data acquired in at least one of steps 15a to 15c above. That is, the target training data is First training data used in training the first model; and second training data obtained from a fifth network element; and inference data obtained from the sixth network element.

[0176] Step 17: The first network element sends model information of the second model to the second network element.

[0177] Model information for the second model is as follows: Model identifier information for the second model; and task identifier information for an inference task to be performed based on the second model; and Information on the scope of the second model; a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during the training phase or the testing phase; Training data for the second model; and and a second model.

[0178] Step 18: The first network element sends model information of the second model to another second network element.

[0179] Step 19: The first network element sends the model information of the second model to the seventh network element.

[0180] When the first network element retrains the first model to obtain a second model or reselects a second model, it may or may not perform steps 17-19 above, and if so, it may perform at least one of steps 17-19.

[0181] For the specific implementation of each step shown in FIG. 4, please refer to the specific implementation of the corresponding step in FIG. 2 and FIG. 3, and the description will not be repeated here.

[0182] In an embodiment of the present application, a first accuracy of a first model when used for actual inference is determined, and if the first accuracy does not satisfy a predetermined condition, the first model can be retrained. Therefore, if the accuracy of the first model when used for actual inference decreases, the accuracy of the first model can be adjusted by retraining the first model or reselecting a second model, thereby improving the accuracy of the first model when used for actual inference, thereby better assisting devices inside and outside the network in making correct policy decisions or taking appropriate actions.

[0183] The data processing method in a communication network provided in the embodiment of the present application can be executed by a data processing device in the communication network. In the embodiment of the present application, the data processing method in a communication network is executed by the data processing device in the communication network as an example, and the data processing device in a communication network provided in the embodiment of the present application will be described.

[0184] 5 is a structural schematic diagram of a data processing device in a communication network according to an embodiment of the present application, which may correspond to the first network element in another embodiment. As shown in FIG. 5, the device 500 includes: A determination module 501 used to determine a first accuracy of a first model, the first accuracy being for indicating a degree of accuracy of the first model used in actual inference; and a model training module 502, which is used to retrain the first model or reselect a second model if the first accuracy satisfies a predetermined condition.

[0185] Optionally, in one embodiment, the first accuracy is: A degree of accuracy of an inference result when the first model is used in an actual inference; and and the degree of error of an inference result when the first model is used for actual inference.

[0186] Optionally, in one embodiment, the determination module 501 is A method is used for obtaining first data and determining the first accuracy based on the first data.

[0187] The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

[0188] Optionally, in one embodiment, the determination module 501 is receiving usage information of the first model transmitted by a second network element; determining source information of the first data in response to the usage information; and The first data is obtained according to the source information.

[0189] Optionally, in one embodiment, the determination module 501 is The network element is adapted to receive the first data transmitted by a second network element, the second network element including a model inference function network element.

[0190] Optionally, in one embodiment, the determination module 501 is and receiving the first data transmitted by a seventh network element, the seventh network element including a data storage function network element.

[0191] Optionally, in one embodiment, the determination module 501 further comprises: and sending a first request message to the second network element, the first request message being for requesting the second network element to obtain usage information of the first model.

[0192] Optionally, in one embodiment, the determination module 501 further comprises: The first network element is used to send a second request message to the second network element, the second request message being for requesting to obtain the first data collected by the second network element.

[0193] Optionally, in one embodiment, the determination module 501 further comprises: and sending a third request message to the seventh network element, the third request message being for obtaining the first data.

[0194] Optionally, in one embodiment, the model training module 502 further comprises: Used for training and obtaining the first model and sending model information of the first model to the second network element; The model information of the first model includes The first model; and and a second accuracy of the first model for indicating a degree of accuracy of model output results presented by the first model during a training phase or a testing phase.

[0195] Optionally, in one embodiment, the usage information comprises: model identifier information of the first model; task identifier information for an inference task to be executed based on the first model; and Conditional limiting information for the inference task; and subject information of the inference task.

[0196] Optionally, in one embodiment, the source information is: a third network element for providing inference input data corresponding to the inference task; and a fourth network element for providing label data corresponding to the inference task.

[0197] Optionally, in one embodiment, the determination module 501 is sending a get input data request message to the third network element, the get input data request message being for requesting to obtain the inference input data; sending a label data obtainment request message to the fourth network element, the label data obtainment request message being for requesting to obtain the label data; The input data acquisition request message includes: type information of the inference input data; object information corresponding to the inference input data; and time information corresponding to the inference input data; The label data acquisition request message includes: type information of the label data; target information corresponding to the label data; and time information corresponding to the label data.

[0198] Optionally, in one embodiment, the determination module 501 is inputting the inference input data into the first model to determine inference result data; and The first accuracy is determined according to the inference result data and the label data.

[0199] Optionally, in one embodiment, the first accuracy satisfies a predetermined condition by: the first accuracy is less than a second accuracy for indicating a degree of accuracy of a model output result provided by the first model during a training phase or a testing phase; The first accuracy is smaller than the second accuracy, and a difference between the first accuracy and the second accuracy is greater than a predetermined value; the first accuracy being less than a predetermined accuracy.

[0200] Optionally, in one embodiment, the second accuracy is: A degree of accuracy of a model output result presented by the first model in a training phase or a testing phase; and and the degree of error of model output results presented by said first model during a training phase or a testing phase.

[0201] Optionally, in one embodiment, the model training module 502: acquiring target training data, the target training data including target input data and target label data corresponding to the target input data; and The first model is retrained based on the target training data.

[0202] Optionally, in one embodiment, the model training module 502 comprises: Obtaining first training data for use in training the first model; determining a fifth network element and obtaining second training data from the fifth network element, the fifth network element being for providing the training data; determining a sixth network element; and obtaining inference data from the sixth network element, the sixth network element being for providing the inference data.

[0203] Optionally, in one embodiment, the model training module 502 comprises: determining the fifth network element according to the second information; The second information includes task identifier information of an inference task executed based on the first model and / or condition limitation information of the inference task.

[0204] Optionally, in one embodiment, the model training module 502 comprises: The sixth network element is determined according to the second information.

[0205] Optionally, in one embodiment, the model training module 502 further comprises: Sending model information of the retrained second model or the reselected second model to a second network element, and performing an inference task based on the second model by the second network element; transmitting model information of the retrained second model or the reselected second model to a seventh network element and storing the model information of the second model by the seventh network element; The model information of the second model is model identifier information of the second model; task identifier information for an inference task to be performed based on the second model; and Applicability information of the second model; and a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during a training phase or a testing phase; and training data for the second model; and and the second model.

[0206] Optionally, in one embodiment, the third accuracy is: A degree of accuracy of a model output result presented by the second model in a training phase or a testing phase; and and the degree of error of model output results presented by said second model during a training phase or a testing phase.

[0207] Optionally, in one embodiment, the first network element includes a model training function network element; The second network element includes a model inference function network element; The third network element includes a source device of inference input data; a fourth network element including a source device for label data; a fifth network element including a training data source device; a sixth network element including a source device of inference data; The seventh network element includes a data storage function network element.

[0208] The apparatus 500 according to an embodiment of the present application can refer to the flow of the method 200 of the corresponding embodiment of the present application, and each unit / module in the apparatus 500 and other operations and / or functions described above are for realizing the corresponding flow in the method 200, respectively, and can achieve similar or equivalent technical effects, and will not be redundantly described here for the sake of brevity.

[0209] 6 is a structural schematic diagram of a data processing device in a communication network according to an embodiment of the present application, which may correspond to a second network element in another embodiment. As shown in FIG. 6, the device 600 includes: A task execution module 601 used for executing an inference task based on a first model, the first model being trained by a first network element, the first network element including a model training function network element; and a transmitting module 602 used for transmitting at least one of the usage information and first data of the first model to the first network element and / or transmitting first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store first data of the inference task, the seventh network element including a data storage function network element.

[0210] Optionally, in one embodiment, the first data comprises: inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

[0211] Optionally, in one embodiment, the device 600 comprises: The device further includes a first receiving module 603 used for receiving a first request message sent by the first network element, where the first request message is for requesting to obtain usage information of the first model by the second network element.

[0212] Optionally, in one embodiment, the device 600 comprises: The device further includes a second receiving module 604 used to receive a second request message sent by the first network element, the second request message being for requesting to obtain the first data collected by the second network element.

[0213] Optionally, in one embodiment, the device 600 comprises: said second network element collecting said inference input data; and wherein the second network element collects label data corresponding to the inference input data.

[0214] Optionally, in one embodiment, the collection module 605 further comprises: collecting the inference input data upon receiving a task request for the inference task by an eighth network element, the eighth network element comprising a consumer network element; the second network element is used for at least one of: actively collecting the inference input data.

[0215] Optionally, in one embodiment, the collection module 605 further comprises: The second network element is used to send a data retrieval request to a fourth network element, the data retrieval request being for requesting to retrieve label data corresponding to the inference input data.

[0216] Optionally, in one embodiment, the task execution module 601 further comprises: Receives model information of a second model sent by the first network element, and the second model is obtained by retraining the first model by the first network element or the second model is a model reselected by the first network element; The model information of the second model is model identifier information of the second model; task identifier information for an inference task to be performed based on the second model; and Applicability information of the second model; and a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during a training phase or a testing phase; and training data for the second model; and and the second model.

[0217] Optionally, in one embodiment, the third accuracy is: A degree of accuracy of a model output result presented by the second model in a training phase or a testing phase; and and the degree of error of model output results presented by said second model during a training phase or a testing phase.

[0218] Optionally, in one embodiment, the task execution module 601 further comprises: sending a model request message to the first network element, the model request message to request obtaining the first model; and Receives model information of the first model sent by the first network element; The model information of the first model includes The first model; and and a second accuracy of the first model for indicating a degree of accuracy of model output results presented by the first model during a training phase or a testing phase.

[0219] Optionally, in one embodiment, the second accuracy is: A degree of accuracy of a model output result presented by the first model in a training phase or a testing phase; and and the degree of error of model output results presented by said first model during a training phase or a testing phase.

[0220] Optionally, in one embodiment, the task execution module 601: receiving a task request for the inference task sent by an eighth network element and performing the inference task based on the first model, the eighth network element comprising a consumer network element; and obtaining a task request of the inference task simulated and triggered by the second network element; and executing the inference task based on the first model.

[0221] Optionally, in one embodiment, the task execution module 601: sending a get input data request message to a third network element, the get input data request message being for requesting to obtain inference input data corresponding to the inference task; receiving the inference input data transmitted by the third network element; and The inference input data is input to the first model to obtain inference result data.

[0222] Optionally, in one embodiment, the task execution module 601 further comprises: The inference result data is sent to the eighth network element.

[0223] Optionally, in one embodiment, the usage information comprises: model identifier information of the first model; task identifier information for an inference task to be executed based on the first model; and Conditional limiting information for the inference task; and subject information of the inference task.

[0224] The apparatus 600 according to an embodiment of the present application can refer to the flow of the method 300 of the corresponding embodiment of the present application, and each unit / module in the apparatus 600 and other operations and / or functions described above are for realizing the corresponding flow in the method 300, respectively, and can achieve similar or equivalent technical effects, and will not be redundantly described here for the sake of brevity.

[0225] The data processing device in the communication network in the embodiment of the present application may be, for example, an electronic device having an operating system, or may be a component in the electronic device, such as an integrated circuit or chip. The electronic device may be a terminal or other device other than a terminal. Exemplarily, the terminal may include, but is not limited to, the types of terminal 11 listed above. The other device may be, for example, a server, a network attached storage (NAS), etc., but is not specifically limited in the embodiment of the present application.

[0226] The data processing device in a communication network provided in the embodiments of the present application can realize each step realized in the method embodiments of Figures 2 to 4 and achieve similar technical effects, and will not be described here repeatedly.

[0227] Optionally, as shown in FIG. 7, an embodiment of the present application further provides a communication device 700, comprising a processor 701 and a memory 702, in which a program or command executable on the processor 701 is stored. When the communication device 700 is a network side device, when the program or command is executed by the processor 701, each step of the embodiment of the data processing method in the above-mentioned communication network can be realized, and similar technical effects can be achieved, which will not be repeated here in order to avoid duplication.

[0228] The embodiment of the present application further provides a network side device including a processor and a communication interface. The processor is used to determine a first accuracy of a first model, the first accuracy being for indicating the accuracy of the first model used in actual inference, and to retrain the first model or reselect a second model when the first accuracy satisfies a predetermined condition. Alternatively, the processor is used to execute an inference task based on a first model, the first model being trained by a first network element, the first network element including a model training function network element. The communication interface is used to transmit at least one of the usage information of the first model and first data to the first network element, and / or transmit first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store the first data of the inference task, the seventh network element including a data storage function network element. The embodiment of the network side device corresponds to the embodiment of the method of the network side device described above, and each implementation step and realization manner of the embodiment of the method described above are all applicable to the embodiment of the network side device, and similar technical effects can be achieved.

[0229] Specifically, an embodiment of the present application further provides a network side device. As shown in Fig. 8, the network side device 800 includes a processor 801, a network interface 802, and a memory 803. The network interface 802 is, for example, a common public radio interface (CPRI).

[0230] Specifically, the network side device 800 of the embodiment of the present invention further includes a command or program stored in the memory 803 and executable on the processor 801, and the processor 801 calls the command or program in the memory 803 and executes the method performed by each module shown in Figure 5 or Figure 6, thereby achieving similar technical effects, which will be omitted here to avoid repeated description.

[0231] An embodiment of the present application further provides a readable storage medium, which stores a program or command, and when the program or command is executed by a processor, it realizes each step of the embodiment of the data processing method in the above-mentioned communication network, and achieves similar technical effects, which will be omitted here to avoid repetitive description.

[0232] The processor is the processor in the terminal described in the above embodiment. The readable storage medium includes a computer readable storage medium such as a computer read only memory ROM, a random access memory RAM, a magnetic disk, or an optical disk.

[0233] An embodiment of the present application further provides a chip, the chip comprising a processor and a communication interface, the communication interface and the processor being coupled to each other, the processor executing a program or command to realize each step of the embodiment of the data processing method in the above-mentioned communication network, and similar technical effects can be achieved, and in order to avoid repetitive description, they will be omitted here.

[0234] The chip described in the embodiments of the present application may be called a system on chip, a system chip, a chip system, an SoC, or the like.

[0235] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to realize each step of the embodiment of the data processing method in the above-mentioned communication network, and achieve similar technical effects, and will not be described again here in order to avoid repetition.

[0236] An embodiment of the present application further provides a data processing system in a communication network, comprising a first network side device and a second network side device, the first network side device being operable to execute steps of the data processing method in a communication network described above in Fig. 2, and the network side device being operable to execute steps of the data processing method in a communication network described above in Fig. 3.

[0237] It should be noted that in this specification, the terms "comprise", "consist of" or any other variants are intended to include a non-exclusive inclusion, whereby a process, method, article, or apparatus that includes a set of elements includes not only those elements, but also other elements not expressly stated or inherent to such process, method, article, or apparatus. Unless otherwise specified, an element limited by the phrase "comprises a" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element. In addition, the scope of the method and apparatus in the embodiments of this application is not limited to performing the functions in the order shown or discussed herein, and may further include performing the functions substantially simultaneously or in the opposite order, depending on the functions involved. For example, the methods described above may be performed in a different order than described, and further, individual steps may be added, omitted, or combined. In addition, features described with reference to some examples may be combined with other examples.

[0238] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be realized in the form of a combination of software and a necessary common hardware platform, and of course, they can be realized by hardware, but in many cases the former is a more preferred embodiment. Based on this view, the technical solution of the present application can be substantially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (e.g., ROM / RAM, magnetic disk, optical disk) and includes a number of commands that cause a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the embodiments of the present application.

[0239] Although the examples of the present application have been described above with reference to the drawings, the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative and not limiting. Based on the suggestions of the present application, many forms that a person skilled in the art can make without departing from the spirit of the present application and the scope of protection of the claims are all within the scope of protection of the present application.

[0240] CROSS-REFERENCE TO RELATED APPLICATIONS The present invention claims priority to Chinese patent applications with application number 202210224517.3, titled "Data processing method and network side equipment in a communication network", filed with the China Patent Office on March 7, 2022, and application number 202210950629.7, titled "Data processing method and network side equipment in a communication network", filed with the China Patent Office on August 9, 2022, the entire contents of which are incorporated herein by reference.

Claims

1. a first network element determining a first accuracy of a first model, said first accuracy being for indicating a degree of accuracy of said first model used in actual inference; if the first accuracy satisfies a predetermined condition, the first network element retraining the first model or reselecting a second model.

2. The first accuracy is a degree of accuracy of an inference result when the first model is used in an actual inference; and and a degree of error in inference results when the first model is used in actual inference.

3. The step of the first network element determining a first accuracy of the first model comprises: the first network element acquiring first data; and the first network element determining the first accuracy based on the first data; The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

4. The step of the first network element obtaining the first data comprises: receiving usage information of the first model transmitted by a second network element; determining source information of the first data in response to the usage information; and obtaining the first data in response to the source information.

5. The step of the first network element obtaining the first data comprises:

4. The method of claim 3, comprising receiving the first data transmitted by a second network element, the second network element comprising a model inference function network element.

6. The step of the first network element obtaining the first data comprises:

4. The method of claim 3, further comprising the step of receiving the first data transmitted by a seventh network element, the seventh network element including a data storage function network element.

7. Prior to the step of receiving the usage information of the first model transmitted by the second network element, 5. The method of claim 4, further comprising the step of: the first network element sending a first request message to the second network element, the first request message being for requesting to obtain usage information of the first model by the second network element.

8. prior to the step of receiving the first data transmitted by a second network element, 6. The method of claim 5, further comprising the step of the first network element sending a second request message to the second network element, the second request message being for requesting to obtain the first data collected by the second network element.

9. prior to the step of receiving the first data transmitted by a seventh network element, 7. The method of claim 6, further comprising the step of the first network element sending a third request message to the seventh network element, the third request message being for obtaining the first data.

10. Prior to the step of receiving the usage information of the first model transmitted by the second network element, The method further includes: training the first model to obtain the first model; and sending model information of the first model to the second network element; The model information of the first model includes The first model; and and a second accuracy of the first model for indicating a degree of accuracy of model output results presented by the first model during a training phase or a testing phase.

11. The usage information is model identifier information of the first model; task identifier information for an inference task to be executed based on the first model; and Conditional limiting information for the inference task; and subject information of the inference task.

12. The source information is a third network element for providing inference input data corresponding to the inference task; and a fourth network element for providing label data corresponding to the inference task.

13. The step of acquiring the first data according to the source information includes: sending a get input data request message to the third network element, the get input data request message being for requesting to obtain the inference input data; sending a label data obtainment request message to the fourth network element, the label data obtainment request message being for requesting to obtain the label data; The input data acquisition request message includes: type information of the inference input data; object information corresponding to the inference input data; and time information corresponding to the inference input data; The label data acquisition request message includes: type information of the label data; target information corresponding to the label data; and time information corresponding to the label data.

14. The step of the first network element determining a first accuracy of the first model comprises: inputting said inference input data into said first model to determine inference result data; and determining the first accuracy in response to the inference result data and the label data.

15. The first accuracy satisfies a predetermined condition, the first accuracy being less than a second accuracy for indicating a degree of accuracy of a model output result provided by the first model during a training phase or a testing phase; The first accuracy is smaller than the second accuracy, and a difference between the first accuracy and the second accuracy is greater than a predetermined value; and the first accuracy being less than a predetermined accuracy.

16. The second accuracy is A degree of accuracy of a model output result presented by the first model in a training phase or a testing phase; and and an error degree of model output results presented by said first model during a training phase or a testing phase.

17. The step of the first network element retraining the first model comprises: obtaining target training data, the target training data including target input data and target label data corresponding to the target input data; and retraining the first model based on the target training data.

18. The step of obtaining target training data comprises: obtaining first training data for use in training the first model; determining a fifth network element and obtaining second training data from the fifth network element, the fifth network element being for providing training data; 20. The method of claim 17, comprising at least one of the steps of: determining a sixth network element; and obtaining inference data from the sixth network element, the sixth network element being for providing the inference data.

19. The step of determining a fifth network element comprises: determining the fifth network element in response to second information; The method of claim 18 , wherein the second information includes task identifier information of an inference task to be executed based on the first model and / or condition constraint information of the inference task.

20. The step of determining a sixth network element further comprises:

20. The method of claim 19, comprising determining the sixth network element in response to the second information.

21. After the step of the first network element retraining the first model or reselecting a second model, transmitting model information of the retrained second model or the reselected second model to a second network element, and executing an inference task based on the second model by the second network element; transmitting model information of the retrained second model or the reselected second model to a seventh network element and storing the model information of the second model by the seventh network element; The model information of the second model is model identifier information of the second model; task identifier information for an inference task to be executed based on the second model; and Applicability information of the second model; and a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during a training phase or a testing phase; and training data for the second model; The method of claim 1 , further comprising at least one of:

22. The third accuracy is A degree of accuracy of a model output result presented by the second model in a training phase or a testing phase; and and an error degree of model output results presented by said second model during a training phase or a testing phase.

23. The first network element includes a model training function network element; the second network element includes a model inference function network element; the third network element includes a source device of inference input data; a fourth network element including a source device of label data; a fifth network element including a source device of training data; the sixth network element includes a source device of the inference data; The method of claim 1 , wherein the seventh network element comprises a data storage function network element.

24. a second network element performing an inference task based on a first model, the first model having been trained by the first network element, the first network element including a model training function network element; A method for processing data in a communications network, comprising: a step of transmitting at least one of the usage information and the first data of the first model to the first network element; and / or a step of transmitting first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store the first data of the inference task, the seventh network element comprising a data storage function network element.

25. The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

26. prior to the step of transmitting usage information of the first model to the first network element, 25. The method of claim 24, further comprising the step of: the second network element receiving a first request message sent by the first network element, the first request message being for requesting to obtain usage information of the first model by the second network element.

27. prior to the step of transmitting the first data to the first network element, 25. The method of claim 24, further comprising the step of: the second network element receiving a second request message sent by the first network element, the second request message being to request to obtain the first data collected by the second network element.

28. prior to the step of transmitting first data to the first network element, said second network element collecting said inference input data; and the second network element collecting label data corresponding to the inference input data.

29. The step of the second network element collecting the inference input data further comprises: collecting the inference input data when the second network element receives a task request for the inference task by an eighth network element, the eighth network element including a consumer network element; and the second network element actively collecting the inference input data.

30. The step of the second network element collecting label data corresponding to the inference input data comprises:

30. The method of claim 28, including the step of the second network element sending a data retrieval request to a fourth network element, the data retrieval request being for requesting to retrieve label data corresponding to the inference input data.

31. The method further includes the step of receiving model information of a second model sent by the first network element, wherein the second model is obtained by retraining the first model by the first network element or the second model is a model reselected by the first network element; The model information of the second model is model identifier information of the second model; task identifier information for an inference task to be executed based on the second model; and Applicability information of the second model; and a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during a training phase or a testing phase; and training data for the second model; 25. The method of claim 24, further comprising at least one of: the first model; and the second model.

32. The third accuracy is A degree of accuracy of a model output result presented by the second model in a training phase or a testing phase; and and an error degree of model output results presented by said second model during a training phase or a testing phase.

33. Prior to the step of the second network element executing an inference task based on the first model, sending a model request message to the first network element, the model request message to request obtaining the first model; receiving model information of the first model transmitted by the first network element; The model information of the first model includes The first model; and and a second accuracy of the first model for indicating a degree of accuracy of model output results presented by the first model during a training phase or a testing phase.

34. The second accuracy is A degree of accuracy of a model output result presented by the first model in a training phase or a testing phase; and and an error degree of model output results presented by said first model during a training phase or a testing phase.

35. The step of the second network element performing an inference task based on the first model comprises: receiving a task request for the inference task sent by an eighth network element and performing the inference task based on the first model, the eighth network element comprising a consumer network element; obtaining a task request of the inference task simulated and triggered by the second network element; and performing the inference task based on the first model.

36. The step of performing the inference task based on the first model comprises: sending a get input data request message to a third network element, the get input data request message being for requesting to obtain inference input data corresponding to the inference task; receiving the inference input data transmitted by the third network element; and inputting the inference input data into the first model to obtain inference result data.

37. receiving a task request for the inference task sent by an eighth network element and performing the inference task using the first model; 36. The method of claim 35, further comprising the step of transmitting the inference result data to the eighth network element.

38. The usage information is model identifier information of the first model; task identifier information for an inference task to be executed based on the first model; and Conditional limiting information for the inference task; and subject information of the inference task.

39. a determination module adapted to determine a first accuracy of a first model, the first accuracy being for indicating a degree of accuracy of the first model used in actual inference; A model training module adapted to retrain the first model or reselect a second model if the first accuracy satisfies a predetermined condition.

40. The first accuracy is a degree of accuracy of an inference result when the first model is used in an actual inference; and and a degree of error in inference results when the first model is used in actual inference.

41. The decision module: A method for determining the first accuracy based on the first data, The first data is inference input data; inference result data corresponding to the inference input data; and label data corresponding to the inference input data.

42. The decision module: receiving usage information of the first model transmitted by a second network element; determining source information of the first data in response to the usage information; and Acquiring the first data according to the source information; The usage information is model identifier information of the first model; task identifier information for an inference task to be executed based on the first model; and Conditional limiting information for the inference task; and subject information of the inference task; The source information is a third network element for providing inference input data corresponding to the inference task; and a fourth network element for providing label data corresponding to the inference task.

43. The decision module:

42. The apparatus of claim 41, adapted to receive the first data transmitted by a second network element, the second network element comprising a model inference function network element.

44. The decision module:

42. The apparatus of claim 41, adapted to receive the first data transmitted by a seventh network element, the seventh network element including a data storage function network element.

45. The decision module further comprises: The apparatus of claim 42, used to send a first request message to the second network element, the first request message being for requesting to obtain usage information of the first model by the second network element.

46. The decision module further comprises:

44. The apparatus of claim 43, used to send a second request message to the second network element, the second request message being for requesting to obtain the first data collected by the second network element.

47. The decision module further comprises:

45. The apparatus of claim 44, further adapted to send a third request message to the seventh network element, the third request message being for obtaining the first data.

48. The model training module further comprises: The method is used for obtaining the first model by training the first model, and for sending model information of the first model to the second network element; The model information of the first model includes The first model; and and a second accuracy of the first model for indicating a degree of accuracy of model output results presented by the first model during a training phase or a testing phase.

49. The decision module: sending a get input data request message to the third network element, the get input data request message being for requesting to obtain the inference input data; sending a label data obtainment request message to the fourth network element, the label data obtainment request message being for requesting to obtain the label data; The input data acquisition request message includes: type information of the inference input data; object information corresponding to the inference input data; and time information corresponding to the inference input data; The label data acquisition request message includes: type information of the label data; target information corresponding to the label data; and time information corresponding to the label data.

50. The decision module: inputting the inference input data into the first model to determine inference result data; and The apparatus of claim 41 , further comprising: determining the first accuracy in response to the inference result data and the label data.

51. The model training module includes: acquiring target training data, the target training data including target input data and target label data corresponding to the target input data; and 40. The apparatus of claim 39, adapted to retrain the first model based on the target training data.

52. The model training module includes: Obtaining first training data for use in training the first model; determining a fifth network element and obtaining second training data from the fifth network element, the fifth network element being for providing the training data; 52. The apparatus of claim 51, for use in at least one of: determining a sixth network element; and obtaining inference data from the sixth network element, the sixth network element being for providing the inference data.

53. The model training module further comprises: determining the fifth network element in response to second information; 53. The apparatus of claim 52, wherein the second information includes task identifier information of an inference task to be performed based on the first model and / or condition constraint information of the inference task.

54. The model training module further comprises:

54. The apparatus of claim 53, adapted to determine the sixth network element in response to the second information.

55. The model training module further comprises: transmitting model information of the retrained second model or the reselected second model to a second network element, and performing an inference task based on the second model by the second network element; transmitting model information of the retrained second model or the reselected second model to a seventh network element and storing the model information of the second model by the seventh network element; The model information of the second model is model identifier information of the second model; task identifier information for an inference task to be executed based on the second model; and Applicability information of the second model; and a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during a training phase or a testing phase; and training data for the second model; 40. The apparatus of claim 39, further comprising at least one of:

56. a task execution module adapted to execute an inference task based on a first model, the first model being trained by a first network element, the first network element including a model training function network element; A data processing device in a communication network comprising: a transmission module used for transmitting at least one of the usage information and first data of the first model to the first network element and / or transmitting first instruction information to a seventh network element, the first instruction information being for instructing the seventh network element to store the first data of the inference task, the seventh network element including a data storage function network element.

57. a first receiving module adapted to receive a first request message sent by the first network element, the first request message being for requesting to obtain usage information of the first model by the second network element; 57. The apparatus of claim 56, further comprising:

58. a second receiving module adapted to receive a second request message sent by the first network element, the second request message being for requesting to obtain the first data collected by the second network element; 57. The apparatus of claim 56, further comprising:

59. collecting said inference input data; and collecting label data corresponding to the inference input data.

57. The apparatus of claim 56, further comprising a collection module.

60. The collection module further comprises: collecting the inference input data upon receiving a task request for the inference task by an eighth network element, the eighth network element comprising a consumer network element; 60. The apparatus of claim 59, adapted for at least one of: actively collecting said inference input data.

61. The collection module further comprises:

60. The apparatus of claim 59, wherein the second network element is adapted to send a data retrieval request to a fourth network element, the data retrieval request being for requesting to retrieve label data corresponding to the inference input data.

62. The task execution module further comprises: Receive model information of a second model sent by the first network element, and the second model is obtained by retraining the first model by the first network element; The model information of the second model is model identifier information of the second model; task identifier information for an inference task to be executed based on the second model; and Applicability information of the second model; and a third accuracy of the second model for indicating a degree of accuracy of a model output result provided by the second model during a training phase or a testing phase; and training data for the second model; 57. The apparatus of claim 56, further comprising at least one of:

63. The task execution module further comprises: sending a model request message to the first network element, the model request message to request obtaining the first model; and Receives model information of the first model transmitted by the first network element; The model information of the first model includes The first model; and and a second accuracy of the first model for indicating a degree of accuracy of model output results presented by the first model during a training phase or a testing phase.

64. A network side device comprising a processor and a memory in which a program or command executable on the processor is stored, and when the program or command is executed by the processor, the steps of the method according to any one of claims 1 to 23 are realized, or the steps of the method according to any one of claims 24 to 38 are realized.

65. A readable storage medium having a program or commands stored thereon, the program or commands being executed by a processor to effectuate the steps of the method according to any one of claims 1 to 23 or to effectuate the steps of the method according to any one of claims 24 to 38.

Images