Model updating method and apparatus, and network-side device

Abstract

The present application relates to the technical field of communications, and discloses a model updating method and apparatus, and a network-side device, the method in embodiments of the present application comprising: a first network element receives a first message from a second network element, the first message being used to request updating of a first model, and the first message comprising information related to a model updating reason, the model updating reason comprising at least one of a model inference quality reason and a model compatibility reason; and the first network element updates the first model on the basis of the first message.

Description

Model update method, device and network-side equipment

[0001] Cross-references

[0002] This application claims priority to Chinese Patent Application No. 202510003193.4, filed on January 2, 2025, entitled “Model Update Method, Apparatus and Network-Side Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application belongs to the field of communication technology, and in particular relates to a model update method, apparatus and network-side equipment. Background Technology

[0004] To improve the performance of communication systems, Artificial Intelligence (AI) technology has been widely used in communication networks, such as using AI models to analyze and predict communication data or indicators.

[0005] However, ensuring the inference performance of AI models remains a critical technical challenge in the application of AI models. Summary of the Invention

[0006] The purpose of this application is to provide a model update method, apparatus, and network-side device.

[0007] In a first aspect, a model update method is provided, comprising: a first network element receiving a first message from a second network element, the first message being used to request an update of a first model, and the first message including relevant information on the reason for the model update, wherein the reason for the model update includes at least one of the reasons for model inference quality and the reasons for model compatibility; and the first network element updating the first model according to the first message.

[0008] Secondly, a model update method is provided, comprising: a second network element determining to update a first model; the second network element sending a first message to a first network element; wherein the first message is used to request an update of the first model, and the first message includes relevant information on the reason for the model update, wherein the reason for the model update includes at least one of the reasons for model inference quality and the reasons for model compatibility.

[0009] Thirdly, a model update apparatus is provided, comprising: a transmission module, configured to receive a first message from a second network element, the first message being used to request an update of a first model, and the first message including relevant information on the reason for the model update, the reason for the model update including at least one of model inference quality reasons and model compatibility reasons; and a processing module, configured to update the first model according to the first message.

[0010] Fourthly, a model update apparatus is provided, comprising: a processing module for determining to update a first model; and a transmission module for sending a first message to a first network element; wherein the first message is used to request an update of the first model, and the first message includes relevant information on the reason for the model update, wherein the reason for the model update includes at least one of the reasons for model inference quality and the reasons for model compatibility.

[0011] Fifthly, embodiments of this application provide a network-side device, including: a memory, a processor, and computer-executable instructions stored in the memory and executable on the processor, wherein the computer-executable instructions, when executed by the processor, implement the steps of the method described in the first aspect or the second aspect.

[0012] In a sixth aspect, embodiments of this application provide a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method described in the first or second aspect. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 is a schematic diagram of the communication network architecture provided in an exemplary embodiment of this application.

[0015] Figure 2 is one of the flowcharts of a model update method provided in an exemplary embodiment of this application.

[0016] Figure 3 is a second schematic flowchart of the model update method provided in an exemplary embodiment of this application.

[0017] Figure 4 is a third flowchart illustrating the model update method provided in an exemplary embodiment of this application.

[0018] Figure 5a is one of the interactive flow diagrams of the model update method provided in an exemplary embodiment of this application.

[0019] Figure 5b is a second schematic diagram of the interaction flow of the model update method provided in an exemplary embodiment of this application.

[0020] Figure 5c is a schematic diagram of the third interactive flow of the model update method provided in an exemplary embodiment of this application.

[0021] Figure 6 is a fourth flowchart illustrating the model update method provided in an exemplary embodiment of this application.

[0022] Figure 7 is one of the structural schematic diagrams of a model update device provided in an exemplary embodiment of this application.

[0023] Figure 8 is a second schematic diagram of the structure of a model update device provided in an exemplary embodiment of this application.

[0024] Figure 9 is a schematic diagram of the structure of a network-side device provided in an exemplary embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0026] Figure 1 shows a communication network architecture provided in an exemplary embodiment of this application. The communication network architecture may include user equipment (UE, also known as a terminal), radio access network (RAN), core network, etc.

[0027] The RAN manages radio resources, transmits user data received via N3 to the UE, and retrieves user data from the UE through the N3 interface. The RAN maps between Quality of Service (QoS) traffic in Dedicated Radio Bearer (DRB) and Protocol Data Unit (PDU) sessions.

[0028] The core network may include, but is not limited to, core network elements such as Access and Mobility Management Function (AMF), Session Management Function (SMF), Advanced User Plane Function (A-UPF), Policy Control Function (PCF), and Data Network (DN).

[0029] The AMF can be used to implement, but is not limited to, functions such as registration management, connection management, reachability management, and mobility management. Furthermore, the AMF can also be used to perform access authentication and access authorization. The AMF is a Non-Access Stratum (NAS) secure endpoint that forwards SM NAS messages between the UE and the SMF.

[0030] SMF can be used to implement, but is not limited to, session establishment, modification and release, UE IP address allocation and management (including optional authorization functions), selection and control of User Plane (UP) functions, downlink data notification and other functions.

[0031] The SMF controls the UPF through N4 association. For example, the SMF can provide the UPF with Packet Detection Rule (PDR), Forwarding Action Rule (FAR), QoS Enforcement Rule (QER), and Usage Reporting Rule (URR). Among them, FAR is used to indicate how to detect user data traffic; QER and URR are used to instruct the UPF how to perform user data traffic forwarding, QoS processing, and usage reporting on user data traffic detected using PDR.

[0032] The UPF can be used to implement, but is not limited to, functions such as serving as an anchor point for intra / inter-Radio Access Technology (RAT) mobility, packet routing and forwarding, traffic usage reporting, user plane QoS processing, downlink packet buffering, and downlink data notification triggering. The GTP-U tunnel is used for the N3 interface between the RAN and the UPF. The GTP-U tunnel operates on a per-PDU session basis. For downlink traffic, the UPF binds the downlink traffic to the QoS traffic within the PDU session's GTP-U tunnel using the FAR received from the SMF. For uplink traffic, the RAN transmits user plane traffic to the QoS stream identified by the UE.

[0033] The PCF is used to provide QoS policy rules to control plane functions for enforcement. The PCF translates Application Function (AF) requests into Policy Control and Charging (PCC) rules applicable to PDU sessions.

[0034] DN can include, but is not limited to, Unified Data Management (UDM). UDM performs tasks such as generating 3GPP AKA authentication credentials, granting access based on subscription data, managing UE service element registration (e.g., providing UE storage AMF, providing UE PDU session storage SMF), and subscription management. UDM accesses the Unified Data Repository (UDR) to retrieve UE subscription data and store the UE context in the UDR. UDM and UDR can be deployed together.

[0035] In addition to the aforementioned core network elements, the core network may also include a Network Data Analytics Function (NWDAF). The NWDAF is a 5G core network (5GC) element located in the control plane, performing statistical data and machine learning-related tasks within the 5G system (5GS). The NWDAF can interact with different entities for various purposes.

[0036] 1) The NWDAF can collect data based on event subscriptions provided by AMF, SMF, UPF, PCF, UDM, Network Slice Admission Control Function (NSACF), AF, and OAM. The NWDAF can interact directly with the AF or through the Network Exposure Function (NEF).

[0037] Optionally, NWDAF uses the Data Collection Coordination Function (DCCF) for analysis and data collection.

[0038] 2) NWDAF retrieves information from the data repository, such as retrieving user-related UDRs via UDM or retrieving PFD information via NEF (or Packet Flow Description Function, PFDF) etc.

[0039] 3) NWDAF collects location information data from the Location Services (LCS) system.

[0040] Optionally, NWDAF stores and retrieves information from the Analytic Data Repository Function (ADRF).

[0041] Optionally, NWDAF analyzes and collects data from the Messaging Framework Adaptor Function (MFAF).

[0042] 4) NWDAF retrieves information about network elements, for example, retrieving network element-related information from the Network Repository Function (NRF).

[0043] 5) NWDAF provides analytics to consumers on demand.

[0044] 6) NWDAF provides batch data related to the analysis ID.

[0045] 7) NWDAF provides information on the accuracy of the analysis ID.

[0046] 8) NWDAF provides information on the accuracy of the AI ​​model or indications of model accuracy degradation. In this embodiment, the AI ​​model may also be referred to as a machine learning (ML) model, AI unit, neural network, etc., without limitation.

[0047] For the deployment of the aforementioned NWDAF, a single instance or multiple instances of the NWDAF can be deployed in a Public Land Mobile Network (PLMN). The NWDAF may include logical functions such as Analytics Logical Function (AnLF) and Model Training Logical Function (MTLF).

[0048] AnLF, as a logical function in NWDAF, is used to perform inference and derive analytical information, that is, to derive statistical data and / or predictions based on consumer requests for analysis, and to expose analytical services, namely Nnwdaf_AnalyticsSubscription or Nnwdaf_Analytics_Request.

[0049] MTLF is used to train AI models and expose new training services, such as providing pre-trained AI models.

[0050] An NWDAF can contain one MTLF or one AnLF, or both MTLF and AnLF, without any restrictions.

[0051] When implementing network communication based on the aforementioned communication network architecture, the application of AI models can improve communication performance in related technologies. However, ensuring the performance of AI models remains a pressing technical problem that needs to be solved in this field. To address this, this application provides a model update scheme to ensure model inference performance in model applications through timely model updates, thereby ensuring the performance of the communication system.

[0052] Optionally, the technical solutions provided in this embodiment can be applied to, but are not limited to, the communication network architecture shown in Figure 1 above. For example, the technical solutions provided in this embodiment of the application can also be applied to 4G communication network architecture, 6G communication network architecture, or other future communication network architectures, etc., without limitation.

[0053] Based on this, the technical solutions provided by the embodiments of this application will be described in detail below with reference to the accompanying drawings and through some embodiments and application scenarios.

[0054] Figure 2 shows a schematic flowchart of a model update method 200 provided in an embodiment of this application. This method 200 can be executed by a first network element. As shown in Figure 2, the method 200 may include the following steps.

[0055] S210, the first network element receives the first message from the second network element.

[0056] S220, the first network element updates the first model according to the first message.

[0057] The first network element is used to provide model training services, etc., that is, the first network element may include, but is not limited to, model training functions. For example, the first network element may be NWDAF, which deploys MTLF in the aforementioned communication network architecture.

[0058] The second network element can be a consumer network element or a network element used to provide model inference services, etc.

[0059] In the case where the second network element is a consumer network element, the second network element may be, but is not limited to, PCF, Location Management Function (LMF), etc.

[0060] In the case where the second network element is a network element used to provide model inference services, the second network element may include, but is not limited to, model inference functions. For example, the second network element may be, but is not limited to, NWDAF deployed in the aforementioned communication network architecture.

[0061] Optionally, for the second network element to provide model inference services, the first and second network elements can be independently deployed core network elements, access network elements, etc., that can interact with each other through the core network interface. For example, the first network element can be NWDAF 1 deploying MTLF, and the second network element can be NWDAF 2 deploying AnLF, where NWDAF 1 and NWDAF 2 are different.

[0062] Alternatively, the first network element and the second network element can be deployed together. For example, the first network element and the second network element can be deployed together in the same NWDAF in the aforementioned communication network architecture, and this embodiment does not impose any limitations on this.

[0063] In this embodiment, the first message sent by the second network element is used to request the first network element to update the first model, and the first message includes information related to the reason for the model update. The reason for the model update includes at least one of the following: model inference quality reasons and model compatibility reasons. That is to say, in this embodiment, the model update process can be triggered based on at least one of the following reasons: model inference quality reasons and model compatibility reasons, so as to realize timely model updates, thereby ensuring the model inference quality during model application and ensuring the performance of the communication system.

[0064] In some embodiments, the relevant information regarding the reasons for the model inference quality may include, but is not limited to, at least one of the following 11)-18).

[0065] 11) First information, which is related information used to indicate the model inference speed of the first model.

[0066] In other words, in this application, the model inference speed can be used as an evaluation factor of the model inference quality to trigger the update process of the first model.

[0067] Optionally, the information related to the model inference speed indicated by the first information may include, but is not limited to, one or more of the following: the model inference speed is below standard or expected, the model inference speed is less than a first threshold, the model inference speed is too slow, and the model inference speed evaluation value.

[0068] In some embodiments, when the first information is used to indicate relevant information about the model's inference speed, it can be an explicit indication or an implicit indication, without limitation.

[0069] In some embodiments, the first threshold and the second threshold mentioned later may be agreed upon by the protocol or configured by a higher layer, and there are no restrictions here.

[0070] 12) Second information, used to indicate that the target inference result has not met the standard (or has not met the expected value), the target inference result including at least one of the model inference results of the first model.

[0071] In this embodiment, considering that in a single model inference based on the first model, the first model may output multiple model inference results (or analysis data, prediction data, etc.) simultaneously, the inference quality of each model inference result can be used as an evaluation factor of the model inference quality to trigger the update process of the first model. This enables finer-grained model quality monitoring in the model update process, thereby ensuring better model inference performance.

[0072] 13) Third information, used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric.

[0073] In this embodiment, the third information enables the first network element to clearly identify which model metric has not met the standard, and / or determine the threshold value of the model metric when the target inference result fails to meet the standard. This allows the first network element to update the first model for the model metric that has not met the standard, making the updated model more compatible with the needs of the second network element.

[0074] In some embodiments, if the third information is only used to indicate the indicator threshold, then when the first network element receives the third information, it can determine that the model metric corresponding to the indicator threshold in the first model has not met the standard, and the first model needs to be updated for the model metric.

[0075] In some embodiments, the model metrics described in this embodiment may include, but are not limited to, at least one of the following: model accuracy, model precision, model recall, model F1 score, mean squared error of the model inference result, and mean absolute percentage error of the model inference result, without limitation herein.

[0076] 14) Fourth information, used to indicate the current value of the unmet model metric corresponding to the target inference result, or used to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold.

[0077] In this embodiment, the indication of the fourth information can be used by the first network element to determine the quality of the model inference of the first model, and then update the first model based on the quality.

[0078] For example, assuming the fourth information is used to indicate the current value, the first network element can determine the quality of the model inference of the first model by determining the difference between the current value and the indicator threshold, and then update the first model based on the quality of the inference.

[0079] 15) The fifth piece of information is used to indicate that the model measurement index is not up to standard when the terminal assists the second network element (such as the consumer network element) in model inference.

[0080] In this embodiment, the fifth information enables the first network element to analyze the possible reasons for model quality issues leading to model updates, such as whether they are related to the inference data provided by the terminal, thereby improving the efficiency of model updates.

[0081] 16) The sixth piece of information is used to indicate that the model metric has not met the standard when the access network equipment assists the second network element (such as the consumer network element) in model inference.

[0082] In this embodiment, the sixth information enables the first network element to analyze the possible reasons for model quality issues leading to model updates, such as whether the issues are related to the inference data provided by the access network device, thereby improving the efficiency of model updates.

[0083] 17) Inference data source information, wherein the inference data source is used to provide the inference data required for the first model to infer.

[0084] In other words, this application uses the inference data source information during the inference of the first model as an evaluation factor for the inference quality of the model, in order to trigger the update process of the first model.

[0085] For example, when the first network element receives the inference data source information, it can directly determine that the poor inference quality of the model is caused by the inference data source, or it can determine whether the poor inference quality of the model is caused by the contamination of the data source, etc., without any restrictions.

[0086] Optionally, the inference data source information may be the network element ID or network element address information of the inference data source, etc., and there is no limitation on this.

[0087] 18) The inference data used by the first model.

[0088] In other words, this application uses the inference data during the inference of the first model as an evaluation factor of the inference quality of the model to trigger the update process of the first model.

[0089] For example, when the first network element receives the inference data used by the first model, it can directly determine that the poor inference quality of the model is caused by the inference data, or it can determine whether the poor inference quality of the model is caused by the contamination of the inference data, etc., without any restrictions.

[0090] In this embodiment, the information in 11)-18) mentioned above included in the first message can be determined by the protocol or configured by higher level, and there is no restriction here.

[0091] In some embodiments, the relevant information regarding the model compatibility reasons may include, but is not limited to, at least one of the following 21)-24).

[0092] 21) The seventh piece of information is used to indicate that the second network element does not support the type of the first model.

[0093] 22) The eighth information is used to indicate that the second network element does not support the model operating environment corresponding to the first model.

[0094] 23) Ninth information, used to indicate the file size information of the model file of the first model.

[0095] The file size information of the model file of the first model may include, but is not limited to, one or more of the following: the model file is too large, the size of the model file exceeds the second threshold, the size of the model file is not up to standard or does not meet expectations.

[0096] In some embodiments, when the file size information of the model file of the first model is indicated by the ninth information, it can be an explicit indication or an implicit indication, and there is no limitation on this.

[0097] 24) Local environment (or local configuration) information of the second network element.

[0098] When the relevant information regarding the reasons for model compatibility includes the local environment information of the second network element, it can be understood that: the incompatibility problem of the first model is caused by changes in the local environment information of the second network element. Therefore, the first network element can update the first model based on the local environment information of the second network element.

[0099] In this embodiment, the specific information included in the first message (21)-24) can be determined by the protocol or configured by higher-level systems, and no restrictions are imposed here.

[0100] In some embodiments, when the second network element sends a first message to the first network element to request the first network element to update the first model, the second network element may determine the way to update the first model in multiple ways, which will be described below in conjunction with different implementation methods.

[0101] Implementation Method 1

[0102] Suppose that the second network element (such as a consumer network element or a network element that includes model inference function) determines or discovers that the local configuration information has changed, thereby causing the first model to be incompatible with the changed local configuration information, such as the second network element not supporting the type of the first model, the second network element not supporting the model running environment corresponding to the first model, or the model file of the first model being too large, then the second network element can determine to update the first model and initiate a model update process for the first model through the first message.

[0103] Implementation Method 2

[0104] Assuming the second network element includes model inference functionality, upon receiving a third message from a consumer network element (such as PCF or LMF), the second network element can determine whether to update the first model based on the third message. The third message is determined by the consumer network element based on the model inference result of the first model provided by the second network element, and serves to indicate model inference quality feedback information corresponding to the first model.

[0105] In some embodiments, the model inference quality feedback information may include, but is not limited to, at least one of the following 31)-36).

[0106] 31) The true data corresponding to the model inference results.

[0107] The true data mentioned above can also be called calibration data or real data.

[0108] In this embodiment, the second network element can determine the model inference quality of the first model based on the truth data and the model inference result, and then determine whether to update the first model based on the model inference quality. If there is an unqualified target inference result in the model inference result, it can be determined to update the first model.

[0109] 32) Second information, used to indicate that the target reasoning result has not met the standard, the target reasoning result including at least one of the model reasoning results of the first model.

[0110] 33) Third information, used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric.

[0111] 34) Fourth information, used to indicate the current value of the unmet model metric corresponding to the target inference result, or used to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold.

[0112] The second, third, and fourth information mentioned in 32)-34) above can be referred to the relevant descriptions in the first message above, and will not be repeated here.

[0113] In this embodiment, for the second network element that receives the second information, the third information, and the fourth information, it can autonomously determine whether to update the first model based on the second information, the third information, and the fourth information. For example, if it is determined that the model inference quality of the first model is not up to standard or not as expected based on one or more of the second information, the first model is updated.

[0114] 35) Twelfth information, which is related to the model inference speed of the first model.

[0115] The twelfth piece of information can be referred to in the description of the first information in the aforementioned first message, and will not be repeated here.

[0116] In this embodiment, the second network element that receives the twelfth information can autonomously determine whether to update the first model based on the twelfth information. For example, if it is determined based on the twelfth information that the model inference speed of the first model does not meet the standard or expectations, the first model is updated.

[0117] 36) Thirteenth message, used to indicate that the model inference quality of the first model is not up to standard.

[0118] In this embodiment, the second network element can determine to update the first model based on the thirteenth information.

[0119] In this embodiment, after receiving the third message, the second network element can evaluate and calculate the model inference quality of the first model based on one or more of the information in the third message, and then determine whether to trigger the model update process. Furthermore, which information from 31)-36) mentioned above is actually included in the third message can be determined by protocol agreement or higher-level configuration, and is not limited here.

[0120] In some embodiments, when the consumer network element sends the third message to the second network element (such as NWDAF including AnLF), it can be understood that: the consumer network element needs to request or subscribe to the model inference result of the first model from the second network element, and after receiving the model inference result from the second network element, the consumer network element can indicate the model inference quality feedback information corresponding to the first model to the second network element, so that the second network element can determine whether to trigger the update process of the first model.

[0121] In some embodiments, the process by which the first network element updates the first model according to the first message in S220 may include S2201 as shown in FIG3.

[0122] S2201, the first network element performs a first operation according to the first message, wherein the first operation includes at least one of the following operations 1-3.

[0123] Operation 1: Retrain the first model.

[0124] In some embodiments, if the first network element determines, based on the information in the first message, that the poor inference quality or model incompatibility is due to insufficient model training or insufficient training samples of the first model, then the first model can be updated by retraining the first model.

[0125] Operation 2: Update the first model to the second model.

[0126] In some embodiments, if the first network element determines, based on the information in the first message, that the poor inference quality or model incompatibility is due to a mismatch in the model structure or model type of the first model, then the first model can be updated by replacing the first model with the second model.

[0127] Operation 3: Determine whether to update or not update the inference data source of the first model.

[0128] In some embodiments, if the first network element determines, based on the information in the first message, that the poor model inference quality or model incompatibility is due to contamination of the model inference data of the first model, it can determine to update the inference data source of the first model and update the first model; otherwise, it determines not to update the inference data source of the first model and can use the aforementioned operation 1 or operation 2 to update the first model.

[0129] In some embodiments, if the first network element determines to update the inference data source of the first model, then the first network element may independently determine a new inference data source and provide it to the second network element, or it may instruct the second network element to determine a new inference data source, without limitation.

[0130] In some embodiments, when updating the first model, the first network element may update only the model itself, only the inference data source, or both simultaneously. That is, the first network element may determine to use one or more of the aforementioned operations 1-3 to update the first model, without limitation.

[0131] In some embodiments, after the first network element completes the update of the first model, the method embodiment 200 of this application may further include S230 as shown in FIG4, the content of which is as follows.

[0132] S230, the first network element sends a second message to the second network element.

[0133] The second message is used to instruct the model to update relevant information, so that the second network element can determine the updated model and / or inference data source based on the second message, and realize subsequent model inference.

[0134] In this embodiment, the second message may include, but is not limited to, at least one of the following 41)-45).

[0135] 41) Updated model files, such as the model file of the first model after retraining, the model file of the second model after replacement, etc.

[0136] 42) The access address of the updated model file, such as the access address of the model file of the first model after retraining, the access address of the model file of the second model after replacement, etc.

[0137] 43) The inference data source corresponding to the updated model file, such as the inference data source corresponding to the first model after retraining, the inference data source corresponding to the second model after replacement, etc.

[0138] 44) The tenth information is used to indicate whether to update the inference data source of the first model.

[0139] If the tenth information indicates that the inference data source of the first model should be updated, then the second network element can independently determine the new inference data source, or it can request the new inference data source from the first network, without any restriction.

[0140] If the tenth information indicates that the inference data source of the first model should not be updated, then the second network element can use the original inference data source and the updated model file (such as the model file of the newly trained first model, the model file of the replaced second model, etc.) for subsequent model inference.

[0141] 45) The eleventh message is used to indicate the inference data source information corresponding to the recollection of inference data.

[0142] Specifically, the eleventh information enables the second network element to determine the new inference data source after the change and to perform model inference based on the new inference data source.

[0143] Optionally, the inference data source information may include, but is not limited to, the network element ID or network element address information of the inference data source or the area information where the data source is located, etc., and there are no restrictions here.

[0144] Based on the description of the aforementioned method embodiment 200, for ease of understanding, the model update scheme provided in this application will be illustrated below with reference to Examples 1-3, as follows.

[0145] Example 1

[0146] S501, a consumer network element (such as PCF, LMF, etc.) sends a subscription request to a second network element (such as NWDAF including AnLF) to subscribe to the model inference results (or analysis results or prediction results) based on the first model.

[0147] S502, the second network element performs local model inference based on the first model and generates model inference results.

[0148] S503, the second network element sends the model inference results to the consumer network element.

[0149] S504, the consumer network element sends a third message to the second network element.

[0150] The third message is determined based on the model inference result of the first model provided by the second network element and is used to indicate the model inference quality feedback information corresponding to the first model.

[0151] S505, the second network element determines whether to update the first model based on the model inference quality feedback information.

[0152] S506, when the second network element determines that the first model needs to be updated, it sends a first message to the first network element (such as an NWDAF including MTLF) to request an update of the first model.

[0153] S507, the first network element updates the first model according to the first message, such as retraining the first model, replacing the first model, or determining whether to replace or not replace the inference data source.

[0154] S508, the first network element sends a second message to the second network element to indicate the updated model-related information.

[0155] S509, the second network element determines the updated model or inference data source information based on the second message.

[0156] Wherein, if the second message includes tenth information and the tenth information is used to indicate updating the inference data source of the first model, or if the second message includes eleventh information and the eleventh message is used to indicate re-collecting the inference data source information corresponding to the inference data, the second network element may subscribe to inference data from the new inference data source.

[0157] If the second message includes an updated model file or the model file access address corresponding to the updated model file, the second network element can obtain a new model, such as the second model, based on the updated model file or the model file access address corresponding to the updated model file.

[0158] S510, the second network element performs model inference based on the updated model or data source.

[0159] In this example 1, the reasons for model inference quality are used as the conditions for model update, thereby implementing the model update process and ensuring the quality of model inference.

[0160] The aforementioned implementation process provided in Example 1 can refer to the relevant description in the aforementioned method embodiment 200 and achieve the same or corresponding technical effects, which will not be repeated here. In addition, Example 1 may include, but is not limited to, the aforementioned S501-S520, and may include more or fewer steps than the aforementioned S501-S510, which will not be repeated here.

[0161] Example 2

[0162] S521, the consumer network element (such as PCF, LMF, etc.) determines to update the first model.

[0163] For example, when a consumer network element performs model inference based on the first model and obtains the model inference result (or analysis result or prediction result), it determines that the model inference quality is substandard or does not meet expectations based on the model inference result, and then determines to update the first model.

[0164] For example, a consumer network element determines or discovers that the first model is incompatible with the updated local configuration due to a local configuration update, and then determines to update the first model.

[0165] S522, the consumer network element sends a first message to the first network element (such as an NWDAF including MTLF) to request an update of the first model.

[0166] The first message includes information about the reason for the model update, which includes at least one of the reasons for model inference quality and model compatibility.

[0167] S523, the first network element updates the first model according to the first message, such as retraining the first model, replacing the first model, or determining whether to replace or not replace the inference data source.

[0168] S524, the first network element sends a second message to the consumer network element to indicate the updated model-related information.

[0169] S525, the consumer network element determines the updated model or inference data source information based on the second message.

[0170] Wherein, if the second message includes tenth information and the tenth information is used to indicate updating the inference data source of the first model, or if the second message includes eleventh information and the eleventh message is used to indicate re-collecting the inference data source information corresponding to the inference data, the consumer network element may subscribe to inference data from the new inference data source.

[0171] If the second message includes an updated model file or the model file access address corresponding to the updated model file, the consumer network element obtains a new model, such as the second model, based on the updated model file or the model file access address corresponding to the updated model file.

[0172] S526, Consumer network elements perform model inference based on the updated model or data source.

[0173] In this Example 2, the reasons for model inference quality and / or model compatibility are used as the conditions for model update to implement the model update process and ensure the quality of model inference.

[0174] The aforementioned implementation process provided in Example 2 can refer to the relevant description in the aforementioned method embodiment 200 and achieve the same or corresponding technical effects, which will not be repeated here. In addition, Example 2 may include, but is not limited to, the aforementioned S521-S526, and may include more or fewer steps than the aforementioned S521-S526, which will not be repeated here.

[0175] Example 3

[0176] S531, the second network element (such as NWDAF including AnLF) determines or discovers that due to a local configuration update, and the updated local configuration is incompatible with the first model, it determines to update the first model.

[0177] S532, the second network element sends a first message to the first network element (such as an NWDAF including MTLF) to request an update of the first model.

[0178] The first message includes information about the reason for the model update, including reasons related to model compatibility.

[0179] S533, the first network element updates the first model according to the first message, such as retraining the first model or replacing the first model.

[0180] S534, the first network element sends a second message to the second network element to indicate the updated model-related information.

[0181] S535, the second network element determines the updated model based on the second message.

[0182] S536, the second network element is based on the updated model or performs model inference.

[0183] In Example 3, for the second network element, model compatibility reasons are used as the model update condition to implement the model update process and ensure the quality of model inference.

[0184] The aforementioned implementation process provided in Example 3 can refer to the relevant description in the aforementioned method embodiment 200 and achieve the same or corresponding technical effects, which will not be repeated here. In addition, Example 3 may include, but is not limited to, the aforementioned S531-S536, and may include more or fewer steps than the aforementioned S531-S536, which will not be repeated here.

[0185] Figure 6 shows a schematic flowchart of a model update method 600 provided in an embodiment of this application. This method 600 can be executed by a second network element. As shown in Figure 2, the method 600 may include the following steps.

[0186] S610, the second network element determines to update the first model.

[0187] S620, the second network element sends a first message to the first network element.

[0188] The first message is used to request an update of the first model, and the first message includes information about the reason for the model update, which includes at least one of the reasons for model inference quality and model compatibility.

[0189] In some embodiments, the information related to the reasons for model inference quality includes at least one of the following: first information, information related to the model inference speed of the first model; second information, information indicating that the target inference result is substandard, the target inference result including at least one of the model inference results of the first model; third information, information indicating at least one of the substandard model metric corresponding to the target inference result and the metric threshold corresponding to the substandard model metric; fourth information, information indicating the current value of the substandard model metric corresponding to the target inference result, or information indicating the difference between the current value of the substandard model metric corresponding to the target inference result and the metric threshold; fifth information, information indicating that the model metric is substandard when the terminal assists the second network element in model inference; sixth information, information indicating that the model metric is substandard when the access network device assists the second network element in model inference; inference data source information, the data source being used to provide the inference data required for the first model inference; and the inference data used by the first model.

[0190] In some embodiments, the model metrics include at least one of model accuracy, model precision, model accuracy, model recall, model F1 score, mean squared error of the model inference result, and mean absolute percentage error of the model inference result.

[0191] In some embodiments, the information related to the model compatibility reasons includes at least one of the following: a seventh piece of information indicating that the second network element does not support the type of the first model; an eighth piece of information indicating that the second network element does not support the model running environment corresponding to the first model; a ninth piece of information indicating file size information related to the model file of the first model; and local environment information of the second network element.

[0192] In some embodiments, the method further includes: the second network element receiving a second message from the first network element; wherein the second message includes at least one of the following: an updated model file; a model file access address corresponding to the updated model file; an inference data source corresponding to the updated model file; a tenth message indicating whether to update the inference data source of the first model; and an eleventh message indicating to re-collect the inference data source information corresponding to the inference data.

[0193] In some embodiments, the second network element determines to update the first model, including: the second network element determines that local configuration information has changed; the second network element determines to update the first model.

[0194] In some embodiments, the second network element determines to update the first model, including: the second network element receiving a third message from a consumer network element, the third message being determined based on the model inference result of the first model provided by the second network element and used to indicate the model inference quality feedback information corresponding to the first model; the second network element determining to update the first model based on the third message; wherein, the second network element is a model inference function.

[0195] In some embodiments, the model inference quality feedback information includes at least one of the following: ground truth data corresponding to the model inference result; second information indicating that the target inference result is substandard, the target inference result including at least one of the model inference results of the first model; third information indicating at least one of the substandard model metric corresponding to the target inference result and the metric threshold corresponding to the substandard model metric; fourth information indicating the current value of the substandard model metric corresponding to the target inference result, or indicating the difference between the current value of the substandard model metric corresponding to the target inference result and the metric threshold; twelfth information indicating relevant information about the model inference speed of the first model; and thirteenth information indicating that the model inference quality of the first model is substandard.

[0196] In some embodiments, the first network element includes a model training function; the second network element includes a consumer network element or a model inference function.

[0197] It is understood that each implementation in this method embodiment 600 has the same or corresponding technical features as each implementation in the aforementioned method embodiment 200. Therefore, the implementation process of each implementation in this method embodiment 600 can refer to the relevant description in the aforementioned method embodiment 200 and achieve the same or corresponding technical effects. To avoid repetition, it will not be described again here.

[0198] Figure 7 shows a schematic diagram of the structure of the model update device 700 provided in an embodiment of this application. The device 700 can be a component of the first network element or applied to the first network element, or it can be the first network element itself; no limitation is made here. In this embodiment, the device 700 may include: a transmission module 710, configured to receive a first message from a second network element, the first message being used to request an update of the first model, and the first message including relevant information about the reason for the model update, the reason for the model update including at least one of model inference quality reasons and model compatibility reasons; and a processing module 720, configured to update the first model according to the first message.

[0199] In some embodiments, the information related to the reasons for model inference quality includes at least one of the following: first information, information related to the model inference speed of the first model; second information, information indicating that the target inference result is substandard, the target inference result including at least one of the model inference results of the first model; third information, information indicating at least one of the substandard model metric corresponding to the target inference result and the metric threshold corresponding to the substandard model metric; fourth information, information indicating the current value of the substandard model metric corresponding to the target inference result, or information indicating the difference between the current value of the substandard model metric corresponding to the target inference result and the metric threshold; fifth information, information indicating that the model metric is substandard when the terminal assists the second network element in model inference; sixth information, information indicating that the model metric is substandard when the access network device assists the second network element in model inference; inference data source information, the data source being used to provide the inference data required for the first model inference; and the inference data used by the first model.

[0200] In some embodiments, the model metrics include at least one of model accuracy, model precision, model accuracy, model recall, model F1 score, mean squared error of the model inference result, and mean absolute percentage error of the model inference result.

[0201] In some embodiments, the information related to the model compatibility reasons includes at least one of the following: a seventh piece of information indicating that the second network element does not support the type of the first model; an eighth piece of information indicating that the second network element does not support the model running environment corresponding to the first model; a ninth piece of information indicating file size information related to the model file of the first model; and local environment information of the second network element.

[0202] In some embodiments, updating the first model according to the first message includes: performing at least one of the following according to the first message: retraining the first model; updating the first model to a second model; determining whether to update or not update the inference data source of the first model.

[0203] In some embodiments, the transmission module 710 is further configured to send a second message to the second network element; wherein the second message includes at least one of the following: an updated model file; a model file access address corresponding to the updated model file; an inference data source corresponding to the updated model file; a tenth message indicating whether to update the inference data source of the first model; and an eleventh message indicating to re-collect the inference data source information corresponding to the inference data.

[0204] The device 700 provided in this application embodiment can execute the methods described in the preceding method embodiments and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0205] Figure 8 shows a schematic diagram of the structure of the model update device 800 provided in an embodiment of this application. The device 800 can be a component of the second network element or applied to the second network element, or it can be the second network element itself; no limitation is made here. In this embodiment, the device 800 may include: a processing module 810, used to determine to update the first model; and a transmission module 820, used to send a first message to the first network element; wherein the first message is used to request an update of the first model, and the first message includes relevant information about the reason for the model update, the reason for the model update including at least one of model inference quality reasons and model compatibility reasons.

[0206] In some embodiments, the information related to the reasons for model inference quality includes at least one of the following: first information, information related to the model inference speed of the first model; second information, information indicating that the target inference result is substandard, the target inference result including at least one of the model inference results of the first model; third information, information indicating at least one of the substandard model metric corresponding to the target inference result and the metric threshold corresponding to the substandard model metric; fourth information, information indicating the current value of the substandard model metric corresponding to the target inference result, or information indicating the difference between the current value of the substandard model metric corresponding to the target inference result and the metric threshold; fifth information, information indicating that the model metric is substandard when the terminal assists the second network element in model inference; sixth information, information indicating that the model metric is substandard when the access network device assists the second network element in model inference; inference data source information, the data source being used to provide the inference data required for the first model inference; and the inference data used by the first model.

[0207] In some embodiments, the model metrics include at least one of model accuracy, model precision, model accuracy, model recall, model F1 score, mean squared error of the model inference result, and mean absolute percentage error of the model inference result.

[0208] In some embodiments, the information related to the model compatibility reasons includes at least one of the following: a seventh piece of information indicating that the second network element does not support the type of the first model; an eighth piece of information indicating that the second network element does not support the model running environment corresponding to the first model; a ninth piece of information indicating file size information related to the model file of the first model; and local environment information of the second network element.

[0209] In some embodiments, the transmission module 820 is further configured to receive a second message from the first network element; wherein the second message includes at least one of the following: an updated model file; a model file access address corresponding to the updated model file; an inference data source corresponding to the updated model file; a tenth message indicating whether to update the inference data source of the first model; and an eleventh message indicating to re-collect the inference data source information corresponding to the inference data.

[0210] In some embodiments, determining to update the first model includes: determining that local configuration information has changed; and determining to update the first model.

[0211] In some embodiments, determining to update the first model includes: receiving a third message from a consumer network element, the third message being determined based on the model inference result of the first model provided by the second network element and used to indicate model inference quality feedback information corresponding to the first model; and determining to update the first model based on the third message.

[0212] In some embodiments, the model inference quality feedback information includes at least one of the following: ground truth data corresponding to the model inference result; second information indicating that the target inference result is substandard, the target inference result including at least one of the model inference results of the first model; third information indicating at least one of the substandard model metric corresponding to the target inference result and the metric threshold corresponding to the substandard model metric; fourth information indicating the current value of the substandard model metric corresponding to the target inference result, or indicating the difference between the current value of the substandard model metric corresponding to the target inference result and the metric threshold; twelfth information indicating relevant information about the model inference speed of the first model; and thirteenth information indicating that the model inference quality of the first model is substandard.

[0213] The device 800 provided in this application embodiment can execute the methods described in the preceding method embodiments and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0214] Figure 9 illustrates a schematic diagram of the hardware structure of the network-side device provided in the embodiments of this application. Referring to the figure, at the hardware level, the network-side device includes a processor and optionally, an internal bus, a network interface, and a memory. The memory may include RAM, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk storage device. Of course, the network-side device may also include other hardware required for other services.

[0215] The processor, network interface, and memory can be interconnected via an internal bus, which can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only a single bidirectional arrow is used in this diagram, but this does not imply that there is only one bus or one type of bus.

[0216] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0217] The processor reads the corresponding computer program from non-volatile memory into memory and then runs it, forming a device for locating a specific user at the logical level. The processor executes the program stored in memory and specifically performs the methods disclosed in the embodiments shown in Figures 2-4, achieving the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0218] The methods disclosed in the embodiments shown in Figures 2-4 of this application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0219] The network-side device can also execute the methods described in the preceding method embodiments and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0220] Of course, besides software implementation, the network-side device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices. This application also proposes a computer-readable storage medium that stores one or more programs. When the one or more programs are executed by a network-side device including multiple applications, the network-side device performs the methods disclosed in the embodiments shown in Figures 2-4 and achieves the functions and beneficial effects of the methods described in the foregoing method embodiments, which will not be repeated here.

[0221] The computer-readable storage medium includes read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc.

[0222] This application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, implement the following process: the method disclosed in the embodiments shown in Figures 2-4, and achieve the functions and beneficial effects of the methods described in the preceding method embodiments, which will not be repeated here.

[0223] Computer-readable storage media include both permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media do not include transient media, such as modulated data signals and carrier waves.

[0224] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0225] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0226] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0227] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A model update method, comprising: The first network element receives a first message from the second network element. The first message is used to request an update of the first model and includes information about the reason for the model update. The reason for the model update includes at least one of the reasons for model inference quality and model compatibility. The first network element updates the first model based on the first message.

2. The method as described in claim 1, wherein, The relevant information regarding the reasons for the quality of the model inference includes at least one of the following: The first information is relevant information used to indicate the model inference speed of the first model; The second piece of information is used to indicate that the target reasoning result has not met the standard, and the target reasoning result includes at least one of the model reasoning results of the first model; The third piece of information is used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric. The fourth piece of information is used to indicate the current value of the unmet model metric corresponding to the target inference result, or to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold. The fifth piece of information is used to indicate that the model measurement indicators when the terminal assists the second network element in model inference have not met the standards; The sixth piece of information is used to indicate that the model metric indicators when the access network equipment assists the second network element in model inference have not met the standards. Inference data source information, wherein the data source is used to provide the inference data required for the first model to perform inference; The inference data used by the first model.

3. The method as described in claim 2, wherein, The model metrics include at least one of the following: model accuracy, model precision, model recall, model F1 score, mean squared error of the model inference results, and mean absolute percentage error of the model inference results.

4. The method of claim 1, wherein, The relevant information regarding the reasons for model compatibility includes at least one of the following: The seventh piece of information is used to indicate that the second network element does not support the type of the first model; The eighth piece of information is used to indicate that the second network element does not support the model operating environment corresponding to the first model; The ninth piece of information is used to indicate the file size information of the model file of the first model; The local environment information of the second network element.

5. The method of claim 1, wherein, The first network element updates the first model according to the first message, including: The first network element performs at least one of the following actions based on the first message: Retrain the first model; Update the first model to the second model; Determine whether to update or not update the inference data source of the first model.

6. The method according to any one of claims 1-5, wherein, The method further includes: The first network element sends a second message to the second network element; The second message includes at least one of the following: Updated model files; The updated model file's access address; The updated model file corresponds to the inference data source; The tenth piece of information is used to indicate whether the inference data source of the first model should be updated; The eleventh piece of information is used to indicate the inference data source information corresponding to the re-collection of inference data.

7. The method according to any one of claims 1-5, wherein, The first network element includes a model training function; The second network element includes a consumer network element or a model inference function.

8. A model update method, comprising: The second network element determines the update of the first model; The second network element sends a first message to the first network element; The first message is used to request an update of the first model, and the first message includes information about the reason for the model update, which includes at least one of the reasons for model inference quality and model compatibility.

9. The method of claim 8, wherein, The relevant information regarding the reasons for the quality of the model inference includes at least one of the following: The first information is relevant information used to indicate the model inference speed of the first model; The second piece of information is used to indicate that the target reasoning result has not met the standard, and the target reasoning result includes at least one of the model reasoning results of the first model; The third piece of information is used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric. The fourth piece of information is used to indicate the current value of the unmet model metric corresponding to the target inference result, or to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold. The fifth piece of information is used to indicate that the model measurement indicators when the terminal assists the second network element in model inference have not met the standards; The sixth piece of information is used to indicate that the model metric indicators when the access network equipment assists the second network element in model inference have not met the standards. Inference data source information, wherein the data source is used to provide the inference data required for the first model to perform inference; The inference data used by the first model.

10. The method of claim 9, wherein, The model metrics include at least one of the following: model accuracy, model precision, model recall, model F1 score, mean squared error of the model inference results, and mean absolute percentage error of the model inference results.

11. The method of claim 8, wherein, The relevant information regarding the reasons for model compatibility includes at least one of the following: The seventh piece of information is used to indicate that the second network element does not support the type of the first model; The eighth piece of information is used to indicate that the second network element does not support the model operating environment corresponding to the first model; The ninth piece of information is used to indicate the file size information of the model file of the first model; The local environment information of the second network element.

12. The method of claim 8, wherein, The method further includes: The second network element receives a second message from the first network element; The second message includes at least one of the following: Updated model files; The updated model file's access address; The updated model file corresponds to the inference data source; The tenth piece of information is used to indicate whether the inference data source of the first model should be updated; The eleventh piece of information is used to indicate the inference data source information corresponding to the re-collection of inference data.

13. The method of claim 8, wherein, The second network element determines to update the first model, including: The second network element determines that the local configuration information has changed; The second network element determines to update the first model.

14. The method of claim 8, wherein, The second network element determines to update the first model, including: The second network element receives a third message from the consumer network element. The third message is determined based on the model inference result of the first model provided by the second network element and is used to indicate the model inference quality feedback information corresponding to the first model. The second network element determines to update the first model based on the third message; The second network element is the model inference function.

15. The method of claim 14, wherein, The model inference quality feedback information includes at least one of the following: The truth data corresponding to the model inference results; The second piece of information is used to indicate that the target reasoning result has not met the standard, and the target reasoning result includes at least one of the model reasoning results of the first model; The third piece of information is used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric. The fourth piece of information is used to indicate the current value of the unmet model metric corresponding to the target inference result, or to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold. The twelfth piece of information is related to information indicating the model inference speed of the first model; The thirteenth piece of information is used to indicate that the model inference quality of the first model is not up to standard.

16. The method according to any one of claims 8-13, wherein, The first network element includes a model training function; The second network element includes a consumer network element or a model inference function.

17. A model updating apparatus, comprising: The transmission module is used to receive a first message from a second network element. The first message is used to request an update of the first model, and the first message includes relevant information about the reason for the model update. The reason for the model update includes at least one of the reasons for model inference quality and model compatibility. The processing module is used to update the first model based on the first message.

18. The apparatus of claim 17, wherein, The relevant information regarding the reasons for the quality of the model inference includes at least one of the following: The first information is relevant information used to indicate the model inference speed of the first model; The second piece of information is used to indicate that the target reasoning result has not met the standard, and the target reasoning result includes at least one of the model reasoning results of the first model; The third piece of information is used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric. The fourth piece of information is used to indicate the current value of the unmet model metric corresponding to the target inference result, or to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold. The fifth piece of information is used to indicate that the model measurement indicators when the terminal assists the second network element in model inference have not met the standards; The sixth piece of information is used to indicate that the model metric indicators when the access network equipment assists the second network element in model inference have not met the standards. Inference data source information, wherein the data source is used to provide the inference data required for the first model to perform inference; The inference data used by the first model.

19. The apparatus of claim 18, wherein, The model metrics include at least one of the following: model accuracy, model precision, model recall, model F1 score, mean squared error of the model inference results, and mean absolute percentage error of the model inference results.

20. The apparatus of claim 17, wherein, The relevant information regarding the reasons for model compatibility includes at least one of the following: The seventh piece of information is used to indicate that the second network element does not support the type of the first model; The eighth piece of information is used to indicate that the second network element does not support the model operating environment corresponding to the first model; The ninth piece of information is used to indicate the file size information of the model file of the first model; The local environment information of the second network element.

21. The apparatus of claim 17, wherein, Updating the first model based on the first message includes: Perform at least one of the following based on the first message: Retrain the first model; Update the first model to the second model; Determine whether to update or not update the inference data source of the first model.

22. The apparatus according to any one of claims 17-21, wherein, The transmission module is also used to send a second message to the second network element; The second message includes at least one of the following: Updated model files; The updated model file's access address; The updated model file corresponds to the inference data source; The tenth piece of information is used to indicate whether the inference data source of the first model should be updated; The eleventh piece of information is used to indicate the inference data source information corresponding to the re-collection of inference data.

23. A model updating device, comprising: The processing module is used to determine whether to update the first model; The transmission module is used to send the first message to the first network element; The first message is used to request an update of the first model, and the first message includes information about the reason for the model update, which includes at least one of the reasons for model inference quality and model compatibility.

24. The apparatus of claim 23, wherein, The relevant information regarding the reasons for the quality of the model inference includes at least one of the following: The first information is relevant information used to indicate the model inference speed of the first model; The second piece of information is used to indicate that the target reasoning result has not met the standard, and the target reasoning result includes at least one of the model reasoning results of the first model; The third piece of information is used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric. The fourth piece of information is used to indicate the current value of the unmet model metric corresponding to the target inference result, or to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold. The fifth piece of information is used to indicate that the model measurement indicators when the terminal assists the second network element in model inference have not met the standards; The sixth piece of information is used to indicate that the model measurement indicators when the access network equipment assists the second network element in model inference have not met the standards. Inference data source information, wherein the data source is used to provide the inference data required for the first model to perform inference; The inference data used by the first model.

25. The apparatus of claim 24, wherein, The model metrics include at least one of the following: model accuracy, model precision, model recall, model F1 score, mean squared error of the model inference results, and mean absolute percentage error of the model inference results.

26. The apparatus of claim 23, wherein, The relevant information regarding the reasons for model compatibility includes at least one of the following: The seventh piece of information is used to indicate that the second network element does not support the type of the first model; The eighth piece of information is used to indicate that the second network element does not support the model operating environment corresponding to the first model; The ninth piece of information is used to indicate the file size information of the model file of the first model; The local environment information of the second network element.

27. The apparatus of claim 23, wherein, The transmission module is also used to receive a second message from the first network element; The second message includes at least one of the following: Updated model files; The updated model file's access address; The updated model file corresponds to the inference data source; The tenth piece of information is used to indicate whether the inference data source of the first model should be updated; The eleventh piece of information is used to indicate the inference data source information corresponding to the re-collection of inference data.

28. The apparatus of claim 23, wherein, The determination to update the first model includes: It has been confirmed that local configuration information has changed; It was determined that the first model would need to be updated.

29. The apparatus of claim 23, wherein, The determination to update the first model includes: Receive a third message from a consumer network element, wherein the third message is determined based on the model inference result of the first model provided by the second network element and is used to indicate the model inference quality feedback information corresponding to the first model; The third message determines that the first model needs to be updated.

30. The apparatus of claim 29, wherein, The model inference quality feedback information includes at least one of the following: The truth data corresponding to the model inference results; The second piece of information is used to indicate that the target reasoning result has not met the standard, and the target reasoning result includes at least one of the model reasoning results of the first model; The third piece of information is used to indicate at least one of the unmet model metric corresponding to the target inference result and the metric threshold corresponding to the unmet model metric. The fourth piece of information is used to indicate the current value of the unmet model metric corresponding to the target inference result, or to indicate the difference between the current value of the unmet model metric corresponding to the target inference result and the metric threshold. The twelfth piece of information is related to information indicating the model inference speed of the first model; The thirteenth piece of information is used to indicate that the model inference quality of the first model is not up to standard.

31. A network-side device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1-16.

32. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1-16.

Images