Model storage management method and apparatus

Abstract

The present application relates to the technical field of communications, and provides a model storage management method and apparatus, which are used for reducing storage requirements. In the method, a model operation functional entity may determine, on the basis of a use value of a first AI model, whether to store the first AI model. For example, the model operation functional entity may determine not to store the first AI model when the use value of the first AI model is low, or the model operation functional entity may determine to store the first AI model when the use value of the first AI model is high. Thus, an AI model having a high probability of being subsequently used can be stored, and an AI model having a low probability of being subsequently used may not be stored, so that storage requirements are reduced, storage efficiency is improved, and the waste of storage resources is avoided.

Description

Model storage management method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 14, 2023, with application number 202311728664.5 and application name “Model Storage Management Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communications, and in particular to a model storage management method and device. Background Art

[0003] Artificial intelligence (AI) can imbue machines with human-like intelligence, for example, by enabling them to simulate certain intelligent human behaviors using computer hardware and software. To implement AI, machine learning (ML) methods can be used to obtain AI models. AI models can be used for inference—that is, to derive output data corresponding to given input data. This enables certain service functions, such as specialized communications services.

[0004] However, as AI models are used, their performance degrades over time, requiring regular retraining and updating. This results in a large number of models being stored in the model library, placing high storage requirements. Summary of the Invention

[0005] Embodiments of the present application provide a model storage management method and apparatus to reduce storage requirements.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a model storage management method is provided, the method comprising: a model operation functional entity receives a first message from a model training functional entity, and determines whether to store a first artificial intelligence (AI) model based on the first message; wherein the first message indicates the use value of the first AI model, and the use value of the first AI model is positively correlated with the probability of storing the first AI model.

[0008] Based on the first aspect, since the use value of an AI model can represent the probability that the AI ​​model may be used in the future, the AI ​​model can be selectively stored according to the use value of the AI ​​model, so that the AI ​​model with a high probability of being used in the future can be stored, while the AI ​​model with a low probability of being used will not be stored, thereby reducing storage requirements, improving storage efficiency, and avoiding wasting storage resources.

[0009] In one possible design, the use value of a first AI model is represented by at least one piece of information about the first AI model: deployment scope, model accuracy, change in accuracy after model updates, or the number of training samples. This allows the use value of an AI model to be evaluated from multiple dimensions. This considers not only the generalizability of the AI ​​model but also its performance, enabling a more comprehensive assessment of the use value of the AI ​​model.

[0010] Optionally, the first message includes at least one item of information about the first AI model, such as the deployment scope, model accuracy, accuracy change after a model update, or the number of samples used for training. In other words, the model operation functional entity needs to calculate the use value of the first AI model independently, while the model training functional entity does not need to calculate the use value of the first AI model, thereby reducing the computational overhead of the model training functional entity.

[0011] Furthermore, the method described in the first aspect further includes: the model operation functional entity sending a second message to the model training functional entity, wherein the second message instructs the model operation functional entity to report the at least one item of information indicating the use value of the updated AI model upon obtaining the updated AI model. In other words, the model operation functional entity can instruct the model training functional entity to report the at least one item of information indicating the use value of the updated AI model via the second message, thereby achieving dynamic evaluation and updating of the use value.

[0012] Furthermore, the second message includes a model value assessment factor, which represents at least one of the above-mentioned information on the use value of the updated AI model, such as the deployment scope, model accuracy, change in accuracy after the model update, or the number of samples used for training. In other words, the model value assessment factor can be used to explicitly indicate the parameters that need to be reported, so as to dynamically adjust the reported parameters according to actual needs. Of course, it can also be implicitly indicated through the message itself.

[0013] Furthermore, the model operation functional entity determines whether to store the first AI model based on the first message, including: the model operation functional entity determines the usage value of the first AI model based on at least one item of information of the first AI model, and determines whether to store the first AI model based on the usage value of the first AI model. It will be understood that the model operation functional entity may determine the usage value of the first AI model by performing a weighted summation of the at least one item of information of the first AI model.

[0014] In one possible design, the use value of a first AI model is determined by taking a weighted sum of at least one of the following pieces of information about the first AI model: deployment scope, model accuracy, change in accuracy after model updates, or the number of training samples. This allows the use value of an AI model to be evaluated from multiple dimensions. This considers not only the generalizability of the AI ​​model but also its performance, enabling a more comprehensive assessment of its use value. Furthermore, this weighted summation approach accurately determines the use value of the AI ​​model.

[0015] Optionally, the first message includes the use value of the first AI model. In other words, the first message can directly carry the use value of the first AI model. In other words, the model training functional entity needs to calculate the use value of the first AI model independently, while the model operation functional entity does not need to calculate the use value of the first AI model, thereby reducing the computational overhead of the model operation functional entity.

[0016] Furthermore, the method described in the first aspect further includes: the model operation functional entity sending a third message to the model training functional entity, wherein the third message instructs the model operation functional entity to report the use value of the updated AI model upon obtaining the updated AI model, where the use value of the updated AI model is determined by weighted summing at least one of the following information about the updated AI model. In other words, the model operation functional entity can instruct the model training functional entity to report the use value of the updated AI model via the third message, thereby achieving dynamic evaluation and updating of the use value.

[0017] Furthermore, the third message includes at least one of the following: a model value evaluation factor, a weight of the model value evaluation factor, or an evaluation method, wherein the model value evaluation factor represents at least one of the following information of the updated AI model, the weight represents the weight corresponding to each of the at least one of the following information of the updated AI model, and the evaluation method represents the determination of the use value of the updated AI model by weighted summation. In this way, the model value evaluation factor, the weight of the model value evaluation factor, and the evaluation method can be used to explicitly indicate how to determine the use value of the AI ​​model, so as to realize a method for dynamically adjusting the use value of the AI ​​model according to actual needs. Of course, the method for determining the use value of the AI ​​model can also be implicitly indicated by the message itself.

[0018] Furthermore, the model operation functional entity determines whether to store the first AI model based on the first message, including: the model operation functional entity determines whether to store the first AI model based on the use value of the first AI model.

[0019] Furthermore, the model operation functional entity determines whether to store the first AI model based on the use value of the first AI model, including: if the use value of the first AI model is greater than or equal to a value threshold, the model operation functional entity determines to store the first AI model; if the use value of the first AI model is less than the value threshold, the model operation functional entity determines not to store the first AI model. It will be understood that the use value of the first AI model can represent the probability of the first AI model being used again in the future. When the use value of the first AI model is greater than or equal to the value threshold, it can indicate that the probability of the first AI model being used again in the future is high, and in this case, the first AI model can be stored; when the use value of the first AI model is less than the value threshold, it can indicate that the probability of the first AI model being used again in the future is low, and in this case, the first AI model can be not stored. In this way, whether to store the first AI model can be quickly and accurately determined.

[0020] Furthermore, the model operation functional entity is a model repository functional entity, and the method of the first aspect further includes: the model repository functional entity receiving a value threshold from the model management functional entity. In other words, the model repository functional entity can obtain the value threshold from the model management functional entity. It is understood that the value threshold can also be preset, predefined by a protocol, or obtained from another functional entity. The specific setting can be based on actual circumstances and is not limited thereto.

[0021] In a possible design scheme, the method described in the first aspect further includes: when it is determined that the first AI model is to be stored, the model operation functional entity sends a fourth message to the model training functional entity, and the fourth message indicates that the first AI model is successfully stored; when it is determined that the first AI model is not to be stored, the model operation functional entity sends a fifth message to the model training functional entity, and the fifth message indicates that the first AI model fails to be stored. That is, after determining whether to store the first AI model, the model operation functional entity can feedback the storage status of the first AI model to the model training functional entity. In this way, the model training functional entity can determine subsequent operations based on the storage status of the first AI model, such as whether to fall back to using the first AI model in the future.

[0022] In a possible design solution, the first message is a model update request message. That is, the first message can adopt a message in the prior art to reduce the difficulty of implementation.

[0023] In one possible design, the first AI model is used to implement specific communication service functions. It is understood that specific communication service functions may include: fault prediction, service type / mode prediction, user location prediction, service experience prediction, interference prediction, network key performance indicator prediction, coverage problem analysis, energy saving optimization, mobility performance analysis, handover optimization, etc. Implementing the above-mentioned predictions or analyses through the first AI model can achieve proactive network management and control optimization, effectively improve network operation and maintenance efficiency and network resource utilization efficiency, and optimize communication services.

[0024] In a possible design scheme, the model operation functional entity is a model library functional entity, and the method described in the first aspect also includes: the model library functional entity sends a sixth message to the model management functional entity, the sixth message indicates the model storage status information of the model library functional entity, and the model storage status information includes at least one of the following: the storage utilization rate of the storage space of the model library functional entity, the storage time of each AI model stored by the model library functional entity, the storage version data of each AI model stored by the model library functional entity, or the use value of each AI model stored by the model library functional entity. In this way, the model management functional entity can update various information and / or methods for determining the use value of the AI ​​model, such as model value factors, weights, evaluation methods, etc., based on the model storage status information, so that the use value of the AI ​​model can be determined more accurately.

[0025] In a second aspect, a model storage management method is provided, which includes: a model training functional entity obtains a first message and sends a first message to a model operation functional entity, the first message indicating the use value of a first artificial intelligence AI model, and the use value of the first AI model is positively correlated with the probability of storage of the first AI model.

[0026] In one possible design scheme, the use value of the first AI model is represented by at least one piece of information of the first AI model: deployment scope, model accuracy, change in accuracy after model update, or the number of samples used in training.

[0027] Optionally, the first message includes at least one item of information about the first AI model, such as deployment scope, model accuracy, accuracy change after model update, or the number of samples used for training.

[0028] Optionally, the method described in the first aspect also includes: the model training functional entity receives a second message from the model operation functional entity, the second message indicates that when the updated AI model is obtained, at least one of the above-mentioned information indicating the use value of the updated AI model is reported; the model training functional entity obtains the first message, including: the model training functional entity obtains the first message based on the second message, and the first AI model belongs to the updated AI model.

[0029] Furthermore, the second message includes a model value assessment factor, which represents at least one of the above information on the use value of the updated AI model.

[0030] In one possible design scheme, the usage value of the first AI model is determined by weighted summing at least one of the following information of the first AI model: deployment scope, model accuracy, change in accuracy after model update, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time.

[0031] Optionally, the first message includes the usage value of the first AI model.

[0032] Optionally, the method described in the first aspect also includes: the model training functional entity receives a third message from the model operation functional entity, the third message indicates that when the updated AI model is obtained, the use value of the updated AI model is reported, and the use value of the updated AI model is determined by weighted summing at least one of the following information of the updated AI model; the model training functional entity obtains the first message, including: the model training functional entity obtains the first message based on the third message, and the first AI model belongs to the updated AI model.

[0033] Furthermore, the third message includes at least one of the following: a model value evaluation factor, the weight of the model value evaluation factor, or an evaluation method. The model value evaluation factor represents at least one of the following information of the updated AI model. The weight represents the weight corresponding to each information in at least one of the following information of the updated AI model. The evaluation method represents determining the use value of the updated AI model by weighted summation.

[0034] In one possible design solution, the first message is a model update request message.

[0035] In one possible design scheme, the first AI model is used to implement specific communication service functions.

[0036] In one possible implementation, the method described in the second aspect also includes: the model training functional entity receives a fourth message or a fifth message from the model operation functional entity, the fourth message indicates that the first AI model is stored successfully, and the fifth message indicates that the first AI model fails to be stored.

[0037] In addition, the technical effects of the method described in the second aspect can also refer to the technical effects of the method described in the first aspect, and will not be repeated here.

[0038] According to a third aspect, a model storage management method is provided, which includes: a model operation functional entity determines to delete at least one AI model among the stored AI models; and the model operation functional entity determines to delete at least one first AI model among the stored AI models based on the usage value of each AI model among the stored AI models, wherein the usage value of each AI model is negatively correlated with the probability of deletion of each AI model.

[0039] Based on the third aspect, since the use value of an AI model can represent the probability that the AI ​​model may be used in the future, the AI ​​models to be deleted can be determined based on the use value of each stored AI model. This can avoid deleting AI models that are likely to be used in the future, which would waste training resources.

[0040] In one possible design, the use value of each AI model is determined by taking a weighted sum of at least one of the following pieces of information: deployment scope, model accuracy, change in accuracy after model updates, number of training samples, number of deployments, average inference accuracy, number of model updates, or average deployment time. This allows the use value of an AI model to be evaluated from multiple dimensions. This considers not only the generalizability of the AI ​​model but also its performance, enabling a more comprehensive assessment of its use value. Furthermore, this weighted summation approach accurately determines the use value of the AI ​​model.

[0041] Optionally, the model operation functional entity is a model library functional entity, and the method described in the third aspect further includes: the model library functional entity receives a first message from the model management functional entity, and the first message indicates that the use value of the AI ​​model is determined by weighted summing at least one of the following information of the AI ​​model, such as by the deployment scope, model accuracy, model accuracy change, or the number of samples used in training, the number of deployments, the average inference accuracy, the number of model updates, or the average deployment time. In other words, the model operation functional entity can indicate the method of determining the use value of each AI model through the first message to achieve dynamic evaluation and update of the use value.

[0042] Furthermore, the first message includes at least one of the following: a model value evaluation factor, a weight of the model value evaluation factor, or an evaluation method, wherein the model value evaluation factor represents at least one of the following information of the AI ​​model, the weight represents the weight corresponding to each of the at least one of the following information of the AI ​​model, and the evaluation method represents the determination of the use value of the AI ​​model by weighted summation. In this way, the model value evaluation factor, the weight of the model value evaluation factor, and the evaluation method can be used to explicitly indicate how to determine the use value of the AI ​​model, so as to realize a method for dynamically adjusting the use value of the AI ​​model according to actual needs. Of course, the method for determining the use value of the AI ​​model can also be implicitly indicated by the message itself.

[0043] In one possible design, the model operation functional entity is a model library functional entity. The model operation functional entity determines to delete at least one stored AI model, including: based on preset conditions, the model library functional entity determines to delete at least one stored AI model, the preset conditions being that the storage time of at least one stored AI model has reached a maximum storage time, and / or the number of stored AI models has reached a maximum storage capacity. In this way, AI models can be deleted in a timely manner to ensure sufficient storage space for new AI models.

[0044] In one possible design, the model operation functional entity determines to delete at least one stored AI model, including: the model management functional entity receives a fourth message from the model library function, the fourth message indicating that a preset condition is satisfied, the preset condition being that the storage time of at least one of the stored AI models has reached a maximum storage time and / or the number of stored AI models has reached a maximum storage number; and the model management functional entity determines to delete at least one of the stored AI models based on the fourth message. In this way, AI models can be deleted in a timely manner to ensure that there is storage space for new AI models.

[0045] In one possible design, the first AI model is the one with the lowest usage value among the stored AI models. It is understood that the usage value of an AI model can represent the probability of the first AI model being used again in the future. For example, the higher the usage value of an AI model, the higher its probability of being used again in the future; conversely, the lower the usage value of an AI model, the lower its probability of being used again in the future. This way, AI models with a higher probability of being used again in the future can be stored, avoiding wasting training resources.

[0046] In a fourth aspect, a model storage management method is provided, which includes: a model operation functional entity determines to delete a first artificial intelligence AI model, where the first AI model is a model corresponding to a first model version, and the first model version only stores one AI model; the model operation functional entity triggers model retraining based on the first AI model; the model operation functional entity triggers deletion of the first AI model and stores a second AI model, where the second AI model is an AI model obtained by model retraining.

[0047] Based on the fourth aspect, when only one AI model is stored in the first model version and it is determined that the first AI model should be deleted, model retraining can be performed to generate a second AI model, and the second AI model can be used to replace the first AI model. In other words, the first AI model is deleted and the second AI model is stored. This ensures that the AI ​​model corresponding to the first model version is always stored, avoiding the situation where the model corresponding to the first model version is unavailable and causing business errors.

[0048] In one possible design, the model operation functional entity triggers model retraining based on a first AI model, including: the model operation functional entity sends a first message to the model training functional entity, where the first message is used to request model retraining based on the first AI model. In other words, model retraining based on the first AI model can be triggered by sending a request message to the model training functional entity.

[0049] Optionally, the model operation functional entity is a model library functional entity. After the model operation functional entity sends a first message to the model training functional entity, the method described in the fourth aspect also includes: the model library functional entity receives a second message from the model training functional entity, and the second message is used to request the storage of a second AI model; the model operation functional entity triggers the deletion of the first AI model and stores the second AI model, including: the model library functional entity triggers the deletion of the first AI model and stores the second AI model according to the second message.

[0050] Furthermore, the method described in the fourth aspect further includes: the model repository functional entity sending a third message to the model training functional entity, wherein the third message indicates that the first AI model has been updated. In this way, the model training functional entity can avoid requesting the model repository functional entity to obtain the first AI model when rolling back, thereby avoiding unnecessary communication overhead.

[0051] In one possible design, the model operation functional entity is a model repository functional entity. The model operation functional entity determines to delete the first AI model, including: determining, based on a preset condition, that the first AI model's storage time has reached a maximum storage time. This allows for timely updates to the first AI model, ensuring normal business operations.

[0052] Optionally, the model operation functional entity is a model management functional entity, and the method described in the fourth aspect also includes: the model management functional entity receives a fourth message from the model training functional entity, and the fourth message indicates the second AI model that has been retrained; the model operation functional entity triggers the deletion of the first AI model and stores the second AI model, including: the model management functional entity sends a fifth message to the model library functional entity according to the fourth message, and the fifth message is used to trigger the deletion of the first AI model and store the second AI model.

[0053] Furthermore, the method described in aspect 4 further includes: the model management functional entity receiving a sixth message from the model repository functional entity, the sixth message indicating that the first AI model has been deleted and the second AI model has been stored. That is, after deleting the first AI model and storing the second AI model, the model repository functional entity may provide feedback to the model management functional entity indicating that the model update has been completed. This facilitates the model management functional entity to perform subsequent operations based on the sixth message, such as deploying the second AI model.

[0054] In one possible design, the model operation functional entity is a model management functional entity. The model operation functional entity determines to delete the first AI model by: the model management functional entity receives a seventh message from the model repository functional entity, the seventh message indicating that a preset condition has been met, where the preset condition is that the storage time of the first AI model has reached a maximum storage time; and the model management functional entity determines to delete the first AI model based on the seventh message. This allows for timely updates to the first AI model, ensuring normal business operations.

[0055] In a fifth aspect, a model storage management method is provided, which includes: a model operation functional entity receives a first message from a first model training functional entity, the first message indicates the use value of the first AI model, and the use value of the first AI model is negatively correlated with the probability of deletion of the first AI model; when the use value of the first AI model is less than or equal to the value threshold, the model operation functional entity triggers model retraining based on the first AI model; the model operation functional entity triggers deletion of the first AI model and stores a second AI model, where the second AI model is the AI ​​model obtained by model retraining.

[0056] Based on the fifth aspect, it can be seen that when the use value of the first AI model is less than or equal to the value threshold, the model operation functional entity can trigger model retraining based on the first AI model to obtain an AI model after model retraining, that is, a second AI model; and replace the first AI model with the second AI model. In this way, when the model version corresponding to the first AI model only stores one AI model, it can avoid the situation where the model library functional entity does not store the model corresponding to the model version after deleting the first AI model, resulting in no model being available. It can also ensure that there are sufficient AI models available in the model library functional entity when the model version corresponding to the first AI model stores multiple AI models.

[0057] In one possible design, the use value of a first AI model is represented by at least one piece of information about the first AI model: deployment scope, model accuracy, model accuracy variation, number of training samples, number of deployments, average inference accuracy, number of model updates, or average deployment time. This allows the use value of an AI model to be evaluated from multiple dimensions. This considers not only the generalizability of the AI ​​model but also its performance, enabling a more comprehensive assessment of the use value of the AI ​​model.

[0058] Optionally, the first message includes at least one item of information about the first AI model, such as deployment scope, model accuracy, accuracy change after model update, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time. In this way, the model operation functional entity needs to calculate the use value of the first AI model independently, while the model training functional entity does not need to calculate the use value of the first AI model based on the at least one item of information about the first AI model, thereby reducing the computational overhead of the model training functional entity.

[0059] Furthermore, the method described in the fifth aspect also includes: the model operation functional entity sending a second message to the first model training functional entity, wherein the second message instructs the model operation functional entity to report at least one item of information indicating the use value of the third AI model upon obtaining the updated AI model or after the model AI is deployed, where the third AI model is the updated AI model or the deployed AI model. In other words, the model operation functional entity can instruct the model training functional entity to report at least one item of information indicating the use value of the updated AI model or the deployed AI model through the second message, thereby achieving dynamic evaluation of the use value.

[0060] Furthermore, the second message includes a model value assessment factor, which represents at least one of the aforementioned information regarding the use value of the third AI model. In other words, the model value assessment factor can be used to explicitly indicate the parameters to be reported, enabling dynamic adjustment of the reported parameters based on actual needs. Of course, this can also be implicitly indicated in the message itself.

[0061] In one possible design, the use value of a first AI model is determined by taking a weighted sum of at least one of the following information about the first AI model: deployment scope, model accuracy, model accuracy variation, number of training samples, number of deployments, average inference accuracy, number of model updates, or average deployment time. In this way, the use value of an AI model can be evaluated from multiple dimensions. This not only considers the generalization of the AI ​​model but also its performance, thereby enabling a more comprehensive assessment of the use value of the AI ​​model. Furthermore, through this weighted summation, the use value of the AI ​​model can be accurately determined.

[0062] Optionally, the first message includes the use value of the first AI model. In other words, the first message can directly carry the use value of the first AI model. In other words, the model training functional entity needs to calculate the use value of the first AI model independently, while the model operation functional entity does not need to calculate the use value of the first AI model, thereby reducing the computational overhead of the model operation functional entity.

[0063] Furthermore, the method described in the fifth aspect also includes: the model operation functional entity sends a third message to the model training functional entity, wherein the third message instructs the model operation functional entity to report the use value of the third AI model when the updated AI model is obtained or after the AI ​​model is deployed. The use value of the third AI model is determined by weighted summing at least one of the following information of the third AI model, and the third AI model is the updated AI model or the deployed AI model. In other words, the model operation functional entity can instruct the model training functional entity to report the use value of the updated AI model or the deployed AI model through the third message, so as to achieve dynamic evaluation and update of the use value.

[0064] Furthermore, the third message includes at least one of the following: a model value evaluation factor, a weight of the model value evaluation factor, or an evaluation method, wherein the model value evaluation factor represents at least one of the following information of the third AI model, the weight represents the weight corresponding to each of the at least one of the following information of the third AI model, and the evaluation method represents determining the use value of the third AI model by weighted summing at least one of the following information of the third AI model. In this way, the model value evaluation factor, the weight of the model value evaluation factor, and the evaluation method can be used to explicitly indicate how to determine the use value of the AI ​​model, so as to realize a method for dynamically adjusting the use value of the AI ​​model according to actual needs. Of course, the method for determining the use value of the AI ​​model can also be implicitly indicated by the message itself.

[0065] In one possible design, when the value of the first AI model is less than or equal to a value threshold, the model operation functional entity triggers model retraining based on the first AI model, including: when the value of the first AI model is less than or equal to the value threshold, the model operation functional entity sends a fourth message to the second model training functional entity, where the fourth message is used to request model retraining based on the first AI model. That is, the model operation functional entity can trigger model retraining based on the first AI model by sending a request message to the second model training functional entity.

[0066] Optionally, the model operation functional entity is a model library functional entity. After the model operation functional entity sends a fourth message to the second model training functional entity, the method described in the fifth aspect also includes: the model library functional entity receives a fifth message from the second model training functional entity, and the fifth message is used to request storage of the second AI model; the model operation functional entity triggers deletion of the first AI model and stores the second AI model, including: the model library functional entity triggers deletion of the first AI model and stores the second AI model according to the fifth message.

[0067] Furthermore, the method described in aspect 5 further includes: the model repository functional entity sending a sixth message to the model management functional entity, where the sixth message indicates that the first AI model has been updated. In other words, after deleting the first AI model and storing the second AI model, the model repository functional entity may send the fifth message to the model management functional entity so that the model management functional entity can perform subsequent operations based on the fifth message, such as deploying the second AI model.

[0068] Optionally, the model operation functional entity is a model management functional entity. After the model operation functional entity sends a fourth message to the second model training functional entity, the method described in the fifth aspect further includes: the model management functional entity receives a seventh message from the second model training functional entity, the seventh message indicating that the model retraining based on the first AI model has been completed; the model operation functional entity triggers the deletion of the first AI model and stores the second AI model, including: the model management functional entity sends an eighth message to the model library functional entity based on the seventh message, the eighth message is used to trigger the deletion of the first AI model and the storage of the second AI model. That is, after the second model training functional entity completes the retraining, the model management functional entity can trigger the deletion of the first AI model and the storage of the second AI model through the eighth message.

[0069] Furthermore, the method described in aspect 5 further includes: the model management functional entity receiving a ninth message from the model repository functional entity, the ninth message indicating that the first AI model has been deleted and the second AI model has been stored. That is, after the model repository functional entity deletes the first AI model and stores the second AI model in accordance with the eighth message, it may indicate to the model management functional entity that the first AI model has been deleted and the second AI model has been stored. This facilitates the model management functional entity to perform subsequent operations based on the ninth message, such as deploying the second AI model.

[0070] In one possible design, after the model operation functional entity receives the first message from the first model training functional entity, the method described in aspect 5 further includes: the model operation functional entity updating the use value of the first AI model based on the first message. This ensures that the use value of the first AI model is up to date, facilitating subsequent operations based on the use value of the first AI model, such as determining whether to delete the first AI model based on the use value of the first AI model.

[0071] In one possible design, the first AI model is used to implement specific communication service functions. It is understood that specific communication service functions may include: fault prediction, service type / mode prediction, user location prediction, service experience prediction, interference prediction, network key performance indicator prediction, coverage problem analysis, energy saving optimization, mobility performance analysis, handover optimization, etc. Implementing the above-mentioned predictions or analyses through the first AI model can achieve proactive network management and control optimization, effectively improve network operation and maintenance efficiency and network resource utilization efficiency, and optimize communication services.

[0072] In the sixth aspect, a model storage management method is provided, which includes: the first model training functional entity obtains a first message, the first message indicates the use value of the first AI model, and the use value of the first AI model is negatively correlated with the probability of deletion of the first AI model; the first model training functional entity sends the first message to the model operation functional entity.

[0073] In one possible design scheme, the usage value of the first AI model is represented by at least one piece of information of the first AI model: deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time.

[0074] Optionally, the first message includes at least one item of information about the first AI model, such as deployment range, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time.

[0075] Optionally, the method described in aspect 6 also includes: the first model training functional entity receives a second message from the model operation functional entity, the second message indicates that when the updated AI model is obtained, or after the AI ​​model is deployed, at least one of the above-mentioned information indicating the use value of the third AI model is reported, and the third AI model is the updated AI model or the deployed AI model; the first model training functional entity obtains the first message, including: the first model training functional entity obtains the first message based on the second message, and the first AI model belongs to the third AI model.

[0076] Furthermore, the second message includes a model value assessment factor, which represents at least one of the above information on the use value of the third AI model.

[0077] In one possible design scheme, the usage value of the first AI model is determined by weighted summing at least one of the following information of the first AI model: deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time.

[0078] Optionally, the first message includes the usage value of the first AI model.

[0079] Optionally, the method described in aspect 6 also includes: the first model training functional entity receives a third message from the model operation functional entity, the third message indicates that when the updated AI model is obtained, or after the AI ​​model is deployed, the use value of the third AI model is reported, and the use value of the third AI model is determined by weighted summing at least one of the following information of the third AI model, and the third AI model is an updated AI model or an AI model that has been deployed; the first model training functional entity obtains the first message, including: the first model training functional entity obtains the first message based on the third message, and the first AI model belongs to the third AI model.

[0080] Furthermore, the third message includes at least one of the following: a model value evaluation factor, the weight of the model value evaluation factor, or an evaluation method. The model value evaluation factor represents at least one of the following information of the third AI model. The weight represents the weight corresponding to each information in at least one of the following information of the third AI model. The evaluation method represents determining the use value of the third AI model by weighted summing at least one of the following information of the third AI model.

[0081] In one possible design scheme, the first AI model is used to implement specific communication service functions.

[0082] In addition, the technical effects of the method described in the sixth aspect can also refer to the technical effects of the method described in the fifth aspect, and will not be repeated here.

[0083] In a seventh aspect, a model storage management method is provided, the method comprising: a model operation functional entity executes the method described in the first aspect, and a model training functional entity executes the method described in the second aspect.

[0084] In addition, the technical effects of the method described in the seventh aspect can also refer to the technical effects of the methods described in the first to second aspects, and will not be repeated here.

[0085] In an eighth aspect, a model storage management method is provided, the method comprising: a model operation functional entity executing the method described in the fifth aspect, and a model training functional entity executing the method described in the sixth aspect.

[0086] In addition, the technical effects of the method described in aspect 8 can also refer to the technical effects of the methods described in aspects 5 to 6, and will not be repeated here.

[0087] In a ninth aspect, a communication device is provided. The communication device includes: a module for executing the method described in any one of aspects 1 to 6, such as a transceiver module and a processing module. For example, the transceiver module is configured to indicate the transceiver function of the communication device, and the processing module is configured to perform functions of the communication device other than the transceiver function.

[0088] Optionally, the transceiver module may include a sending module and a receiving module, wherein the sending module is used to implement the sending function of the communication device described in the ninth aspect, and the receiving module is used to implement the receiving function of the communication device described in the ninth aspect.

[0089] Optionally, the communication device described in the ninth aspect may further include a storage module, wherein the storage module stores a program or instruction. When the processing module executes the program or instruction, the communication device may execute the method described in any one of the first to sixth aspects.

[0090] It can be understood that the communication device described in the ninth aspect can be a terminal or a network device, or a chip (system) or other parts or components that can be set in the terminal or the network device, or a device that includes the terminal or the network device. This application does not limit this.

[0091] In addition, the technical effects of the communication device described in the ninth aspect can refer to the technical effects of the method described in any one of the implementation methods of the first to sixth aspects, and will not be repeated here.

[0092] In a tenth aspect, a communication device is provided, comprising: a processor, wherein when the processor executes computer instructions, the communication device executes the method described in any possible implementation manner of the first to sixth aspects.

[0093] In one possible design solution, the communication device described in the tenth aspect may further include a transceiver. The transceiver may be a transceiver circuit or an interface circuit. The transceiver may be used for the communication device described in the tenth aspect to communicate with other communication devices.

[0094] In one possible design, the communication device described in aspect 10 may further include a memory. The memory may be integrated with the processor or provided separately. The memory may be used to store the computer program and / or data involved in the method described in any one of aspects 1 to 6.

[0095] In an embodiment of the present application, the communication device described in the tenth aspect may be the terminal or network device described in any one of the first to sixth aspects, or a chip (system) or other parts or components that can be set in the terminal or the network device, or a device that includes the terminal or the network device.

[0096] In addition, the technical effects of the communication device described in the tenth aspect can refer to the technical effects of the method described in any one of the implementation methods of the first aspect to the sixth aspect, and will not be repeated here.

[0097] In an eleventh aspect, a communication device is provided. The communication device includes: a processor coupled to a memory, the processor being configured to execute a computer program stored in the memory, so that the communication device performs the method described in any possible implementation of the first to sixth aspects.

[0098] In one possible design solution, the communication device described in the eleventh aspect may further include a transceiver. The transceiver may be a transceiver circuit or an interface circuit. The transceiver may be used for the communication device described in the eleventh aspect to communicate with other communication devices.

[0099] In an embodiment of the present application, the communication device described in the eleventh aspect may be the terminal or network device described in any one of the first to sixth aspects, or a chip (system) or other parts or components that may be set in the terminal or the network device, or a device that includes the terminal or the network device.

[0100] In addition, the technical effects of the communication device described in the eleventh aspect can refer to the technical effects of the method described in any one of the implementation methods of the first aspect to the sixth aspect, and will not be repeated here.

[0101] In the twelfth aspect, a communication device is provided, comprising: a processor and a memory; the memory is used to store a computer program, and when the processor executes the computer program, the communication device executes the method described in any one of the implementation methods of the first to sixth aspects.

[0102] In one possible design solution, the communication device described in aspect 12 may further include a transceiver. The transceiver may be a transceiver circuit or an interface circuit. The transceiver may be used for the communication device described in aspect 12 to communicate with other communication devices.

[0103] In an embodiment of the present application, the communication device described in aspect 12 may be the terminal or network device described in any one of aspects 1 to 6, or a chip (system) or other parts or components that may be provided in the terminal or the network device, or a device that includes the terminal or the network device.

[0104] In addition, the technical effects of the communication device described in the twelfth aspect can refer to the technical effects of the method described in any one of the implementation methods of the first aspect to the sixth aspect, and will not be repeated here.

[0105] In a thirteenth aspect, a communication chip is provided, in which instructions are stored. When the chip runs on a communication device, the method described in any one of the implementation methods of the first to sixth aspects is implemented.

[0106] In the fourteenth aspect, a communication chip is provided, comprising: a logic circuit and a communication interface, wherein the logic circuit is used to execute computer instructions, and the communication interface is used for the communication chip to communicate with other devices or chips, and when the logic circuit executes the computer instructions, the method described in any one of the implementation methods of the first to sixth aspects is implemented.

[0107] In the fifteenth aspect, a communication system is provided, which includes: a model operation functional entity for executing the method described in the first aspect, and / or a model training functional entity for executing the method described in the second aspect; or, a model operation functional entity for executing the method described in the fifth aspect, and / or a model training functional entity for executing the method described in the sixth aspect.

[0108] In the sixteenth aspect, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are run on a computer, the computer executes the method described in any possible implementation method of the first to sixth aspects.

[0109] In the seventeenth aspect, a computer program product is provided, comprising a computer program or instructions, which, when executed on a computer, enables the computer to execute the method described in any one of the possible implementations of aspects one to six. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] FIG1 is a schematic diagram of the architecture of an open access network (O-RAN) provided in an embodiment of the present application;

[0111] FIG2 is a schematic diagram of an artificial intelligence (AI) workflow provided by an embodiment of the present application;

[0112] FIG3 is a schematic diagram of the architecture of a communication system according to an embodiment of the present application;

[0113] FIG4 is a second schematic diagram of the architecture of the communication system provided in an embodiment of the present application;

[0114] FIG5 is a third schematic diagram of the architecture of the communication system provided in an embodiment of the present application;

[0115] FIG6 is a fourth schematic diagram of the architecture of the communication system provided in an embodiment of the present application;

[0116] FIG7 is a fifth schematic diagram of the architecture of the communication system provided in an embodiment of the present application;

[0117] FIG8 is a sixth schematic diagram of the architecture of the communication system provided in an embodiment of the present application;

[0118] FIG9 is a flowchart of a model storage management method according to an embodiment of the present application;

[0119] FIG10 is a second flow chart of the model storage management method provided in an embodiment of the present application;

[0120] FIG11 is a third flow chart of the model storage management method provided in an embodiment of the present application;

[0121] FIG12 is a fourth flow chart of the model storage management method provided in an embodiment of the present application;

[0122] FIG13 is a fifth flow chart of the model storage management method provided in an embodiment of the present application;

[0123] FIG14 is a sixth flow chart of the model storage management method provided in an embodiment of the present application;

[0124] FIG15 is a seventh flow chart of the model storage management method provided in an embodiment of the present application;

[0125] FIG16 is a flowchart of the model storage management method according to an embodiment of the present application;

[0126] FIG17 is a ninth flowchart of a model storage management method according to an embodiment of the present application;

[0127] FIG18 is a flowchart diagram 10 of a model storage management method provided in an embodiment of the present application;

[0128] FIG19 is a flowchart diagram 11 of the model storage management method provided in an embodiment of the present application;

[0129] FIG20 is a flowchart diagram 12 of the model storage management method provided in an embodiment of the present application;

[0130] FIG21 is a first structural diagram of a communication device provided in an embodiment of the present application;

[0131] FIG22 is a second structural diagram of the communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0132] For ease of understanding, the technical terms involved in the embodiments of this application are first introduced below.

[0133] 1. Open radio access network (O-RAN)

[0134] O-RAN aims to achieve an intelligent, open access network. It is based on the concepts of radio access network (RAN) interoperability and standardization, and provides unified interconnection standards for white-box hardware and software from different vendors. As shown in Figure 1, the O-RAN architecture divides the functions of the RAN node into multiple logical functional modules: the O-RAN control unit (O-CU), the O-RAN distributed unit (O-DU), the O-RAN radio unit (O-RU), the O-RAN central unit control plane (O-CU-CP), the O-RAN central unit user plane (O-CU-UP), and the O-RAN evolved NodeB (O-eNB). The O-CU implements the radio resource control (RRC), packet data convergence protocol (PDCP), and service data adaptation protocol (SDAP) layers, as specified in the 3GPP standard, as well as other control functions. O-DU is used to implement the radio link control (RLC) layer, media access control (MAC) layer, and upper layers of the physical layer in the 3rd generation partnership project (3GPP) standard; the high-layer functions of the physical layer include one or more of the following: forward error correction (FEC) encoding / decoding, scrambling / descrambling, or modulation / demodulation. O-RU is used to implement the low-layer (close to the radio frequency) functions and radio frequency functions of the physical layer (PHY) in the 3GPP standard. It can be understood that each logical function module is mainly implemented by software. O-CU-CP is used to implement the functions of the RRC layer and the control plane functions of the PDCP layer, which can also be called the O-RAN control unit control plane.The O-CU-UP implements the user plane functions of the service data adaptation protocol (SDAP) layer and the packet data convergence protocol (PDCP) layer, and is also called the O-RAN control unit user plane. The O-eNB implements the base station functions.

[0135] O-RAN can include a service management and orchestration framework (SMO), a non-real time RAN intelligent controller (Non-RT RIC), and a near-real time RAN intelligent controller (Near-RT RIC). The SMO's functions are similar to those of a network management system, and it is primarily responsible for the operation, maintenance, and management of each communication module in the O-RAN system. The Non-RT RIC is used to implement non-real-time intelligent management of RAN functions. It can implement workflows including model training and model updates, and can guide applications / functions in the Near-RT RIC based on policies. The Non-RT RIC can be set in the SMO. The Near-RT RIC is used to implement near-real-time intelligent management of the RAN. It can achieve near-real-time control and optimization of O-RAN modules and resources through data collection and related operations on the E2 interface.

[0136] It can be understood that O1, O2, A1, E1, E2, F1-c, F1-u, X2-c, X2-u, NG-u, Xn-u, Xn- and NG-c in Figure 1 are different interfaces, and you can refer to the existing technology and will not go into details here.

[0137] 2. AI

[0138] O-RAN incorporates AI. AI can imbue machines with human intelligence, for example, by enabling them to simulate certain intelligent human behaviors using computer hardware and software. To implement AI, machine learning (ML) methods can be employed. For example, a machine uses training data to learn or train an ML model, known as model training. The model obtained through model training can represent the mapping between input data and output data and can be used for inference, known as model inference. In other words, the model can be used to obtain output data corresponding to given input data. It is understood that the model obtained through model training can be an AI model, an ML model, or other model, and an AI model can include an ML model, meaning that an ML model is a type of AI model. An AI model can be considered a specific method for implementing an AI function, representing a mapping relationship or function between the input and output of the AI ​​model. This AI function, also known as an AI operation, can include at least one of the following: data collection, model training, model learning, model information release, model inference, model reasoning, or model prediction.

[0139] In O-RAN, key AI functions (i.e., model training and model inference) are primarily implemented by the Non-RT RIC and Near-RT RIC. Model training and model inference can be deployed in a variety of ways, including: deploying both model training and model inference in the SMO / Non-RT RIC; deploying model training in the SMO / Non-RT RIC and model inference in the Near-RT RIC; or deploying model training in both the SMO / Non-RT RIC and the Near-RT RIC, with model inference in the Near-RT RIC. The SMO / Non-RT RIC can be understood as the Non-RT RIC deployed in the SMO. The following describes the AI ​​workflow in O-RAN, using the examples of deploying model training in the Non-RT RIC and Near-RT RIC in the SMO, model inference in the Near-RT RIC, and ML.

[0140] As shown in Figure 2, the AI ​​workflow in O-RAN mainly includes the following 10 steps, namely S201-S210. They are introduced below.

[0141] S201, Non-RT RIC performs offline data collection and model training, and stores the trained model in the ML model repository functional entity.

[0142] S202, the model trained by the Non-RT RIC is downloaded to the Near-RT RIC.

[0143] S203, the ML model is deployed to the ML inference functional entity of the Near-RT RIC.

[0144] S204, the ML reasoning function entity of the Near-RT RIC collects reasoning data from the O-CU / O-DU.

[0145] S205 , the ML reasoning function entity of the Near-RT RIC performs reasoning operations based on the reasoning data.

[0146] S206, the ML reasoning function entity of the Near-RT RIC issues a control operation based on the reasoning result.

[0147] S207, the ML training functional entity of the Near-RT RIC collects online data from the O-CU / O-DU.

[0148] S208 , the ML training functional entity of the Near-RT RIC obtains reasoning performance feedback from the ML reasoning functional entity of the Near-RT RIC.

[0149] S209, the ML training functional entity of Near-RT RIC performs online model training based on online data and inference performance feedback.

[0150] At step S210, the ML training functional entity of the Near-RT RIC stores the online trained model in the ML model library functional entity. The online trained model can be understood as a model that has been updated for the ML model currently in use. The model can be a model trained based on the ML model currently in use or a retrained model.

[0151] It is understood that the aforementioned ML model library functional entity can be responsible for storing trained ML models; the aforementioned ML training functional entity can be responsible for training ML models and generating ML models after training; and the ML inference functional entity can be responsible for ML model inference, that is, inputting input data into the ML model to obtain output data. It is also understood that the ML training functional entity in the Non-RT RIC can train a basic model for use, and the ML training functional entity in the Near-RT RIC can update the basic model to generate an updated model.

[0152] As can be seen, after model training is completed, the model needs to be stored in the ML model library functional entity. That is, the aforementioned ML model library can be responsible for storing models trained with Non-RT RIC and Near-RT. Because the performance of the model will decline over time, the model needs to be retrained and updated regularly. This will cause the model library to store a large number of models, resulting in high storage requirements for the model library functional entity. In addition, some models stored in the model library functional entity may no longer be used, which will result in a waste of storage space.

[0153] In response to the above technical problems, the embodiments of the present application propose the following technical solutions to reduce storage requirements.

[0154] The technical solution in this application will be described below with reference to the accompanying drawings.

[0155] The technical solutions of the embodiments of the present application can be applied to various communication systems, such as 4th generation (4G) mobile communication systems, such as long term evolution (LTE) systems, fifth generation (5G) mobile communication systems, such as new radio (NR) systems, and communication systems evolved after 5G, such as sixth generation (6G) mobile communication systems. They can also be applied to wireless fidelity (WiFi) systems, vehicle to everything (V2X) communication systems, device-to-device (D2D) communication systems, and Internet of Vehicles communication systems.

[0156] This application presents various aspects, embodiments, or features in the context of systems that may include multiple devices, components, modules, and the like. It should be understood and appreciated that each system may include additional devices, components, modules, and the like, and / or may not include all of the devices, components, modules, and the like discussed in connection with the accompanying figures. Furthermore, combinations of these approaches may also be used. Furthermore, in the embodiments of this application, words such as "exemplary," "for example," and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described in this application as "exemplary" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. In the embodiments of this application, the terms "information," "signal," "message," "channel," and "signaling" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the intended meanings are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the intended meanings are the same. In addition, the “ / ” mentioned in this application can be used to represent an “or” relationship.

[0157] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field will know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0158] To facilitate understanding of the embodiments of the present application, a communication system applicable to the embodiments of the present application is first described in detail using the communication system shown in Figure 3 as an example. For example, Figure 3 is a schematic diagram of the architecture of a communication system applicable to the model storage management method provided in the embodiments of the present application.

[0159] As shown in Figure 3, the communication system includes: a model operation functional entity and a model training functional entity. It is understood that the "functional entity" mentioned in the embodiments of the present application is only an exemplary expression, and the "functional entity" can also be replaced by any possible expression, such as "network element", "module" or "logical function module", etc., without limitation.

[0160] The communication system can be used in different communication system architectures, such as the O-RAN architecture, the 3GPP architecture, or the mobile intelligent function (MIF) architecture. In different architectures, the model operation functional entity and the model training functional entity can be different functional entities, which are described below.

[0161] The above-mentioned communication system can be applied to the O-RAN architecture. As shown in Figures 4 and 5, the O-RAN architecture may include a model management functional entity, a first model training functional entity, a model library functional entity, a second model training functional entity, a model reasoning functional entity, and an O-CU / O-DU / O-RU. The first model training functional entity may be set in a Non-RT RIC, and the Non-RT RIC and the model management functional entity may be set in an SMO; the model library functional entity may be set in the Non-RT RIC or the Near-RT RIC; the second model training functional entity and the model reasoning functional entity may be set in the Near-RT RIC. The model management functional entity may be responsible for model lifecycle management, such as model training management and model update management; the model training functional entity may be responsible for training the model and generating the model after the model training is completed; the model library functional entity may be responsible for storing the trained model; and the model reasoning functional entity may be responsible for model reasoning, that is, inputting input data into the model to obtain output data. In this case, the model operation functional entity may be a model management functional entity or a model library functional entity; the model training functional entity may be a first model training functional entity or a second model training functional entity.

[0162] The above-mentioned communication system can also be applied to the 3GPP architecture. As shown in Figures 6 and 7, the 3GPP architecture may include a model management functional entity, a first model training functional entity, a model inference functional entity, a model repository functional entity, and a RAN. The model management functional entity can be located in a network management system (NMS); the first model training functional entity and the model repository functional entity can be located in an element management system (EMS); and the model inference functional entity can be located in either the EMS or the RAN. It is understood that the NMS can be responsible for network operation, management, and maintenance and can also be referred to as a cross-domain management system. The EMS can be used to manage at least one network element or functional entity of a certain category and can also be referred to as a domain management system or a single-domain management system. In a mobile communication system, the RAN can connect the fixed portion (such as core network equipment, transmission network equipment, base station equipment, etc.) with the wireless portion and can connect to terminals via wireless channels over the air. For details about each of the above-mentioned functional entities, please refer to the relevant descriptions above and will not be repeated here. In this case, the model operation functional entity can be the model management functional entity or the model repository functional entity; and the model training functional entity can be the first model training functional entity.

[0163] Optionally, the 3GPP architecture may further include a second model training functional entity, which may be provided in the NMS. In this case, the model training functional entity may also be the second model training functional entity.

[0164] The above-mentioned communication system can also be applied to the MIF architecture. As shown in Figure 8, the MIF architecture may include a model management functional entity, a first model training functional entity, a model reasoning functional entity, and a model library functional entity. Among them, the model management functional entity can be set in the MIF, and the first model training functional entity, the model reasoning functional entity and the model library functional entity can be set in the RAN. It can be understood that the MIF is the AI ​​intelligent business processing logic functional entity of the RAN network, which is used to realize the intelligent data analysis function of the RAN domain. The above-mentioned functional entities can refer to the above-mentioned relevant introduction and will not be repeated here. In this case, the model operation functional entity can be a model management functional entity or a model library functional entity; the model training functional entity can be a first model training functional entity.

[0165] Optionally, the MIF architecture may further include a second model training functional entity, which may be provided in the NMS. In this case, the model training functional entity may also be the second model training functional entity.

[0166] In the communication system, the model training functional entity can send a first message indicating the use value of the first AI model to the model operation functional entity. The model operation functional entity can then determine whether to store the first AI model based on the use value of the first AI model. For example, if the use value of the first AI model is high, the first AI model may be stored; if the use value of the first AI model is low, the first AI model may not be stored. This ensures that AI models with a high probability of being used in the future are stored, while AI models with a low probability of being used are not stored, thereby reducing storage requirements, improving storage efficiency, and avoiding wasting storage resources.

[0167] It can be understood that FIG3 is a simplified schematic diagram for ease of understanding, and the communication system may also include other network devices and / or other terminal devices, which are not shown in FIG3 .

[0168] The following will specifically introduce the interaction process between the functional entities in the above communication system through a method embodiment in conjunction with Figures 9 to 20. The model storage management method provided in the embodiment of the present application can be applied to the above communication system and is described in detail below.

[0169] Figure 9 is a flow chart of the model storage management method provided in an embodiment of the present application. The method can be applied to the interaction between the model operation function entity and the model training function entity in the above communication system.

[0170] As shown in Figure 9, the process of the model storage management method is as follows:

[0171] S901: The model training functional entity obtains a first message.

[0172] S902: The model training functional entity sends a first message to the model operation functional entity. Correspondingly, the model operation functional entity receives the first message from the model training functional entity.

[0173] S903: The model operation functional entity determines whether to store the first AI model based on the first message.

[0174] S901 to S903 are introduced below respectively.

[0175] For S901:

[0176] The first message may indicate the use value of the first AI model, and the first message may be a new message, or a reused existing message, such as a model update request message.

[0177] The first AI model can be used to implement specific communication service functions, such as fault prediction, service type / mode prediction, user location prediction, service experience prediction, interference prediction, network key performance indicators (KPI) prediction, coverage problem analysis, energy saving optimization, mobility performance analysis, and handover optimization. Based on these predictions or analyses, proactive network management and control optimization can be achieved, effectively improving network operation and maintenance efficiency and network resource utilization efficiency, and optimizing communication services.

[0178] The use value of a first AI model can represent the probability of the first AI model being subsequently reused. That is, the higher the use value of the first AI model, the higher the probability of its subsequent reuse; conversely, the lower the probability of the first AI model being subsequently reused. It can be understood that the higher the probability of the first AI model being subsequently reused, the more necessary it is to store the first AI model. In other words, the probability of the first AI model being subsequently reused is positively correlated with the probability of the first AI model being stored. In other words, the use value of the first AI model is positively correlated with the probability of the first AI model being stored. That is, the higher the use value of the first AI model, the greater the probability of the first AI model being stored.

[0179] It can be understood that the use value of the first AI model can be represented by information, or determined by weighted summation of various information, which are explained below.

[0180] Method 1: The use value of the first AI model can be represented by at least one piece of information about the first AI model: deployment scope, model accuracy, accuracy change after model update, or the number of samples used in training.

[0181] The deployment scope can represent the scope of model deployment. The deployment scope can be the number of network elements (NEs) where the model is deployed. These NEs can be those that are used for model inference and effectively function. It is understood that a larger scope, i.e., a greater number of deployed NEs, indicates greater model generalization, meaning the model can be applied to more NEs.

[0182] Model accuracy can represent the accuracy of model reasoning. Model accuracy can be determined based on training data after model training, such as initial training or retraining. Model accuracy can also be determined based on data generated during model use, such as after the model is deployed to a network element and has been in use for a period of time. It is understood that model accuracy can be determined based on actual circumstances without limitation.

[0183] The change in accuracy after the model update can represent the change in the accuracy of the model before the update and the accuracy of the model after the update, that is, the change in accuracy between the old model before the update and the new model after the update. The value of the accuracy change can be positive or negative. That is, when the accuracy of the model after the update is higher than the accuracy before the model update, the value of the accuracy change can be a positive number; when the accuracy of the model after the update is lower than the accuracy before the model update, the value of the accuracy change can be a negative number. The accuracy change can reflect the performance of the model, that is, when the value of the accuracy change is larger, the performance of the model is better; conversely, the performance of the model is worse. The accuracy change after the model update can also be called accuracy gain, or can be replaced by any possible name without limitation.

[0184] The number of training samples can be the number of training data samples used during model training. This number of samples can indicate the sufficiency of the training samples; that is, a higher number of samples indicates a greater sufficiency of the training samples. Generally, a greater sufficiency of training samples leads to better model performance, i.e., higher model accuracy.

[0185] It can be seen that the deployment range can characterize the generalization of the model, and the model accuracy, the change in accuracy after the model update, and the number of samples used in training can characterize the performance of the model. The generalization and performance of the model can both characterize the use value of the model, that is, the better the generalization and performance of the model, the greater the probability that the model will be used again in the future, and the higher the use value of the model. In other words, the use value of the model can be reflected by the above parameters. It can be understood that the above-mentioned model can be an AI model, such as the first AI model mentioned above, without limitation.

[0186] The first message may include at least one item of information about the first AI model, that is, the first message may include at least one item of information about the first AI model, such as deployment range, model accuracy, accuracy change after model update, or the number of samples used for training. In other words, the first message may directly carry at least one item of information about the first AI model. In this case, the model operation functional entity needs to calculate the use value of the first AI model by itself, while the model training functional entity does not need to calculate the use value of the first AI model, thereby reducing the computing overhead of the model training functional entity.

[0187] It can be understood that the model training functional entity can send a first message according to the instructions of the model operation functional entity, such as instructing that when the updated AI model is obtained, various information indicating the use value of the updated AI model is reported.

[0188] Specifically, before the model training functional entity obtains the first message, the above-mentioned model storage management method may also include: the model operation functional entity sends a second message to the model training functional entity, and accordingly, the model training functional entity receives the second message from the model operation functional entity, and the second message indicates that when the updated AI model is obtained, at least one of the above-mentioned information indicating the use value of the updated AI model is reported; the above-mentioned model training functional entity obtaining the first message may specifically include: the model training functional entity obtains the first message based on the second message, and the first AI model belongs to the updated AI model. In other words, the model operation functional entity can instruct the model training functional entity through the second message to report the information indicating the use value of the AI ​​model when the updated AI model is obtained, so that the model operation functional entity can determine whether to store the AI ​​model based on the information.

[0189] It can be understood that at least one item of information on the use value of the updated AI model is the same as at least one item of information on the first AI model mentioned above. For example, if at least one item of information on the use value of the updated AI model is the deployment scope and model accuracy, then the use value of the first AI model is represented by the deployment scope and model accuracy of the first AI model.

[0190] The second message may include a model value assessment factor, which represents at least one of the above-mentioned information on the use value of the updated AI model. The at least one information may include: deployment scope, model accuracy, change in accuracy after the model update, or the number of samples used for training. In other words, the model value assessment factor can be used to explicitly indicate the parameters that need to be reported, so as to dynamically adjust the reported parameters according to actual needs. Of course, it can also be implicitly indicated through the message itself.

[0191] It is understood that in this case, the model training functional entity can be used to update the AI ​​model. And the model training functional entity can be set in the Near-RT RIC.

[0192] In addition, the above content introduces that the model operation functional entity instructs the model training functional entity to obtain the first message through the second message. It can be understood that when the model operation functional entity is a model library functional entity, each piece of information indicated by the second message can be pre-set in the model library functional entity; or, each piece of information indicated by the second message can be predefined by agreement; or, each piece of information indicated by the second message can be obtained from the model management functional entity. Each piece of information indicated by the second message can be at least one of the above-mentioned information that is reported to indicate the use value of the updated AI model when the updated AI model is obtained. It can also be understood that there are many ways for the model library functional entity to obtain each piece of information from the model management functional entity, such as the model library functional entity sending a message for requesting each piece of information to the model management functional entity, and in the initial configuration stage, the model management functional entity first sends each piece of information to the model library functional entity.

[0193] Method 2: The usage value of the first AI model can be determined by weighted summing at least one of the following information of the first AI model: deployment scope, model accuracy, change in accuracy after model update, or number of samples used in training. The above-mentioned information can be referred to for details and will not be repeated here.

[0194] Weighted summation can be understood as setting different weight coefficients for different information, multiplying each weight coefficient by each piece of information to obtain a value, and then adding up the values ​​to obtain the usage value of the first AI model. The sum of the weight coefficients is 1.

[0195] For example, the weight coefficient corresponding to the model accuracy is w21, the weight coefficient corresponding to the deployment range is w22, the weight coefficient corresponding to the change in accuracy after the model update is w23, and the weight coefficient corresponding to the number of samples used in training is w24, w21+w22+w23+w24=1; at this time, the use value of the first AI model is w21×model accuracy+w22×deployment range+w23×change in accuracy after the model update+w14×number of samples used in training.

[0196] It can be understood that the weight coefficients in the weighted summation can be set according to actual conditions without limitation.

[0197] The first message may include the use value of the first AI model. In other words, the first message may directly carry the use value of the first AI model. In other words, the use value of the first AI model may be calculated by the model training functional entity. In this way, upon receiving the first message from the model training functional entity, the model operation functional entity does not need to calculate the use value of the first AI model, thereby reducing the computational overhead of the model operation functional entity.

[0198] It can be understood that the model training functional entity can send a first message according to the instructions of the model operation functional entity, such as instructing to report the usage value of the updated AI model when the updated AI model is obtained.

[0199] Specifically, before the model training functional entity obtains the first message, the above-mentioned model storage management method may also include: the model operation functional entity sends a third message to the model training functional entity, and accordingly, the model training functional entity receives the third message from the model operation functional entity, the third message indicating that when the updated AI model is obtained, the use value of the updated AI model is reported, and the use value of the updated AI model is determined by weighted summing at least one of the following information of the updated AI model; the above-mentioned model training functional entity obtaining the first message may specifically include: the model training functional entity obtains the first message according to the third message, and the first AI model belongs to the updated AI model. That is to say, the model operation functional entity can instruct the model training functional entity through the third message to report the use value of the AI ​​model when the updated AI model is obtained, so that the model operation functional entity can determine whether to store the AI ​​model based on the use value.

[0200] It can be understood that at least one of the following information of the use value of the updated AI model is the same as at least one of the following information of the first AI model mentioned above. For example, if at least one of the following information of the use value of the updated AI model is the deployment scope, model accuracy, and the number of samples used for training, then the use value of the first AI model is determined by the weighted sum of the deployment scope, model accuracy, and the number of samples used for training of the first AI model.

[0201] The third message may include at least one of the following: a model value evaluation factor, a weight of the model value evaluation factor, or an evaluation method.

[0202] The model value assessment factor may represent at least one of the following information about the updated AI model: deployment scope, model accuracy, change in accuracy after the model update, or the number of training samples. It is understood that the model value factor is the same as the at least one of the following information about the first AI model.

[0203] The weight of the model value assessment factor may include a weight coefficient corresponding to the model value assessment factor, or may represent the weight corresponding to each of at least one of the following information of the updated AI model, the at least one of which is: deployment range, model accuracy, change in accuracy after the model update, or the number of samples used for training. For example, when the information representing the updated AI model is deployment range, model accuracy, change in accuracy after the model update, and the number of samples used for training, the weight may include a weight coefficient corresponding to the deployment range, a weight coefficient corresponding to the model accuracy, a weight coefficient corresponding to the change in accuracy after the model update, and a weight coefficient corresponding to the number of samples used for training.

[0204] The evaluation method indicates that the use value of the updated AI model is determined by weighted summation. That is, the use value of the updated AI model can be obtained by weighted summation of the weights and the model value evaluation factors. For example, the model value factor includes the deployment scope, model accuracy, the change in accuracy after the model update, and the number of samples used for training. The weight of the model value evaluation factor includes the weight coefficient w31 corresponding to the deployment scope, the weight coefficient w32 corresponding to the model accuracy, the weight coefficient w33 corresponding to the change in accuracy after the model update, and the weight coefficient w34 corresponding to the number of samples used for training. w31+w32+w33+w34=1; at this time, the use value of the AI ​​model is w31×deployment scope+w32×model accuracy+w33×change in accuracy after the model update+w34×number of samples used for training.

[0205] It can be understood that the model operation functional entity can send the model value evaluation factor, the weight of the model value evaluation factor, or the evaluation method to the model training functional entity through a third message, so that the model training functional entity obtains the use value of the first AI model, that is, the first message, after the model is updated according to the third message. In this case, the model training functional entity can be used to update the AI ​​model. And the model training functional entity can be set in the Near-RT RIC.

[0206] In addition, the above content introduces that the model operation functional entity instructs the model training functional entity to obtain the first message through the third message. It can be understood that when the model operation functional entity is a model library functional entity, the various information indicated by the third message can be pre-set in the model library functional entity; or, the various information indicated by the third message can be predefined by agreement; or, the various information indicated by the third message can be obtained from the model management functional entity. The various information indicated by the third message can be the use value of the updated AI model reported when the updated AI model is obtained. It can also be understood that there are many ways for the model library functional entity to obtain various information from the model management functional entity, such as the model library functional entity sending a message for requesting various information to the model management functional entity, and the model management functional entity first sending various information to the model library functional entity in the initial configuration stage.

[0207] The first message may also include information indicating the first AI model and / or the first AI model. The information about the first AI model may include at least one of the following: an identifier of the first AI model, a task identifier, or a storage address of the first AI model. The task identifier corresponds to the first AI model, meaning that the first AI model can be identified by the task identifier. This clearly indicates that the first message refers to the usefulness of the first AI model, avoiding errors. Furthermore, this facilitates triggering the storage of the first AI model by the model operation functional entity.

[0208] For S902:

[0209] After obtaining the first message, the model training functional entity may send the first message to the model operation functional entity. At this time, the model operation functional entity may receive the first message from the model training functional entity.

[0210] For S903:

[0211] After receiving the first message, the model operation functional entity can determine whether to store the first AI model based on the first message. It can be understood that the aforementioned "S901" introduces that the use value of the first AI model can be represented by at least one item of information of the first AI model, or the use value of the first AI model is determined by weighted summing of at least one item of information of the first AI model. That is to say, the first message may include at least one item of information of the first AI model, or the first message may include the use value of the first AI model. When the first message includes at least one item of information of the first AI model, the model operation functional entity also needs to determine the use value of the first AI model based on at least one item of information of the first AI model; when the first message includes the use value of the first AI model, the model operation functional entity can directly use the use value of the first AI model, that is, there is no need to calculate the use value of the first AI model. The following are explained separately.

[0212] Regarding the above-mentioned method 1: the model operation functional entity determines whether to store the first AI model based on the first message, which can specifically include: the model operation functional entity determines the use value of the first AI model based on at least one item of information of the first AI model; the model operation functional entity determines whether to store the first AI model based on the use value of the first AI model.

[0213] It is understood that in this case, the model operation functional entity can obtain the weight coefficient corresponding to at least one item of information of the first AI model and can obtain the usage value of the first AI model through weighted summation. The operation of the model operation functional entity obtaining the usage value of the first AI model is similar to the operation of obtaining the usage value of the first AI model described in "S901" above. Please refer to the relevant description above and will not be repeated here.

[0214] Regarding the aforementioned approach 2: the model operation functional entity determining whether to store the first AI model based on the first message may specifically include: the model operation functional entity determining whether to store the first AI model based on the usage value of the first AI model. In this case, the model operation functional entity can directly use the usage value of the first AI model in the first message, i.e., there is no need to calculate the usage value of the first AI model, thereby reducing the computational overhead of the model operation functional entity.

[0215] After obtaining the use value of the first AI model based on the first message, the model operation functional entity may determine whether to store the first AI model based on the use value of the first AI model. Specifically, the model operation functional entity determining whether to store the first AI model based on the use value of the first AI model may include: determining to store the first AI model if the use value of the first AI model is greater than or equal to a value threshold; and determining not to store the first AI model if the use value of the first AI model is less than the value threshold.

[0216] The above-mentioned value threshold can be set according to actual conditions, and the embodiment of the present application does not limit the value of the value threshold. When the model operation functional entity is a model library functional entity, the model library functional entity can obtain the value threshold from the model management functional entity. Exemplarily, the model operation functional entity is a model library functional entity, and the above-mentioned model storage management method can also include: the model management functional entity sends the value threshold to the model library functional entity, and correspondingly, the model library functional entity receives the value threshold from the model management functional entity. It can be understood that the model management functional entity can actively send the value threshold to the model library functional entity, and can also send the value threshold to the model library functional entity after receiving the message from the model library functional entity for requesting the value threshold. The value threshold can also be pre-set, or pre-defined by the protocol, or sent to the model library functional entity by other functional entities. It can be set specifically according to actual conditions without limitation.

[0217] By comparing the usage value of the first AI model with a value threshold, the first AI model can be stored when the usage value of the first AI model is high, and not stored when the usage value of the first AI model is low. This reduces storage requirements and avoids wasting storage space.

[0218] In summary, in the embodiments of the present application, the model operation functional entity can determine whether to store the first AI model based on the usage value of the first AI model. For example, the first AI model is stored only when the usage value of the first AI model is greater than or equal to the value threshold, so that the AI ​​model that is likely to be used subsequently can be stored, while the AI ​​model that is unlikely to be used will not be stored, thereby reducing storage requirements, improving storage efficiency, and avoiding wasting storage resources.

[0219] Optionally, in combination with the above embodiment, the above model storage management method may further include: when it is determined that the first AI model is to be stored, the model operation function entity sends a fourth message to the model training function entity, and accordingly, the model training function entity receives the fourth message from the model operation function entity, and the fourth message indicates that the first AI model is successfully stored; when it is determined that the first AI model is not to be stored, the model operation function entity sends a fifth message to the model training function entity, and accordingly, the model training function entity receives the fifth message from the model operation function entity, and the fifth message indicates that the first AI model fails to be stored. In this way, the model training function entity can know whether the first AI model is stored, and avoid the model training function entity from obtaining the first AI model again from the model library function entity when the first AI model is not stored, thereby avoiding unnecessary communication overhead.

[0220] Optionally, in combination with the above embodiments, when the model operation functional entity is a model library functional entity, the above-mentioned model storage management method may also include: the model library functional entity sends a sixth message to the model management functional entity, and accordingly, the model management functional entity receives the sixth message from the model library functional entity, and the sixth message indicates the model storage status information of the model library functional entity, and the model storage status information includes at least one of the following: the storage utilization rate of the storage space of the model library functional entity, the storage time of each AI model stored by the model library functional entity, the storage version data of each AI model stored by the model library functional entity, or the use value of each AI model stored by the model library functional entity.

[0221] The storage utilization rate of the storage space of the above-mentioned model library functional entity can be the ratio of the number of stored models to the number of models that can be stored. For example, if the stored model data is 50 and the number of models that can be stored is 100, then the storage utilization rate is 50%. It can also be the ratio of the storage space occupied by the stored models to the total storage space. For example, if the storage space occupied by the stored models is 20% of the total storage space, then the storage utilization rate is 20%.

[0222] The storage time of the AI ​​model stored by the model library functional entity can be the storage time of each AI model stored by the model library functional entity, or it can be the maximum storage time of the model generated by Non-RT RIC stored by the model library functional entity, and / or the maximum storage time of the model generated by Near-RT RIC.

[0223] The storage version data of each AI model stored by the model library functional entity can be the number of stored versions corresponding to each AI model stored by the model library functional entity, or the total number of stored versions corresponding to each AI model.

[0224] The use value of each AI model stored in the model library functional entity can be the latest use value corresponding to each AI model stored in the model library functional entity, or it can be all the use values ​​corresponding to each AI model.

[0225] By sending the model storage status information to the model management functional entity, the model management functional entity can update the various information and / or methods for determining the use value of the model based on the model storage status information, such as the above-mentioned model value factor, weight, or evaluation method. In this way, the determined use value of the model can be made more accurate. It can be understood that when the model operation functional entity is a model library functional entity, after the model management functional entity updates the various information and / or methods for determining the use value of the model based on the model storage status information, the model management functional entity can send the updated various information and / or methods for determining the use value of the model to the model library functional entity to accurately determine the use value of the AI ​​model.

[0226] Scenario 1:

[0227] Figure 10 is a second flow diagram of the model storage management method provided in an embodiment of the present application. This method is applicable to the aforementioned communication system and primarily involves interaction between the model repository functional entity and the first model training functional entity. In scenario 1, the first model training functional entity can send message #9 to the model repository functional entity, indicating the usefulness of model #2. The model repository functional entity can then determine whether to store model #2 based on message #9.

[0228] As shown in Figure 10, the process of the model storage management method is as follows:

[0229] S1001: The model management functional entity sends a message #1 to the model repository functional entity. Correspondingly, the model repository functional entity receives the message #1 from the model management functional entity.

[0230] Message #1 may indicate a model value assessment factor, the weight of the model value assessment factor, and an assessment method; the model value assessment factor represents at least one of the following information about the AI ​​model, such as the deployment scope, model accuracy, accuracy change after model update, or the number of samples used for training. For details, please refer to the aforementioned "S901" related introduction and will not be repeated here.

[0231] After receiving message #1, the model library functional entity can determine the method of obtaining the usage value of the AI ​​model based on message #1.

[0232] S1002: The model repository functional entity sends a message #2 to the first model training functional entity. Correspondingly, the first model training functional entity receives the message #2 from the model repository functional entity.

[0233] Message #2 may be the aforementioned second message or third message. For details, please refer to the aforementioned related introduction and will not be repeated here. The first model training functional entity may be a model training functional entity provided in the Near-RT RIC, and the first model training functional entity is the model training functional entity in the embodiment shown in FIG9 . By sending message #2, the first model training functional entity may be instructed to report information on the use value of the updated model after the model is updated.

[0234] It can be understood that message #2 can also be sent by the model management functional entity to the first model training functional entity. The specific settings can be made according to actual conditions without any restrictions.

[0235] S1003: The second model training functional entity collects offline data and performs model training.

[0236] The second model training functional entity may be a model training functional entity in a non-RT RIC. The second model training functional entity may obtain Model #1 through model training. Model #1 may be an AI model or an ML model. The operations of collecting offline data and performing model training by the second model training functional entity may refer to existing technologies and will not be further described here.

[0237] S1004: The second model training functional entity sends message #3 to the model library functional entity. Correspondingly, the model library functional entity receives message #3 from the second model training functional entity.

[0238] Message #3 can be used to request the storage of model #1. That is, message #3 can be a model storage request message. Message #3 can carry the identifier of model #1 and / or model #1, so that the model repository functional entity stores model #1 after receiving message #3. The specific implementation of S1004 can refer to existing technologies and will not be further described here.

[0239] S1005: The second model training function entity sends message #4 to the model management function entity. Correspondingly, the model management function entity receives message #4 from the second model training function entity.

[0240] Message #4 can indicate that model #1 training is complete. This means that message #4 can be a model training completion notification message. By sending message #4, the model management function entity can obtain the current status of model #1, facilitating subsequent operations such as sending a request to deploy model #1.

[0241] S1006: The model management function entity sends message #5 to the first model training function entity. Correspondingly, the first model training function entity receives message #5 from the model management function entity.

[0242] Message #5 instructs the first model training function entity to deploy model #1, that is, message #5 can be a model deployment request message. In this way, the first model training function entity can deploy model #1 after receiving message #5.

[0243] S1007, the first model training functional entity obtains model #1 from the model library functional entity.

[0244] It is understood that after the training is completed, model #1 is stored in the model library functional entity. Therefore, the first model training functional entity can download model #1 from the model library functional entity to complete the deployment of model #1.

[0245] S1008: The first model training functional entity sends message #6 to the model reasoning functional entity. Correspondingly, the model reasoning functional entity receives message #6 from the first model training functional entity.

[0246] Message #6 may indicate the deployment of model #1, i.e., message #6 may be a model deployment request. Message #6 may include the identifier of model #1 and / or model #1, so that upon receiving message #6, the model inference function entity can deploy model #1 according to message #6.

[0247] S1009, the model reasoning functional entity performs reasoning based on model #1.

[0248] It can be understood that the specific implementation of S1009 can refer to the existing technology and will not be repeated here.

[0249] S1010: The model reasoning function entity sends a message #7 to the first model training function entity. Correspondingly, the first model training function entity receives the message #7 from the model reasoning function entity.

[0250] Message #7 may indicate the performance information and online data of model #1. This performance information may represent the performance of model #1, such as the accuracy of model #1. This online data may be data generated by model #1 during inference, such as input data and output data. In other words, during the inference process based on model #1, the model inference function may obtain the performance information and online data of model #1 and may send this performance information and online data to the first model training function, so that the first model training function can update the model based on this data.

[0251] S1011: The O-CU / O-DU sends message #8 to the first model training functional entity. Correspondingly, the first model training functional entity receives message #8 from the O-CU / O-DU.

[0252] Message #8 may indicate relevant data of model #1, such as input data and output data of model #1. For details, please refer to the existing technology and will not be repeated here.

[0253] S1012: The first model training functional entity performs model training based on message #7 and message #8.

[0254] That is, the first model training functional entity can update, i.e., train, model #1 based on the above-mentioned performance information, online data, and related data to obtain an updated model, i.e., model #2.

[0255] It is understood that Model #2 can be a model trained based on Model #1, or a newly trained model. The specific setting can be based on the actual situation and is not limited. In addition, Model #2 can be used as an updated model of Model #1.

[0256] S1013: The first model training functional entity sends message #9 to the model library functional entity. Correspondingly, the model library functional entity receives message #9 from the first model training functional entity.

[0257] Message #9 can be the first message in the embodiment shown in Figure 9, the model library functional entity can be the model operation functional entity in the embodiment shown in Figure 9, and the specific implementation of S1013 can refer to the relevant introduction of the aforementioned "S901" and "S902", which will not be repeated here.

[0258] S1014, the model library functional entity determines the use value of model #2 based on message #9, and determines whether to store model #2 according to the use value.

[0259] For the specific implementation of S1014, please refer to the relevant introduction of "S903" mentioned above, which will not be repeated here.

[0260] S1015: The model repository functional entity sends a message #10 to the first model training functional entity. Correspondingly, the first model training functional entity receives the message #10 from the model repository functional entity.

[0261] Message #10 may be the aforementioned fourth message or fifth message. For details, please refer to the aforementioned related introduction, and the specific implementation of S1015 may refer to the aforementioned related introduction, which will not be repeated here.

[0262] S1016: The first model training function entity sends a message #11 to the model management function entity. Correspondingly, the model management function entity receives the message #11 from the first model training function entity.

[0263] Message #11 may indicate that model #1 has been updated, i.e., message #11 may be a model update notification message. It is understood that S1016 may be implemented after the first model training functional entity acquires model #2, i.e., after S1012. In other words, the order of S1016 and S1013-S1015 is not limited; S1016 and S1013-S1015 may be performed simultaneously or sequentially.

[0264] It can be understood that the specific implementation of S1001, S1002, S1013-S1015 can refer to the relevant introduction of the aforementioned S901-S903, which will not be repeated here.

[0265] Scenario 2:

[0266] Figure 11 is a third flow diagram of the model storage management method provided in an embodiment of the present application. This method is applicable to the aforementioned communication system and primarily involves interaction between a first model training functional entity and a model management functional entity. In scenario 2, the first model training functional entity can send message #9 to the model management functional entity, indicating the usefulness of model #2. The model management functional entity then determines whether to store model #2 based on message #9.

[0267] As shown in Figure 11, the process of the model storage management method is as follows:

[0268] S1101: The model management functional entity sends a message #1 to the model repository functional entity. Correspondingly, the model repository functional entity receives the message #1 from the model management functional entity.

[0269] For S1101, please refer to the relevant introduction of "S1001" mentioned above, which will not be repeated here.

[0270] S1102: The model management function entity sends a message #2 to the first model training function entity. Correspondingly, the first model training function entity receives the message #2 from the model management function entity.

[0271] Message #2 may be the aforementioned second message or third message. For details, please refer to the aforementioned related introduction and will not be repeated here. The first model training functional entity may be a model training functional entity provided in the Near-RT RIC, and the first model training functional entity is the model training functional entity in the embodiment shown in FIG9 . By sending message #2, the first model training functional entity may be instructed to report information on the use value of the updated model after the model is updated.

[0272] It can be understood that message #2 can also be sent by the model library functional entity to the first model training functional entity. The specific settings can be made according to actual conditions without any restrictions.

[0273] S1103: The second model training functional entity collects offline data and performs model training.

[0274] It can be understood that after the second model training functional entity performs model training based on offline data, model #1 can be obtained.

[0275] S1104: The second model training functional entity sends message #3 to the model repository functional entity. Correspondingly, the model repository functional entity receives message #3 from the second model training functional entity.

[0276] S1105: The second model training function entity sends message #4 to the model management function entity. Correspondingly, the model management function entity receives message #4 from the second model training function entity.

[0277] S1106: The model management function entity sends message #5 to the first model training function entity. Correspondingly, the first model training function entity receives message #5 from the model management function entity.

[0278] S1107, the first model training functional entity obtains model #1 from the model library functional entity.

[0279] S1108: The first model training functional entity sends message #6 to the model reasoning functional entity. Correspondingly, the model reasoning functional entity receives message #6 from the first model training functional entity.

[0280] S1109, the model reasoning functional entity performs reasoning based on model #1.

[0281] S1110: The model reasoning function entity sends message #7 to the first model training function entity. Correspondingly, the first model training function entity receives message #7 from the model reasoning function entity.

[0282] S1111: The O-CU / O-DU sends message #8 to the first model training functional entity. Correspondingly, the first model training functional entity receives message #8 from the O-CU / O-DU.

[0283] S1112, the first model training functional entity performs model training based on message #7 and message #8.

[0284] The first model training functional entity performs model training based on message #7 and message #8 to obtain model #2.

[0285] It can be understood that S1103-S1112 can refer to the relevant introduction of the aforementioned S1003-S1012 and will not be repeated here.

[0286] S1113: The first model training function entity sends message #9 to the model management function entity. Correspondingly, the model management function entity receives message #9 from the first model training function entity.

[0287] Message #9 can be the first message in the embodiment shown in Figure 9, and the model management functional entity can be the model operation functional entity in the embodiment shown in Figure 9. The specific implementation of S1113 can refer to the relevant introduction of the aforementioned "S901" and "S902", which will not be repeated here.

[0288] S1114: The model management function entity determines the usage value of model #2 based on message #9, and determines whether to store model #2 based on the usage value.

[0289] For the specific implementation of S1114, please refer to the relevant introduction of the aforementioned "S903", which will not be repeated here.

[0290] S1115: When determining to store model #2, the model management function entity sends message #10 to the model repository function entity. Correspondingly, the model repository function entity receives message #10 from the model management function entity.

[0291] Message #10 can be used to request the storage of model #2, that is, message #10 can be a model storage request message. In this way, the model library functional entity can store model #2 after receiving message #10.

[0292] S1116: The model management function entity sends a message #11 to the first model training function entity. Correspondingly, the first model training function entity receives the message #11 from the model management function entity.

[0293] Message #11 can be the aforementioned fourth message or fifth message. For details, please refer to the aforementioned related introduction, and the specific implementation of S1116 can refer to the aforementioned related introduction, which will not be repeated here.

[0294] It can be understood that the specific implementation of S1101, S1102, S1113-S1115 can refer to the relevant introduction of the aforementioned S901-S903, which will not be repeated here.

[0295] Figure 12 is a fourth flow chart of the model storage management method provided in an embodiment of the present application. The method mainly involves a model operation functional entity.

[0296] As shown in Figure 12, the process of the model storage management method is as follows:

[0297] S1201: The model operation functional entity determines to delete at least one AI model among the stored AI models.

[0298] The AI ​​model can be used to implement specific communication business functions. For details, please refer to the relevant introduction of the aforementioned "S901" and will not be repeated here.

[0299] The stored AI model may be an AI model stored in the model library functional entity, and the stored AI model may include at least one AI model. It is understood that the stored AI model may also be an AI model stored in other functional entities, without limitation.

[0300] There are various conditions under which the model operation functional entity determines to delete at least one AI model from the stored AI models. For example, the model operation functional entity may determine to delete the at least one AI model based on a preset time period, such as after each preset time period, the model operation functional entity determines to delete the at least one AI model. Another example is that the model operation functional entity may determine to delete the at least one AI model when a preset condition is met, such as when the storage time of at least one AI model in the stored AI models reaches the maximum storage time, and / or when the number of stored AI models reaches the maximum storage number, the model operation functional entity determines to delete the at least one AI model. It is understandable that when the model version stores only one AI model, the preset condition may be that the storage time of the AI ​​model reaches the maximum storage time. In this way, at least one AI model from the stored AI models can be deleted in a timely manner to ensure that there is sufficient storage space for the newly generated AI model.

[0301] It is understood that the model operation functional entity can be a model library functional entity or a model management functional entity. When the model operation functional entity is a different functional entity, the model operation functional entity determines the deletion of at least one of the stored AI models in different ways based on preset conditions. The following describes each of these.

[0302] In one possible implementation, the model operation functional entity is a model library functional entity, and the model operation functional entity determines to delete at least one AI model among the stored AI models, which can specifically include: according to a preset condition, the model library functional entity determines to delete at least one model among the stored AI models, and the preset condition is that the storage time of at least one AI model among the stored AI models reaches the maximum storage time, and / or the number of stored AI models reaches the maximum storage number.

[0303] The storage time can be determined by the current time and the time when the AI ​​model is stored in the model library functional entity. For example, if the AI ​​model is stored in the model library functional entity at 12:00 and the current time is 14:00, the storage time is 2 hours. The maximum storage time and maximum storage quantity can be preset, predefined by the protocol, or sent to the model library functional entity by other functional entities (such as the model management functional entity). They can be set according to actual conditions and are not restricted.

[0304] It is understandable that the model library functional entity can be used to store AI models, such as trained AI models, updated AI models, etc. In other words, the model library functional entity can determine, based on the stored AI models, whether the storage time of at least one AI model has reached the maximum storage time, and / or whether the number of stored AI models has reached the maximum storage number. In other words, the model library functional entity does not need to determine whether the preset conditions are met through other functional entities.

[0305] In another possible implementation, the model operation functional entity is a model management functional entity, and the model operation functional entity determines to delete at least one AI model from the stored AI models, which can specifically include: the model library functional entity sends a fourth message to the model management functional entity, and accordingly, the model management functional entity receives a fourth message from the model library function, the fourth message indicating that a preset condition is met, the preset condition being that the storage time of at least one AI model in the stored AI models reaches the maximum storage time, and / or the number of stored AI models reaches the maximum storage number; the model management functional entity determines to delete at least one model from the stored AI models based on the fourth message. For the storage time, maximum storage time, and maximum storage number, please refer to the above-mentioned relevant introduction and will not be repeated here.

[0306] It is understood that the model repository functional entity can be used to store AI models, that is, the model repository functional entity can determine whether a preset condition is met. When the preset condition is met, the model repository functional entity can send a fourth message to the model management functional entity to notify the model management functional entity that the preset condition is currently met, so that the model management functional entity can promptly determine to delete at least one of the stored AI models.

[0307] S1202: The model operation functional entity determines to delete at least one first AI model among the stored AI models based on the usage value of each AI model among the stored AI models.

[0308] The use value of each AI model can represent the probability of subsequent reuse of each AI model. That is, the higher the use value of the AI ​​model, the higher the probability of its subsequent reuse; conversely, the lower the probability of the AI ​​model being subsequently reused. It can be understood that the lower the probability of the AI ​​model being subsequently reused, the higher the probability of the AI ​​model being deleted. In other words, the probability of each AI model being subsequently reused is negatively correlated with the probability of each AI model being deleted. In other words, the use value of each AI model is negatively correlated with the probability of each AI model being deleted. That is, the lower the use value of each AI model, the higher the probability of each AI model being deleted, and conversely, the lower the probability of each AI model being deleted.

[0309] The usage value of each AI model can be determined by weighted summing at least one of the following information of each AI model: deployment scope, model accuracy, change in accuracy after model update, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time.

[0310] For details about the deployment scope, model accuracy, accuracy changes after model updates, and the number of samples used for training, please refer to the aforementioned "S901" and will not be repeated here.

[0311] The number of deployments can be the number of times a model is deployed, which can represent the generalization level of the model. That is, the more times a model is deployed, the higher the generalization level of the model, i.e., the more network elements that can use the model. Conversely, the lower the generalization level of the model, the fewer network elements that can use the model. It is understood that the number of deployments can be determined based on deployment requests. In other words, before a network element deploys a model, it will send a deployment request. Therefore, the number of deployment requests can be used to determine the number of model deployments.

[0312] The average inference accuracy can be the average of multiple accuracies corresponding to the model, and it can characterize the performance of the model, that is, the higher the average inference accuracy, the better the model performance; conversely, the worse the model performance. The multiple accuracies can include at least two of the following: the accuracy of the model after the initial training is completed, the accuracy of the model after the retraining is completed, or the accuracy determined during the use of the model. It can be understood that the model can correspond to an accuracy after the training is completed, and the training can be the initial training or the retraining; the model can also correspond to at least one accuracy during use, such as when the model is in use, obtaining an accuracy during use every first preset time period. The first preset time period can be set according to actual conditions and is not limited.

[0313] The number of model updates may be the number of model updates within a second preset time period, and may indicate a rate of model deterioration. Specifically, a greater number of model updates indicates that the model cannot be used for a longer period of time, i.e., a faster rate of model deterioration. Conversely, a greater number of model updates indicates that the model can be used for a longer period of time, i.e., a slower rate of model deterioration. The second preset time period may be set accordingly based on actual circumstances and is not subject to limitation.

[0314] The average deployment time is the average time between updates and deletions after a model is deployed to each NE. For example, if model #1 is deployed in NE #1 and NE #2, and model #1 is updated 2 hours after being deployed to NE #1 and 1 hour after being deployed to NE #2, the average deployment time is 1.5 hours. Alternatively, if model #1 is deleted 4 hours after being deployed to NE #1 and 6 hours after being deployed to NE #2, the average deployment time is 5 hours. The average deployment time can reflect the degree of model degradation: shorter average deployment time indicates faster model degradation; conversely, shorter average deployment time indicates slower model degradation.

[0315] It can be understood that the method of determining the use value of each AI model is similar to the method of determining the use value of the first AI model in the aforementioned "S901". For details, please refer to the aforementioned relevant introduction and will not be repeated here.

[0316] When the model operation functional entity is a model library functional entity, the model library functional entity can obtain a specific way to determine the use value of each AI model from the model management functional entity. In this case, the above-mentioned model storage management method may also include: the model management functional entity sends a first message to the model library functional entity, and accordingly, the model library functional entity receives a first message from the model management functional entity, and the first message indicates that the use value of the AI ​​model is determined by weighted summing at least one of the following information of the AI ​​model, and the following at least one information may include: deployment range, model accuracy, model accuracy change, or the number of samples used for training, number of deployments, average inference accuracy, number of model updates, or average deployment time. In this way, the model library functional entity can obtain a specific way to determine the use value of each AI model.

[0317] The first message may include at least one of the following: a model value assessment factor, a weight of the model value assessment factor, or an evaluation method. The model value assessment factor may represent at least one of the following information about the AI ​​model, namely, the at least one of the following information may be: deployment scope, model accuracy, model accuracy variation, number of training samples, number of deployments, average inference accuracy, number of model updates, or average deployment time. The weight of the model value assessment factor may represent the weight corresponding to each of the at least one of the following information about the AI ​​model, namely, the weight of the model value assessment factor may include a weight coefficient corresponding to the at least one of the following information about the AI ​​model. The evaluation method may represent determining the use value of the AI ​​model through a weighted summation method. It is understood that the AI ​​model may be a trained AI model, an updated AI model, or another AI model, without limitation. The model value assessment factor, the weight of the model value assessment factor, and the evaluation method in the embodiment of the present application are similar to those in the embodiment shown in FIG. 9 . For details, please refer to the relevant description of "S901" above and will not be repeated here.

[0318] It can be understood that the model management functional entity can send a first message to the model library functional entity before the task starts, that is, pre-configure the specific method of determining the use value of each AI model to the model library functional entity. Alternatively, when the specific method of determining the use value of each AI model needs to be determined, the model library functional entity can send a message to the model management functional entity for requesting the specific method to obtain the specific method. In addition, the specific method of the use value of each AI model can also be pre-set in the model library functional entity, or pre-defined by the protocol, that is, in this case, the model library functional entity does not need to obtain the specific method of determining the use value of each AI model from the model management functional entity.

[0319] After determining the usage value of each stored AI model, the model operation functional entity may determine to delete at least one first AI model from the stored AI models based on the usage value of each stored AI model and a preset value condition. The preset value condition may be to delete the AI ​​model with the lowest value, or to sort the stored AI models from low to high usage value and delete the top N AI models, where N is a positive integer.

[0320] For example, five AI models are stored, namely Model #1a to Model #5a, and the usage values ​​of these five AI models are 10, 10, 50, 40, and 80, respectively. When the value preset condition is to delete the AI ​​model with the lowest value, Model #1a and Model #2a are deleted, that is, the first AI model is now Model #1a or Model #2a. Alternatively, when the value preset condition is to sort the usage values ​​of the stored AI models from low to high and delete the top three AI models, Model #1a, Model #2a, and Model #4a are deleted, that is, the first AI model is now Model #1a, Model #2a, or Model #4a.

[0321] As can be seen, the first AI model can be an AI model that meets a preset value condition. This preset value condition can be the lowest usage value, or the AI ​​model ranked in the top N of the stored AI models sorted by usage value from low to high, where N is a positive integer. In other words, the first AI model can be the AI ​​model with the lowest or relatively low usage value among the stored AI models. This allows at least one AI model with the lowest or relatively low value among the stored AI models to be deleted, avoiding the deletion of AI models that are likely to be used later.

[0322] In summary, in the embodiment of the present application, the model operation functional entity can determine the AI ​​model to be deleted based on the usage value of each stored AI model. In this way, it is possible to avoid deleting AI models that are likely to be needed later, which would cause a waste of training resources.

[0323] It is understood that after determining to delete at least one first AI model from the stored AI models, when the model operation functional entity is a model library functional entity, the model library functional entity may directly delete the at least one first AI model; when the model operation functional entity is a model management functional entity, the model management functional entity may send a fifth message to the model library functional entity, indicating the deletion of the at least one first AI model, thereby instructing the model library functional entity to delete the at least one first AI model. After receiving the fifth message, the model library functional entity may perform a model deletion operation, i.e., delete the at least one first AI model according to the fifth message. Furthermore, after deleting the at least one first AI model, the model library functional entity may send a sixth message to the model management functional entity, indicating the deletion of the at least one first AI model, thereby notifying the model management functional entity of the deletion. It is also understood that when deleting at least one first AI model from the stored AI models, the model library functional entity may clear the storage time corresponding to the at least one first AI model.

[0324] Optionally, in combination with the above embodiment, after the model operation functional entity determines to delete at least one first AI model from the stored AI models based on the usage value of each AI model, the above model storage management method may further include: the model operation functional entity sends a seventh message to the model training functional entity, and the model training functional entity receives the seventh message from the model operation functional entity, the seventh message indicating that at least one first AI model from the stored AI models has been deleted. In this way, after deleting the at least one first AI model, the model training functional entity can be notified of the deletion of the at least one first AI model, avoiding the model training functional entity requesting the deleted AI model when rolling back, which would cause unnecessary overhead.

[0325] Scenario 3:

[0326] Figure 13 is a flowchart diagram of the model storage management method provided in an embodiment of the present application. This method is applicable to the above-mentioned communication system and mainly involves the interaction between the model library functional entity, the model training functional entity, and the model management functional entity. In scenario 3, when determining to delete at least one AI model from the stored AI models, the model library functional entity can determine to delete at least one first AI model from the stored AI models based on the usage value of each AI model in the stored AI models.

[0327] As shown in Figure 13, the process of the model storage management method is as follows:

[0328] S1301: The model management functional entity sends a message #1 to the model repository functional entity. Correspondingly, the model repository functional entity receives the message #1 from the model management functional entity.

[0329] Message #1 is the first message in the embodiment shown in Figure 12. The specific implementation of S1301 can refer to the relevant introduction in the aforementioned "S1001", which will not be repeated here.

[0330] S1302: The model library functional entity determines to delete at least one AI model among the stored AI models.

[0331] S1303: The model library functional entity determines to delete at least one first AI model from the stored AI models based on the usage value of each AI model from the stored AI models.

[0332] S1304: The model library functional entity deletes at least one first AI model from the stored AI models.

[0333] That is, after the model operation functional entity determines to delete at least one first AI model among the stored AI models based on the usage value of each AI model among the stored AI models, it can delete the at least one first AI model, that is, it can directly perform the model deletion operation.

[0334] S1305: The model library functional entity sends message #2 to the model training functional entity. Correspondingly, the model training functional entity receives message #2 from the model operation functional entity.

[0335] Message #2 may be the seventh message in the embodiment shown in Figure 12. S1305 may refer to the above related introduction and will not be described again here.

[0336] It is understood that the specific implementation of S1302-S1303 can refer to the relevant introduction of S1201-S1202 above, which will not be repeated here. In this case, the model library functional entity in scenario 3 can be the model operation functional entity in the embodiment shown in FIG12.

[0337] Scenario 4:

[0338] Figure 14 is a flowchart diagram of the model storage management method provided in an embodiment of the present application. This method is applicable to the above-mentioned communication system and mainly involves the interaction between the model library functional entity, the model training functional entity, and the model management functional entity. In scenario 4, when determining to delete at least one AI model from the stored AI models, the model management functional entity may determine to delete at least one first AI model from the stored AI models based on the usage value of each AI model in the stored AI models.

[0339] As shown in Figure 14, the process of the model storage management method is as follows:

[0340] S1401, the model library functional entity determines whether the preset conditions are met.

[0341] The preset conditions can be referred to the relevant introduction in the aforementioned "S1201", which will not be repeated here.

[0342] S1402: The model repository functional entity sends message #1 to the model management functional entity. Correspondingly, the model management functional entity receives message #1 from the model repository functional entity.

[0343] Message #1 is the fourth message in the embodiment shown in Figure 12. For details of S1402, please refer to the relevant introduction in the aforementioned "S1201", which will not be repeated here.

[0344] S1403: The model management function entity determines to delete at least one AI model among the stored AI models according to message #1.

[0345] S1404: The model management function entity determines to delete at least one first AI model among the stored AI models based on the usage value of each AI model among the stored AI models.

[0346] S1405: The model management functional entity sends message #2 to the model repository functional entity. Correspondingly, the model repository functional entity receives message #2 from the model management functional entity.

[0347] Message #2 is the fifth message in the embodiment shown in Figure 12. The specific implementation of S1405 can refer to the above related introduction and will not be repeated here.

[0348] S1406: The model library functional entity deletes at least one first AI model from the stored AI models according to message #2.

[0349] S1407, after deleting at least one first AI model from the stored AI models, the model library functional entity sends message #3 to the model management functional entity. Correspondingly, the model management functional entity receives message #3 from the model library functional entity.

[0350] Message #3 is the sixth message in the embodiment shown in Figure 12. The specific implementation of S1407 can refer to the above related introduction and will not be repeated here.

[0351] It is understood that the specific implementation of S1401-S1404 can refer to the relevant introduction of S1201-S1202 above, which will not be repeated here. In this case, the model management functional entity in scenario 4 can be the model operation functional entity in the embodiment shown in FIG12.

[0352] Figure 15 is a flow chart of the model storage management method provided in an embodiment of the present application. The method may mainly involve a model operation functional entity.

[0353] As shown in Figure 15, the process of the model storage management method is as follows:

[0354] S1501: The model operation functional entity determines to delete the first AI model.

[0355] The first AI model can be used to implement specific communication service functions. For details, please refer to the relevant introduction of the aforementioned "S901" and will not be repeated here. The first AI model can be a model corresponding to the first model version. The model version can be different versions of the same model that implement the same function. For example, after the model is trained for the first time, the version is V1, and after a retraining and update, the version is V2. The first model version only stores one AI model. That is, for the first model version, only one corresponding AI model is stored in the model library functional entity, namely the first AI model.

[0356] The model operation functional entity may determine to delete the first AI model based on a preset condition. The preset condition may be that the storage time of the AI ​​model reaches the maximum storage time. The storage time and maximum storage time can be described in the aforementioned "S1201" and will not be repeated here. The preset condition may also be the number of times the AI ​​model has been deployed. The number of deployments can be described in the aforementioned "S901" and will not be repeated here. It is understood that the preset condition may also be other conditions without limitation.

[0357] It is understood that the model operation functional entity can be a model library functional entity or a model management functional entity. When the model operation functional entity is a different functional entity, the model operation functional entity determines the deletion of the first AI model in different ways based on preset conditions. The following describes them separately.

[0358] In one possible implementation, the model operation functional entity is a model repository functional entity. The model operation functional entity determining to delete the first AI model may specifically include: based on a preset condition, the model repository functional entity determining to delete the first AI model, where the preset condition is that the storage time of the first AI model has reached a maximum storage time. It is understood that the model repository functional entity may be used to store the first AI model. That is, the model repository functional entity may independently determine whether the preset condition is met based on stored AI models, and thereby determine to delete the first AI model if the preset condition is met.

[0359] In another possible implementation, the model operation functional entity is a model management functional entity, and the model operation functional entity determines to delete the first AI model, which may specifically include: the model library functional entity sends a seventh message to the model management functional entity, and accordingly, the model management functional entity receives the seventh message from the model library functional entity, the seventh message indicating that a preset condition is met, and the preset condition is that the storage time of the first AI model reaches the maximum storage time; the model management functional entity determines to delete the first AI model based on the seventh message. It can be understood that the model library functional entity can be used to store the first AI model, that is, the model library functional entity can determine whether the preset condition is met. When the preset condition is met, the model library functional entity can send the seventh message to the model management functional entity to notify the model management functional entity that the preset condition is currently met, so that the model management functional entity determines to delete the first AI model based on the seventh message.

[0360] S1502, the model operation functional entity triggers model retraining based on the first AI model.

[0361] The model operation functional entity triggering model retraining based on the first AI model can specifically include: the model operation functional entity sending a first message to the model training functional entity, the first message is used to request model retraining based on the first AI model, and the first message can be a model retraining request message. In this way, after the model training functional entity receives the first message, it can retrain the model based on the first AI model to obtain a second AI model, and the second AI model can be an AI model obtained by model retraining. It can be understood that the model training functional entity can refer to the existing technology for model retraining, which will not be repeated here. In addition, the second AI model can be a model trained based on the first AI model, or it can be a retrained model. It can be set according to actual conditions without limitation.

[0362] It is understood that the model operation functional entity can be a model library functional entity or a model management functional entity. When the model operation functional entity is a different functional entity, after the model training functional entity retrains the second AI model, the message sent to the model library functional entity and the model management functional entity is different. The following describes them separately.

[0363] In one possible implementation, the model operation functional entity is a model library functional entity. After the model operation functional entity sends a first message to the model training functional entity, the above-mentioned model storage management method may further include: the model training functional entity sends a second message to the model library functional entity, and accordingly, the model library functional entity receives the second message from the model training functional entity, where the second message is used to request storage of a second AI model. It is understood that the model library functional entity can store AI models. Therefore, after retraining to obtain the second AI model, the model training functional entity can request the model library functional entity to store the second AI model.

[0364] In another possible implementation, the model operation functional entity is a model management functional entity, and the above-mentioned model storage management method may further include: the model training functional entity sends a fourth message to the model library functional entity, and accordingly, the model management functional entity receives a fourth message from the model training functional entity, where the fourth message indicates the second AI model that has been retrained. It can be understood that the model management functional entity can be responsible for the lifecycle management of the model. Therefore, after the model training functional entity retrains the second AI model, it can notify the model management functional entity that the retraining is complete, and the model management functional entity determines whether to store the second AI model.

[0365] S1503: The model operation functional entity triggers deletion of the first AI model and stores the second AI model.

[0366] When the model operation functional entity is a model repository functional entity, triggering the deletion of the first AI model and storing the second AI model by the model operation functional entity may specifically include: the model repository functional entity triggering the deletion of the first AI model and storing the second AI model based on the second message. That is, after receiving the second message, the model repository functional entity may trigger the deletion of the first AI model and the storage of the second AI model.

[0367] When the model operation functional entity is a model management functional entity, triggering the deletion of the first AI model and storing the second AI model by the model operation functional entity may specifically include: the model management functional entity sending a fifth message to the model repository functional entity based on the fourth message, and the model repository functional entity correspondingly receiving the fifth message from the model management functional entity, the fifth message being used to trigger the deletion of the first AI model and the storage of the second AI model. In this case, after receiving the fifth message, the model repository functional entity may, based on the fifth message, delete the first AI model and store the second AI model.

[0368] In summary, in the embodiments of the present application, when only one AI model is stored in the first model version, after retraining to generate a new AI model, the pre-retrained AI model can be deleted and the newly generated AI model can be stored. This ensures that the AI ​​model corresponding to the first model version is always stored, avoiding the situation where no AI model is available.

[0369] It can be understood that when deleting the AI ​​model before retraining (i.e., the first AI model), the storage time corresponding to the AI ​​model can be cleared to save storage space.

[0370] Optionally, in combination with the above embodiment, the model operation functional entity is a model library functional entity. After the model operation functional entity triggers the deletion of the first AI model and stores the second AI model, the above model storage management method may further include: the model library functional entity sends a third message to the model training functional entity. In response, the model training functional entity receives the third message from the model library functional entity, and the third message indicates that the first AI model has been updated. In this way, the model training functional entity can avoid requesting the first AI model from the model library functional entity when rolling back, thereby avoiding unnecessary overhead.

[0371] Optionally, in combination with the above embodiment, the model operation functional entity is a model management functional entity. After the model library functional entity deletes the first AI model and stores the second AI model, the above model storage management method may further include: the model library functional entity sends a sixth message to the model management functional entity. In response, the model management functional entity receives the sixth message from the model library functional entity, the sixth message indicating that the first AI model has been deleted and the second AI model has been stored. In this way, the model management functional entity can know that the first AI model has been updated, thereby facilitating subsequent operations of the model management functional entity, such as deploying the second AI model.

[0372] Optionally, in combination with the above embodiments, after the model training functional entity obtains the second AI model, the above-mentioned model storage management method may also include: the model training functional entity sends an eighth message to the model management functional entity, and accordingly, the model management functional entity receives the eighth message from the model training functional entity, and the eighth message indicates that the retraining of the first AI model is completed.

[0373] Scenario 5:

[0374] Figure 16 is a flowchart of the model storage management method provided in an embodiment of the present application. This method is applicable to the above-mentioned communication system and primarily involves a model repository functional entity, a model training functional entity, and a model management functional entity. In scenario 5, the model repository functional entity can retrain model #2 based on model #1, delete model #1, and store model #2.

[0375] As shown in Figure 16, the process of the model storage management method is as follows:

[0376] S1601, the model library functional entity determines to delete model #1.

[0377] Model #1 may be the first AI model in the aforementioned S1501.

[0378] S1602: The model repository functional entity sends message #1 to the model training functional entity. Correspondingly, the model training functional entity receives message #1 from the model repository functional entity.

[0379] Message #1 is the first message in the aforementioned S1502.

[0380] S1603, the model training functional entity performs model retraining according to message #1.

[0381] It is understood that after the model training function entity performs model retraining according to message #1, model #2 can be obtained. Model #2 can be the second AI model in the aforementioned S1502.

[0382] S1604: The model training functional entity sends message #2 to the model repository functional entity. Correspondingly, the model repository functional entity receives message #2 from the model training functional entity.

[0383] Message #2 may be the second message in the aforementioned S1502.

[0384] S1605: The model library functional entity deletes model #1 according to message #2 and stores model #2.

[0385] S1606: The model repository functional entity sends message #3 to the model training functional entity. Correspondingly, the model training functional entity receives message #3 from the model repository functional entity.

[0386] Message #3 is the third message in the embodiment shown in FIG. 15 .

[0387] S1607: The model training functional entity sends message #4 to the model management functional entity. Correspondingly, the model management functional entity receives message #4 from the model training functional entity.

[0388] Message #4 is the eighth message in the embodiment shown in FIG. 15 .

[0389] It is understood that the specific implementation of S1601-S1607 can refer to the relevant introduction of S1501-S1503 above, which will not be repeated here. In this case, the model library functional entity in scenario 5 can be the model operation functional entity in the embodiment shown in FIG15.

[0390] Scenario 6:

[0391] Figure 17 is a ninth flow chart of the model storage management method provided in an embodiment of the present application. This method is applicable to the aforementioned communication system and primarily involves a model library functional entity, a model training functional entity, and a model management functional entity. In scenario 6, the model management functional entity can retrain model #2 based on model #1, triggering the deletion of model #1 and storing model #2.

[0392] As shown in Figure 17, the process of the model storage management method is as follows:

[0393] S1701: The model library functional entity determines whether the preset conditions are met.

[0394] S1702: The model repository functional entity sends message #1 to the model management functional entity. Correspondingly, the model management functional entity receives message #1 from the model repository functional entity.

[0395] Message #1 may be the seventh message in the aforementioned S1501.

[0396] S1703: The model management function entity sends message #2 to the model training function entity. Correspondingly, the model training function entity receives message #2 from the model management function entity.

[0397] Message #2 may be the first message in the aforementioned S1502.

[0398] S1704: The model training functional entity performs model retraining according to message #2.

[0399] S1705: The model training functional entity sends message #3 to the model management functional entity. Correspondingly, the model management functional entity receives message #3 from the model training functional entity.

[0400] Message #3 may be the fourth message in the embodiment shown in FIG. 15 .

[0401] S1706: The model management functional entity sends message #4 to the model repository functional entity. Correspondingly, the model repository functional entity receives message #4 from the model management functional entity.

[0402] Message #4 may be the fifth message in the embodiment shown in FIG. 15 .

[0403] S1707, the model library functional entity deletes model #1 according to message #4 and stores model #2.

[0404] S1708: The model repository functional entity sends message #5 to the model management functional entity. Correspondingly, the model management functional entity receives message #5 from the model repository functional entity.

[0405] Message #5 may be the sixth message in the embodiment shown in FIG. 15 .

[0406] It is understood that the specific implementation of S1701-S1708 can refer to the relevant introduction of S1501-S1503 above, which will not be repeated here. In this case, the model management functional entity in scenario 6 can be the model operation functional entity in the embodiment shown in Figure 15.

[0407] Figure 18 is a flowchart diagram 10 of the model storage management method provided by an embodiment of the present application. The method can be applied to the interaction between the model operation function entity and the model training function entity in the above communication system.

[0408] As shown in Figure 18, the process of the model storage management method is as follows:

[0409] S1801: The first model training functional entity obtains a first message.

[0410] The first message may indicate the use value of the first AI model.

[0411] The first AI model can be used to implement specific communication business functions. For details, please refer to the relevant introduction of the aforementioned "S901" and will not be repeated here. The use value of the first AI model can represent the probability of the first AI model being used again in the future. For example, the higher the use value of the first AI model, the higher the probability of its subsequent use; conversely, the lower the probability of the first AI model being used again in the future. It can be understood that the lower the probability of the first AI model being used again in the future, the higher the probability of the first AI model being deleted. In other words, the probability of the first AI model being used again in the future is negatively correlated with the probability of the first AI model being deleted, that is, the use value of the first AI model is negatively correlated with the probability of the first AI model being deleted, that is, the lower the use value of the first AI model, the higher the probability of the first AI model being deleted, and conversely, the lower the probability of the first AI model being deleted.

[0412] It can be understood that the use value of the first AI model can be represented by information, or determined by weighted summation of various information, which are explained below.

[0413] In the first possible design scheme, the use value of the first AI model can be represented by at least one item of information of the first AI model: deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time. For details, please refer to the relevant introduction of the aforementioned "S901" and "1202", which will not be repeated here.

[0414] The first message may include at least one item of information about the first AI model, such as deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time. For details, please refer to the relevant introduction of the aforementioned "S901", which will not be repeated here.

[0415] Before the first model training functional entity obtains the first message, the above-mentioned model storage management method may also include: the model operation functional entity sends a second message to the first model training functional entity, and accordingly, the first model training functional entity receives the second message from the model operation functional entity, the second message indicating that when the updated AI model is obtained or after the AI ​​model is deployed, at least one of the above-mentioned information representing the use value of the third AI model is reported: the number of deployments, the average inference accuracy, the number of model updates, or the average deployment time, and the third AI model is an updated AI model or an AI model that has been deployed; the above-mentioned first model training functional entity obtaining the first message may specifically include: the first model training functional entity obtains the first message based on the second message, and the first AI model belongs to the third AI model. That is, the model operation functional entity can instruct the first model training functional entity through the second message to report information representing the use value of the corresponding AI model after obtaining the AI ​​model that has been deployed or the updated AI model, so that the model operation functional entity can determine whether to delete the AI ​​model based on the information.

[0416] The second message may include a model value assessment factor, which may represent at least one of the above-mentioned information on the use value of the third AI model: deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time. That is, the second message may directly carry this information to instruct the model training functional entity to report various information representing the use value of the AI ​​model after obtaining the deployed AI model or the updated AI model. In this way, the first model training functional entity can be prevented from obtaining the model value assessment factor from other places, thereby reducing the overhead of the first model training functional entity.

[0417] In addition, the above content introduces that the model operation functional entity instructs the model training functional entity to obtain the first message through the second message. It can be understood that when the model operation functional entity is a model library functional entity, the various information indicated by the second message can be pre-set in the model library functional entity; or, the various information indicated by the second message can be predefined by agreement; or, the various information indicated by the second message can be obtained from the model management functional entity. The various information indicated by the second message can be at least one information indicating the use value of the deployed AI model after the deployed AI model is obtained; and / or, when the updated AI model is obtained, at least one information indicating the use value of the updated AI model is reported. It can also be understood that there are many ways for the model library functional entity to obtain various information from the model management functional entity, such as the model library functional entity sending a message for requesting various information to the model management functional entity, and in the initial configuration stage, the model management functional entity first sends various information to the model library functional entity.

[0418] In the second possible design scheme, the usage value of the first AI model can be determined by weighted summing at least one of the following information of the first AI model: deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time. For details, please refer to the relevant introductions of the aforementioned "S901" and "1202", which will not be repeated here.

[0419] The first message may include the usage value of the first AI model. For details, please refer to the relevant introduction of the aforementioned "S901" and will not be repeated here.

[0420] Before the first model training functional entity obtains the first message, the model storage management method may further include: the model operation functional entity sends a third message to the first model training functional entity, and accordingly, the first model training functional entity receives the third message from the model operation functional entity, wherein the third message indicates that upon obtaining an updated AI model or after deploying the AI ​​model, the use value of the third AI model should be reported, where the use value of the third AI model is determined by weighted summing at least one of the following information of the third AI model: deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time, where the third AI model is an updated AI model or a fully deployed AI model; and the first model training functional entity obtaining the first message may specifically include: the first model training functional entity obtaining the first message based on the third message, where the first AI model belongs to the third AI model. That is, the model operation functional entity may instruct the first model training functional entity, through the second message, to report the use value of the corresponding AI model upon obtaining a fully deployed AI model or an updated AI model, so that the model operation functional entity can determine whether to delete the AI ​​model based on the use value.

[0421] The third message may include at least one of the following: a model value evaluation factor, a weight of the model value evaluation factor, or an evaluation method.

[0422] The model value evaluation factor may represent at least one of the following information of the third AI model, and the at least one of the following information may include: deployment scope, model accuracy, model accuracy change, number of samples used in training, number of deployments, average inference accuracy, number of model updates, or average deployment time. The weight of the model value evaluation factor may represent the weight corresponding to each of the following at least one information of the third AI model, that is, the weight of the model value evaluation factor may include the weight coefficient corresponding to the at least one of the following information of the third AI model. The evaluation method represents determining the use value of the third AI model by weighted summing the at least one of the following information of the third AI model. The model operation functional entity may send the model value evaluation factor, the weight of the model value evaluation factor, or the evaluation method to the model training functional entity through a third message, so that the model training functional entity obtains the use value of the corresponding AI model according to the third message after the model deployment is completed or the model is updated.

[0423] It can be understood that the model value assessment factors, the weights of the model value assessment factors, and the assessment methods in the embodiments of the present application are similar to the model value assessment factors, the weights of the model value assessment factors, and the assessment methods in the embodiment shown in FIG9 . For details, please refer to the relevant introduction of the aforementioned “S901” and will not be repeated here.

[0424] In addition, the above content introduces that the model operation functional entity instructs the model training functional entity to obtain the first message through the third message. It can be understood that when the model operation functional entity is a model library functional entity, the various information indicated by the third message can be pre-set in the model library functional entity; or, the various information indicated by the third message can be predefined by agreement; or, the various information indicated by the third message can be obtained from the model management functional entity. The various information indicated by the third message can be reporting the use value of the deployed AI model when the deployed AI model is obtained; and / or reporting the use value of the updated AI model when the updated AI model is obtained. It can also be understood that there are many ways for the model library functional entity to obtain various information from the model management functional entity, such as the model library functional entity sending a message for requesting various information to the model management functional entity, and in the initial configuration stage, the model management functional entity first sends various information to the model library functional entity.

[0425] S1802: The first model training functional entity sends a first message to the model operation functional entity. Correspondingly, the model operation functional entity receives the first message from the first model training functional entity.

[0426] After acquiring the first message, the first model training functional entity may send the first message to the model operation functional entity. At this time, the model operation functional entity may receive the first message from the first model training functional entity.

[0427] S1803: When the usage value of the first AI model is less than or equal to the value threshold, the model operation functional entity triggers model retraining based on the first AI model.

[0428] The value threshold can be used to determine whether to delete the first AI model. For example, when the usage value of the first AI model is less than or equal to the value threshold, the first AI model can be deleted; when the usage value of the first AI model is greater than the value threshold, the first AI model can be retained, i.e., the first AI model continues to be stored. The value threshold can be pre-set, pre-defined by the protocol, or obtained from other functional entities (such as a model operation functional entity), such as by carrying the value threshold in the second message or the third message mentioned above.

[0429] When the value of the first AI model is less than or equal to the value threshold, the model operation functional entity triggers model retraining based on the first AI model, which can specifically include: when the value of the first AI model is less than or equal to the value threshold, the model operation functional entity sends a fourth message to the second model training functional entity, and the fourth message is used to request model retraining based on the first AI model.

[0430] The second model training functional entity may be the same as or different from the first model training functional entity. Exemplarily, the first model training functional entity and the second model training functional entity may both be the same model training functional entity set in the Non-RT RIC or the Near-RT RIC; or, the first model training functional entity may be a model training functional entity set in the Near-RT RIC, and the second model training functional entity may be a model training functional entity set in the Non-RT RIC; or, the first model training functional entity may be a model training functional entity set in the Non-RT RIC, and the second model training functional entity may be a model training functional entity set in the Near-RT RIC. The specific settings may be made according to actual conditions and are not limited.

[0431] It can be understood that the first AI model can be the model corresponding to the first model version. When the first model version stores only one AI model, if the value of the first AI model is less than or equal to the value threshold, a new AI model can be retrained first, and then the old AI model (i.e., the first AI model) can be deleted, that is, the old AI model is replaced with the new AI model. In this way, it can be avoided that after the first AI model is deleted, the model library functional entity does not store the model corresponding to the first model version, resulting in no model available. When the first model version can store multiple AI models, if the value of the first AI model is less than or equal to the value threshold, a new AI model can be retrained first, and then the old AI model (i.e., the first AI model) can be deleted, that is, the old AI model is replaced with the new AI model. In this way, it can be ensured that there are sufficient AI models for use in the model library functional entity.

[0432] It is also understood that the model operation functional entity can be a model library functional entity or a model management functional entity. When the model operation functional entity is a different functional entity, the second model training functional entity will also feedback different content based on the fourth message. The following are respectively explained.

[0433] Method 1: The model operation functional entity is a model library functional entity. After the model operation functional entity sends the fourth message to the second model training functional entity, the above-mentioned model storage management method may also include: the second model training functional entity sends a fifth message to the model library functional entity. Correspondingly, the model library functional entity receives the fifth message from the second model training functional entity, and the fifth message is used to request the storage of the second AI model. It can be understood that the model library functional entity can be used to store AI models. Therefore, after the second model training function obtains the second AI model through retraining (described below), it can directly request the model library functional entity to store the second AI model. At this time, the model library functional entity can also determine that the model retraining is completed based on the request (i.e., the fifth message).

[0434] Method 2: The model operation functional entity is a model management functional entity. After the model operation functional entity sends a fourth message to the second model training functional entity, the model storage management method may further include: the second model training functional entity sends a seventh message to the model management functional entity. In response, the model management functional entity receives the seventh message from the second model training functional entity, indicating that model retraining based on the first AI model has been completed. In this way, the model management functional entity can determine that a new AI model has been obtained through model retraining based on the seventh message.

[0435] S1804: The model operation functional entity triggers deletion of the first AI model and stores the second AI model.

[0436] The second AI model can be an AI model obtained by model retraining. The second AI model can be a model trained based on the first AI model, that is, a model obtained by continuing training on the basis of the first AI model, or a retrained model. The specific setting can be based on actual conditions and is not limited.

[0437] It is understood that the model operation functional entity can be a model library functional entity or a model management functional entity. When the model operation functional entity is a different functional entity, the model operation functional entity triggers the deletion of the first AI model and the storage of the second AI model in different ways. The following describes them separately.

[0438] Regarding the aforementioned approach 1: where the model operation functional entity is a model repository functional entity, triggering the deletion of the first AI model and storing the second AI model by the aforementioned model operation functional entity may specifically include: the model repository functional entity triggering the deletion of the first AI model and storing the second AI model based on the fifth message. In other words, after receiving the fifth message, the model repository functional entity may trigger the deletion of the first AI model and the storage of the second AI model. It will be understood that the model repository functional entity can be used to store AI models. Therefore, after triggering the deletion of the first AI model and the storage of the second AI model, the model repository functional entity may directly delete the first AI model and store the second AI model.

[0439] Regarding the aforementioned approach 2: where the model operation functional entity is a model management functional entity, triggering the deletion of the first AI model and storage of the second AI model by the aforementioned model operation functional entity may specifically include: the model management functional entity sending an eighth message to the model repository functional entity based on the seventh message; and the model repository functional entity correspondingly receiving the eighth message from the model management functional entity, where the eighth message is used to trigger the deletion of the first AI model and storage of the second AI model. It is understood that, after receiving the eighth message, the model repository functional entity may delete the first AI model and store the second AI model.

[0440] It can be understood that when the first AI model is deleted, the storage time corresponding to the first AI model can be cleared to save storage space.

[0441] In summary, in the embodiments of the present application, when the model operation functional entity determines to delete the first AI model based on its usefulness, it can trigger model retraining based on the first AI model to obtain a retrained AI model, i.e., a second AI model. Furthermore, the first AI model can be replaced with the second AI model, i.e., the first AI model is deleted and the second AI model is stored. This avoids a situation where there is no model available after deleting the first AI model, or ensures that the model library functional entity has sufficient AI models available for use.

[0442] Optionally, in conjunction with the above embodiment, the model operation functional entity is a model repository functional entity. After the model repository functional entity deletes the first AI model and stores the second AI model, the above model storage management method may further include: the model repository functional entity sends a sixth message to the model management functional entity. In response, the model management functional entity receives the sixth message from the model repository functional entity, the sixth message indicating that the first AI model has been updated. This facilitates the model management functional entity to perform subsequent operations based on the sixth message, such as deploying the second AI model, to avoid errors.

[0443] Optionally, in conjunction with the above embodiment, the model operation functional entity is a model management functional entity. After the model repository functional entity deletes the first AI model and stores the second AI model, the above model storage management method may further include: the model repository functional entity sends a ninth message to the model management functional entity. In response, the model management functional entity receives the ninth message from the model repository functional entity, indicating that the first AI model has been deleted and the second AI model has been stored. This facilitates the model management functional entity to perform subsequent operations based on the sixth message, such as deploying the second AI model, and avoids errors.

[0444] Optionally, after the model operation functional entity receives the first message from the first model training functional entity, the model storage management method may further include: the model operation functional entity updating the use value of the first AI model based on the first message. This ensures that the use value of the first AI model is up to date, facilitating subsequent operations based on the use value of the first AI model, such as determining whether to delete the first AI model based on the use value of the first AI model.

[0445] It can be understood that the method of obtaining the first message in the embodiment of the present application is similar to the method of obtaining the first message in the embodiment described in Figure 9, and can refer to each other. And similar parts of each embodiment can refer to each other. In addition, the above-mentioned embodiments can be used in combination with each other without limitation. When the embodiments are used in combination with each other, the management function entity can send information such as reasoning type, model source, model storage constraints, model value evaluation method, value threshold, etc. to the model library function entity.

[0446] The inference type can be used to indicate the inference type corresponding to the model, such as cell traffic prediction, coverage problem prediction, energy saving optimization, etc.

[0447] The inference source can indicate where the model is trained, such as trained by Non-RT RIC or trained by Near-RT RIC.

[0448] The model storage constraint may indicate the conditions for model storage, such as the maximum storage time or the maximum number of stored versions of the model. It is understood that the model storage constraint may be set separately for models trained by non-RT RIC and models trained by near-RT RIC. For example, the model storage constraint for a model trained by non-RT RIC may be 1, meaning that a maximum of 1 version may be stored; or the model storage constraint for a model trained by non-RT RIC may be 5, meaning that a maximum of 5 versions may be stored.

[0449] The value threshold can be used to indicate the threshold value for storing or deleting a model. For details, please refer to the above related introduction and will not be repeated here.

[0450] The model value assessment method can be used to indicate an assessment method for determining the use value of a model, which may include a model value assessment factor, a weight of the model value assessment factor, and an assessment method. For details, please refer to the aforementioned related introduction. It can be understood that the model value assessment method can be set separately for the model obtained by Non-RT RIC training and the model obtained by Near-RT RIC training. For example, the model value assessment factor of the model corresponding to the model obtained by Non-RT RIC training can be at least one of the following: model accuracy, number of deployments, average inference accuracy, number of model updates, or average deployment time; the model value assessment factor of the model corresponding to the model obtained by Near-RT RIC training can be at least one of the following: deployment range, model accuracy, accuracy change after model update, or the number of samples used for training.

[0451] It is understood that the aforementioned inference type, model source, model storage constraint, model value assessment method, and value threshold have a mapping relationship, i.e., different inference types correspond to different model sources, model storage constraints, model value assessment methods, and value thresholds. Furthermore, the specific content of each piece of information can be determined based on actual circumstances and is not limited.

[0452] Scenario 7:

[0453] Figure 19 is a flowchart diagram 11 of the model storage management method provided in an embodiment of the present application. This method is applicable to the aforementioned communication system and primarily involves a model repository functional entity, a model training functional entity, and a model management functional entity. In scenario 7, the model repository functional entity can trigger model retraining based on model #1 based on the usage value of model #1 to obtain model #2; and can also trigger the deletion of model #1 and the storage of model #2.

[0454] As shown in Figure 19, the process of the model storage management method is as follows:

[0455] S1901, the first model training functional entity obtains message #1.

[0456] Message #1 may indicate the use value of model #1. Message #1 may be the first message in S1801, and model #1 may be the first AI model in S1801. The first model training functional entity may be a model training functional entity provided in the Near-RT RIC.

[0457] S1902: The first model training functional entity sends a message #1 to the model repository functional entity. Correspondingly, the model repository functional entity receives the message #1 from the first model training functional entity.

[0458] S1903, the model library functional entity determines whether to trigger model retraining based on model #1 according to the usage value of model #1.

[0459] S1904: When the usage value of model #1 is less than or equal to the value threshold, the model library functional entity sends message #2 to the second model training functional entity. Correspondingly, the second model training functional entity receives message #2 from the model library functional entity.

[0460] Message #2 is used to request model retraining based on model #1. Message #2 may be the fourth message in S1803. For details, please refer to the above related introduction and will not be repeated here. The second model training function entity may be a model training function entity set in the Non-RT RIC.

[0461] S1905, the second model training functional entity performs model retraining based on model #1.

[0462] Model #2 can be obtained by retraining the model based on model #1. This model #2 can be the second AI model in S1803. It is understood that the specific method of model retraining can refer to the existing technology and will not be repeated here.

[0463] S1906: The second model training functional entity sends message #3 to the model repository functional entity. Correspondingly, the model repository functional entity receives message #3 from the second model training functional entity.

[0464] Message #3 may be used to request storage of model #2. Message #3 may be the fifth message in the aforementioned S1802.

[0465] S1907: The model library functional entity triggers deletion of model #1 according to message #3 and stores model #2.

[0466] S1908, the model library functional entity deletes model #1 and stores model #2.

[0467] S1909: The model repository functional entity sends message #4 to the second model training functional entity. Correspondingly, the second model training functional entity receives message #4 from the model repository functional entity.

[0468] Message #4 may be used to indicate that model #2 has been stored, ie, model #1 has been updated.

[0469] S1910: The second model training function entity sends message #5 to the model management function entity. Correspondingly, the model management function entity receives message #5 from the second model training function entity.

[0470] Message #5 may indicate that model #1 has been updated.

[0471] It is understood that the specific implementation of S1901-S1910 can refer to the relevant introduction of S1801-S1804 above, which will not be repeated here. In this case, the model library functional entity in scenario 7 can be the model operation functional entity in the embodiment shown in Figure 18.

[0472] Scenario 8:

[0473] Figure 20 is a flowchart diagram 12 of the model storage management method provided in an embodiment of the present application. This method is applicable to the aforementioned communication system and primarily involves a model library functional entity, a model training functional entity, and a model management functional entity. In scenario 8, the model management functional entity can trigger model retraining based on model #1 based on the usage value of model #1 to obtain model #2; and can also trigger the deletion of model #1 and the storage of model #2.

[0474] As shown in Figure 20, the process of the model storage management method is as follows:

[0475] S2001, the first model training functional entity obtains message #1.

[0476] Message #1 may indicate the use value of model #1. Message #1 may be the first message in S1801, and model #1 may be the first AI model in S1801. The first model training functional entity may be a model training functional entity provided in the Near-RT RIC.

[0477] S2002: The first model training function entity sends a message #1 to the model management function entity. Correspondingly, the model management function entity receives the message #1 from the first model training function entity.

[0478] S2003, the model management functional entity determines whether to trigger model retraining based on model #1 according to the usage value of model #1.

[0479] S2004: When the usage value of model #1 is less than or equal to the value threshold, the model management function entity sends message #2 to the second model training function entity. Correspondingly, the second model training function entity receives message #2 from the model management function entity.

[0480] Message #2 is used to request model retraining based on model #1. This message #2 may be the fourth message in S1803. For details, please refer to the above related introduction and will not be repeated here. The second model training functional entity may be a model training functional entity set in the Non-RT RIC.

[0481] S2005: The second model training functional entity performs model retraining based on model #1.

[0482] Model #2 can be obtained by retraining the model based on model #1. This model #2 can be the second AI model in S1803. It is understood that the specific method of model retraining can refer to the existing technology and will not be repeated here.

[0483] S2006: The second model training function entity sends message #3 to the model management function entity. Correspondingly, the model management function entity receives message #3 from the second model training function entity.

[0484] Message #3: Model retraining based on model #1 has been completed. This message #3 may be the seventh message in the aforementioned S1803.

[0485] S2007: The model management functional entity sends message #4 to the model repository functional entity. Correspondingly, the model repository functional entity receives message #4 from the model management functional entity.

[0486] Message #4 is used to trigger the deletion of model #1 and the storage of model #2. This message #4 may be the eighth message in the aforementioned S1804.

[0487] S2008: The model library functional entity deletes model #1 according to message #4 and stores model #2.

[0488] S2009: After deleting model #1 and storing model #2, the model repository functional entity sends message #5 to the model management functional entity. Correspondingly, the model management functional entity receives message #5 from the model repository functional entity.

[0489] Message #5 may indicate that model #1 has been deleted and model #2 has been stored. Message #5 may be the ninth message in the embodiment shown in FIG.

[0490] It is understood that the specific implementation of S2001-S2009 can refer to the relevant introduction of S1801-S1804 above, which will not be repeated here. In this case, the model management function entity in scenario 7 can be the model operation function entity in the embodiment shown in Figure 18.

[0491] The above describes in detail the model storage management method provided by the embodiment of the present application in conjunction with Figures 9 to 20. The following describes in detail the communication device for executing the model storage management method provided by the embodiment of the present application in conjunction with Figures 21 and 22.

[0492] Figure 21 is a structural diagram of a communication device according to an embodiment of the present application. As shown in Figure 21 , the communication device 2100 includes a transceiver module 2101 and a processing module 2102. For ease of illustration, Figure 21 only shows the main components of the communication device.

[0493] Among them, the transceiver module 2101 is used to perform the transceiver function of the method shown in Figures 9 to 20 above, and the processing module 2102 is used to perform other functions of the method shown in Figures 9 to 20 except the transceiver function.

[0494] Optionally, the transceiver module 2101 may include a sending module (not shown in FIG21 ) and a receiving module (not shown in FIG21 ). The sending module is used to implement the sending function of the communication device 2100 , and the receiving module is used to implement the receiving function of the communication device 2100 .

[0495] Optionally, the communication device 2100 may further include a storage module (not shown in FIG. 21 ) storing a program or instruction. When the processing module 2102 executes the program or instruction, the communication device 2100 may perform the functions of the terminal or network device (such as the model operation functional entity or the model training functional entity) in the method shown in FIG. 9-FIG . 20 in the above method.

[0496] It can be understood that the communication device 2100 can be a terminal or a network device, or a chip (system) or other parts or components that can be set in a terminal or a network device, or a device that includes a terminal or a network device. This application does not limit this.

[0497] In addition, the technical effects of the communication device 2100 can refer to the technical effects of the model storage management method shown in Figures 9-20, and will not be repeated here.

[0498] Figure 22 is a second structural diagram of a communication device provided in an embodiment of the present application. Exemplarily, the communication device may be a terminal or a network device, or a chip (system) or other component or assembly that can be provided in a terminal or a network device. As shown in Figure 22, the communication device 2200 may include a processor 2201. Optionally, the communication device 2200 may further include a memory 2202 and / or a transceiver 2203. The processor 2201 is coupled to the memory 2202 and the transceiver 2203, such as by a communication bus.

[0499] The following is a detailed introduction to the various components of the communication device 2200 with reference to FIG22 :

[0500] The processor 2201 is the control center of the communication device 2200 and can be a single processor or a collective term for multiple processing elements. For example, the processor 2201 can be one or more central processing units (CPUs), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSPs) or one or more field programmable gate arrays (FPGAs).

[0501] Optionally, the processor 2201 can execute various functions of the communication device 2200 by running or executing software programs stored in the memory 2202 and calling data stored in the memory 2202, such as executing the model storage management method shown in Figures 9-20 above.

[0502] In a specific implementation, as an embodiment, the processor 2201 may include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 22 .

[0503] In a specific implementation, as an embodiment, the communication device 2200 may also include multiple processors, such as the processor 2201 and the processor 2204 shown in FIG22 . Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0504] Among them, the memory 2202 is used to store the software program for executing the solution of this application, and the execution is controlled by the processor 2201. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0505] Alternatively, the memory 2202 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2202 may be integrated with the processor 2201 or exist independently and be coupled to the processor 2201 via an interface circuit (not shown in FIG. 22 ) of the communication device 2200, which is not specifically limited in this embodiment of the present application.

[0506] Transceiver 2203 is used for communication with other communication devices. For example, if communication device 2200 is a terminal, transceiver 2203 can be used to communicate with a network device or another terminal device. For another example, if communication device 2200 is a network device, transceiver 2203 can be used to communicate with a terminal or another network device.

[0507] Optionally, the transceiver 2203 may include a receiver and a transmitter (not shown separately in FIG22 ), wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0508] Optionally, the transceiver 2203 can be integrated with the processor 2201, or can exist independently and be coupled to the processor 2201 through the interface circuit of the communication device 2200 (not shown in Figure 22). This embodiment of the present application does not specifically limit this.

[0509] It is understandable that the structure of the communication device 2200 shown in FIG22 does not constitute a limitation on the communication device, and the actual communication device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0510] In addition, the technical effects of the communication device 2200 can refer to the technical effects of the method described in the above method embodiment, and will not be repeated here.

[0511] It should be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0512] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory, a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM) or a flash memory. The volatile memory can be a random access memory, which is used as an external cache. By way of example but not limitation, many forms of random access memory are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM) and direct rambus RAM (direct rambus RAM, DR RAM).

[0513] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (such as infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0514] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0515] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0516] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0517] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0518] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0519] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical functional division. In actual implementation, other division methods may be used, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not implemented. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection between devices or units, and can be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment. In addition, the functional units in the various embodiments of this application can be integrated into a processing unit, each unit can exist physically separately, or two or more units can be integrated into a single unit.

[0520] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0521] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A model storage management method, characterized in that: The method comprises: The model operation functional entity receives a first message from the model training functional entity, the first message indicating the use value of the first artificial intelligence AI model, and the use value of the first AI model is positively correlated with the probability of storage of the first AI model; The model operation functional entity determines whether to store the first AI model according to the first message.

2. The method according to claim 1, characterized in that The use value of the first AI model is represented by at least one item of information of the first AI model: deployment scope, model accuracy, change in accuracy after model update, or number of samples used for training.

3. The method according to claim 2, characterized in that The first message includes at least one item of information of the first AI model.

4. The method according to claim 3, characterized in that The method further comprises: The model operation functional entity sends a second message to the model training functional entity, and the second message indicates that when the updated AI model is obtained, at least one item of information representing the use value of the updated AI model should be reported.

5. The method according to claim 4, characterized in that The second message includes a model value assessment factor, where the model value assessment factor represents at least one item of information on the use value of the updated AI model.

6. The method according to any one of claims 3 to 5, characterized in that: The model operation function entity determines, according to the first message, whether to store the first AI model, including: The model operation functional entity determines the use value of the first AI model according to at least one item of information of the first AI model; The model operation functional entity determines whether to store the first AI model according to the use value of the first AI model.

7. The method according to claim 1, characterized in that The usage value of the first AI model is determined by weighted summing at least one of the following information of the first AI model: deployment scope, model accuracy, change in accuracy after model update, or number of samples used for training.

8. The method according to claim 7, characterized in that The first message includes the usage value of the first AI model.

9. The method according to claim 8, characterized in that The method further comprises: The model operation functional entity sends a third message to the model training functional entity, and the third message indicates that when the updated AI model is obtained, the use value of the updated AI model should be reported, and the use value of the updated AI model is determined by weighted summing at least one of the following information of the updated AI model.

10. The method according to claim 9, characterized in that The third message includes at least one of the following: a model value evaluation factor, a weight of the model value evaluation factor, or an evaluation method, wherein the model value evaluation factor represents at least one of the following information of the updated AI model, the weight represents the weight corresponding to each of the at least one of the following information of the updated AI model, and the evaluation method represents determining the use value of the updated AI model by weighted summation.

11. The method according to any one of claims 8 to 10, characterized in that: The model operation function entity determines, according to the first message, whether to store the first AI model, including: The model operation functional entity determines whether to store the first AI model according to the use value of the first AI model.

12. The method according to claim 6 or 11, characterized in that: The model operation functional entity determines whether to store the first AI model according to the use value of the first AI model, including: When the use value of the first AI model is greater than or equal to a value threshold, the model operation functional entity determines to store the first AI model; When the use value of the first AI model is less than the value threshold, the model operation functional entity determines not to store the first AI model.

13. The method according to claim 12, characterized in that The model operation functional entity is a model library functional entity, and the method further includes: The model repository functional entity receives the value threshold from the model management functional entity.

14. A model storage management method, characterized in that: The method comprises: The model training functional entity obtains a first message, where the first message indicates the use value of the first artificial intelligence AI model, and the use value of the first AI model is positively correlated with the probability of storing the first AI model; The model training functional entity sends the first message to the model operation functional entity.

15. The method according to claim 14, characterized in that The use value of the first AI model is represented by at least one item of information of the first AI model: deployment scope, model accuracy, change in accuracy after model update, or number of samples used for training.

16. The method according to claim 15, characterized in that The first message includes at least one item of information of the first AI model.

17. The method according to claim 15 or 16, characterized in that The method further comprises: The model training functional entity receives a second message from the model operation functional entity, wherein the second message indicates that when the updated AI model is obtained, the at least one item of information indicating the use value of the updated AI model is reported; The model training function entity obtains a first message, including: The model training functional entity obtains the first message according to the second message, and the first AI model belongs to the updated AI model.

18. The method according to claim 17, characterized in that The second message includes a model value assessment factor, where the model value assessment factor represents at least one item of information on the use value of the updated AI model.

19. The method according to claim 14, characterized in that The usage value of the first AI model is determined by weighted summing at least one of the following information of the first AI model: deployment scope, model accuracy, change in accuracy after model update, or number of samples used for training.

20. The method according to claim 19, characterized in that The first message includes the usage value of the first AI model.

21. The method according to claim 19 or 20, characterized in that The method further comprises: The model training functional entity receives a third message from the model operation functional entity, wherein the third message indicates that when an updated AI model is obtained, the use value of the updated AI model is reported, and the use value of the updated AI model is determined by weighted summing at least one of the following information of the updated AI model; The model training function entity obtains a first message, including: The model training functional entity obtains the first message according to the third message, and the first AI model belongs to the updated AI model.

22. The method according to claim 21, characterized in that The third message includes at least one of the following: a model value evaluation factor, a weight of the model value evaluation factor, or an evaluation method, wherein the model value evaluation factor represents at least one of the following information of the updated AI model, the weight represents the weight corresponding to each of the at least one of the following information of the updated AI model, and the evaluation method represents determining the use value of the updated AI model by weighted summation.

23. A communication device, characterized in that: The apparatus comprises: a module for executing the method as claimed in any one of claims 1-22.

24. A communication device, characterized in that: The communication device comprises: a processor and a memory; the memory is used to store computer instructions, and when the processor executes the instructions, the communication device executes the method according to any one of claims 1 to 22.

25. A communication system, characterized in that: include: A model operation functional entity for executing the method as described in any one of claims 1-13, and / or a model training functional entity for executing the method as described in any one of claims 14-22.

26. A communication chip, characterized in that: Instructions are stored therein, and the communication chip includes: a logic circuit and a communication interface, the logic circuit is used to execute computer instructions, and the communication interface is used for the communication chip to communicate with other devices or chips. When the logic circuit executes the computer instructions, the method described in any one of claims 1-22 is implemented.

27. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program or instructions, and when the computer program or instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 22.

28. A computer program product, characterized in that The computer program product comprises a computer program or instructions, and when the computer program or instructions are executed by a communication device, the method according to any one of claims 1 to 22 is executed.

Images