Machine learning model collaboration method and system for 5G communication system

Abstract

The invention provides a 5G communication system-oriented machine learning model collaboration method and system. The method comprises the following steps of: receiving a model use request sent by a user through a model user matched with a 5G communication system; when the model user determines that the 5G communication system meets a model cooperation event triggering condition, the model user determines a model cooperation party matched with the model user according to the model use request, and generates a model cooperation request according to the model use request so as to send the model cooperation request to the model cooperation party; receiving a model collaboration request through a model collaboration party, and judging whether the 5G communication system meets a collaboration triggering condition or not; if yes, generating a corresponding collaborative operation request according to the model collaborative request through the model collaborative party, and sending the corresponding collaborative operation request to the corresponding model generator; corresponding collaborative response information is fed back to the model collaborative party through the model generator according to the collaborative operation request; and executing corresponding collaborative operation according to the collaborative response information through the model collaborative party.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, and more particularly, to a machine learning model coordination method and system for a 5G communication system. BACKGROUND

[0002] With the development of technology, artificial intelligence (AI) / machine learning (ML) technology and related applications are widely used in various industries, especially in the telecommunications industry. Among them, AI / ML functions are used in various fields of 5G communication systems (5GS), and the AI / ML functions defined in the 5G communication system include: AI / ML management and orchestration, management data analysis (Management Data Analytics Function, MDAF), network data analysis function (Network Data Analytics Function, NWDAF), etc. MDAF is a function defined in the 3rd Generation Partnership Project (3GPP) standard, which involves multiple management data analysis capabilities.

[0003] In the prior art, the overall performance of the 5G communication system can be improved by coordinating the machine learning capabilities in the 5G core network (5GC) or the radio access network (RAN) with the 3GPP management analysis and other possible aspects.

[0004] However, due to the performance of the model in the 5GC or RAN may be biased in some aspects, resulting in some bias in the type of data collected by using the 5GC or RAN model training / inference. The data collected by the 3GPP management system is more extensive and involves more RAN nodes / 5GC, which means that the prediction made by the 3GPP management system is unbiased relative to the overall RAN node / 5GC. However, the prediction of the 3GPP management system may lack insight into specific node behavior-related patterns and more fine-grained time granularity. Therefore, by coordinating the machine learning capabilities among the 5GC, RAN and 3GPP management system, the overall performance of the 5G communication system can be further improved, and the accuracy of the prediction can be improved.

[0005] Therefore, how to provide a coordination method to ensure that the machine learning models corresponding to the 5GC, RAN and 3GPP management system can work efficiently in coordination, that is, to coordinate the machine learning models in the 5G communication system, and to improve the overall performance of the 5G communication system, is a problem that needs to be solved urgently at present. SUMMARY

[0006] In view of this, the present application provides a machine learning model coordination method and system for a 5G communication system to realize coordination between machine learning models of the 5G communication system and improve the overall performance of the 5G communication system.

[0007] The first aspect of the present application provides a machine learning model coordination method for a 5G communication system, which is suitable for the 5G communication system and includes the following steps:

[0008] determining a model user matched with the 5G communication system and receiving a model use request sent by a user through the model user; wherein the model user is a first MDAF or a first NWDAF;

[0009] determining whether the 5G communication system meets a model coordination event trigger condition through the model user;

[0010] if the model coordination event trigger condition is met, determining a model coordination partner matched with the model user according to the model use request through the model user, and generating a model coordination request according to the model use request to send the model coordination request to the model coordination partner; wherein the model coordination partner is a second MDAF or a second NWDAF or a third MDAF;

[0011] receiving the model coordination request through the model coordination partner and determining whether the 5G communication system meets a coordination trigger condition;

[0012] if the coordination trigger condition is met, generating a corresponding coordination operation request according to the model coordination request through the model coordination partner, and sending the corresponding coordination operation request to a corresponding model generator;

[0013] feeding back a corresponding coordination response information to the model coordination partner according to the coordination operation request through the model generator;

[0014] executing a corresponding coordination operation according to the coordination response information through the model coordination partner.

[0015] Optionally, the step of determining a model user matched with the 5G communication system and receiving a model use request sent by a user through the model user includes the following steps:

[0016] determining a domain to which the 5G communication system belongs, and taking an artificial intelligence service producer corresponding to the domain as the model user matched with the 5G communication system; wherein the domain is a RAN domain, and the artificial intelligence service producer corresponding to the domain is a first MDAF corresponding to the domain; or the domain is a CN domain, and the artificial intelligence service producer corresponding to the domain is a first NWDAF corresponding to the domain;

[0017] receiving, by the model user, a model use request sent by a user based on the request information corresponding to the domain; wherein the model use request at least includes the request information.

[0018] Optionally, determining, by the model user, a model collaboration party matched with the model user according to the model use request, and generating a model collaboration request according to the model use request, to send the model collaboration request to the model collaboration party, includes:

[0019] analyzing, by the model user, the request information to determine a machine learning model corresponding to the model use request and a collaboration stage thereof, and judging whether the machine learning model needs cross-domain collaboration or same-domain collaboration in the collaboration stage;

[0020] If same-domain collaboration is needed, and the domain is a RAN domain or a CN domain, determining, by the model user, a second MDAF or a second NWDAF corresponding to the RAN domain or the CN domain as the model collaboration party matched with the model user;

[0021] If cross-domain collaboration is needed, and the domain is a RAN domain or a CN domain, determining, by the model user, a third MDAF corresponding to the management domain as the model collaboration party matched with the model user;

[0022] generating, by the model user, a model collaboration request according to the machine learning model, the collaboration stage and the request information, and sending the model collaboration request to the model collaboration party; wherein the model collaboration request includes a request type identifier, a task identifier and description, a model requirement specification and a performance expectation index.

[0023] Optionally, determining, by the model user, whether the 5G communication system meets a model collaboration event trigger condition, includes:

[0024] acquiring, by the model user, network state information, system information and network information of the 5G communication system;

[0025] judging, by the model user, whether the 5G communication system meets a network performance threshold trigger condition according to the network state information, whether the 5G communication system meets a time-related trigger condition according to the service processing information, and whether the 5G communication system meets an event-driven trigger condition according to the network device information;

[0026] If the 5G communication system meets the network performance threshold trigger condition, and / or, meets the time-related trigger condition, and / or, meets the event-driven trigger condition, determining, by the model user, that the 5G communication system meets the model collaboration event trigger condition.

[0027] If the 5G communication system does not satisfy the network performance threshold trigger condition, does not satisfy the time-related trigger condition, and does not satisfy the time-driven trigger condition, it is determined by the model usage side that the 5G communication system does not satisfy the model collaboration event trigger condition.

[0028] Optionally, the model collaboration side receiving the model collaboration request and determining whether the 5G communication system satisfies a collaboration trigger condition comprises:

[0029] The model collaboration side obtaining a plurality of network key performance indicators, data flow information, and service type information of the 5G communication system;

[0030] The model collaboration side determining whether each of the network key performance indicators is within a corresponding preset range, and determining whether there is a service with a growth rate of service flow not less than a second preset growth rate within a third preset time period in the 5G communication system according to the data flow information and the service type information;

[0031] If any one or more of the network key performance indicators is not within the corresponding preset range, and / or there is a service with a growth rate of service flow not less than the second preset growth rate within the third preset time period, the model collaboration side determines that the 5G communication system satisfies the collaboration trigger condition;

[0032] If each of the network key performance indicators is within the corresponding preset range, and there is no service with a growth rate of service flow not less than the second preset growth rate within the third preset time period, the model collaboration side determines that the 5G communication system does not satisfy the collaboration trigger condition.

[0033] Optionally, the model collaboration side generating a corresponding collaboration operation request according to the model collaboration request and sending the corresponding collaboration operation request to the corresponding model generation side comprises:

[0034] The model collaboration side determining a collaboration stage of the corresponding machine learning model according to the request type identifier, and generating collaboration task details and resource and time estimation of the collaboration stage under the machine learning model according to the task identifier and description, the performance expectation indicator, and the model requirement specification;

[0035] The model collaboration side generating a collaboration operation request according to the collaboration task details and the resource and time estimation, and sending the collaboration operation request to the corresponding model generation side.

[0036] Optionally, the model collaboration side performing a corresponding collaboration operation according to the collaboration response information comprises:

[0037] determining, by the model coordination party, whether the collaborative response information contains a refusal identifier or an agreement identifier;

[0038] If the refusal identifier is contained, generating and outputting, by the model coordination party, corresponding prompt information according to response information in the collaborative response information;

[0039] If the agreement identifier is contained, establishing, by the model coordination party, a corresponding model group object according to model information in the collaborative response information, and generating a corresponding collaborative learning report, and feeding back the collaborative learning report to the model user, so that the model user makes a corresponding decision according to the collaborative learning report.

[0040] The second aspect of the present application provides a machine learning model coordination system for a 5G communication system, which is suitable for a 5G communication system, and the system comprises:

[0041] A receiving module is configured to determine a model user matched with the 5G communication system, and receive a model use request sent by a user through the model user; wherein the model user is a first MDAF or a first NWDAF;

[0042] A first determining module is configured to determine, by the model user, whether the 5G communication system meets a model coordination event trigger condition;

[0043] A first sending module is configured to, if the model coordination event trigger condition is met, determine, by the model user, a model coordination party matched with the model user according to the model use request, and generate a model coordination request according to the model use request, so as to send the model coordination request to the model coordination party; wherein the model coordination party is a second MDAF or a second NWDAF or a third MDAF;

[0044] A second determining module is configured to receive, by the model coordination party, the model coordination request, and determine whether the 5G communication system meets a coordination trigger condition;

[0045] A second sending module is configured to, if the coordination trigger condition is met, generate, by the model coordination party, a corresponding coordination operation request according to the model coordination request, and send the corresponding coordination operation request to a corresponding model generation party;

[0046] A feedback module is configured to feed back, by the model generation party, corresponding collaborative response information to the model coordination party according to the coordination operation request;

[0047] A coordination module is configured to execute, by the model coordination party, a corresponding coordination operation according to the collaborative response information.

[0048] Optionally, the receiving module is specifically configured to:

[0049] determine a domain to which the 5G communication system belongs, and use an artificial intelligence service producer corresponding to the domain as a model usage party matched with the 5G communication system; wherein the domain is a RAN domain, and the artificial intelligence service producer corresponding to the domain is a first MDAF corresponding to the domain; or the domain is a CN domain, and the artificial intelligence service producer corresponding to the domain is a first NWDAF corresponding to the domain;

[0050] receive, by the model usage party, a model usage request sent by a user based on request information corresponding to the domain; wherein the model usage request at least includes the request information.

[0051] Optionally, the first sending module is specifically configured to:

[0052] analyze, by the model usage party, the request information to determine a machine learning model corresponding to the model usage request and a collaborative stage thereof, and determine whether the machine learning model needs cross-domain collaboration or same-domain collaboration in the collaborative stage;

[0053] if same-domain collaboration is needed, and the domain is a RAN domain or a CN domain, determine, by the model usage party, a second MDAF or a second NWDAF corresponding to the RAN domain or the CN domain as a model collaboration party matched with the model usage party;

[0054] if cross-domain collaboration is needed, and the domain is a RAN domain or a CN domain, determine, by the model usage party, a third MDAF corresponding to a management domain as a model collaboration party matched with the model usage party;

[0055] generate, by the model usage party, a model collaboration request according to the machine learning model, the collaborative stage and the request information, and send the model collaboration request to the model collaboration party; wherein the model collaboration request includes a request type identifier, a task identifier and description, a model requirement specification and a performance expectation index.

[0056] The application provides a machine learning model cooperation method and system for a 5G communication system, a model user is determined to match the 5G communication system, and a model use request sent by a user is received through the model user; wherein the model user is a first MDAF or a first NWDAF; whether the 5G communication system meets a model cooperation event trigger condition is determined through the model user; if the model cooperation event trigger condition is met, a model cooperation party matching the model user is determined according to the model use request through the model user, and a model cooperation request is generated according to the model use request, so as to send the model cooperation request to the model cooperation party; wherein the model cooperation party is a second MDAF or a second NWDAF or a third MDAF; the model cooperation request is received through the model cooperation party, and whether the 5G communication system meets a cooperation trigger condition is judged; if the cooperation trigger condition is met, a corresponding cooperation operation request is generated according to the model cooperation request through the model cooperation party, and a corresponding cooperation operation request is sent to a corresponding model generation party; the model generation party feeds back a corresponding cooperation response information to the model cooperation party according to the cooperation operation request, so that the model cooperation party performs a corresponding cooperation operation according to the cooperation response information. As can be seen, the model user, the model cooperation party and the model generation party are introduced, so as to complete the cooperation between the machine learning models of the 5G communication system by using the model user, the model cooperation party and the model generation party, thereby improving the overall performance of the 5G communication system. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0058] Figure 1 A flowchart of a machine learning model cooperation method for a 5G communication system provided by the embodiments of the present application;

[0059] Figure 2 A structure diagram of a machine learning model cooperation system for a 5G communication system provided by the embodiments of the present application. DETAILED DESCRIPTION

[0060] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0061] In the present application, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the sentence "including a…" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0062] Referring to Figure 1 , a flowchart of a machine learning model coordination method for a 5G communication system is shown, which is suitable for a 5G communication system, and the method specifically includes the following steps:

[0063] S101: Determine the model user matched with the 5G communication system, and receive the model use request sent by the user through the model user.

[0064] In the process of specifically executing step 101, when detecting that the model use request initiated by the user based on the 5G communication is detected, the model user matched with the 5G communication system can be determined according to the domain to which the 5G communication system belongs, so as to receive the model use request initiated by the user by using the determined model user.

[0065] It should be noted that the domain to which the 5G communication system belongs can be a core network (Core Network, CN) domain or a RAN domain; in the case where the domain to which the 5G communication system belongs is the CN domain, the user initiating the model use request based on the 5G communication system can be a consumer of the CN domain; in the case where the domain to which the 5G communication system belongs is the RAN domain, the user initiating the model use request based on the 5G communication system can be a consumer of the RAN domain.

[0066] In some embodiments, in the case that the domain to which the 5G communication system belongs is a RAN domain, the consumer of the RAN domain can initiate a corresponding model usage request based on the wireless access network data and service requirements local to the 5G communication system, wherein the model usage request includes request information, and the request information at least includes the wireless access network data and the service requirements.

[0067] For example, the consumer of the RAN domain can initiate a corresponding model usage request (service requirements) and wireless access network data for model application in aspects such as signal optimization of the base station, user access management, base station energy saving analysis, etc.

[0068] In other embodiments, in the case that the domain to which the 5G communication system belongs is a CN domain, the consumer of the CN domain can initiate a corresponding model usage request around service requirements such as corresponding slice load analysis, UE mobility analysis, service experience, etc., wherein the model usage request includes request information, and the request information at least includes the service requirements.

[0069] Optionally, the process of determining the model usage party matched with the 5G communication system and receiving the model usage request sent by the user through the model usage party can be: determining the domain to which the 5G communication system belongs, and taking the artificial intelligence service producer corresponding to the domain as the model usage party matched with the 5G communication system; wherein the domain is a RAN domain, and the artificial intelligence service producer corresponding to the domain is a first MDAF (for easy distinction, the MDAF in the RAN domain matched with the 5G communication system can be referred to as the first MDAF) corresponding to the domain; or the domain is a CN domain, and the artificial intelligence service producer corresponding to the domain is a first NWDAF (for easy distinction, the NWDAF in the CN domain matched with the 5G communication system can be referred to as the first NWDAF) corresponding to the domain; and receiving the model usage request sent by the user based on the request information corresponding to the domain through the model usage party.

[0070] S102: Determine whether the 5G communication system meets the model coordination event trigger condition through the model usage party; if the model coordination event trigger condition is met, execute step S103;

[0071] In the embodiments of the present application, the corresponding model collaboration event trigger condition can be pre-set, so that after the model user matched with the 5G communication system receives the model use request, the network state information, service processing information and network device information of the 5G communication system can be further obtained, and it is determined whether the 5G communication system meets the pre-set model collaboration event trigger condition according to the obtained network state information, service processing information and network device information; if it meets, step S103 can be executed to initiate the corresponding model collaboration request; if it does not meet, the network state information, service processing information and network device information of the 5G communication system can be continuously obtained until the 5G communication system meets the model collaboration event trigger condition according to the obtained network state information, service processing information and network device information, and then step S103 is executed.

[0072] It should be noted that the model collaboration event trigger condition can include a network performance threshold trigger condition, a time-related trigger condition and an event-driven trigger condition.

[0073] Optionally, the process of determining whether the 5G communication system meets the model collaboration event trigger condition by the model user can be: obtaining the network state information, system information and network information of the 5G communication system by the model user; determining whether the 5G communication system meets the network performance threshold trigger condition according to the network state information, whether the 5G communication system meets the time-related trigger condition according to the system information, and whether the 5G communication system meets the event-driven trigger condition according to the network information by the model user; if the 5G communication system meets the network performance threshold trigger condition, and / or, meets the time-related trigger condition, and / or, meets the event-driven trigger condition, it is determined by the model user that the 5G communication system meets the model collaboration event trigger condition; if the 5G communication system does not meet the network performance threshold trigger condition, does not meet the time-related trigger condition, and does not meet the time-driven trigger condition, it is determined by the model user that the 5G communication system does not meet the model collaboration event trigger condition.

[0074] In some embodiments, the network performance threshold trigger condition indicates that the network delay of the 5G communication system exceeds the preset network delay threshold, and the connection number of the base station corresponding to the 5G communication system grows at a speed not less than the first preset growth speed in the first preset time period.

[0075] In the actual application process, the network state information at least includes network delay of the 5G communication system and growth rate of the connection number of the corresponding base station in the 5G communication system within a preset time period; the model user can determine whether the network delay of the 5G communication system exceeds a preset network delay threshold and whether the growth rate of the connection number of the corresponding base station in the 5G communication system within a first preset time period is less than a first preset growth rate; if the preset network delay threshold is exceeded and the first preset growth rate is not less than the first preset growth rate, it is determined that the 5G communication system satisfies the network performance threshold triggering condition; otherwise, it is determined that the 5G communication system does not satisfy the network performance threshold triggering condition.

[0076] It should be noted that when each threshold indicated by the network performance threshold triggering condition is satisfied, it indicates that the current network state of the 5G communication system may need to start the corresponding model collaboration process (model training, model testing or model inference), and thus the corresponding model collaboration process can be triggered.

[0077] It should be further noted that the preset network delay threshold can be 50 ms, the first preset time period can be 1 minute, and the first preset growth rate can be 30%, and the corresponding preset network delay threshold, first preset time period and first preset growth rate can be set according to actual application, which is not limited herein.

[0078] In some embodiments, the time-related triggering condition indicates that the system time of the 5G communication system is within a second preset time period.

[0079] In the actual application process, the system information includes the current system time of the 5G communication system; the model user can determine whether the current system time of the 5G communication system is within the second preset time period; if so, it is determined that the 5G communication system satisfies the time-related triggering condition, otherwise, it is determined that the 5G communication system does not satisfy the time-related triggering condition.

[0080] It should be noted that the second preset time period can be a business peak period (such as 7-10 pm) every day or a specific time period after a new service is put online (such as two hours after the new service is put online), which is not limited herein.

[0081] For example, after a new high-definition video service is put online, the training / inference / testing of the related network bandwidth prediction model is automatically triggered within the next 2 hours, that is, the corresponding model collaboration process is triggered, so as to better provide resource guarantee for the service.

[0082] In some embodiments, the event-driven triggering condition indicates that any event in a preset event set occurs in the network of the 5G communication system. The preset event set can be an event of a specific type of error code occurring in the network of the 5G communication system, an event of a new device accessing the network of the 5G communication system. In some embodiments, the event-driven triggering condition indicates that any event in a preset event set occurs in the network of the 5G communication system. The preset event set can be an event of a specific type of error code occurring in the network of the 5G communication system, an event of a new device accessing the network of the 5G communication system.

[0083] It should be noted that in the 5G network, a specific type of error code is a kind of identification code generated by the network device or system in the 5G communication system when encountering specific problems or abnormal situations during operation. These error codes are defined according to certain standards and specifications, and the purpose is to quickly and accurately convey the type of problem and related information in the network. For example, for the device running state, error code "D010" may indicate that the hardware module (such as the radio frequency module) of a certain base station has failed, affecting signal transmission and reception; error code "D020" may indicate that the software system of the device has an abnormality, such as memory overflow, process crash, etc. When a specific network connection error code appears, the test or inference of the network fault diagnosis model is triggered, that is, the corresponding model collaboration process is triggered to quickly locate the problem.

[0084] In actual application process, the model user can judge whether any event in the preset event set exists in the network information, if exists, determine that the 5G communication system meets the event-driven trigger condition, otherwise, determine that the 5G communication system does not meet the event-driven trigger condition.

[0085] S103: Determine the model collaboration party matched with the model user according to the model use request, and generate a model collaboration request according to the model use request, to send the model collaboration request to the model collaboration party.

[0086] In the process of specifically executing step S103, the model user can further determine the corresponding machine learning model and the collaboration stage of the machine learning model according to the model use request in the case that the 5G communication system meets the model collaboration event trigger condition, and determine the model collaboration party matched with the model user according to the domain to which the model user belongs and the collaboration stage of the machine learning model, and finally generate a model collaboration request according to the machine learning model and the model use request, to send the model collaboration request to the model collaboration party.

[0087] Optionally, the process that the model usage side determines the model collaboration side matched with the model usage side according to the model usage request and generates the model collaboration request according to the model usage request to send the model collaboration request to the model collaboration side through the model usage side can be: the model usage side analyzes the request information to determine the machine learning model corresponding to the model usage request and its collaboration stage, and judges whether the machine learning model needs cross-domain collaboration or same-domain collaboration in the collaboration stage; if it needs same-domain collaboration, and the domain is RAN domain or CN domain, the model usage side determines the second MDAF or the second NWDAF corresponding to the RAN domain or the CN domain as the model collaboration side matched with the model usage side, that is, the model usage side determines the second MDAF in the RAN domain as the model collaboration side or determines the second NWDAF in the CN domain as the model collaboration side; if it needs cross-domain collaboration, and the domain is RAN domain or CN domain, the model usage side determines the third MDAF corresponding to the management domain as the model collaboration side matched with the model usage side; the model usage side generates the model collaboration request according to the machine learning model, the collaboration stage and the request information, and sends the model collaboration request to the model collaboration side; wherein the model collaboration request includes request type identifier, task identifier and description, model requirement specification and performance expectation index.

[0088] In some embodiments, the model usage side can further determine the machine learning model indicated by the user according to the service requirement in the request information in the model usage request and determine the collaboration stage in which the machine learning model currently locates in the case of determining that the 5G communication system meets the model collaboration event triggering condition. It is judged whether the machine learning model needs data collaboration of other domains to complete in the collaboration stage it currently locates in; if it needs, it is determined that the machine learning model needs cross-domain collaboration in the collaboration stage, otherwise, it is determined that the machine learning model does not need cross-domain collaboration, that is, needs same-domain collaboration.

[0089] It should be noted that the collaboration stage of the machine learning model can be the inference stage, or the training stage, or the test stage.

[0090] In some embodiments, the model usage side can generate the request type identifier corresponding to the collaboration stage of the machine learning model to clarify which stage to collaborate in; analyze the service requirement in the request information to determine the corresponding request task, and generate the task identifier and description corresponding to the request task, and generate the model requirement specification corresponding to the request task; determine the performance expectation index corresponding to the request task, and finally generate the corresponding model collaboration request according to the request type identifier, the task identifier and the description, the model requirement specification and the performance expectation index.

[0091] It should be noted that the request task can be: a task network congestion prediction task (corresponding to the request task of the inference stage), a network traffic prediction model training task (corresponding to the request task of the training stage), and a network fault diagnosis model test task (corresponding to the request task of the test stage); accordingly, the task identifier and the description clearly describe the core target of the request task, so that the model coordination party and the model generation party can quickly understand the nature of the task.

[0092] In some embodiments, the model requirement specification can be: a model requirement specification for a request task of a training stage, or a model requirement specification for a request task of a test stage, or a model requirement specification for a request task of an inference stage.

[0093] It should be noted that the model requirement specification for the request task of the training stage can include: a description of the business scenario to which the required machine learning model is directed (such as traffic prediction business for a specific base station, resource allocation business for a core network slice, etc.) and the key capabilities that the machine learning model is expected to have (such as the ability to effectively process a specific type of network data, good scalability, etc.). For example, in the base station traffic prediction model training task, it is specified that the model needs to accurately process real-time traffic data and historical traffic trend information of the base station, and the model structure is convenient for subsequent optimization and expansion.

[0094] The model requirement specification for the request task of the test stage can include: the business field to which the machine learning model to be tested is applied and the main functions it should have in that field. For example, in the test task of the network fault diagnosis model, it is mentioned that the model needs to accurately identify common network fault types and quickly diagnose based on specific network data.

[0095] The model requirement specification for the request task of the inference stage can include: the characteristics of the business data to be processed by the required inference machine learning model and the expected inference effect. For example, in the inference request of the network congestion prediction model, it is indicated that the model needs to be able to quickly process real-time traffic and connection state data of the current network, and accurately predict whether congestion will occur in the short term.

[0096] It should be noted that the performance expectation index corresponding to the request task of the training stage can include: model convergence index (such as training loss value reduced to a certain level, accuracy reached to a certain level, etc.), training time upper limit index, etc. For example, the network traffic prediction model is expected to reduce the loss value to below 0.1 within 10 training cycles.

[0097] The performance expectation index corresponding to the request task in the test phase can include: an acceptable test accuracy range (such as not less than 80%), an upper limit of false positive rate, and other key performance indicators. For example, in the test phase corresponding to the network fault diagnosis model, the accuracy is required to be not less than 90%, and the false positive rate is required to be not more than 5%.

[0098] The performance expectation index corresponding to the request task in the inference phase can include: an acceptable prediction accuracy range (such as not less than 80%), a maximum response time (such as within 5 minutes), and other key performance indicators, which provide a reference for subsequent model selection and collaboration.

[0099] In step S104, the model collaboration party receives the model collaboration request and determines whether the 5G communication system meets the collaboration trigger condition. If the collaboration trigger condition is met, step S105 is performed.

[0100] In the process of specifically performing step S104, after receiving the model collaboration request sent by the model user, the model collaboration party can further obtain a plurality of network key performance indicators, data flow information and service type information of the 5G communication system, so as to determine whether the 5G communication system meets the collaboration trigger condition according to the plurality of network key performance indicators, data flow information and service type information; if the collaboration trigger condition is met, step S105 is performed; otherwise, the plurality of network key performance indicators, data flow information and service type information of the 5G communication system are reacquired until it is determined that the 5G communication system meets the collaboration trigger condition according to the obtained network key performance indicators, data flow information and service type information, and then step S105 is performed.

[0101] It should be noted that the collaboration trigger condition indicates that any one or more network key performance indicators of the 5G communication system is not located in the corresponding preset range, and / or there is a service with a growth rate of service traffic in a third preset time period not less than a second preset growth rate in the 5G communication system.

[0102] Optionally, the process that the model coordination party receives the model coordination request and determines whether the 5G communication system meets the coordination trigger condition can be: the model coordination party acquires a plurality of network key performance indicators, data traffic information, and service type information of the 5G communication system; the model coordination party determines whether each network key performance indicator is within the corresponding preset range, and determines whether there is a service whose traffic growth rate is not less than a second preset growth rate in a third preset time period in the 5G communication system according to the data traffic information and the service type information; if any one or more network key performance indicators is not within the corresponding preset range, and / or there is a service whose traffic growth rate is not less than the second preset growth rate in the third preset time period, the model coordination party determines that the 5G communication system meets the coordination trigger condition; if each network key performance indicator is within the corresponding preset range, and there is no service whose traffic growth rate is not less than the second preset growth rate in the third preset time period, the model coordination party determines that the 5G communication system does not meet the coordination trigger condition.

[0103] In some embodiments, the corresponding preset range of each network key performance indicator is preset, wherein the plurality of network key performance indicators can include key performance indicators of the 5GC and the RAN.

[0104] For example, the plurality of network key performance indicators can include network throughput, packet loss rate, base station load, and the like. When any one or more of these indicators abnormally fluctuates or exceeds the preset range, the corresponding model coordination process (model training, model testing, or model inference) is triggered. For example, when the network throughput is less than 40% (corresponding to the preset range), the model coordination party initiates the corresponding model coordination process to send the corresponding model coordination request to the model generation party.

[0105] In some embodiments, the model coordination party can analyze the data traffic change trend and the service type distribution in the network of the 5G communication system according to the data traffic information and the service type information, so as to determine the traffic growth rate of each service in the third preset time period according to the data traffic change trend and the service type distribution, and determine whether there is a service whose traffic growth rate is not less than the second preset growth rate in the third preset time period in the 5G communication system; if there is a service whose traffic growth rate is not less than the second preset growth rate in the third preset time period, the model coordination party initiates the corresponding model coordination process to send the corresponding model coordination request to the model generation party.

[0106] It should be noted that the third preset time period can be a specific time period preset, and the second preset growth rate can be 50%. The third preset time period and the second preset growth rate can be set according to actual application, which is not limited herein.

[0107] For example, if the service traffic of an online game service increases by more than 50% in a certain time period, the model coordination party can organize the corresponding model to cooperatively analyze the network demand and potential problems of the service, which may involve training a new traffic prediction model or testing and optimizing the existing model, i.e., initiating a corresponding model coordination process to send a corresponding model coordination request to the model generating party to make corresponding coordination operations.

[0108] S105: The model coordination party generates a corresponding coordination operation request according to the model coordination request and sends the corresponding coordination operation request to the corresponding model generating party.

[0109] In the process of specifically performing step S105, the model coordination party can generate a corresponding coordination operation request according to the request type identifier, task identifier and description, model requirement description and performance expectation index in the received model coordination request, and send the coordination operation request to the model generating party when it is determined that the coordination condition is met.

[0110] Optionally, the process of generating a corresponding coordination operation request by the model coordination party according to the model coordination request and sending the corresponding coordination operation request to the corresponding model generating party can be: determining the coordination stage of the corresponding machine learning model according to the request type identifier by the model coordination party, and generating the coordination task details and resource and time estimation of the machine learning model in the coordination stage according to the task identifier and description, performance expectation index and model requirement description; generating the coordination operation request by the model coordination party according to the coordination task details and resource and time estimation, and sending the coordination operation request to the corresponding model generating party.

[0111] In some embodiments, after the model coordination party determines the coordination stage of the machine learning model according to the request type identifier, it can refine the task details of the machine learning model in the coordination stage according to the task identifier and description, performance expectation index and model requirement description of the machine learning model to generate the corresponding coordination task details and resource and time estimation. The coordination task details can include information describing the task target, model requirement and performance expectation of the machine learning model, and the resource and time estimation can include the resource estimation and time estimation required for the model generating party to participate in the coordination.

[0112] It should be noted that, in the case of the model training phase as the collaboration phase, the collaboration task details can include the number of iterations of the machine learning model training, the optimization algorithm, and the like; in the case of the model testing phase as the collaboration phase, the collaboration task details can include the method of testing the machine learning model and the evaluation standard, and the like; in the case of the model inference phase as the collaboration phase, the collaboration task details can include the real-time requirement of the inference corresponding to the machine learning model and the result output format.

[0113] It should also be noted that the resource estimation can include the number of CPU cores and the size of memory required for the model generation party to collaborate, and the time estimation can include the estimated duration required by the model generation party, the upper limit of time for the model training / test / inference phase, so that the model generation party can evaluate whether its own ability meets the requirements according to the resource and time estimation.

[0114] It should be noted that, in the case of the model training phase as the collaboration phase, in determining the corresponding resource and time estimation, factors such as data preparation and iteration period of model training need to be considered; in the case of the model testing phase as the collaboration phase, in determining the corresponding resource and time estimation, the processing and evaluation time of the test data need to be considered; in the case of the model inference phase as the collaboration phase, in determining the corresponding resource and time estimation, the real-time requirement of the inference and the possible concurrent processing time need to be considered.

[0115] S106: The model generation party feeds back the corresponding collaboration response information to the model collaboration party according to the collaboration operation request.

[0116] In the process of specifically executing step S106, after receiving the collaboration operation request sent by the model collaboration party, the model generation party can evaluate whether it can participate in the collaboration according to the collaboration task details in the collaboration operation request and the resource and time estimation; if it can participate, it feeds back the collaboration response information including the consent identifier indicating the consent to participate in the collaboration work and the model information of the corresponding machine learning model; if it cannot participate, it feeds back the collaboration response information including the refusal identifier indicating the refusal to participate in the collaboration work and the response information.

[0117] It should be noted that the response information can include the reason why the model generator cannot participate in the collaboration; the model information includes the model version information of the available machine learning model, the performance indicators (such as historical accuracy, recall rate, etc. in similar tasks), the load condition (such as CPU usage, memory occupancy), and the required resource support (such as at least 2GB memory, 1 CPU core). For different request types, the focus of the model information is also different. In the model training stage, the scalability and adaptability of the model to new data can be emphasized; in the model testing stage, the performance of the model on the existing test data is highlighted; in the model inference stage, the inference efficiency and stability of the model are concerned.

[0118] S107: performing a corresponding collaborative operation according to the collaborative response information by the model collaboration party.

[0119] In the specific execution process of step S107, after receiving the collaborative response information fed back by the model generator, the model collaboration party can perform the response collaborative operation according to the identification and information in the collaborative response information. The identification can be an approval identification or a rejection identification, and the information can be response information or model information.

[0120] Optionally, the process of performing a corresponding collaborative operation according to the collaborative response information by the model collaboration party can be: judging by the model collaboration party whether the collaborative response information contains a rejection identification or an approval identification; if the rejection identification is contained, generating and outputting corresponding prompt information according to the response information in the collaborative response information by the model collaboration party; if the approval identification is contained, establishing a corresponding model group object according to the model information in the collaborative response information by the model collaboration party, and generating a corresponding collaborative learning report, and feeding back the collaborative learning report to the model user.

[0121] It should be noted that the model group object can include a model member list and a model collaboration mode.

[0122] In some embodiments, the model member list includes the model identification, model version, and model generator to which each machine learning model participating in the collaboration belongs. Specifically, the model collaboration party can reasonably select machine learning models participating in the collaboration according to the corresponding collaborative stage (model training stage / model testing stage / model inference stage), and combine according to the selected machine learning models to generate a corresponding model member list.

[0123] For example, in the model training phase, a more accurate network slice future load prediction model is collaboratively trained to guide resource dynamic adjustment. The corresponding model member list can include: a NWDAF network slice instance load historical data analysis model that analyzes the historical load data of the network slice instance stored in the corresponding NWDAF (the historical load data includes CPU utilization, memory occupancy, traffic load, etc.) to extract the trend and periodicity of load changes according to the historical load data (corresponding to the NWDAF's "network slice instance load level calculation and prediction"). Or include a MDAF network slice traffic prediction model that uses the capabilities of MDA to analyze historical traffic data of the network slice and predict future traffic trends as an important input for load prediction.

[0124] In the model testing phase, the performance of the UE abnormal behavior detection model is collaboratively tested. The corresponding model member list can include: a MDAF-assisted fault management-fault prediction model to use the fault prediction capabilities of MDAF to analyze network fault patterns related to UE abnormal behavior to provide a reference benchmark for testing the UE abnormal behavior detection model (if MDAF predicts that congestion may occur in a certain cell, the performance of the UE abnormal behavior detection model in the high load situation of the cell can be tested (corresponding to the MDAF's "MDA-assisted fault management including fault prediction and service fault recovery"). Or include a MDAF management data prediction and statistical UE-related KPI prediction model to use the management data prediction capabilities of MDAF to predict future changes in KPI indicators related to UE behavior (such as access success rate, drop rate, etc.) and compare these prediction results with the output of the UE abnormal behavior detection model of NWDAF to evaluate the prediction accuracy of the model.

[0125] In the model inference phase, the precise prediction of current and future congestion information in a specific location is collaboratively performed. The corresponding model member list can include a NWDAF specific location current and future congestion information prediction model to use the capabilities of NWDAF to directly predict the current and future congestion information of a specific location, such as the congestion probability and degree at the cell level (corresponding to the NWDAF's "specific location current and future congestion information prediction"). Or it can include a NWDAF network load performance calculation and future load prediction model to analyze the network load performance data stored in the NWDAF to predict the network load situation in the future period of time to provide macro background information for congestion prediction in a specific location (corresponding to the NWDAF's "network load performance calculation and future load prediction"). Or include a MDAF coverage-related analysis-coverage problem analysis model to use the coverage analysis capabilities of MDAF to identify coverage problems that may cause congestion in a specific location, such as weak coverage areas, signal interference, etc. to provide potential cause analysis for congestion prediction.

[0126] It should be noted that the model coordination mode can include a serial mode, a parallel mode, and a mixed coordination mode.

[0127] The serial mode includes serial coordination and serial collaboration processes. In serial coordination, the execution order of each machine learning model is clear, ensuring that the output of the previous machine learning model can be correctly used as the input of the subsequent machine learning model. In the model member list, the execution order of each machine learning model needs to be clear.

[0128] For example, predicting the future QoS change of a specific UE and providing protection, the execution order of each model in the model member list of the corresponding serial mode can be: model 1, NWDAF UE expected behavior prediction model, model 2, NWDAF predicted QoS change model, and model 3, management domain MDAF resource allocation optimization model.

[0129] In the serial collaboration process, the NWDAF UE expected behavior prediction model analyzes the historical data of the UE and outputs the future behavior prediction of the UE; the NWDAF predicted QoS change model receives the UE behavior prediction and predicts the QoS change trend in combination with the current network state; and the management domain MDAF resource allocation optimization model receives the QoS change prediction and formulates a corresponding resource allocation strategy to protect the QoS.

[0130] The parallel mode includes parallel coordination and parallel collaboration processes. In parallel coordination, multiple machine learning models simultaneously receive the same input data and independently perform inference or calculation. In the model member list, all parallel-executed machine learning models need to be listed, and the weighting method of the results needs to be defined. For example, to evaluate the service level agreement (SLA) compliance of a network slice in multiple dimensions, the corresponding model member list can include: model 1, NWDAF network slice instance load level calculation model, for calculating the real-time load level of the current network slice instance (complying with the NWDAF “network slice instance load level calculation and prediction”); model 2, management domain MDAF network slice throughput analysis model, for analyzing the actual throughput performance of the network slice; and model 3, management domain MDAF end-to-end delay analysis model, for analyzing the end-to-end delay of the network slice.

[0131] In the parallel collaboration process, the model coordinator requests the NWDAF and the management domain MDAF (model 1, model 2, and model 3) to simultaneously analyze a specific network slice. The NWDAF calculates the real-time load level of the slice instance, and the management domain MDAF analyzes the throughput and end-to-end delay of the slice. The model coordinator receives the analysis results of the three models, and according to the indicators and weights defined in the SLA, comprehensively evaluates the load level, throughput, and delay to determine whether the SLA is met.

[0132] In the hybrid collaborative mode, part of the machine learning models are executed in parallel, and their output results are used as the input of one or more subsequent serial machine learning models. Among them, the hybrid collaborative mode specifies the model group that needs to be executed in parallel in the model member list and the order of serial execution of each model. For example, based on the predicted network load to make active RAN node software upgrade decision, the model member group can include: model 1 NWDAF network load performance calculation and future load prediction model, which is used to predict the network load in the future period of time; model 2 management domain MDAF resource related analysis model, which is used to analyze the resource utilization of the current RAN node; model 3 management domain MDAF assisted key maintenance management-RAN node software upgrade recommendation model, which is used to receive the output of model 1 and model 2, combine the predicted load growth trend and the resource utilization of the RAN node, judge whether software upgrade is needed, and give upgrade suggestion.

[0133] In the collaborative process, model 1 NWDAF network load performance calculation and future load prediction model predicts future network load. Model 2 management domain MDAF resource related analysis model analyzes the resource utilization of the current RAN node. These two models are executed in parallel, and the model coordinator receives the load prediction result and the RAN node resource utilization analysis result, and sends them to model 3 management domain MDAF assisted key maintenance management-RAN node software upgrade recommendation model; among them, model 3 management domain MDAF assisted key maintenance management-RAN node software upgrade recommendation model judges whether RAN node software upgrade is needed according to the received information and gives suggestion.

[0134] In some embodiments, the model coordination party can also generate a corresponding collaborative learning report, wherein the collaborative learning can include task overview, model collaboration details, result analysis, and problems and suggestions.

[0135] It should be noted that the task overview can include the task name of the current request task, the trigger condition (model collaboration event trigger condition and collaboration trigger condition), and the collaboration time, etc. basic information. It is clearly stated that the request task is a training task, a test task or an inference task.

[0136] The model collaboration details include: listing each machine learning model involved and its performance in the collaboration process (such as running time, output result), data interaction between models (such as data transfer times, data size). For different request tasks, the specific performance of the model in each task is described in detail. For example, in the training task, the training loss value change, convergence condition, etc. of the model are reported; in the test task, the test accuracy, recall rate, etc. of the model are presented; in the inference task, the accuracy of the inference result and the response time are recorded.

[0137] Result analysis includes: detailed analysis of the collaborative results of each machine learning model in the model member list according to the model collaborative mode, whether the expected performance indicators are reached, etc. According to the task type (training / test / anti-thrust), targeted analysis is performed, such as training task to evaluate whether the model achieves the expected training effect; test task to determine whether the model meets the test standard; reasoning task to verify whether the reasoning result meets the actual network demand.

[0138] Problems and suggestions: summarize the problems (such as model compatibility problems) that occur during the collaboration process and provide suggestions for future collaboration work (such as model update requirements). For different task types, the problems and suggestions are different. In the training task, suggestions such as model structure adjustment may be involved; in the test task, suggestions such as improving the test method or optimizing the model parameters are proposed; in the reasoning task, the improvement direction of how to improve the reasoning efficiency and accuracy is focused on.

[0139] In some embodiments, after the model user receives the corresponding collaborative learning report, it makes decisions according to the content in the collaborative learning report. For example, if the model user determines that the collaborative result of this time is satisfactory according to the collaborative learning report, it can end this collaborative process; if the result is not satisfactory, it can decide to re-initiate a new round of request collaboration process, that is, to re-initiate a new model collaboration request to the model collaboration party, and in the new request, the information such as task identification and description, model requirement description and trigger condition can be adjusted according to the experience of the last time.

[0140] In summary, by introducing the roles of model user, model collaboration party and model generation party, the model member list is created based on the model user, model collaboration party and model generation party, and the collaborative operation of each machine learning model in the model member list is controlled according to the model collaboration mode, and the corresponding collaborative report information is generated. As can be seen, by constructing a tightly coupled cross-domain / same-domain model collaboration system, the traditional domain barriers are broken down, thereby realizing the deep integration and efficient collaboration between 5GC, RAN and 3GPP management system, that is, the collaboration of each machine learning model in the 5G communication system, which provides a solid foundation for the overall improvement of network performance. For example, in the network optimization task, the advantages of models in different domains can be fully integrated to realize all-round optimization from wireless access to core network, significantly improving the throughput and stability of the network.

[0141] The present application can also select the most suitable collaboration mode (serial mode, parallel mode or mixed mode) according to the characteristics of the requested task and the real-time state of the network. In processing complex tasks, such as network fault diagnosis, multiple models can be dynamically combined for collaborative work, fully utilizing the advantages of different models in different stages, greatly improving the accuracy and efficiency of diagnosis, and effectively improving the reliability and maintainability of the network.

[0142] The application provides a machine learning model cooperation method for a 5G communication system, a model user is determined to match the 5G communication system, and a model use request sent by a user is received through the model user; wherein the model user is a first MDAF or a first NWDAF; whether the 5G communication system meets a model cooperation event trigger condition is determined through the model user; if the model cooperation event trigger condition is met, a model cooperation party matching the model user is determined through the model user according to the model use request, and a model cooperation request is generated according to the model use request to send the model cooperation request to the model cooperation party; wherein the model cooperation party is a second MDAF or a second NWDAF or a third MDAF; the model cooperation request is received through the model cooperation party, and whether the 5G communication system meets a cooperation trigger condition is determined; if the cooperation trigger condition is met, a corresponding cooperation operation request is generated through the model cooperation party according to the model cooperation request, and the corresponding cooperation operation request is sent to a corresponding model generation party; the corresponding cooperation response information is fed back to the model cooperation party through the model generation party according to the cooperation operation request, so that the model cooperation party performs corresponding cooperation operations according to the cooperation response information. As can be seen, the model user, the model cooperation party and the model generation party are introduced to complete the cooperation between the machine learning models of the 5G communication system by using the model user, the model cooperation party and the model generation party, so as to improve the overall performance of the 5G communication system.

[0143] Based on the machine learning model cooperation method for the 5G communication system provided in the above embodiments of the application, the embodiments of the application also provide a machine learning model cooperation system for the 5G communication system, as shown in Figure 2 The machine learning model cooperation system for the 5G communication system includes:

[0144] The receiving module 21 is used to determine a model user matching the 5G communication system, and receive a model use request sent by a user through the model user; wherein the model user is a first MDAF or a first NWDAF;

[0145] The first determining module 22 is used to determine whether the 5G communication system meets a model cooperation event trigger condition through the model user;

[0146] The first sending module 23 is used to determine a model cooperation party matching the model user through the model user according to the model use request if the model cooperation event trigger condition is met, generate a model cooperation request according to the model use request, and send the model cooperation request to the model cooperation party; wherein the model cooperation party is a second MDAF or a second NWDAF or a third MDAF;

[0147] The second determining module 24 is configured to receive a model coordination request through a model coordination party, and determine whether the 5G communication system satisfies a coordination trigger condition.

[0148] The second sending module 25 is configured to, if the coordination trigger condition is satisfied, generate a corresponding coordination operation request according to the model coordination request through the model coordination party, and send the corresponding coordination operation request to a corresponding model generation party.

[0149] The feedback module 26 is configured to feed back corresponding coordination response information to the model coordination party according to the coordination operation request through the model generation party.

[0150] The coordination module 27 is configured to perform corresponding coordination operations according to the coordination response information through the model coordination party.

[0151] The above-mentioned embodiments of the present application disclose the specific principles and execution processes of each unit in the machine learning model coordination system for the 5G communication system, which are the same as the machine learning model coordination method for the 5G communication system disclosed by the above-mentioned embodiments of the present application. For details, refer to the corresponding part in the machine learning model coordination method for the 5G communication system disclosed by the above-mentioned embodiments of the present application, which will not be repeated here.

[0152] The present application provides a machine learning model coordination method for a 5G communication system, which determines a model user matched with the 5G communication system, and receives a model use request sent by a user through the model user; wherein the model user is a first MDAF or a first NWDAF; determines whether the 5G communication system satisfies a model coordination event trigger condition through the model user; if the model coordination event trigger condition is satisfied, determines a model coordination party matched with the model user according to the model use request through the model user, and generates a model coordination request according to the model use request to send the model coordination request to the model coordination party; wherein the model coordination party is a second MDAF or a second NWDAF or a third MDAF; receives the model coordination request through the model coordination party, and determines whether the 5G communication system satisfies a coordination trigger condition; if the coordination trigger condition is satisfied, generates a corresponding coordination operation request according to the model coordination request through the model coordination party, and sends the corresponding coordination operation request to a corresponding model generation party; feeds back corresponding coordination response information to the model coordination party according to the coordination operation request through the model generation party, so that the model coordination party performs corresponding coordination operations according to the coordination response information. As can be seen, the present application introduces a model user, a model coordination party and a model generation party, so as to complete the coordination between the machine learning models of the 5G communication system by using the model user, the model coordination party and the model generation party, thereby improving the overall performance of the 5G communication system.

[0153] Optionally, the receiving module is specifically configured to:

[0154] Determine the domain to which the 5G communication system belongs, and use the domain corresponding artificial intelligence service producer as the model usage side matched with the 5G communication system; wherein, the domain is the RAN domain, and the domain corresponding artificial intelligence service producer is the first MDAF corresponding to the domain; or the domain is the CN domain, and the domain corresponding artificial intelligence service producer is the first NWDAF corresponding to the domain;

[0155] Receive the model usage request sent by the user based on the domain corresponding request information through the model usage side; wherein, the model usage request at least includes the request information.

[0156] Optionally, the first sending module is specifically used for:

[0157] Analyze the request information through the model usage side to determine the machine learning model corresponding to the model usage request and its collaborative stage, and judge whether the machine learning model needs cross-domain collaboration or same-domain collaboration in the collaborative stage;

[0158] If same-domain collaboration is needed, and the domain is the RAN domain or the CN domain, determine the second MDAF or the second NWDAF corresponding to the RAN domain or the CN domain as the model collaboration side matched with the model usage side through the model usage side;

[0159] If cross-domain collaboration is needed, and the domain is the RAN domain or the CN domain, determine the third MDAF corresponding to the management domain as the model collaboration side matched with the model usage side through the model usage side;

[0160] Generate the model collaboration request according to the machine learning model, the collaborative stage and the request information through the model usage side, and send the model collaboration request to the model collaboration side; wherein, the model collaboration request includes the request type identifier, the task identifier and description, the model requirement description and the performance expectation index.

[0161] Optionally, the first judging module is specifically used for:

[0162] Obtain the network state information, system information and network information of the 5G communication system through the model usage side;

[0163] Judge whether the 5G communication system meets the network performance threshold trigger condition according to the network state information, whether the 5G communication system meets the time related trigger condition according to the business processing information, and whether the 5G communication system meets the event driven trigger condition according to the network device information through the model usage side;

[0164] If the 5G communication system meets the network performance threshold trigger condition, and / or, meets the time related trigger condition, and / or, meets the event driven trigger condition, determine that the 5G communication system meets the model collaboration event trigger condition through the model usage side;

[0165] If the 5G communication system does not satisfy the network performance threshold trigger condition, does not satisfy the time-related trigger condition, and does not satisfy the time-driven trigger condition, it is determined by the model usage side that the 5G communication system does not satisfy the model collaborative event trigger condition.

[0166] Optionally, the second determining module is specifically configured to:

[0167] The model collaborator obtains the plurality of network key performance indicators, the data traffic information, and the service type information of the 5G communication system.

[0168] The model collaborator determines whether each network key performance indicator is within the corresponding preset range, and determines whether there is a service with a growth rate of service traffic not less than a second preset growth rate within a third preset time period in the 5G communication system according to the data traffic information and the service type information.

[0169] If any one or more network key performance indicators is not within the corresponding preset range, and / or there is a service with a growth rate of service traffic not less than the second preset growth rate within the third preset time period, the model collaborator determines that the 5G communication system satisfies the collaborative trigger condition.

[0170] If each network key performance indicator is within the corresponding preset range, and there is no service with a growth rate of service traffic not less than the second preset growth rate within the third preset time period, the model collaborator determines that the 5G communication system does not satisfy the collaborative trigger condition.

[0171] Optionally, the second sending module is specifically configured to:

[0172] The model collaborator determines the collaborative stage of the corresponding machine learning model according to the request type identifier, and generates collaborative task details and resource and time estimates in the collaborative stage under the machine learning model according to the task identifier and description, the performance expectation indicator, and the model requirement specification.

[0173] The model collaborator generates a collaborative operation request according to the collaborative task details and the resource and time estimates, and sends the collaborative operation request to the corresponding model generator.

[0174] Optionally, the collaboration module is specifically configured to:

[0175] The model collaborator determines whether the collaborative response information contains a refusal identifier or an agreement identifier.

[0176] If the refusal identifier is contained, the model collaborator generates and outputs corresponding prompt information according to the response information in the collaborative response information.

[0177] If the consent identifier is included, the corresponding model group object is established by the model coordination party according to the model information in the collaborative response information, and the corresponding collaborative learning report is generated, and the collaborative learning report is fed back to the model user, so that the model user makes corresponding decisions according to the collaborative learning report.

[0178] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, it is described more simply, and the related parts can be referred to the part of the method embodiment. The above-described system and system embodiment are only illustrative, and the units described as separate components can be or can not be physically separated, and the components shown as units can be or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0179] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0180] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application should not be limited to the embodiments shown herein, but should be consistent with the widest scope of principles and novel features disclosed herein.

[0181] The above is only the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, several improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method of machine learning model collaboration for a 5G communication system, the method comprising: The method is suitable for a 5G communication system, and comprises the following steps: A model user matched with the 5G communication system is determined, and a model use request sent by a user is received through the model user; wherein the model user is a first MDAF or a first NWDAF; It is determined through the model user whether the 5G communication system meets a model coordination event trigger condition; If the model coordination event trigger condition is met, a model coordination party matched with the model user is determined through the model user according to the model use request, and a model coordination request is generated according to the model use request, so as to send the model coordination request to the model coordination party; wherein the model coordination party is a second MDAF or a second NWDAF or a third MDAF; The model coordination request is received through the model coordination party, and it is determined whether the 5G communication system meets a coordination trigger condition; If the coordination trigger condition is met, a corresponding coordination operation request is generated through the model coordination party according to the model coordination request, and a corresponding coordination operation request is sent to a corresponding model generation party; A corresponding coordination response information is fed back to the model coordination party through the model generation party according to the coordination operation request; A corresponding coordination operation is performed through the model coordination party according to the coordination response information.

2. The method of claim 1, wherein, The determination of the model user matched with the 5G communication system and the reception of the model use request sent by the user through the model user comprise the following steps: A domain to which the 5G communication system belongs is determined, and an artificial intelligence service producer corresponding to the domain is taken as the model user matched with the 5G communication system; wherein the domain is a RAN domain, and the artificial intelligence service producer corresponding to the domain is a first MDAF corresponding to the domain; or the domain is a CN domain, and the artificial intelligence service producer corresponding to the domain is a first NWDAF corresponding to the domain; The model use request sent by the user based on request information corresponding to the domain is received through the model user; wherein the model use request at least comprises the request information.

3. The method of claim 2, wherein, The determination of the model coordination party matched with the model user through the model user according to the model use request and the generation of the model coordination request according to the model use request to send the model coordination request to the model coordination party comprise the following steps: The request information is analyzed through the model user to determine a machine learning model corresponding to the model use request and a coordination stage thereof, and it is determined whether the machine learning model needs cross-domain coordination or same-domain coordination in the coordination stage; If same-domain coordination is needed, and the domain is a RAN domain or a CN domain, a second MDAF or a second NWDAF corresponding to the RAN domain or the CN domain is determined as the model coordination party matched with the model user through the model user; If cross-domain coordination is needed, and the domain is a RAN domain or a CN domain, a third MDAF corresponding to a management domain is determined as the model coordination party matched with the model user through the model user; The model user determines whether the 5G communication system meets the model collaboration event trigger condition, and sends a model collaboration request to the model collaboration party according to the model learning model, the collaboration stage and the request information; wherein the model collaboration request includes request type identification, task identification and description, model requirement specification and performance expectation index.

4. The method of claim 1, wherein, The model user determines whether the 5G communication system meets the model collaboration event trigger condition, and sends a model collaboration request to the model collaboration party according to the model learning model, the collaboration stage and the request information; wherein the model collaboration request includes request type identification, task identification and description, model requirement specification and performance expectation index. The model user obtains network state information, system information and network information of the 5G communication system; The model user determines whether the 5G communication system meets the network performance threshold trigger condition according to the network state information, whether the 5G communication system meets the time-related trigger condition according to the business processing information, and whether the 5G communication system meets the event-driven trigger condition according to the network device information; If the 5G communication system meets the network performance threshold trigger condition, and / or meets the time-related trigger condition, and / or meets the event-driven trigger condition, the model user determines that the 5G communication system meets the model collaboration event trigger condition; If the 5G communication system does not meet the network performance threshold trigger condition, does not meet the time-related trigger condition, and does not meet the time-driven trigger condition, the model user determines that the 5G communication system does not meet the model collaboration event trigger condition.

5. The method of claim 1, wherein, The model collaboration party receives the model collaboration request and determines whether the 5G communication system meets the collaboration trigger condition, including: The model collaboration party obtains a plurality of network key performance indicators, data flow information and service type information of the 5G communication system; The model collaboration party determines whether each network key performance indicator is within the corresponding preset range, and determines whether there is a service in the 5G communication system whose service flow growth rate is not less than a second preset growth rate within a third preset time period according to the data flow information and service type information; If any one or more of the network key performance indicators is not within the corresponding preset range, and / or there is a service whose service flow growth rate is not less than a second preset growth rate within a third preset time period, the model collaboration party determines that the 5G communication system meets the collaboration trigger condition; If each network key performance indicator is within the corresponding preset range, and there is no service whose service flow growth rate is not less than a second preset growth rate within a third preset time period, the model collaboration party determines that the 5G communication system does not meet the collaboration trigger condition.

6. The method of claim 3, wherein, The model collaboration party generates a corresponding collaboration operation request according to the model collaboration request, and sends the corresponding collaboration operation request to the corresponding model generation party, including: The model coordination party identifies a corresponding collaborative stage of a machine learning model according to the request type identifier, and generates collaborative task details and resource and time estimates of the collaborative stage under the machine learning model according to the task identifier and description, the performance expectation indicator, and the model requirement specification; The model coordination party generates a collaborative operation request according to the collaborative task details and the resource and time estimates, and sends the collaborative operation request to a corresponding model generation party.

7. The method of claim 1, wherein, The model coordination party performs corresponding collaborative operations according to the collaborative response information, including: The model coordination party determines whether the collaborative response information contains a refusal identifier or an agreement identifier; If the refusal identifier is contained, the model coordination party generates and outputs corresponding prompt information according to response information in the collaborative response information; If the agreement identifier is contained, the model coordination party establishes a corresponding model group object according to model information in the collaborative response information, generates a corresponding collaborative learning report, and feeds back the collaborative learning report to the model user, so that the model user makes corresponding decisions according to the collaborative learning report. 8.A machine learning model coordination system for a 5G communication system, characterized by, The system is suitable for a 5G communication system, and the system comprises: A receiving module is configured to determine a model user matched with the 5G communication system, and receive a model use request sent by a user through the model user; the model user is a first MDAF or a first NWDAF; A first determining module is configured to determine, through the model user, whether the 5G communication system meets a model coordination event trigger condition; A first sending module is configured to, if the model coordination event trigger condition is met, determine, through the model user, a model coordination party matched with the model user according to the model use request, generate a model coordination request according to the model use request, and send the model coordination request to the model coordination party; the model coordination party is a second MDAF or a second NWDAF or a third MDAF; A second determining module is configured to receive the model coordination request through the model coordination party, and determine whether the 5G communication system meets a coordination trigger condition; A second sending module is configured to, if the coordination trigger condition is met, generate a corresponding collaborative operation request according to the model coordination request through the model coordination party, and send the corresponding collaborative operation request to a corresponding model generation party; A feedback module is configured to feed back, through the model generation party, corresponding collaborative response information to the model coordination party according to the collaborative operation request; A coordination module is configured to perform corresponding collaborative operations according to the collaborative response information through the model coordination party.

9. The system of claim 8, wherein, The receiving module is specifically configured to: determine a domain to which the 5G communication system belongs, and use an artificial intelligence service producer corresponding to the domain as a model usage side matched with the 5G communication system; wherein the domain is a RAN domain, and the artificial intelligence service producer corresponding to the domain is a first MDAF corresponding to the domain; or the domain is a CN domain, and the artificial intelligence service producer corresponding to the domain is a first NWDAF corresponding to the domain; receive, by the model usage side, a model usage request sent by a user based on request information corresponding to the domain; wherein the model usage request at least includes the request information.

10. The system of claim 9, wherein, The first sending module is specifically configured to: analyze, by the model usage side, the request information to determine a machine learning model corresponding to the model usage request and a collaborative stage thereof, and determine whether the machine learning model needs cross-domain collaboration or same-domain collaboration in the collaborative stage; if same-domain collaboration is needed, and the domain is a RAN domain or a CN domain, determine, by the model usage side, a second MDAF or a second NWDAF corresponding to the RAN domain or the CN domain as a model collaboration side matched with the model usage side; if cross-domain collaboration is needed, and the domain is a RAN domain or a CN domain, determine, by the model usage side, a third MDAF corresponding to a management domain as a model collaboration side matched with the model usage side; generate, by the model usage side, a model collaboration request according to the machine learning model, the collaborative stage and the request information, and send the model collaboration request to the model collaboration side; wherein the model collaboration request includes a request type identifier, a task identifier and description, a model requirement specification and a performance expectation index.

Images