Network operation and maintenance methods and devices, storage media and electronic equipment

Abstract

This disclosure provides a network operation and maintenance method, apparatus, storage medium, and electronic device, relating to the field of intelligent network operation and management technology. The method includes: determining network operation and maintenance requirements analysis results based on network operation and maintenance objectives; these results include a requirement for artificial intelligence (AI) capabilities; the AI ​​capabilities include: an AI model and feature data; retraining AI models whose performance indicators do not meet a set threshold, thereby determining an updated AI model; the retraining involves adjusting the model parameters of the AI ​​model and training it using the feature data until the performance indicators of the AI ​​model meet the set threshold; determining a target AI model based on the network operation and maintenance requirements analysis results and the updated AI model; and executing the network operation and maintenance objectives using the target AI model based on the feature data to determine the network operation and maintenance results. This disclosure achieves unified management and application of AI models.

Description

Technical Field

[0001] This disclosure relates to the field of network intelligent operation and management technology, and in particular to a network operation and maintenance method and apparatus, storage medium and electronic equipment. Background Technology

[0002] With the rapid development of intelligent network operation and maintenance, more and more AI (Artificial Intelligence) models are being applied to network operation and maintenance.

[0003] However, each AI model has different predictive capabilities and performance metrics, making it impossible to achieve unified management and shared use of AI models.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a network operation and maintenance method, apparatus, storage medium, and electronic device, which at least to some extent overcomes the problem that AI models in related technologies cannot be uniformly managed and used.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a network operation and maintenance method is provided, comprising:

[0008] Based on the network operation and maintenance objectives, the network operation and maintenance requirements analysis results are determined; the network operation and maintenance requirements analysis results include the requirement for artificial intelligence (AI) capabilities; the AI ​​capabilities include: AI models and feature data;

[0009] The AI ​​model whose performance metrics do not meet the set threshold is retrained to determine the updated AI model; the retraining is to adjust the model parameters of the AI ​​model and train it using the feature data until the performance metrics of the AI ​​model meet the set threshold.

[0010] Based on the network operation and maintenance requirements analysis results and the updated AI model, the target AI model is determined;

[0011] Based on the feature data, the target AI model is used to execute the network operation and maintenance objectives and determine the network operation and maintenance results.

[0012] In some embodiments, the network operation and maintenance requirements analysis results are determined based on network operation and maintenance objectives, including:

[0013] Based on network operation and maintenance goals, define the subject sample feature set and target set;

[0014] Based on the subject sample feature set and the target set, the subject classification result is determined;

[0015] Based on the subject classification results, the semantic similarity calculation results are determined;

[0016] Based on the subject classification results and the semantic similarity calculation results, the network operation and maintenance requirements analysis results are determined.

[0017] In some embodiments, determining the subject classification result based on the subject sample feature set and the target set includes:

[0018] The subject sample feature set is vectorized to determine the vectorized subject sample feature set; the vectorized subject sample feature set includes multiple subject sample feature vectors.

[0019] Based on the target set, the subject classification result is determined by predicting the maximum probability that the feature vector of the subject sample in the vectorized subject sample feature set belongs to the target set using a subject classification algorithm.

[0020] In some embodiments, determining the semantic similarity calculation result based on the subject classification result includes:

[0021] Based on the keyword extraction algorithm, the first keyword of the subject classification result is determined;

[0022] The second keyword for the AI ​​capability management library is determined; the AI ​​capability management library is used to manage existing AI capabilities; the AI ​​capabilities also include: tag data and application data;

[0023] The semantic similarity calculation result is determined based on the semantic similarity between the first keyword and the second keyword.

[0024] In some embodiments, the network operation and maintenance requirements parsing result is determined based on the subject classification result and the semantic similarity calculation result, including:

[0025] The maximum value between the subject classification result and the semantic similarity calculation result is determined as the network operation and maintenance requirements analysis result.

[0026] In some embodiments, retraining an AI model whose performance metrics in the original AI model do not meet a set threshold, and determining the updated AI model, includes:

[0027] Obtain the performance parameters of the original AI model;

[0028] Based on the performance parameters, determine the performance metrics for each AI model;

[0029] AI models whose performance metrics do not meet the set threshold are identified as retrained AI models.

[0030] The retrained AI model is retrained using the feature data until the performance index of the retrained AI model reaches or exceeds the set threshold, and the retrained AI model is output.

[0031] The original AI model and the retrained AI model are identified as the updated AI model.

[0032] In some embodiments, determining the target AI model based on the network operation and maintenance requirements analysis results and the updated AI model includes:

[0033] The network operation and maintenance requirements analysis results are used as matching indicators and matched with the performance indicators corresponding to the updated AI models to determine the first set of AI models that meet the matching indicators.

[0034] The first set of AI models is sorted according to performance metrics other than the matching metric, and the best AI model in the sorting results is determined as the target AI model.

[0035] In some embodiments, based on the feature data, the target AI model is used to execute the network operation and maintenance objectives to determine the network operation and maintenance results, including:

[0036] The target AI model and the feature data are shared according to AI capability recommendations; the AI ​​capability recommendations include at least one or more of the following combinations: standardized sharing, retraining sharing, inference invocation, model embedded reference, and image download;

[0037] The network operation and maintenance objectives are executed using the shared target AI model to determine the network operation and maintenance results;

[0038] Specifically, the standardization sharing is used to directly share the target AI model; the retraining sharing is used to adaptively retrain the target AI model according to usage requirements; the inference call is used to configure the target AI model for model inference in the cloud and call it through the API interface; the model embedded reference is used to encapsulate and embed the target AI model into another system different from the system where the AI ​​model is located; and the image download is used to encapsulate the target AI model and its operating environment into another system different from the system where the AI ​​model is located, with the computing resources provided by the other system.

[0039] In some embodiments, after determining the network operation and maintenance requirements analysis results, the method further includes:

[0040] The network operation and maintenance requirements analysis results will be sent to the user for verification.

[0041] In some embodiments, it also includes:

[0042] In response to the user's approval of the network operation and maintenance requirement analysis results, the network operation and maintenance requirement analysis results are submitted to the subscription application;

[0043] In response to the approval of the subscription application, the network operation and maintenance requirements analysis results are downloaded to the local system in the form of an image or model.

[0044] In some embodiments, the performance metrics include at least one or more combinations of the following: AI model precision, AI model recall, AI model accuracy, AI model error, and AI model computation latency.

[0045] According to another aspect of this disclosure, a network operation and maintenance device is also provided, comprising:

[0046] The network operation and maintenance requirements analysis result determination module is used to determine the network operation and maintenance requirements analysis results based on the network operation and maintenance objectives.

[0047] The original AI model update module is used to retrain AI models whose performance indicators do not meet a set threshold in the original AI model, and to determine the updated AI model; the retraining is to adjust the model parameters of the AI ​​model and train it using the feature data until the performance indicators of the AI ​​model meet the set threshold.

[0048] The target AI model determination module is used to determine the target AI model based on the network operation and maintenance requirements analysis results and the updated AI model.

[0049] The network operation and maintenance result determination module is used to determine the network operation and maintenance result by executing the network operation and maintenance target using the target AI model based on the feature data.

[0050] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform a network operation and maintenance method as described in any of the preceding claims by executing the executable instructions.

[0051] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements a network operation and maintenance method as described in any of the preceding claims.

[0052] According to another aspect of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements a network operation and maintenance method according to any one of the above.

[0053] The network operation and maintenance method, apparatus, storage medium, and electronic device provided in the embodiments of this disclosure first analyze the network operation and maintenance objectives to obtain the network operation and maintenance requirements analysis results, thereby obtaining the requirements for AI capabilities. Then, the AI ​​models whose performance indicators do not meet the set thresholds in the original AI model are retrained to update the original AI model, so that the performance indicators of each AI model in the updated AI model can reach the set thresholds, thereby expanding the application scope of the AI ​​model and facilitating the unified management and application of the AI ​​model. Next, based on the network operation and maintenance requirements analysis results and the updated AI model, the target AI model is determined. Finally, based on feature data, the network operation and maintenance objectives are executed using the target AI model to determine the network operation and maintenance results. This disclosure comprehensively utilizes the network operation and maintenance requirements analysis results to screen the capabilities of the target AI model in the updated AI model. Under the premise of meeting the network operation and maintenance requirements for AI performance indicators, it can obtain the AI ​​model with the optimal performance indicators. Executing the network operation and maintenance objectives using the target AI model can effectively improve the efficiency of network operation and maintenance engineers in network maintenance, improve network quality, meet user needs, and effectively enhance user experience. Meanwhile, telecom operators are among the earliest companies to explore intelligent network operation and maintenance, possessing advantages in users, data, network technology, and system platforms, which facilitates the implementation and application of the technology. This disclosure can be applied to standalone systems and can interact with other internal and external systems through interfaces, providing efficient and high-quality AI models and feature data services, improving the efficiency of network operation and management, and enhancing customer experience.

[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0056] Figure 1 This diagram illustrates the system architecture of a network operation and maintenance method according to an embodiment of the present disclosure.

[0057] Figure 2 A schematic diagram of a network operation and maintenance method in an embodiment of this disclosure is shown.

[0058] Figure 3 This diagram illustrates the process of determining the network operation and maintenance requirements analysis results according to an embodiment of the present disclosure.

[0059] Figure 4 This diagram illustrates the process of determining the subject classification results in a network operation and maintenance method according to an embodiment of the present disclosure.

[0060] Figure 5 This diagram illustrates the process of determining the semantic similarity calculation result of a network operation and maintenance method according to an embodiment of the present disclosure.

[0061] Figure 6 This document illustrates a flowchart of the requirements analysis process for a network operation and maintenance method according to an embodiment of the present disclosure.

[0062] Figure 7 This diagram illustrates the verification and auditing process of a network operation and maintenance method according to an embodiment of the present disclosure.

[0063] Figure 8 This diagram illustrates the AI ​​model retraining process of a network operation and maintenance method according to an embodiment of the present disclosure.

[0064] Figure 9 This diagram illustrates the process of determining the target AI model in a network operation and maintenance method according to an embodiment of the present disclosure.

[0065] Figure 10 An interactive diagram of a network operation and maintenance method according to an embodiment of this disclosure is shown.

[0066] Figure 11 A flowchart of a network operation and maintenance method according to an embodiment of this disclosure is shown.

[0067] Figure 12 A schematic diagram of a network operation and maintenance device is shown in an embodiment of this disclosure.

[0068] Figure 13 This diagram illustrates the structural block diagram of a computer device for a network operation and maintenance method according to an embodiment of the present disclosure. Detailed Implementation

[0069] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0070] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0071] To facilitate understanding, before introducing the embodiments of this disclosure, the following explanations are provided for several terms involved in the embodiments of this disclosure:

[0072] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0073] Figure 1 A schematic diagram of an exemplary application system architecture to which a network operation and maintenance method according to embodiments of this disclosure can be applied is shown. For example... Figure 1 As shown, the system architecture may include terminal device 101, network 102 and server 103.

[0074] Network 102 is a medium used to provide a communication link between terminal device 101 and server 103, and can be a wired network or a wireless network.

[0075] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0076] Terminal device 101 can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.

[0077] Optionally, the client of the application installed on different terminal devices 101 may be the same, or the client of the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client may also be different; for example, the application client may be a mobile client, a PC client, etc.

[0078] Server 103 can be a server that provides various services, such as a backend management server that supports the device operated by the user using terminal device 101. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal device.

[0079] Optionally, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0080] Those skilled in the art will know that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative; any number of terminal devices, networks, and servers can be included depending on actual needs. This disclosure does not limit the scope of the embodiments.

[0081] Under the above system architecture, this disclosure provides a network operation and maintenance method that can be executed by any electronic device with computing power.

[0082] In some embodiments, the network operation and maintenance method provided in this disclosure can be executed by a terminal device in the above-described system architecture; in other embodiments, the network operation and maintenance method provided in this disclosure can be executed by a server in the above-described system architecture; in still other embodiments, the network operation and maintenance method provided in this disclosure can be implemented by the terminal device and the server in the above-described system architecture through interaction.

[0083] With the development of intelligent network operation and maintenance technology, various platforms or systems for managing and hosting AI models have been developed. These platforms or systems manage multiple AI capability resources containing AI models and feature tag data. Because these platforms or systems operate independently, there is significant duplication and fragmentation of AI models and capability resources. Many AI models, due to differences in their predictive capabilities, management, and performance metrics, are scattered across individual platforms or systems, making unified management and access impossible. Therefore, to facilitate unified management of AI models, it is necessary to integrate these scattered models across individual platforms or systems into a unified management system, enabling centralized access and management of AI models to provide more efficient services and effectively improve the efficiency of network operation and management.

[0084] Figure 2 This invention discloses a flowchart of a network operation and maintenance method according to an embodiment of the present invention, as shown below. Figure 2 As shown in the embodiments of this disclosure, a network operation and maintenance method includes the following steps:

[0085] S202: Based on the network operation and maintenance objectives, determine the network operation and maintenance requirements analysis results; the network operation and maintenance requirements analysis results include the requirements for artificial intelligence (AI) capabilities; the AI ​​capabilities include: AI models and feature data;

[0086] S204: Retrain the AI ​​model whose performance metrics in the original AI model do not meet the set threshold, and determine the updated AI model; the retraining is to adjust the model parameters of the AI ​​model and train it using the feature data until the performance metrics of the AI ​​model meet the set threshold.

[0087] S206: Determine the target AI model based on the network operation and maintenance requirements analysis results and the updated AI model;

[0088] S208: Based on the feature data, execute the network operation and maintenance objectives using the target AI model to determine the network operation and maintenance results.

[0089] This embodiment of the disclosure achieves network operation and maintenance requirement analysis by parsing network operation and maintenance objectives, thereby obtaining the requirement for AI capabilities. It then retrains and updates the original AI models by re-training those whose performance indicators do not meet set thresholds. This ensures that the performance indicators of each AI model in the updated model meet the set thresholds, expanding the application scope of the AI ​​models and facilitating unified management and application. By comprehensively utilizing the network operation and maintenance requirement analysis results to screen the updated AI models for target AI model capabilities, it obtains the AI ​​model with optimal performance indicators while meeting the network operation and maintenance requirements for AI performance indicators. Based on feature data, the target AI model is used to execute network operation and maintenance objectives, effectively improving the efficiency of network operation and maintenance engineers in network maintenance, enhancing network quality, meeting user needs, and effectively improving user experience.

[0090] Furthermore, when selecting target AI models, AI models can be classified and managed according to specific network operation and maintenance scenarios. Based on the analysis of the needs of network operation and maintenance objectives, accurate and effective recommendations of AI models can be achieved, realizing the precise sharing of AI models and feature data in AI capabilities.

[0091] like Figure 3 As shown in the embodiment, the step S202 above, which determines the network operation and maintenance requirements parsing result based on the network operation and maintenance objectives, may include:

[0092] S302: Based on network operation and maintenance goals, define the subject sample feature set and target set;

[0093] S304: Determine the subject classification result based on the subject sample feature set and the target set;

[0094] S306: Determine the semantic similarity calculation result based on the subject classification result;

[0095] S308: Determine the network operation and maintenance requirements parsing result based on the subject classification result and the semantic similarity calculation result.

[0096] In this embodiment, network operation and maintenance goals are generally formed in text form, such as being recorded in a document. To accurately understand these goals and obtain precise AI capability requirements, it is necessary to perform requirement analysis on the network operation and maintenance goals. First, the network operation and maintenance goals are defined by setting a subject sample feature set X = {x1, x2, ... x...}. n} and the target set Y = {y1, y2, ... y m Next, the subject is classified based on the subject sample feature set and the target set to obtain the subject classification result. Then, the semantic similarity of the subject classification result is calculated to obtain the semantic similarity calculation result. Finally, the network operation and maintenance requirements analysis result is determined based on the subject classification result and the semantic similarity calculation result.

[0097] By classifying network operation and maintenance objectives by subject and calculating semantic similarity, accurate requirement analysis can be performed. The network operation and maintenance requirement analysis results obtained by combining the above two methods are more in line with the actual network operation and maintenance requirements for AI capabilities, thus improving the accuracy of requirement analysis.

[0098] like Figure 4 As shown in the embodiment, the step S304 above, which determines the subject classification result based on the subject sample feature set and the target set, may include:

[0099] S402: Vectorize the subject sample feature set to determine the vectorized subject sample feature set; the vectorized subject sample feature set includes multiple subject sample feature vectors;

[0100] S404: Based on the target set, predict the maximum probability that the feature vector of the subject sample in the vectorized subject sample feature set belongs to the target set using the subject classification algorithm, and determine the subject classification result.

[0101] When classifying subjects, the first step is to define the subject sample feature set X = {x1, x2, ... x...}. n Vectorization is performed to obtain a vectorized subject sample feature set, which consists of the vectorized subject sample feature vectors. Next, based on a subject classification algorithm, the subject sample feature vectors in the vectorized subject sample feature set are predicted to belong to the target set Y = {y1, y2, ... y...}. mThe subject classification result is determined by the maximum probability of the given condition. The subject classification algorithm described above can employ the Naive Bayes algorithm or other types of subject classification algorithms.

[0102] In this embodiment, the subject classification result is determined in the following manner:

[0103]

[0104]

[0105] Where max(P(y) i |x)) is the subject classification result, used to indicate that the subject sample feature vector x in the vectorized subject sample feature set belongs to the target set Y={y1,y2,…y m The maximum probability of each element y in}; P(y i To classify an item by classifying it into elements y in the target set, the classifier is used to classify the item to be classified. i The probability of P(x|y) i To calculate whether the item to be classified belongs to the element y in the target set i And element y i The probability that the sample contains the feature vector x of the subject is given; P(x) is the joint probability; P(x) k |y i To calculate whether the item to be classified belongs to the element y in the target set i And element y i Includes subject sample feature vector x k The probability of ; n is the number of subject sample feature vectors x in the vectorized subject sample feature set; k is the number of products.

[0106] When determining the subject classification result, if P(y) i |x)=max{P(y1|x)×P(y2|x)…×P(y i If |x)}, then the subject sample feature vector x belongs to the target set Y={y1,y2,…y m Each element y in} i Output P(y) at this time. i |x), and obtain the subject classification result based on the Naive Bayes algorithm.

[0107] like Figure 5 As shown in the embodiment, the step S306 above, which determines the semantic similarity calculation result based on the subject classification result, may include:

[0108] S502: Based on the keyword extraction algorithm, determine the first keyword of the subject classification result;

[0109] S504: Determine the second keyword for the AI ​​capability management library; the AI ​​capability management library is used to manage existing AI capabilities; the AI ​​capabilities also include: tag data and application data;

[0110] S506: Determine the semantic similarity calculation result based on the semantic similarity between the first keyword and the second keyword.

[0111] When performing similarity matching to calculate semantic similarity, the first keyword n1 of the subject classification result needs to be calculated based on the keyword extraction algorithm. The keyword extraction algorithm mentioned above can be the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm, or other keyword extraction algorithms can be used.

[0112] Regarding the TF-IDF algorithm, Term Frequency (TF) represents the frequency of a keyword (term) appearing in the text, while Inverse Document Frequency (IDF) refers to the IDF of a specific word, which can be obtained by dividing the total number of documents by the number of documents containing that word, and then taking the logarithm of the quotient.

[0113] Specifically, the first keyword can be calculated and extracted as follows:

[0114]

[0115]

[0116] TF-IDF = TF(t) × IDF(t) (5)

[0117] Where TF(t) is the word frequency; n t The number of times the selected word appears in the file; n k The number of times each word appears in the file; IDF(t) is the frequency of the reverse file; n n is the number of files containing the words; N is the total number of files; TF-IDF is the result of the TF-IDF algorithm, i.e., the extracted keywords.

[0118] After calculating the first keyword n1, the second keyword n2 is determined from the AI ​​capability management library. This library includes various types of data such as original AI models, feature data, tag data, and application data, enabling unified management of AI capabilities. The AI ​​capability requirements in network operations and maintenance can be for AI models; for example, if an AI model is needed for link traffic prediction, the required AI capability is an AI model capable of link traffic prediction. The original AI models in the AI ​​capability management library include multiple AI models of various types, enabling applications in different types and scenarios. For example, if a project requires training an existing AI model for multiple scenarios, then the AI ​​capability... The capability requirement refers to the need for feature data across multiple scenarios. The feature data in the AI ​​capability management library can be used for training AI models, and this feature data can be data from various types of scenarios. For example, if an AI model lacks labeled data during initial training, then the AI ​​capability requirement is labeled data. The labeled data in the AI ​​capability management library can be used for the initial training of the AI ​​model; for example, the labeled data can be divided into training and validation sets for initial training. Similarly, if an AI model lacks auxiliary operating procedures, then the AI ​​capability requirement is application data. The application data in the AI ​​capability management library can be auxiliary programs for running the AI ​​model, such as the AI ​​model's operating environment. In another embodiment, a keyword extraction algorithm can also be used to calculate the second keyword of the AI ​​capability management library.

[0119] Calculate the semantic similarity between the first keyword n1 and the second keyword n2 to obtain the semantic similarity result. Specifically, the semantic similarity result can be calculated as follows:

[0120]

[0121] Where Sim(n1, n2) represents the semantic similarity calculation result; n1 is the first keyword; and n2 is the second keyword. α is the semantic distance; α is an adjustment parameter that can represent the semantic distance value when the similarity is 0.5 in a single instance.

[0122] In this embodiment, determining the network operation and maintenance requirement parsing result based on the subject classification result and the semantic similarity calculation result in step S308 above may include:

[0123] The maximum value between the subject classification result and the semantic similarity calculation result is determined as the network operation and maintenance requirements analysis result.

[0124] In the embodiment, the maximum value between the subject classification result and the semantic similarity calculation result is calculated using a maximum value function, for example, {max(max(P(y)} i The maximum value obtained by calculating |x)),(Sim(n1,n2)))} is determined as the result of the network operation and maintenance requirements analysis.

[0125] like Figure 6 As shown, the network operation and maintenance requirements are analyzed to obtain the network operation and maintenance requirements analysis results. The main process includes the following steps.

[0126] First, based on the network operation and maintenance goals, define the subject sample feature set and the target set;

[0127] On the one hand, based on the Naive Bayes algorithm, P(x|y) is calculated. i ), and then calculate max(P(y) i |x)) yields the subject classification results;

[0128] On the other hand, based on the TF-IDF algorithm, the semantic similarity Sim(n1, n2) between the first keyword n1 and the second keyword n2 is calculated. In one way, the maximum value of Sim(n1, n2) is used as the semantic similarity calculation result. In another way, the value of Sim(n1, n2) can be directly taken as the semantic similarity calculation result.

[0129] Finally, the maximum value of the subject classification result and the semantic similarity calculation result is combined to obtain the network operation and maintenance requirements analysis result.

[0130] like Figure 7 As shown in the embodiment, after determining the network operation and maintenance requirements analysis result in step S202, the method further includes:

[0131] S702: Send the network operation and maintenance requirements parsing results to the user for verification;

[0132] S704: In response to the user's approval of the network operation and maintenance requirement parsing result, the network operation and maintenance requirement parsing result is submitted to a subscription application;

[0133] S706: In response to the approval of the subscription application, the network operation and maintenance requirements analysis results are downloaded to the local system in the form of an image or model.

[0134] To ensure the accuracy of the network operation and maintenance (O&M) requirement analysis results, these results can be sent to users for verification. User manual verification further confirms the correctness of the requirement analysis. If the results meet the user's requirements, the user approves the verification. This verification process can be displayed to the user on the interface. After approval, the user clicks the "Approval" button, indicating that the user has approved the network O&M requirement analysis results. A subscription request is then submitted. This subscription request allows the user to subscribe to the desired network O&M objective, eliminating the need for repeated requirement analysis for the same objective. Upon receiving the subscription request, the administrator reviews it according to user permissions or AI model access permissions. Upon approval, the network O&M requirement analysis results are downloaded to the local system as an image or model, which can then be directly used.

[0135] like Figure 8 As shown in the embodiment, the step S204 above, which involves retraining the AI ​​model whose performance metrics do not meet a set threshold to determine the updated AI model, may include:

[0136] S802: Obtain the performance parameters of the original AI model;

[0137] S804: Determine the performance metrics for each AI model based on the aforementioned performance parameters;

[0138] S806: Identify AI models whose performance indicators do not meet the set threshold as retrained AI models;

[0139] S808: Use the feature data to retrain the retrained AI model until the performance index of the retrained AI model reaches or exceeds the set threshold, and output the retrained AI model.

[0140] S8010: Combine the original AI model and the retrained AI model to determine the updated AI model.

[0141] In this embodiment, due to differences in application scenarios, computational objectives, and computational objects within the original AI models, multiple AI models of various types are included. To achieve unified management and use of AI models, it is necessary to retrain any AI models that do not meet the requirements, thereby achieving standardized management of the original AI models. First, the performance parameters of the original AI models are obtained, and the performance metrics for each AI model are calculated. These performance metrics include at least one or more combinations of the following: AI model precision, AI model recall, AI model accuracy, AI model error, and AI model computation latency. These performance metrics can be used for value assessment; for example, the effectiveness of the AI ​​models can be evaluated to obtain value assessment metrics.

[0142] Taking the performance metric of AI model error as an example, the error can be calculated based on algorithms such as Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE).

[0143] Taking the Mean Absolute Percentage Error (MAPE) method as an example, the error is calculated as follows:

[0144]

[0145] Where MAPE is the mean absolute percentage error; Y i For the actual value, F i denoted as , and g represents the number of predicted values.

[0146] Next, a threshold is set, and AI models whose performance metrics do not meet the threshold are identified as retrained AI models and need to be retrained. The next step is to retrain the retrained AI models using feature data until their performance metrics reach or exceed the set threshold, at which point the retrained AI model is output. Alternatively, a learning rate is set, and the retrained model is trained using feature data, adjusting the model's parameters until its performance metrics reach or exceed the set threshold, thus achieving the retraining goal and outputting the retrained AI model. The original AI model and the retrained AI model are combined to construct a new set, which serves as the updated AI model. This combined update process results in an updated AI model that includes both the original and retrained AI models, better covering various network operation and maintenance scenarios, expanding the application scope of the AI ​​model, and facilitating unified management and application of the AI ​​model. Another approach is to remove the AI ​​models that need retraining from the original AI models, determine the remaining original AI models, and combine the remaining original AI models with the retrained AI models to construct a new AI model as the updated AI model. This approach reduces the size of the updated AI model and lowers the difficulty of AI model management.

[0147] like Figure 9 As shown in the embodiment, the step S206 above, which determines the target AI model based on the network operation and maintenance requirements analysis result and the updated AI model, may include:

[0148] S902: The network operation and maintenance requirements analysis results are used as matching indicators and matched with the performance indicators corresponding to the updated AI models to determine the first set of AI models that meet the matching indicators.

[0149] S904: The first AI model set is sorted according to performance indicators other than the matching index, and the best AI model in the sorting result is determined as the target AI model.

[0150] In this embodiment, when selecting target AI models, the first step is to use the network operation and maintenance requirements analysis results as a matching metric, and match them with the performance metrics corresponding to the updated AI models to determine the first set of AI models that meet the matching metrics. For example, if the network operation and maintenance goal is to predict the results of IP link L3 link traffic over the next 5 minutes, and the accuracy of the prediction needs to be greater than 90%, then the network operation and maintenance requirements analysis results obtained after requirements analysis are IP link traffic predictions with an accuracy greater than 90%. This IP link traffic prediction with an accuracy greater than 90% is used as a matching metric and matched with the performance metrics corresponding to the updated AI models. The matching object is a model capable of traffic prediction, and the performance metrics of the AI ​​models need to be examined. The accuracy of AI models used for traffic prediction is used to select the first set of AI models that meet the matching criteria. The next step is to sort the AI ​​models in the first set of AI models according to other performance indicators besides accuracy. For example, they can be sorted according to the precision, recall, error, and computation latency of the AI ​​models. The top 10 models in each of these sorts are selected. The AI ​​model that appears in the top 10 of each sort and ranks highest is selected as the optimal AI model. The optimal AI model has the highest precision, lowest recall, lowest error, and lowest computation latency. Therefore, the optimal AI model can be determined as the target AI model.

[0151] In this embodiment, step S208 above, which involves using the target AI model to execute the network operation and maintenance objective based on the feature data and determine the network operation and maintenance result, may include:

[0152] The target AI model and the feature data are shared according to AI capability recommendations; the AI ​​capability recommendations include at least one or more of the following combinations: standardized sharing, retraining sharing, inference invocation, model embedded reference, and image download;

[0153] The network operation and maintenance objectives are executed using the shared target AI model to determine the network operation and maintenance results;

[0154] The standardization sharing is used to directly share the target AI model; the retraining sharing is used to adaptively retrain the target AI model according to usage requirements; the inference call is used to configure the target AI model in the cloud for model inference and call it through an API interface; the model embedded reference is used to package and embed the target AI model into another system different from the system where the AI ​​model resides; the image download is used to package the target AI model and its runtime environment into another system different from the system where the AI ​​model resides, with the other system providing computing resources. Further, the standardization sharing is used to directly share the feature data; the retraining sharing is used to modify the feature data for adaptive retraining according to usage requirements; the inference call is used to configure the feature data in the cloud for model inference and call it through an API interface; the model embedded reference is used to package and embed the feature data into another system different from the system where the AI ​​model resides; the image download is used to package the feature data and its runtime environment into another system different from the system where the AI ​​model resides, with the other system providing computing resources.

[0155] In this embodiment, after obtaining the target AI model, the target AI model and feature data are shared according to AI capability recommendations. This enables the use of the AI ​​model in different scenarios, achieving multi-scenario sharing of AI capabilities and adapting to various network operation and maintenance scenarios. Next, the shared target AI model is used to execute the aforementioned network operation and maintenance objectives, determining the network operation and maintenance results. The updated AI model can be used to screen for target AI model capabilities by comprehensively utilizing the network operation and maintenance requirements analysis results. This allows for obtaining the AI ​​model with optimal performance indicators while meeting the network operation and maintenance requirements for AI performance metrics. Using the target AI model to execute network operation and maintenance objectives can effectively improve the efficiency of network operation and maintenance engineers in network maintenance, enhance network quality, meet user needs, and effectively improve user experience.

[0156] Standardized sharing refers to the direct sharing of the target AI model without modification to achieve network operation and maintenance goals. Retraining sharing refers to reusing the target AI model for different regions, professions, and scenarios by adaptively retraining it according to usage requirements. This retraining adapts the model to different regions, professions, and scenarios, achieving precise adaptation to network operation and maintenance goals. Inference invocation refers to configuring the AI ​​model in the cloud and providing an API (Application Programming Interface) for inference, enabling edge-side calls. Model embedded referencing involves packaging the target AI model and embedding it into a different system, enabling migration between systems while utilizing the computing resources of the original system. Mirror download refers to packaging the target AI model and its runtime environment into a different system, with the latter providing computing resources. These various AI capability recommendations can be selected individually or combined based on specific needs. For example, if the requirement is to retrain based on the application scenario, then retraining sharing should be selected. Alternatively, if the requirement is to retrain in another system based on the application scenario, then a combination of retraining sharing and model embedding can be chosen.

[0157] Operators were among the first to explore intelligent network operation and maintenance, possessing advantages in users, data, network technology, and system platforms, which facilitates the implementation and application of the technology. This disclosure can be applied to standalone systems and can interact with other internal and external systems through interfaces, providing efficient and high-quality AI model services, improving the efficiency of network operation and management, and enhancing customer experience.

[0158] like Figure 10 As shown, the network operation and maintenance method provided in this disclosure can run independently in a separate system. By interacting with other systems through interfaces, it provides efficient and high-quality intelligent services, improves the efficiency of telecommunications network operation and management, and enhances customer experience. Furthermore, the network operation and maintenance method of this disclosure can be deployed on a cloud server, an edge server, a standalone server, a server-side application, or a client-side application. This disclosure can run on various devices to achieve accurate recommendations from AI models.

[0159] This disclosure also provides specific examples of a network operation and maintenance method, such as... Figure 11As shown in the flowchart, the user is inputting a network operation and maintenance (O&M) goal. The algorithm then analyzes the user's application intent to obtain the O&M requirement analysis results, which are fed back to the user for verification. For example, if the O&M goal is for the user to predict the IP link L3 traffic flow for the next 5 minutes, the O&M requirement analysis result would be an AI capability requirement of an IP link traffic prediction model with a computation latency of less than 30 seconds. After verifying that the O&M requirement analysis results are correct, the user submits a subscription request to the AI ​​management system. Once approved, the O&M requirement analysis results are downloaded to the local system as an image or model. Evaluate the AI ​​model's performance based on performance metrics; compare the AI ​​model's performance metrics with a set threshold (e.g., a computational accuracy of 90%), and retrain AI models that fall below the threshold until they are greater than or equal to the threshold; (e.g., if the threshold is 95%, and the model is greater than the threshold, no retraining is needed; if the threshold is 85%, and the model is less than the threshold, retraining is required); based on user intent, share the optimal target AI model (highest accuracy / lowest computational latency, etc.) that has passed the AI ​​model performance evaluation through one of the following methods: standardized sharing / retraining sharing / inference invocation / model embedded reference / mirror download. (e.g., if the user requires a computational latency of less than 30 seconds, then the AI ​​model that meets the conditions of highest precision, lowest recall, and smallest error under a 30-second latency condition is recommended).

[0160] This disclosure classifies and analyzes user-input network operation and maintenance goals based on a combination of Naive Bayes and TF-IDF algorithms. The optimal AI model is then recommended based on the analyzed requirements and a value-based intelligent recommendation algorithm. This method comprehensively considers user needs for AI capabilities, the highest accuracy, lowest error, and minimum computation latency to recommend the best AI capabilities. The user input requirement classification and analysis method based on Naive Bayes + TF-IDF algorithms combines the best results of both algorithms, resulting in a more accurate analysis that better meets user needs. It recommends the most effective AI capabilities while satisfying user requirements.

[0161] This disclosure can enhance the policy decision verification capabilities of the cloud network operation system, improve the overall system performance, and ensure a better user experience. It can also help improve the intelligence level of the cloud network operation management system, thereby raising its overall intelligence level.

[0162] It should be noted that the acquisition, storage, use, and processing of data in this disclosed technical solution comply with the relevant provisions of national laws and regulations. The various types of data, such as personal identity data, operational data, and behavioral data related to individuals, customers, and groups, obtained in the embodiments of this disclosure have all been authorized.

[0163] Based on the same inventive concept, this disclosure also provides a network operation and maintenance device, as described in the following embodiments. Since the principle by which this device solves the problem is similar to that of the above-described method embodiments, the implementation of this device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0164] Figure 12 This diagram illustrates a network operation and maintenance device according to an embodiment of the present disclosure, such as... Figure 12 As shown, the device includes:

[0165] The network operation and maintenance requirements analysis result determination module 1201 is used to determine the network operation and maintenance requirements analysis result based on the network operation and maintenance objectives; the network operation and maintenance requirements analysis result includes the requirement for artificial intelligence (AI) capabilities; the AI ​​capabilities include: AI models and feature data;

[0166] The original AI model update module 1202 is used to retrain the AI ​​model whose performance indicators do not meet the set threshold in the original AI model, and determine the updated AI model; the retraining is to adjust the model parameters of the AI ​​model and train it using the feature data until the performance indicators of the AI ​​model meet the set threshold.

[0167] The target AI model determination module 1203 is used to determine the target AI model based on the network operation and maintenance requirements analysis results and the updated AI model.

[0168] The network operation and maintenance result determination module 1204 is used to determine the network operation and maintenance result by executing the network operation and maintenance target using the target AI model based on the feature data.

[0169] It should be noted that the network operation and maintenance requirement analysis result determination module 1201, the original AI model update module 1202, the target AI model determination module 1203, and the network operation and maintenance result determination module 1204 mentioned above correspond to S202 to S208 in the method embodiment. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0170] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0171] The following reference Figure 13 To describe an electronic device 1300 according to such an embodiment of the present disclosure. Figure 13 The electronic device 1300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0172] like Figure 13 As shown, the electronic device 1300 is manifested in the form of a general-purpose computing device. The components of the electronic device 1300 may include, but are not limited to: at least one processing unit 1310, at least one storage unit 1320, and a bus 1330 connecting different system components (including storage unit 1320 and processing unit 1310).

[0173] The storage unit stores program code that can be executed by the processing unit 1310, causing the processing unit 1310 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1310 can perform the following steps of the above method embodiment: determining the network operation and maintenance requirements analysis result based on the network operation and maintenance objectives; the network operation and maintenance requirements analysis result includes the requirement for artificial intelligence (AI) capabilities; the AI ​​capabilities include: an AI model and feature data; retraining the AI ​​model whose performance indicators do not meet a set threshold in the original AI model to determine an updated AI model; the retraining involves adjusting the model parameters of the AI ​​model and training it using the feature data until the performance indicators of the AI ​​model meet the set threshold; determining a target AI model based on the network operation and maintenance requirements analysis result and the updated AI model; and executing the network operation and maintenance objectives using the target AI model based on the feature data to determine the network operation and maintenance result.

[0174] Storage unit 1320 may include readable media in the form of volatile storage units, such as random access memory (RAM) 13201 and / or cache memory 13202, and may further include read-only memory (ROM) 13203.

[0175] Storage unit 1320 may also include a program / utility 13204 having a set (at least one) of program modules 13205, such program modules 13205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0176] Bus 1330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0177] Electronic device 1300 can also communicate with one or more external devices 1340 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1300, and / or with any device that enables electronic device 1300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1350. Furthermore, electronic device 1300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1360. As shown, network adapter 1360 communicates with other modules of electronic device 1300 via bus 1330. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0178] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0179] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described network operation and maintenance method.

[0180] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the methods described above is stored thereon. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0181] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0182] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0183] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0184] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0185] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0186] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0187] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0188] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A network operation and maintenance method, characterized in that, include: Based on the network operation and maintenance objectives, determine the network operation and maintenance requirements analysis results; The network operation and maintenance requirements analysis results include the requirement for artificial intelligence (AI) capabilities; The AI ​​capabilities include: AI models and feature data; The AI ​​model whose performance metrics do not meet the set threshold is retrained to determine the updated AI model; the retraining is to adjust the model parameters of the AI ​​model and train it using the feature data until the performance metrics of the AI ​​model meet the set threshold. Based on the network operation and maintenance requirements analysis results and the updated AI model, the target AI model is determined; Based on the feature data, the target AI model is used to execute the network operation and maintenance objectives and determine the network operation and maintenance results. The process of determining the network operation and maintenance requirements analysis results based on network operation and maintenance objectives includes: setting a subject sample feature set and a target set based on network operation and maintenance objectives; determining the subject classification results based on the subject sample feature set and the target set; determining the semantic similarity calculation results based on the subject classification results; and determining the network operation and maintenance requirements analysis results based on the subject classification results and the semantic similarity calculation results. Specifically, determining the target AI model based on the network operation and maintenance requirements analysis results and the updated AI model includes: using the network operation and maintenance requirements analysis results as a matching indicator, matching them with the performance indicators corresponding to the updated AI model, and determining a first set of AI models that meet the matching indicator; sorting the first set of AI models according to other performance indicators besides the matching indicator, and determining the optimal AI model in the sorting results as the target AI model.

2. The network operation and maintenance method according to claim 1, characterized in that, Based on the subject sample feature set and the target set, the subject classification result is determined, including: The subject sample feature set is vectorized to determine the vectorized subject sample feature set; the vectorized subject sample feature set includes multiple subject sample feature vectors. Based on the target set, the subject classification result is determined by predicting the maximum probability that the feature vector of the subject sample in the vectorized subject sample feature set belongs to the target set using a subject classification algorithm.

3. The network operation and maintenance method according to claim 1, characterized in that, Based on the subject classification results, the semantic similarity calculation results are determined, including: Based on the keyword extraction algorithm, the first keyword of the subject classification result is determined; The second keyword for the AI ​​capability management library is determined; the AI ​​capability management library is used to manage existing AI capabilities; the AI ​​capabilities also include: tag data and application data; The semantic similarity calculation result is determined based on the semantic similarity between the first keyword and the second keyword.

4. The network operation and maintenance method according to claim 1, characterized in that, Based on the subject classification results and the semantic similarity calculation results, the network operation and maintenance requirements parsing results are determined, including: The maximum value between the subject classification result and the semantic similarity calculation result is determined as the network operation and maintenance requirements analysis result.

5. The network operation and maintenance method according to claim 1, characterized in that, Retraining is performed on AI models whose performance metrics do not meet a set threshold in the original AI model to determine the updated AI model, including: Obtain the performance parameters of the original AI model; Based on the performance parameters, determine the performance metrics for each AI model; AI models whose performance metrics do not meet the set threshold are identified as retrained AI models. The retrained AI model is retrained using the feature data until the performance index of the retrained AI model reaches or exceeds the set threshold, and the retrained AI model is output. The original AI model and the retrained AI model are combined to determine the updated AI model.

6. The network operation and maintenance method according to claim 1, characterized in that, Based on the network operation and maintenance requirements analysis results and the updated AI model, the target AI model is determined, including: The network operation and maintenance requirements analysis results are used as matching indicators and matched with the performance indicators corresponding to the updated AI models to determine the first set of AI models that meet the matching indicators. The first set of AI models is sorted according to performance metrics other than the matching metric, and the best AI model in the sorting results is determined as the target AI model.

7. The network operation and maintenance method according to claim 1, characterized in that, Based on the aforementioned feature data, the target AI model is used to execute the network operation and maintenance objectives, and the network operation and maintenance results are determined, including: The target AI model and the feature data are shared according to AI capability recommendations; the AI ​​capability recommendations include at least one or more of the following combinations: standardized sharing, retraining sharing, inference invocation, model embedded reference, and image download; The network operation and maintenance objectives are executed using the shared target AI model to determine the network operation and maintenance results; Specifically, the standardization sharing is used to directly share the target AI model; the retraining sharing is used to adaptively retrain the target AI model according to usage requirements; the inference call is used to configure the target AI model for model inference in the cloud and call it through the API interface; the model embedded reference is used to encapsulate and embed the target AI model into another system different from the system where the AI ​​model is located; and the image download is used to encapsulate the target AI model and its operating environment into another system different from the system where the AI ​​model is located, with the computing resources provided by the other system.

8. The network operation and maintenance method according to claim 1, characterized in that, After determining the network operation and maintenance requirements analysis results, the following is also included: The network operation and maintenance requirements analysis results will be sent to the user for verification.

9. The network operation and maintenance method according to claim 8, characterized in that, Also includes: In response to the user's approval of the network operation and maintenance requirement analysis results, the network operation and maintenance requirement analysis results are submitted to the subscription application; In response to the approval of the subscription application, the network operation and maintenance requirements analysis results are downloaded to the local system in the form of an image or model.

10. The network operation and maintenance method according to any one of claims 1-9, characterized in that, The performance metrics include at least one or more combinations of the following: AI model precision, AI model recall, AI model accuracy, AI model error, and AI model computation latency.

11. A network operation and maintenance device, characterized in that, include: The network operation and maintenance requirements analysis result determination module is used to determine the network operation and maintenance requirements analysis results based on the network operation and maintenance objectives. The network operation and maintenance requirements analysis results include the requirement for artificial intelligence (AI) capabilities; the AI ​​capabilities include: AI models and feature data; The original AI model update module is used to retrain AI models whose performance indicators do not meet a set threshold in the original AI model, and to determine the updated AI model; the retraining is to adjust the model parameters of the AI ​​model and train it using the feature data until the performance indicators of the AI ​​model meet the set threshold. The target AI model determination module is used to determine the target AI model based on the network operation and maintenance requirements analysis results and the updated AI model. The network operation and maintenance result determination module is used to determine the network operation and maintenance result by executing the network operation and maintenance target using the target AI model based on the feature data. The network operation and maintenance requirement analysis result determination module is further configured to: set a subject sample feature set and a target set according to the network operation and maintenance objectives; determine the subject classification result according to the subject sample feature set and the target set; determine the semantic similarity calculation result according to the subject classification result; and determine the network operation and maintenance requirement analysis result according to the subject classification result and the semantic similarity calculation result. The target AI model determination module is further configured to: use the network operation and maintenance requirement analysis result as a matching indicator, match it with the performance indicator corresponding to the updated AI model, and determine a first set of AI models that meet the matching indicator; sort the first set of AI models according to other performance indicators besides the matching indicator, and determine the optimal AI model in the sorting result as the target AI model.

12. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the network operation and maintenance method according to any one of claims 1 to 10 by executing the executable instructions.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the network operation and maintenance method according to any one of claims 1 to 10.

Images