Optical transport network digital twin model management method and system

Abstract

The present invention provides a method and system for managing a digital twin model of an optical transmission network, the method comprising: collecting real-time data streams of equipment running in the optical transmission network; an online reasoning instance management module sending a reasoning environment configuration request of a selected trained machine or non-machine learning model to a data management module; providing reasoning environment configuration parameters according to the reasoning environment configuration request, and configuring the reasoning operating environment according to the reasoning environment configuration parameters; a basic environment management module generating a reasoning instance according to the reasoning operating environment, the reasoning instance sending a retrieval request for a trained machine or non-machine learning model code file to the data management module, and the data management module sending the trained machine or non-machine learning model code file to the reasoning instance according to the retrieval request; a real-time data transmission module sending a real-time data stream to a topic, listening to new messages of the topic and routing them to the reasoning instance, the reasoning instance generating results based on data stream reasoning and feeding them back to the real-time data transmission module.

Description

Technical Field

[0001] The present invention relates to the technical field of digital twin optical networks, and in particular to a method and system for managing a digital twin model of an optical transmission network. Background Art

[0002] Optical fiber communication technology, with its high bandwidth, low loss and anti-interference characteristics, has become the key to achieving large-capacity and long-distance data transmission in the era of big data. With the rapid development of related technologies, optical communication systems have shown new features such as multi-band ultra-wideband, long-distance large capacity and dynamic scheduling, which have promoted the widespread application of optical networks, but also brought challenges to operation and maintenance. In order to meet these challenges, network digitization, real-time perception, full-process automation and intelligence are increasingly valued by the industry.

[0003] Digital twin technology strengthens the network's systematic simulation, optimization, verification and control capabilities by building a real-time digital mirror of the physical network. The application of this technology in network digitization, real-time perception, full-process automation and intelligence provides support for the deployment of new technologies in optical communication networks and improves the efficiency of responding to network problems. In view of the technical challenges facing optical communications, digital twin technology has the potential to provide low-cost, lightweight solutions to meet these challenges and has broad application prospects in optical communications.

[0004] In digital twin optical networks, model management capabilities are crucial. It not only involves accurate virtual mapping of physical entities, but also covers the integration, update and optimization of models, which plays a decisive role in the overall performance and adaptability of the platform. In the application practice of digital twin management platforms, given the diversity and complexity of model types, an innovative management technology framework is urgently needed. At present, there is no mature model management technology and platform in the field of digital twin optical networks. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and system for managing a digital twin model of an optical transport network to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present invention provides a method for managing a digital twin model of an optical transport network, the method comprising:

[0007] Based on user needs and different types of virtual models in the optical transport network digital twin platform, select a machine learning model or non-machine learning model that implements functions corresponding to user needs;

[0008] Upload the corresponding inference environment configuration parameters based on the selected machine learning model, upload the corresponding model code file and inference environment configuration parameters based on the selected non-machine learning model, and store the inference environment configuration parameters and model code files persistently in the data management module;

[0009] The data collection module collects the real-time data stream of the equipment running in the optical transmission network;

[0010] After the selected machine learning model is deployed and trained or the model code file of the non-machine learning model is uploaded, the online reasoning instance management module sends a reasoning environment configuration request of the selected trained machine learning model or non-machine learning model to the data management module;

[0011] The data management module provides corresponding reasoning environment configuration parameters to the online reasoning instance management module according to the reasoning environment configuration request of the selected trained machine learning model or non-machine learning model, so that the online reasoning instance management module configures the reasoning operating environment according to the reasoning environment configuration parameters;

[0012] The basic environment management module generates an inference instance according to the configured inference running environment, and the inference instance sends a retrieval request for the model code file of the selected trained machine learning model or non-machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected trained machine learning model or non-machine learning model to the inference instance; the model code file of the trained machine learning model is a model code file generated by the machine learning model after training, containing the model training results and persistently stored in the data management module;

[0013] The inference instance receives the selected trained machine learning model or non-machine learning model model code file to complete the model inference instantiation;

[0014] The real-time data transmission module receives the real-time data stream from the data acquisition module, sends the real-time data stream to the corresponding topic, and monitors the topic. When a new message is monitored, the real-time data stream is routed to the reasoning instance so that the reasoning instance performs reasoning based on the real-time data stream, generates reasoning results, and sends the reasoning results and real-time data stream to the real-time data transmission module.

[0015] In some embodiments of the present invention, the method further comprises:

[0016] Upload the corresponding model code file and training environment configuration parameters based on the selected machine learning model, and store the model code file and training environment configuration parameters persistently in the data management module;

[0017] The data collection module collects the historical data of the operation of the equipment in the optical transmission network, and stores the historical data persistently in the data management module;

[0018] The online training instance management module receives an online training request for the machine learning model selected by the user, and sends a training environment configuration request for the selected machine learning model to the data management module based on the online training request;

[0019] The data management module provides corresponding training environment configuration parameters to the online training instance management module according to the training environment configuration request of the selected machine learning model, so that the online training instance management module configures the training operation environment according to the training environment configuration parameters;

[0020] The basic environment management module generates a training instance according to the configured training operating environment, and the training instance sends a retrieval request for the model code file of the selected machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected machine learning model to the training instance;

[0021] The training instance receives the model code file of the selected machine learning model, completes the model training instantiation, and receives the historical data of device operation pushed by the data management module. The selected machine learning model is trained online based on the historical data, and the generated model code file containing the model training results is persistently stored in the data management module.

[0022] In some embodiments of the present invention, the method further comprises:

[0023] Receive an offline training request for a machine learning model selected by a user, and upload a model code file containing offline training results and training environment configuration parameters of the selected machine learning model based on the offline training request;

[0024] The model code file containing the offline training results and the training environment configuration parameters of the selected machine learning model are persistently stored in the data management module.

[0025] In some embodiments of the present invention, the method further comprises:

[0026] The online training instance management module receives an incremental training request for a trained machine learning model selected by a user, and sends a training environment configuration request for the selected trained machine learning model to the data management module based on the incremental training request;

[0027] The data management module provides corresponding training environment configuration parameters to the online training instance management module according to the training environment configuration request of the selected trained machine learning model, so that the online training instance management module configures the training operating environment according to the training environment configuration parameters;

[0028] The basic environment management module generates a training instance according to the configured training operating environment, and the training instance sends a retrieval request for a model code file of a selected trained machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected trained machine learning model to the training instance;

[0029] The training instance receives the model code file of the selected trained machine learning model, completes the model update instantiation, and receives the real-time data stream of the device running transmitted by the real-time data transmission module. The selected trained machine learning model is incrementally trained according to the real-time data stream, and the generated model code file containing the model update result is persistently stored in the data management module.

[0030] In some embodiments of the present invention, the real-time data transmission module is implemented by stream framework technology, and the basic environment management module is implemented by containerization technology and container orchestration technology.

[0031] In some embodiments of the present invention, after the data collection module collects historical data or real-time data streams of equipment operation in the optical transmission network, the method further includes:

[0032] Perform data cleaning and / or data conversion and / or time series processing on historical data or real-time data streams, wherein the data cleaning includes missing value, outlier, duplicate removal and format standardization, the data conversion includes one-hot encoding and label encoding processing of normalized numerical data and encoded categorical data, and the time series processing includes sliding window feature extraction and time decomposition processing of time series data;

[0033] The real-time data transmission module receives the real-time data stream of the device operation after data cleaning and / or data conversion and / or time series processing.

[0034] In some embodiments of the present invention, the virtual model includes a basic model, a transmission model and a functional model.

[0035] Another aspect of the present invention provides an optical transport network digital twin model management system, the system comprising: a processor, a memory, and computer instructions stored in the memory, the processor being used to execute the computer instructions, and when the computer instructions are executed, the system implements the steps of the aforementioned method.

[0036] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the aforementioned method when executed by a processor.

[0037] Another aspect of the present invention provides a computer program product, comprising computer instructions, which implement the steps of the above method when executed by a processor.

[0038] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0039] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0041] Figure 1 It is a flowchart of a method for managing a digital twin model of an optical transport network in one embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a model reasoning sequence in one embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the timing of online training of a model in one embodiment of the present invention;

[0044] Figure 4 A schematic diagram of a flow chart of front-end user operations in one embodiment of the present invention;

[0045] Figure 5 It is a structural schematic diagram of an optical transport network digital twin model management system in one embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0047] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0048] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0049] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0050] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0051] The data type generated by the physical entities of each device in the optical transmission network or optical transmission system in the field of digital twin optical network is a continuous data stream generated in real time. Since most of the existing model management system frameworks are focused on processing persistent and static data sets, the processing capabilities of real-time data streams are very limited. Therefore, these model management frameworks cannot be applied to the optical transmission network or optical transmission system in the field of digital twin optical network. In addition, the existing model management framework focuses only on the definition and training of machine learning models, and the training method is relatively simple, and it is not possible to achieve the full life cycle management and sharing of machine learning model data and non-machine learning model data. To this end, an optical transmission network digital twin model management method and system are proposed in an embodiment of the present invention, and a comprehensive framework is proposed. The framework integrates multiple modules, and through the interaction between the front and back ends and the collaborative work between the modules, all the above problems can be effectively solved to ensure the effective management and optimized operation of the model.

[0052] Figure 1 : is a flow chart of a method for managing a digital twin model of an optical transport network in one embodiment of the present invention. Figure 2 FIG. 1 is a schematic diagram of a model reasoning sequence in an embodiment of the present invention. Figure 1 and Figure 2 As shown, a method for managing an optical transport network digital twin model according to an embodiment of the present invention includes the following steps:

[0053] Step S110, based on user needs and different types of virtual models in the optical transport network digital twin platform, select a machine learning model or a non-machine learning model that implements functions corresponding to user needs.

[0054] Specifically, different types of virtual models include basic models, transmission models, and functional models. Basic models refer to the virtual entities of various basic device units that make up an optical transmission network or optical transmission system, such as optical amplifier devices, wavelength selection switch devices, etc. Transmission models refer to the process in digital space where the simulated optical signal reaches the receiving end after single-span / multi-span optical fiber transmission and relay amplification from the transmitting end, as well as the noise introduced by various physical effects during the transmission process, including polarization effects, spontaneous radiation noise of optical amplifiers, transceiver noise, nonlinear noise, filtering effects, etc., and finally accurately simulate the optical signal transmission process in the time domain and frequency domain, and respond dynamically with the real network. The functional model calls and trains the model based on the status and data information of the link, system, and network provided by the data layer. The output results can provide auxiliary references for optimizing device models, network deployment decisions, configuration simulation, etc. Functional models can include the following five categories: health assessment model, performance prediction model, transmission optimization model, fault simulation model and dynamic configuration model. Specifically, the health assessment model can be used to evaluate the health of the equipment, the performance prediction model can be used to predict the performance of the equipment, the transmission optimization model can be used to decide whether the transmission link is suitable for transmission, the fault simulation model can be used to simulate the detection and prediction of equipment failures, and the dynamic configuration model can be used to dynamically configure and adjust the parameters of the model.

[0055] The modeling methods of optical transmission systems can be summarized into two categories: model-driven and data-driven. Model-driven modeling methods use theoretical knowledge, physical theorems and scientific principles to build mathematical models, that is, non-machine learning models. Data-driven modeling methods mainly use artificial intelligence technology to learn internal laws by collecting system operation data, thereby realizing the modeling of optical transmission systems. As a core branch of artificial intelligence, machine learning is widely used in many fields. The application of machine learning algorithms depends on various frameworks, libraries and environments, so it is necessary to support various machine learning environments. In order to realize the modeling of optical transmission systems, the optical transmission network digital twin platform must be able to manage and deploy multiple machine learning models and mathematical models. For the optical transmission network digital twin platform, the basic model is modeled in a model-driven manner, corresponding to the mathematical model, part of the transmission model is modeled in a model-driven manner, and the other part is modeled in a data-driven manner, corresponding to the mathematical model or machine learning model, and the functional model is modeled in a data-driven manner, corresponding to the machine learning model.

[0056] Step S120: Upload the corresponding inference environment configuration parameters based on the selected machine learning model, upload the corresponding model code file and inference environment configuration parameters based on the selected non-machine learning model, and persistently store the inference environment configuration parameters and model code files in the data management module.

[0057] In this step, the inference environment configuration parameters of the machine and non-machine learning models include relevant parameters for model inference, such as the format of the input real-time data stream, the framework version information required for the inference runtime, and the configuration parameters of the real-time data transmission module used for data transmission; the model code file of the non-machine learning model includes information such as the model's relevant algorithms and parameters.

[0058] The data management module will store various types of data including the inference environment configuration parameters of the above-mentioned machine and non-machine learning models and the model code files of the non-machine learning models in a structured manner for easy access, query and management. The same is true for the data stored persistently in the data management module involved in the subsequent steps, which will not be described in detail later. In the database, data is mapped to relational tables. Each table consists of rows (records) and columns (fields). Rows represent a collection of data items, and columns define the attributes and characteristics of the data. In addition to data information, the database also contains logical relationships with other tables.

[0059] Step S130: The data collection module collects the real-time data stream of the equipment running in the optical transmission network.

[0060] Specifically, based on the large-scale, high-performance network monitoring requirements in optical transmission networks, this embodiment uses network telemetry technology in active push mode to collect real-time data streams of equipment to meet the real-time, high-speed, and precise requirements of the optical network digital twin system for data.

[0061] Step S140: After the selected machine learning model is deployed and trained or the model code file of the non-machine learning model is uploaded, the online reasoning instance management module sends an inference environment configuration request for the selected trained machine learning model or non-machine learning model to the data management module.

[0062] In this step, the training method of the machine learning model may include online training, offline training, and incremental training. Among them, when the training method of the machine learning model is offline training, uploading the model code file containing the offline training results and the training environment configuration parameters of the selected machine learning model marks the end of the machine learning model deployment and offline training. In other words, the machine learning model that has undergone online training, offline training, and incremental training can perform the reasoning process of this step S140 to step S180. In this step, the online reasoning instance management module sends the reasoning environment configuration request of the selected trained machine learning model or non-machine learning model to the data management module. Figure 2 Step (1).

[0063] In step S150, the data management module provides corresponding reasoning environment configuration parameters to the online reasoning instance management module according to the reasoning environment configuration request of the selected trained machine learning model or non-machine learning model, so that the online reasoning instance management module configures the reasoning operating environment according to the reasoning environment configuration parameters.

[0064] In this step, the online inference instance management module first parses the inference environment configuration parameters after receiving them, and then configures the model's inference operating environment based on the relevant parameters of the model inference obtained after parsing, the framework version information required for the inference runtime, and the configuration parameters of the real-time data transmission module. This can be achieved by setting the necessary environment variables, which can define the various operating configurations in the container. This step includes Figure 2 Steps (2) and (3).

[0065] Step S160, the basic environment management module generates an inference instance according to the configured inference running environment, and the inference instance sends a retrieval request for the model code file of the selected trained machine learning model or non-machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected trained machine learning model or non-machine learning model to the inference instance; the model code file of the trained machine learning model is a model code file generated by the machine learning model after training, which contains the model training results and is persistently stored in the data management module.

[0066] In one embodiment of the present invention, the basic environment management module is implemented through containerization technology and container orchestration technology.

[0067] In this step, the basic environment management module can provide different basic environments for different training and reasoning tasks performed by different deployed machine learning models or non-machine learning models, achieve environmental isolation, and avoid mutual interference between different models, providing security protection. Specifically, the basic environment management module generates an inference instance by starting a container. A lightweight, portable container is created through the adopted container technology (such as Docker), which will contain all the dependencies and environments required for the inference instance. Then, according to the version information in the inference environment configuration parameters, a basic container image is selected or created (if it does not exist, it is created). The image is pre-installed with the operating system, programming language environment and other basic software required to perform the inference task, and then the containerization process is started to create the inference instance. This step includes Figure 2 Steps (4) and (5).

[0068] The data management module interacts with the database through the database interface and uses structured query language (SQL) to query the database. SQL is a standard language for database operations that allows various operations such as retrieving (SELECT), inserting (INSERT), updating (UPDATE), and deleting (DELETE) data. The query can be a simple single-table query or a complex multi-table join query. Therefore, the data management module can query the required data through corresponding operations based on different instructions issued by different needs and return it to the data management module, and then transmit it to the reasoning instance.

[0069] The model training results include the performance of the machine learning model after training and learning on a training set consisting of historical equipment operation data, as well as model weight parameters. The performance includes but is not limited to evaluation indicators such as accuracy, recall rate, and F1 score.

[0070] Step S170: The inference instance receives the selected trained machine learning model or non-machine learning model model code file to complete the model inference instantiation.

[0071] This step is Figure 2 Step (5).

[0072] Step S180, the real-time data transmission module receives the real-time data stream from the data acquisition module, sends the real-time data stream to the corresponding topic, and monitors the topic. When a new message is monitored, the real-time data stream is routed to the reasoning instance so that the reasoning instance performs reasoning based on the real-time data stream, generates reasoning results, and sends the reasoning results and real-time data stream to the real-time data transmission module.

[0073] The reasoning process in this step is to use the trained machine learning model to predict the real-time data stream to achieve the function of outputting prediction results or decisions corresponding to user needs, that is, the process of meeting user needs. In the real-time data stream system-real-time data transmission module, the input topic serves as the channel for data inflow, allowing data producers to publish messages, that is, messages of the real-time data stream. The output topic is the publishing point of the data processing results (inference results) for data consumers to subscribe and read, realizing real-time data processing and distribution. This step includes Figure 2 Steps (6) to (9).

[0074] In one embodiment of the present invention, the real-time data transmission module is implemented by a stream framework technology. Specifically, the key technology implemented by the real-time data transmission module is a stream processing framework, which is used to receive, process and analyze data streams in real time, and supports buffering and transmission of real-time data streams. Unlike traditional batch processing systems, the stream processing framework focuses on the immediate processing of real-time data that continues to flow in, without the need to store the data in a database or data warehouse waiting for batch processing. This framework is particularly suitable for scenarios that require rapid response, such as real-time monitoring systems, equipment data management, etc. Therefore, the real-time data transmission module is mainly used for the transmission of real-time data streams in the inference link, mainly including the input of inference data and the output of inference results. Data transmission can be carried out between modules other than the real-time data transmission module through API calls or data transmission protocols, such as RESTful API, and different services can exchange data through network requests using the HTTP protocol.

[0075] Based on the above scheme, the optical transport network digital twin model management method can integrate multiple non-machine learning models (mathematical models) and machine learning models, and realize the selection, deployment and inference of multiple mathematical models and machine learning models.

[0076] In order to adapt to different business needs and scenarios, the framework of this method can support multiple training methods of machine learning models, including online training, offline training and incremental training, to improve the flexibility and adaptability of the model.

[0077] Figure 3 Schematic diagram of the online training sequence of the model in one embodiment of the present invention. The optical transport network digital twin model management method can also realize the selection, deployment and online training of the machine learning model. Therefore, in one embodiment of the present invention, Figure 3 As shown, the method also includes the following steps:

[0078] Step S210, upload the corresponding model code file and training environment configuration parameters based on the selected machine learning model, and persistently store the model code file and training environment configuration parameters in the data management module.

[0079] In this step, the training environment configuration parameters include, but are not limited to, the batch size, number of iterations, Python version, number of containers, and other related information for machine learning model training. The model code file of the machine learning model includes the relevant algorithms and parameters of the machine learning model.

[0080] Step S220: The data collection module collects historical data of the operation of the equipment in the optical transmission network, and stores the historical data persistently in the data management module.

[0081] Step S230: The online training instance management module receives an online training request for the machine learning model selected by the user, and sends a training environment configuration request for the selected machine learning model to the data management module based on the online training request. This step includes Figure 3 Step (1).

[0082] In step S240, the data management module provides corresponding training environment configuration parameters to the online training instance management module according to the training environment configuration request of the selected machine learning model, so that the online training instance management module configures the training operating environment according to the training environment configuration parameters.

[0083] In this step, the online training instance management module parses the training environment configuration parameters after receiving them, and then configures the model training running environment according to the parameter information such as the batch processing size, number of iterations, Python version and number of containers obtained after parsing. This can be achieved by setting the necessary environment variables, which can define the various running configurations in the container. This step includes Figure 3 Steps (2) and (3).

[0084] In step S250, the basic environment management module generates a training instance according to the configured training operating environment, and the training instance sends a retrieval request for the model code file of the selected machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected machine learning model to the training instance.

[0085] In one embodiment of the present invention, the basic environment management module is implemented through containerization technology and container orchestration technology.

[0086] In this step, the basic environment management module can provide different basic environments for different tasks of training and reasoning performed by different deployed machine learning models or non-machine learning models, realize environmental isolation, and avoid mutual interference problems for different models and different tasks, and provide security. Specifically, the basic environment management module generates a training instance by starting a container. A lightweight, portable container is created through the adopted container technology (such as Docker), which will contain all the dependencies and environments required for the training instance. Then, according to the version information in the training environment configuration parameters, a basic container image is selected or created (if it does not exist, it is created). The image is pre-installed with the operating system, programming language environment and other basic software required to perform the training task, and then the containerization process is started to create a training instance. This step includes Figure 3 Steps (4) and (5).

[0087] Step S260: The training instance receives the model code file of the selected machine learning model, completes the model training instantiation, and receives the historical data of the device operation pushed by the data management module. The selected machine learning model is trained online based on the historical data, and the generated model code file containing the model training results is persistently stored in the data management module, so as to facilitate further model analysis, update and inference of the machine learning model after the online training is completed. This step includes Figure 3 Steps (5) to (8).

[0088] In one embodiment of the present invention, the optical transport network digital twin model management method can also support offline or local training of machine learning models. Therefore, the method also includes the following steps:

[0089] Step S310, receiving an offline training request for a machine learning model selected by a user, and uploading a model code file containing offline training results and training environment configuration parameters of the selected machine learning model based on the offline training request. The offline training results include the performance of the machine learning model after local training and the model weight parameters, etc.

[0090] Step S320, persistently store the model code file containing the offline training results and the training environment configuration parameters of the selected machine learning model in the data management module to facilitate further model analysis, updating and inference of the machine learning model after the offline training is completed.

[0091] In one embodiment of the present invention, since the machine learning model will continue to learn from the newly collected real-time data stream, the optical transport network digital twin model management method can also perform incremental training on the machine learning model that has been trained online or offline. During this process, the model is regularly updated and iterated in batches to adjust and optimize the weights and parameters of the model, so as to adapt to the ever-changing new real-time data, improve its generalization ability, and reduce the risk of overfitting. Therefore, the method also includes the following steps:

[0092] In step S410, the online training instance management module receives an incremental training request for a trained machine learning model selected by the user, and sends a training environment configuration request for the selected trained machine learning model to the data management module based on the incremental training request.

[0093] In step S420, the data management module provides corresponding training environment configuration parameters to the online training instance management module according to the training environment configuration request of the selected trained machine learning model, so that the online training instance management module configures the training operating environment according to the training environment configuration parameters.

[0094] Step S430, the basic environment management module generates a training instance according to the configured training operating environment, and the training instance sends a retrieval request for the model code file of the selected trained machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected trained machine learning model to the training instance. Specifically, the model code file of the trained machine learning model is a model code file containing the model training results generated by the machine learning model after online training or local training.

[0095] Step S440: The training instance receives the model code file of the selected trained machine learning model, completes the model update instantiation, and receives the real-time data stream of the device running transmitted by the real-time data transmission module. The selected trained machine learning model is incrementally trained according to the real-time data stream, and the generated model code file containing the model update result is persistently stored in the data management module to facilitate further inference of the machine learning model after the incremental training is completed.

[0096] In the schemes of each of the above-mentioned embodiments, the method also supports operations such as adding, deleting and updating the generated training instances and reasoning instances, and can also provide a secure data destruction mechanism. When the data is no longer needed, the data can be completely deleted and the backup can be cleared to prevent data leakage.

[0097] In one embodiment of the present invention, after the data collection module collects historical data or real-time data streams of equipment operation in the optical transmission network, the method further includes:

[0098] Perform data cleaning and / or data conversion and / or time series processing on historical data or real-time data streams, wherein the data cleaning includes missing values, outliers, deduplication processing and format standardization, the data conversion includes normalizing numerical data and encoding categorical data through one-hot encoding and label encoding, and the time series processing includes sliding window feature extraction and time decomposition processing of time series data.

[0099] In this embodiment, the real-time data transmission module receives the real-time data stream of the device operation after data cleaning and / or data conversion and / or time series processing. The data management module also persistently stores the processed data.

[0100] Based on the solutions of the aforementioned embodiments of the present invention, this method can deploy and manage a variety of machine learning models and non-machine learning models, realize the processing of real-time data streams, and realize the full life cycle management of model data, namely, model selection, deployment, training, update iteration and reasoning evaluation. After the training is completed, the machine learning model will persistently store the generated model code file containing the model training and update results in the data management module, which can facilitate user query and acquisition, realize the sharing of deployed and trained updated models, and quickly complete reasoning, thereby avoiding the problem of inefficient model management caused by retraining models with lengthy training time and avoiding waste of resources.

[0101] Corresponding to the above-mentioned embodiments of the present invention, the operation process of the front-end user is as follows: Figure 4 As shown. The user first selects and specifies the virtual model to be managed. The virtual module includes three categories: basic model, functional model and transmission model. Then, according to the specified virtual model, the machine learning model or non-machine learning model is further selected. If a machine learning model is selected, the model training method is selected as online training, offline training or incremental training. After selecting different model training methods, the corresponding steps will be executed according to different training processes. If a non-machine learning model is selected, the model code file of the model is uploaded and then online inference is performed.

[0102] Corresponding to the above method, an embodiment of the present invention also provides an optical transport network digital twin model management system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the above method.

[0103] like Figure 5 As shown in the figure, the optical transport network digital twin model management system is realized through the interaction between the front-end and the back-end. The back-end includes a data acquisition module, a data management module, an online reasoning instance management module, a basic environment management module, a real-time data transmission module, and an online training instance management module. The specific interactions between the modules and the collaborative working process are detailed in the above content.

[0104] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0105] An embodiment of the present invention further provides a computer program product, comprising computer instructions, which implement the steps of the aforementioned method when executed by a processor.

[0106] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0107] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0108] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0109] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for managing a digital twin model of an optical transport network, characterized in that: The method comprises: Based on user needs and different types of virtual models in the optical transport network digital twin platform, select a machine learning model or non-machine learning model that implements functions corresponding to user needs; Upload the corresponding inference environment configuration parameters based on the selected machine learning model, upload the corresponding model code file and inference environment configuration parameters based on the selected non-machine learning model, and store the inference environment configuration parameters and model code files persistently in the data management module; The data collection module collects the real-time data stream of the equipment running in the optical transmission network; After the selected machine learning model is deployed and trained or the model code file of the non-machine learning model is uploaded, the online reasoning instance management module sends a reasoning environment configuration request of the selected trained machine learning model or non-machine learning model to the data management module; The data management module provides corresponding reasoning environment configuration parameters to the online reasoning instance management module according to the reasoning environment configuration request of the selected trained machine learning model or non-machine learning model, so that the online reasoning instance management module configures the reasoning operating environment according to the reasoning environment configuration parameters; The basic environment management module generates an inference instance according to the configured inference operating environment, and the inference instance sends a retrieval request for a model code file of a selected trained machine learning model or non-machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the selected trained machine learning model or non-machine learning model model code file to the inference instance; the model code file of the trained machine learning model is a model code file generated by the machine learning model after training and contains a model training result and is persistently stored in the data management module; The inference instance receives the selected trained machine learning model or non-machine learning model model code file to complete the model inference instantiation; The real-time data transmission module receives the real-time data stream from the data acquisition module, sends the real-time data stream to the corresponding topic, and monitors the topic. When a new message is monitored, the real-time data stream is routed to the reasoning instance so that the reasoning instance performs reasoning based on the real-time data stream, generates reasoning results, and sends the reasoning results and real-time data stream to the real-time data transmission module.

2. The method according to claim 1, characterized in that The method further comprises: Upload the corresponding model code file and training environment configuration parameters based on the selected machine learning model, and store the model code file and training environment configuration parameters persistently in the data management module; The data collection module collects the historical data of the operation of the equipment in the optical transmission network, and stores the historical data persistently in the data management module; The online training instance management module receives an online training request for the machine learning model selected by the user, and sends a training environment configuration request for the selected machine learning model to the data management module based on the online training request; The data management module provides corresponding training environment configuration parameters to the online training instance management module according to the training environment configuration request of the selected machine learning model, so that the online training instance management module configures the training operation environment according to the training environment configuration parameters; The basic environment management module generates a training instance according to the configured training operating environment, and the training instance sends a retrieval request for the model code file of the selected machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected machine learning model to the training instance; The training instance receives the model code file of the selected machine learning model, completes the model training instantiation, and receives the historical data of device operation pushed by the data management module. The selected machine learning model is trained online based on the historical data, and the generated model code file containing the model training results is persistently stored in the data management module.

3. The method according to claim 1, characterized in that The method further comprises: Receive an offline training request for a machine learning model selected by a user, and upload a model code file containing offline training results and training environment configuration parameters of the selected machine learning model based on the offline training request; The model code file containing the offline training results and the training environment configuration parameters of the selected machine learning model are persistently stored in the data management module.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: The online training instance management module receives an incremental training request for a trained machine learning model selected by a user, and sends a training environment configuration request for the selected trained machine learning model to the data management module based on the incremental training request; The data management module provides corresponding training environment configuration parameters to the online training instance management module according to the training environment configuration request of the selected trained machine learning model, so that the online training instance management module configures the training operating environment according to the training environment configuration parameters; The basic environment management module generates a training instance according to the configured training operating environment, and the training instance sends a retrieval request for a model code file of a selected trained machine learning model to the data management module, so that the data management module performs a retrieval operation according to the retrieval request and sends the model code file of the selected trained machine learning model to the training instance; The training instance receives the model code file of the selected trained machine learning model, completes the model update instantiation, and receives the real-time data stream of the device running transmitted by the real-time data transmission module. The selected trained machine learning model is incrementally trained according to the real-time data stream, and the generated model code file containing the model update result is persistently stored in the data management module.

5. The method according to any one of claims 1 to 3, characterized in that The real-time data transmission module is implemented through the stream framework technology, and the basic environment management module is implemented through the containerization technology and the container orchestration technology.

6. The method according to claim 1 or 2, characterized in that: After the data collection module collects historical data or real-time data streams of equipment operation in the optical transmission network, the method further includes: Perform data cleaning and / or data conversion and / or time series processing on historical data or real-time data streams, wherein the data cleaning includes missing value, outlier, duplicate removal and format standardization, the data conversion includes one-hot encoding and label encoding processing of normalized numerical data and encoded categorical data, and the time series processing includes sliding window feature extraction and time decomposition processing of time series data; The real-time data transmission module receives the real-time data stream of the device operation after data cleaning and / or data conversion and / or time series processing.

7. The method according to any one of claims 1 to 3, characterized in that The virtual model includes a basic model, a transmission model and a functional model.

8. An optical transport network digital twin model management system, comprising a processor, a memory, and computer instructions stored in the memory, characterized in that: The processor is used to execute the computer instructions, and when the computer instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images