Digital twinning for AI / ML training and testing

Abstract

A method of simulating a network for machine learning and training is provided. The method may include generating a simulation model (digital twin) of the network, wherein the simulation model is based on receiving network data from the network. The method may also include training a machine learning model based on data received from the simulation model, and operating the trained machine learning model within the network. In addition, the network may be based on an open radio access network (O-RAN). Further, the method may include operating the simulation model within a non-real-time (non-RT) Radio Access Network Intelligent Controller (RIC) framework. Further, the method may include operating the simulation model within a near real-time (near RT) radio access network intelligent controller (RIC) framework. In addition, the method may include operating the simulation model in parallel with the O-RAN.

Description

Technical Field

[0001] The present disclosure described herein relates to simulating an Open Radio Access Network (O-RAN) for machine learning and training. Background Art

[0002] Radio Access Network (RAN) automation, management, orchestration and optimization technologies based on artificial intelligence (AI) and machine learning (ML) are key factors in the foundation of the Open Radio Access Network (O-RAN) architecture. In particular, non-real-time (RT) and near-RT RAN intelligent controllers (RIC) are currently the two main hosts for implementing RAN intelligence. However, conventional systems have many problems and challenges that need to be solved across the RAN industry before any AI / ML-based solutions can be commercially deployed and create real business value.

[0003] In particular, data availability for AI / ML model training is one of the major challenges facing the industry. Conventional AI / ML models are usually trained using real network data captured through vendor-prepared or standardized key performance indicator (KPI) exposure interfaces, which have the following challenges: limited data access, limited real-world sunny and rainy day scenarios, and limited dynamic interactions between AI / ML models and the network.

[0004] Furthermore, testing of rApps / xApps (which may be automated tools and applications) hosted by non-RT or near-RT RICs or any application running on any virtual RAN platform is challenging when executed on a live network as it may negatively impact the performance of the network itself. For example, any application that implements automatic antenna tilt methods to reduce inter-cell interference or increase cell coverage may result in significant performance degradation on the live network if the parameters, configurations or logic of these methods are not extensively tested on a simulation platform and if the simulation platform does not adequately represent the real-world network. Summary of the invention

[0005] According to example embodiments, the disclosure described herein provides a novel architecture for introducing a "digital twin" module in an AI / ML training host of a non-RT RIC. Here, the digital twin implemented by digital simulation and modeling can represent a digital replica of a physical O-RAN network connected to the RIC. The AI / ML model before deployment in the rAppl / xApp can be trained based on a training data set generated by the digital twin, which can supplement the limitations of real data captured from the physical network. The digital twin module can be calibrated with physical network data to create an accurate replica of the network not only for the historical state when the physical network data was captured, but also for any future or unknown state used to generate and construct training scenarios. According to one or more example embodiments, the digital twin module can be deployed in a near-RT RIC, rApp, xApp, or outside the RIC to support both offline and online training of AI / ML models.

[0006] In other example embodiments, one of the many applications of the present disclosure described herein is to create a lightweight digital replica (or virtual environment, model or simulation) of a physical O-RAN network that is as realistic as a real physical environment for efficient AI / ML model training and testing. This can be achieved via advanced wireless network modeling and data-driven model calibration techniques. For example, billions of training and test scenarios can be automatically generated from the digital twin module of the present disclosure described herein, which can significantly improve the performance and reliability of AI / ML solutions in RIC, while greatly reducing costs and overcoming data availability challenges, as well as other advantages or technical improvements. Here, the methods and systems of the present disclosure described herein can help accelerate the maturity and commercial deployment of O-RAN technology and RAN intelligent technology. In addition, the methods and systems of the present disclosure described herein can also be applied to any other type of wireless network with intelligence, and bring significant value to it, including but not limited to 4G, 5G and 6G networks that are not based on O-RAN standards, as well as WiFi, Bluetooth, LoRa, V2X and D2D.

[0007] In other example embodiments, a method of creating a lightweight, realistic digital replica of a network for machine learning and training includes generating a digital twin of the network, wherein the digital twin is calibrated based on receiving performance metric data from the network; training a machine learning model based on data generated from the digital twin; and operating the trained machine learning model within the network.

[0008] Additionally, the method may include wherein the network is based on an Open Radio Access Network (O-RAN).

[0009] Additionally, the method may include operating the digital twin within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

[0010] Furthermore, the method may include operating the digital twin within a near real-time (near RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

[0011] In addition, the method may include: operating the digital twin in parallel with O-RAN, wherein the digital twin is also operated outside of a non-real-time (non-RT) radio access network intelligent controller (RIC) framework and a near real-time (near-RT) radio access network intelligent controller (RIC) framework.

[0012] In addition, the method may include: the step of generating a digital twin of the network further includes:

[0013] In addition, the method may include analyzing performance of the machine learning model; and generating one or more network scenarios via a radio access network (RAN) scenario generator based on the performance of the machine learning model.

[0014] Furthermore, the method may include modeling the network based on network data received from the network and the generated one or more network scenarios from the RAN scenario generator.

[0015] Furthermore, the method may include: monitoring performance of the modeled network; providing feedback to the RAN scenario generator based on the monitored performance of the modeled network; and

[0016] Furthermore, the method may include optimizing the modeled network based on feedback provided to the RAN scenario generator.

[0017] Additionally, the method may include: wherein the digital twin includes an offline simulation module and a runtime simulation module.

[0018] In addition, the method may include: generating a user equipment (UE) mobility pattern within an offline simulation module; simulating radio frequency (RF) propagation within the offline simulation module and generating an RF map that at least partially represents power and interference at each location within a geographic area; and loading the UE mobility pattern and the generated RF map into an artificial intelligence (AI) or machine learning (ML) model being trained or tested at runtime to generate training or test data.

[0019] In other example embodiments, an apparatus for creating a lightweight, lifelike digital replica of a network for machine learning and training includes: a memory storage device storing computer-executable instructions; and a processor communicatively coupled to the memory storage device, wherein the processor is configured to execute the computer-executable instructions and cause the apparatus to: generate a digital twin of the network, wherein the digital twin is calibrated based on receiving performance metric data from the network; train a machine learning model based on data generated from the digital twin; and operate the trained machine learning model within the network.

[0020] Additionally, the apparatus may include wherein the network is based on an Open Radio Access Network (O-RAN).

[0021] In addition, the computer executable instructions, when executed by the processor, may also cause the device to: operate a digital twin within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

[0022] In addition, the computer executable instructions, when executed by the processor, may also cause the device to: operate a digital twin within a near real-time (near RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

[0023] In addition, the computer executable instructions, when executed by the processor, may also cause the device to: operate the digital twin in parallel with O-RAN, wherein the digital twin is also operated outside a non-real-time (non-RT) radio access network intelligent controller (RIC) framework and a near real-time (near RT) radio access network intelligent controller (RIC) framework.

[0024] In addition, a step of generating a digital twin of the network, wherein the computer executable instructions, when executed by the processor, may also cause the device to: analyze the performance of the machine learning model; and based on the performance of the machine learning model, generate one or more network scenarios via a radio access network (RAN) scenario generator.

[0025] Furthermore, the computer executable instructions, when executed by the processor, may also cause the apparatus to: model the network based on the network data received from the network and the one or more network scenarios generated from the RAN scenario generator.

[0026] In addition, the computer executable instructions, when executed by the processor, may also cause the device to: monitor the performance of the modeled network; provide feedback to the RAN scenario generator based on the monitored performance of the modeled network; and optimize the modeled network based on the feedback provided to the RAN scenario generator.

[0027] In addition, the digital twin can include offline simulation modules and runtime simulation modules.

[0028] In other example embodiments, a non-transitory computer-readable medium includes computer-executable instructions for creating, by an apparatus, a lightweight, lifelike digital replica of a network for machine learning and training, wherein the computer-executable instructions, when executed by at least one processor of the apparatus, cause the apparatus to: generate a digital twin of the network, wherein the digital twin is calibrated based on performance metric data received from the network; train a machine learning model based on data generated from the digital twin; and operate the trained machine learning model within the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The features, advantages, and significance of exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings, in which like symbols represent like elements, and in the drawings:

[0030] Figure 1 A diagram illustrating a general system architecture of the disclosed network simulation and machine learning methods and systems described herein according to one or more embodiments;

[0031] Figure 2 A diagram illustrating the process flow and various modules of the disclosed network simulation and machine learning method and system described herein according to one or more embodiments;

[0032] Figure 3 Another diagram illustrating the process flow and various modules of the disclosed network simulation and machine learning methods and systems described herein according to one or more embodiments;

[0033] Figure 4 Another diagram illustrating the process flow and various modules of the network simulation and machine learning platform method and system of the present disclosure described herein according to one or more embodiments;

[0034] Figure 5 Another diagram illustrating the process flow and various modules of the disclosed network simulation and machine learning methods and systems described herein according to one or more embodiments;

[0035] Figure 6 Another diagram illustrating the process flow and various modules of the disclosed network simulation and machine learning methods and systems described herein according to one or more embodiments;

[0036] Figure 7 Another diagram illustrating the process flow and various modules of the disclosed network simulation and machine learning methods and systems described herein according to one or more embodiments;

[0037] Figure 8 Another diagram illustrating the process flow and various modules of the network simulation and machine learning methods and systems of the present disclosure described herein according to one or more embodiments; and

[0038] Fig. 9 Another diagram illustrating the process flow and various units / modules of the disclosed network simulation and machine learning methods and systems described herein according to one or more embodiments. DETAILED DESCRIPTION

[0039] The following detailed description of example embodiments refers to the accompanying drawings.The same reference numbers in different drawings may identify the same or similar elements.

[0040] The above disclosure provides illustration and description, but is not intended to be exhaustive or to limit implementation to the disclosed precise form. According to the above disclosure, modification and variation are possible, or can be obtained from the practice of implementation. In addition, one or more features or parts of an embodiment can be incorporated into another embodiment (or one or more features of another embodiment) or combined with it. In addition, in the flow chart and the operation description provided below, it can be understood that one or more operations can be omitted, one or more operations can be added, one or more operations can be performed simultaneously (at least in part), and the order of one or more operations can be switched.

[0041] It is obvious that the system and / or method described herein can be implemented with different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code for implementing these systems and / or methods does not limit these implementations. Therefore, the operation and behavior of the system and / or method are described herein without reference to specific software codes. It should be understood that software and hardware can be designed to implement these systems and / or methods based on the description herein.

[0042] Even though particular combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features may be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

[0043] Unless explicitly stated, any element, act, or instruction used herein should not be construed as critical or essential. In addition, as used herein, the terms "a" and "an" are intended to include one or more items and can be used interchangeably with "one or more". Figure 1items, the term "only one" or similar language is used. In addition, as used herein, the terms "has," "have," "having," "include," "including," and the like are intended to be open-ended terms. In addition, unless otherwise expressly stated, the term "based on" means "based at least in part on." In addition, expressions such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include only A, only B, or both A and B.

[0044] In this specification, reference to "one embodiment," "an embodiment," or "a non-limiting exemplary embodiment" indicates that a particular feature, structure, or characteristic described in connection with the illustrated embodiment is included in at least one embodiment of the present solution. Thus, the phrases "in one embodiment," "in an embodiment," "in a non-limiting exemplary embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0045] In addition, the above-mentioned features, advantages and characteristics of the present disclosure can be combined in any suitable manner in one or more embodiments. Based on the description herein, those skilled in the relevant art will recognize that the present disclosure can be practiced without one or more specific features or advantages of a particular embodiment. In other cases, additional features and advantages that may not be present in all embodiments of the present disclosure may be recognized in certain embodiments.

[0046] In one implementation of the present disclosure described herein, a display page may include information residing in a memory of a computing device, which may be sent from the computing device to a database center via a network, or vice versa. The information may be stored in a memory at each computing device, in a data storage device residing at the edge of the network, or on a server at a database center. A computing device or mobile device may receive a non-transient computer-readable medium, which may contain instructions, logic, data, or code that may be stored in a persistent or transient memory of a mobile device, or may affect or initiate an action of the mobile device in some way. Similarly, one or more servers may communicate with one or more mobile devices via a network, and may send a computer file residing in a memory. For example, a network may include the Internet, a wireless communication network, or any other network for connecting one or more mobile devices to one or more servers.

[0047] Any discussion of computing or mobile devices may also apply to any type of network device, including, but not limited to, mobile devices and cell phones, such as mobile phones (e.g., any "smartphone"), personal computers, server computers, or laptop computers; personal digital assistants (PDAs); nomadic devices, such as network-connected nomadic devices; wireless devices, such as wireless email devices, or other devices capable of wireless communication with a computer network; or any other type of network device that can communicate over a network and conduct electronic transactions. Any discussion of any mobile device mentioned may also apply to other devices, such as devices including short-range ultra-high frequency (UHF) devices, near field communication (NFC), infrared (IR), and Wi-Fi capabilities.

[0048] Phrases and terms like "software," "application," "app," and "firmware" may include any non-transitory computer-readable medium having stored thereon a program that, when executed by a computer, causes the computer to perform a method, function, or control operation.

[0049] Phrases and terms similar to "network" may include one or more data links that enable electronic data to be transmitted between computer systems and / or modules. When information is transmitted or provided to a computer via a network or other communication connection (hardwired, wireless, or a combination of hardwired or wireless), the computer uses the connection as a computer-readable medium. Therefore, by way of example and not limitation, a computer-readable medium may also include a network or data link that may be used to carry or store a desired program code device in the form of a computer-executable instruction or data structure and may be accessed by a general-purpose or special-purpose computer.

[0050] Phrases and terms similar to "portal" or "terminal" may include an intranet page, an Internet page, a locally resident software or application, a mobile device graphical user interface, or a digital presentation of a user. A portal may also be any graphical user interface for accessing the various modules, components, features, options, and / or properties of the present disclosure described herein. For example, a portal may be a web page accessed using a web browser, a mobile device application, or any application or software resident on a computing device.

[0051] Figure 1 A diagram illustrating a general network architecture according to one or more embodiments is shown. Figure 1According to one or more embodiments, end users 110, network support team users 120, and management terminal / dashboard users 130 (collectively referred to herein as users 110, 120, and 130) can communicate bidirectionally with a central server or application server 100 via a secure network. In addition, according to one or more embodiments, users 110, 120, and 130 can also communicate directly bidirectionally with each other via the network system of the present disclosure described herein. Here, user 110 can be any type of customer of a network or telecommunications service provider, a network service provider agent or supplier, etc., such as a user operating a computing device and user terminals A, B, and C. Each user 110 can communicate with server 100 via its corresponding terminal or portal, wherein server 110 can provide or automatically operate the network impact prediction engine system and method of the present disclosure described herein. User 120 may include an application development member or support agent of a network service provider, which is used to develop, integrate, and monitor the network simulation and machine learning methods and systems of the present disclosure described herein, including assisting, scheduling / modifying network events, and providing support services to end users 110. The management terminal / dashboard user 130 may be any type of user with access rights to a dashboard or management portal of the present disclosure described herein, wherein the dashboard portal may provide various user tools, GUI information, maps, open / closed / pending support tickets, charts, and customer support options. It is contemplated within the scope of the present disclosure described herein that any of the users 110 and 120 may also access the management terminal / dashboard 130 of the present disclosure described herein.

[0052] Still refer to Figure 1 According to one or more embodiments, the central server 100 of the present disclosure described herein may further bidirectionally communicate with a database / third-party server 140, which may also include a user. Here, the server 140 may include a supplier and a database on which various captured, collected, or aggregated data, such as current, real-time, and past network-related history and KPI data, may be stored and retrieved for network analysis, RCA, artificial intelligence (AI) processing, neural network models, machine learning, prediction, and simulation performed by the server 100. In addition, the server 100 may include a digital twin module of the present disclosure described herein. However, within the scope of the present disclosure described herein, it is contemplated that the network simulation and machine learning methods and systems of the present disclosure described herein may include any type of general network architecture.

[0053] Still refer to Figure 1One or more servers or terminals of elements 100-140 may include a personal computer (PC), a printed circuit board including a computing device, a minicomputer, a mainframe computer, a microcomputer, a telephone computing device, a wired / wireless computing device (e.g., a smartphone, a personal digital assistant (PDA)), a laptop computer, a tablet computer, a smart device, a wearable device, or any other device with similar functionality.

[0054] In some embodiments, Figure 1 As shown, one or more servers, terminals and users 100-140 may include a set of components such as a processor, a memory, a storage component, an input component, an output component, a communication interface and a JSON UI rendering component. The component set of the device may be communicatively coupled via a bus.

[0055] The bus may include one or more components that allow communication between a group of components of one or more servers or terminals of elements 100-140. For example, the bus may be a communication bus, a crossbar, a network, etc. The bus may be implemented using a single or multiple (two or more) connections between a group of components of one or more servers or terminals of elements 100-140. The present disclosure is not limited in this regard.

[0056] One or more servers or terminals of element 100-140 may include one or more processors. One or more processors may be implemented with a combination of hardware, firmware and / or hardware and software. For example, one or more processors may include a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a general purpose single chip or multi-chip processor or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, or any conventional processor, controller, microcontroller or state machine. One or more processors may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In some embodiments, a particular process and method may be performed by a circuit system specific to a given function.

[0057] One or more processors may control the overall operation of one or more servers or terminals of elements 100-140 and / or a collection of components (e.g., memory, storage components, input components, output components, communication interfaces, rendering components) of one or more servers or terminals of elements 100-140.

[0058] One or more servers or terminals of elements 100-140 may also include memory. In some embodiments, the memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic memory, optical memory, and / or another type of dynamic or static storage device. The memory may store information and / or instructions for use (e.g., execution) by the processor.

[0059] The storage component of one or more servers or terminals of elements 100-140 may store information and / or computer-readable instructions and / or codes related to the operation and use of one or more servers or terminals of elements 100-140. For example, the storage component may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a universal serial bus (USB) flash drive, a personal computer memory card international association (PCMCIA) card, a floppy disk, a cassette, a magnetic tape, and / or another type of non-transitory computer-readable medium, and a corresponding drive.

[0060] One or more servers or terminals of elements 100-140 may also include an input component. The input component may include one or more components that allow one or more servers and terminals 100-140 to receive information, such as via user input (e.g., a touch screen, keyboard, keypad, mouse, stylus, button, switch, microphone, camera, etc.). Alternatively or additionally, the input component may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.).

[0061] The output components of any one or more servers or terminals of elements 100-140 may include one or more components (e.g., a display, a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), a tactile feedback device, a speaker, etc.) that can provide output information from device 100.

[0062] One or more servers or terminals of element 100-140 may also include a communication interface. The communication interface may include a receiver component, a transmitter component and / or a transceiver component. The communication interface may enable one or more servers or terminals of element 100-140 to establish a connection and / or transmit communication with other devices (e.g., a server, another device). Communication may be realized via a combination of wired connection, wireless connection or wired and wireless connection. The communication interface may allow one or more servers or terminals of element 100-140 to receive information from another device and / or provide information to another device. In some embodiments, the communication interface may provide communication with another device via a network, such as a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a private network, an ad hoc network, an intranet, the Internet, a fiber-based network, a cellular network (e.g., a fifth generation (5G) network, a sixth generation (6G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a telephone network (e.g., a public switched telephone network (PSTN)), etc., and / or a combination of these or other types of networks. Alternatively or additionally, the communication interface may provide communication with another device via a device-to-device (D2D) communication link, such as FlashLinQ, WiMedia, Bluetooth, ZigBee, Wi-Fi, LTE, 5G, etc. In other embodiments, the communication interface may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, etc. It should be understood that other embodiments are not limited in this regard and may be implemented in a variety of different architectures (e.g., bare metal architecture, any cloud-based architecture or deployment architecture, such as Kubernetes, Docker, OpenStack, etc.).

[0063] Figure 2 The process flow and various modules of the O-RAN modeling and training platform method and system of the present disclosure described herein according to one or more exemplary embodiments are illustrated. Specifically, Figure 2The service, management and orchestration (SMO) and non-RT RIC architecture and framework 200 of the present disclosure described herein implementing a digital twin module 202 (i.e., a simulation / replica model of a physical network) and a near-RT RIC framework related to a physical O-RAN are illustrated. Part of the goal of the framework 200 is to improve the ML model training module 204A and the testing process in the non-RT RIC framework. Here, the O-RAN Central Unit (O-CU) and the O-RAN Distributed Unit (O-DU) (which can be logical nodes) send network data, performance feedback for offline / online training to the ML training module / host 204 (via the O1 interface), send network data for ML reasoning to the rApp module 210 (via the O1 interface), send network data for calibration with respect to the digital twin module 202 (via the O1 interface), send network data for online learning to the ML training module / host 222 (via the E2 interface), and send network data for ML reasoning to the ML reasoning module / host 220.

[0064] Still refer to Figure 2 , the ML training module / host 204 can upload and download ML model data from the rApp module 210, such as data from the ML reasoning module / host 212 and the ML training module or host 214. In particular, after the model is trained and tested, it can be deployed in the rApp module 210A for additional ML training via module 214 and ML model reasoning via module 212. In addition, the ML training module / host 20A can communicate bidirectionally with the ML model repository 206 to send and receive modeling data. Here, by using data from the digital twin module 202 and the collected physical network data from the O-CU / O-DU module 250, the system can be used to train and test ML models offline via the ML training module / host 204 or online via the ML model module 222 within a real physical network.

[0065] Note that real network data from O-RAN interfaces is often limited and cannot provide suitable training for models to address specific scenarios or cover all situations to ensure reliable operation supported by AI / ML technologies. In addition, ML models may need to be tested and proven reliable in all possible situations before real network problems occur. Figure 2As shown, the digital twin module 202 of the present disclosure described herein provided within the non-RT RIC framework 200 can improve the performance of the training and testing process by executing and generating all possible scenarios offline within the ML training module / host before the ML model is implemented within the real physical network, thereby providing a mature and tested ML model to operate within the real physical network. Here, the digital twin model 202 implemented by computer simulation can represent a digital replica of the physical O-RAN network through advanced simulation and modeling, just as if it were real.

[0066] In addition, rApp and xApp ( Figure 2-Figure 6 ) can access the digital twin module 202 (and its functions) in the non-RT RIC framework and the near-RT RIC framework via the R1 and near-RT RIC API interfaces, respectively, and access AI / ML workflow-related services for AI / ML model training and testing. rApp can directly access the training data generated based on the digital twin via the R1 interface for AI / ML model training in the rApp itself, or load the AI / ML model into the training host in the non-RT RIC framework for platform layer model training and testing, and specify via the R1 interface whether and when physical network data or digital twin data should be used. xApp AI / ML model training occurs in the non-RT RIC framework, training host, and the xApp vendor or operator can specify whether and when physical or digital twin module data should be used for training and testing through some standard interface. xApp AI / ML model training can also occur in the near-RT RIC framework, training host, and the xApp can specify whether and when physical or digital twin module data should be used for training and testing through the near-RT RIC API interface. The xApp can also access training data generated based on the digital twin via the near-RT RIC API interface for AI / ML model training in the xApp itself.

[0067] Still refer to Figure 2 In one method of operation, the digital twin module 202 can be implemented at a slower time scale relative to real time. For example, in the 3GPP 4G-LTE standard, a frame can have a duration of 10msec, and when a signal waveform is implemented on a digital twin module, its frame duration can be much longer than 10msec (e.g., 100msec, 1 second or longer, or any number that is not a multiple of 10msec). In fact, real-time processing may require a large amount of computing resources, multiple CPUs, and multiple threads to generate LTE waveforms in real time. In contrast, the systems and methods of the present disclosure described herein can be implemented with a limited number of CPUs, cores, threads, processes, or any other type of processing unit because real-time constraints can be relaxed and expanded.

[0068] Figure 3 Picture shows Figure 2 An alternative embodiment in which the digital twin module 202 can be implemented in a near-RT RIC framework 260. In this embodiment, the O-CU / O-DU module 250 can send real network data (via the E2 interface) to the ML training module / host 222 for training and testing used offline or online by the xApp module 210, where the xApp module 210 may include an inference module 224 and an ML training module or host 226. The performance of the digital twin module 202 can be calibrated with real data captured from the O-CU / O-DU module 250 (via the E2 interface) to ensure that the behavior of the digital twin module is as close to the real network deployment as possible, thereby performing reliable RIC AI / ML model training and testing. The xApp module 210 can also access the digital twin module 202 in the near-RT RIC framework 260 via the near-RT RIC API / SDK interface 232 for AI / ML model training and testing. The xApp module 210 can also directly access the training data generated from the digital twin module 202 via the near-RT RIC API / SDK 232, or load the AI / ML model (from the ML training module / host 222 or the ML model library 206) into the training host in the near-RT RIC framework for platform layer model training and testing, and specify whether and when physical network data or digital twin module data should be used via the near-RT RIC API / SDK 234. Figure 2 Other modules and methods discussed about Figure 3 are incorporated into this article.

[0069] Figure 4 and Figure 5 Pictured Figure 2 Other alternative embodiments, in which the digital twin module 202 can be implemented in the application layer of the non-RT RIC 200 via the rApp module 210 for AI / ML model training and testing, such as Figure 4 As shown, or implemented in the application layer of the near-RT RIC framework 260 via the xApp module 230 for AI / ML model training and detection, as shown Figure 5 As shown. Figure 4 or Figure 5In any embodiment of the present invention, AI / ML training and testing tasks are performed in the application layer, rather than in the non-RT or near-RT RIC framework / platform layer. This can allow xApp or rApp vendors or third parties to have greater flexibility in the implementation choices of AI / ML model training, rather than relying on non-RT RIC or near-RT RIC framework AI / ML training services provided via standard API interfaces (e.g., R1 interfaces and near-RT RIC APIs). Here, Figure 4 and Figure 5 The embodiments allow xApp and rApp vendors to provide AI / ML training and testing services based on the digital twin module 202 to other vendors and other applications via standard interfaces. Figure 2-Figure 3 The other modules and methods discussed are about Figure 4 and Figure 5 are incorporated into this article.

[0070] Figure 6 Picture shows Figure 2 Another alternative embodiment, in which the digital twin module 202 can be implemented outside the near-RT framework 260 and the non-RT RIC framework 200. In this embodiment, the digital twin module 202 can simulate a real physical network environment and can also run in parallel with the real physical network. From the perspective of the RIC framework 200 or 260, there is no substantial difference between the physical network and the simulated network provided by the digital twin module 202. Here, the RIC framework 200 or 260 can use the O-CU / O-DU module 250 (via standard O-RAN interfaces O1, O2 and E2) to communicate with the digital twin module 202 and the physical network. Here, Figure 6 The embodiment of may allow the digital twin module 202 to be provided via a third party or supplier instead of the RIC platform supplier. In addition, for this embodiment, during online ML training where multiple training scenarios will be generated, interoperability issues and performance overhead on external interfaces (e.g., O1, O2, and E2) may need to be considered. Figure 2-Figure 5 The other modules and methods discussed are about Figure 6 are incorporated into this article.

[0071] Figure 7The process flow and various modules of the digital twin module 202 of the present disclosure described herein according to some exemplary embodiments are illustrated. Specifically, the digital twin module 202 may include a RAN scenario generator module 210, which may create various network-related scenarios and configure them to the modeling module 300 for modeling and simulation. Specifically, the module 300 may include a mobility / RF model module 302, a cloud model module 304, a RAN model module 306, and a business model module 308. The modeling module may receive data from the O-RAN network 320 via an O-RAN interface 326, and may also send data to and receive data from the AI / ML model module 324. In addition, the AI / ML model module 324 may also receive data from the O-RAN network 320 via an O-RAN interface 322. In addition, the digital twin module 202 may also include a RAN analysis module 312, which includes an analysis engine module 314 (for performance feedback), which may receive data from the modeling module 300 and further send the data as input to the RAN scenario generator module 310.

[0072] Still refer to Figure 7 In one exemplary method of operation, the RAN scenario generator module 310 (which may be powered by AI / ML technology) may configure parameters of a digital twin model or simulated O-RAN network to automatically generate one or more test scenarios or network events (data sets) to challenge the RIC AI / ML model module 324 being trained and tested. The RAN scenario generator module 310 may also be trained and further evolved based on performance feedback from the RAN analysis module 312 to form a generative adversarial network (GAN). The evolution process of the digital twin module 202 may run continuously in a loop and provide challenging network scenarios (such as node downtime / inoperability, network coverage issues, etc.) to the RIC AI / ML model being trained and tested. In an example embodiment, as the performance of the RIC AI / ML model module 324 improves, the training and test scenarios generated by the RAN scenario generator module 310 may automatically become more and more challenging until a certain level of intelligence and reliability is reached. The above method can be applied to all stages of the AI / ML training and testing process in RIC, such as offline before the AI / ML model is deployed for operation, online after the AI / ML model is deployed but before the AI / ML model issues control actions and instructions to the network, and online after the AI / ML model is deployed and after the AI / ML model issues control actions and instructions.

[0073] Figure 8 Pictured Figure 7Alternative embodiments. In particular, the digital twin module 202 can also be accessed and operated in the cloud (or via an application server) via one or more third parties or suppliers. Specifically, the O-RAN network 320 can send network data to the modeling module 300, wherein the O-RAN network 320 can also receive information from the rApp / xApp module 330. In addition, the rApp / xApp module 330 can also send information to and receive information from the modeling module 300 via the O-RAN interface 326. In addition, the rApp / xApp module 300 can also send information to the RAN analysis module 312 via the internal interface 328. In an exemplary operating method, the O1, O2, and E2 interfaces can be used to collect real network data from the O-RAN network to train and test the AI / ML model used by the rApp / xApp module 330 offline or online. The performance of the digital twin module 202 can be calibrated with real data captured from the network through standard O1, O2, and E2 interfaces or proprietary interfaces to ensure that the behavior of the digital twin module 202 is as close as possible to the real network deployment to achieve reliable cloud RIC platform 240 and its AI / ML model training and testing. The digital twin module 202 can provide generated training data and interact with the AI / ML model in the cloud RIC platform 240 through standard O1, O2, and E2 interfaces (O-RAN interface 326) or proprietary interfaces. The digital twin module 202 can also be deployed in the cloud RIC platform 240 or within the application layer. Here, about Figure 7 Other modules and methods discussed about Figure 8 are incorporated into this article.

[0074] refer to Fig. 9, the digital twin module 202 system may include one or more units or modules. In some example embodiments, there may be two different types of units: 1) "offline" units or modules implemented at a slower time scale relative to the real-time execution of the system; 2) "runtime" units or modules implemented at the same or faster time scale relative to the real-time execution of the system. In an exemplary embodiment, the offline units or modules may include, but are not limited to, an RF grid generator module 410 and a user equipment / equipment (UE) mobility pattern generator module 402. Specifically, the RF grid generator module 410 may include a free space path loss model module 412, a statistical fading model module 414, a ray tracing model module 416, and an AI / ML RF model module 418. Models can be flexibly selected based on modeling accuracy requirements in order to train specific types of AI / ML models for specific use cases. In another exemplary embodiment, the runtime units or modules may include, but are not limited to, a mobility / RF model runtime module 402, a RAN model module 404, a core / business model module 406, and an O-Cloud model module 408. However, it is contemplated that any unit that may belong to the digital twin module 202 system may be an offline unit or a runtime unit within the scope of the present disclosure described herein.

[0075] like Fig. 9 As shown, the RF mesh propagation generator module 410 and the user equipment (UE) mobility pattern behavior module 400 can be simulated by the mobility / RF model module to realistically replicate the RF environment using advanced ray tracing based on RF modeling techniques from the RF model module 418 or AI / ML from the ray tracing model module 416. The network protocol stack functions (i.e., physical layer, L2 and L3) are simulated using the RAN model using existing RAN function implementation and simulation techniques. The business patterns and core behaviors of high-level applications are simulated by the business model or module 406. The cloud infrastructure of the O-RAN network is simulated with the O-Cloud model or module 408. Here, the performance of the digital twin module 202 (i.e., the simulated O-RAN network) can be calibrated with real data captured from the open network interfaces O1, O2 and A1 or any proprietary interface to ensure that the behavior of the digital twin module 202 is as close to the real network deployment as possible to achieve reliable RIC AI / ML model training and testing.

[0076] Still refer to Fig. 9, the runtime unit can interact directly with the RIC AI / ML unit at a relatively fast speed during training and testing for large interactive training and dataset generation. For true replication, the behavior of the digital twin module model can be the same as the real network, and therefore may encounter the same computational complexity (e.g., a full L1, L2, and L3 stack implementation). However, in order to make the digital twin module lightweight, by relaxing its constraints relative to real-time constraints (such as time scales), the clock speed used to drive the model can be much slower than the real-time requirements of the network for training and testing purposes. It is conceivable that within the scope of the present disclosure described herein, if the hard real-time timing constraints are removed, soft real-time or faster than real-time is also possible for large-scale training dataset generation, and the amount of CPU and memory resources is smaller.

[0077] Still refer to Fig. 9 , within the scope of the present disclosure described herein, it is contemplated that some behaviors of the real physical network may be simplified and abstracted to further reduce the computational cost of the digital twin module 202 model. For example, RF environment techniques such as ray tracing (via module 416) or AI / ML-based RF models (via module 418) may involve high computational complexity. Such modeling calculations may be performed offline. Fig. 9 As shown, the offline RF mesh generator module 410 is based on ray tracing and AI / ML techniques, using limited CPU resources, to derive RF signal power and interference strength in a real-world environment at a much slower speed than its runtime counterpart. This offline process is acceptable because the RF large-scale environment does not change as frequently as other parts of the network that interact with the RIC AI / ML model. Large-scale RF environments typically change when antenna and beam configurations change at a relatively slow rate (e.g., antenna down tilt, azimuth, gain, and beam pattern, etc.). In addition, UE mobility pattern calculations (via module 400) can also be performed at a low speed, which can generate UE movement trajectories offline for runtime layer reading and playback. In addition, the calculation of the RF mesh and UE mobility patterns can be accelerated by GPU or FGPA, which can greatly increase the speed with greater parallelism without hard delays or real-time requirements in training and testing environments. In some AI / ML training and testing scenarios where the details of the RF environment are not required, the RF model can also be simplified in its simplest form using statistical-based RF modeling techniques (via module 414) or free space path loss models (via module 412) to minimize complexity.

[0078] It should be understood that the specific order or hierarchy of blocks in the process / flowchart disclosed herein is an illustration of an exemplary method. It is understood that, based on design preferences, the specific order or hierarchy of blocks in the process / flowchart can be rearranged. In addition, some blocks can be combined or omitted. The attached method requires the elements of each block to be presented in a sample order, and does not represent being limited to the specific order or hierarchy presented.

[0079] Some embodiments may involve systems, methods, and / or computer-readable media at any possible level of integrated technical detail. In addition, one or more of the above components may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include (multiple) computer-readable non-transitory storage media having computer-readable program instructions thereon for causing the processor to perform operations.

[0080] Computer readable storage medium can be a tangible device, which can retain and store instructions for use by instruction execution devices. Computer readable storage medium can be, for example, but not limited to, electronic storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage medium includes: portable computer floppy disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device (such as a convex structure in a punch card or groove) with instructions recorded thereon, and any suitable combination of the above. Computer readable storage medium as used herein itself should not be interpreted as transient signal, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse passing through optical fiber cable) or electrical signal sent by wire.

[0081] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a corresponding computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or downloaded to an external computer or external storage device. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network, and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0082] The computer readable program code / instruction for performing the operation can be an assembly instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-related instruction, a microcode, a firmware instruction, a state setting parameter, the configuration data of the integrated circuit system, or a source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and process programming languages ​​(such as "C" programming languages ​​or similar programming languages). The computer readable program instruction can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection to an external computer can be executed (for example, using an Internet service provider through the Internet). In some embodiments, an electronic circuit system (including, for example, a programmable logic circuit system, a field programmable gate array (FPGA) or a programmable logic array (PLA)) can execute computer readable program instructions by using the state information of the computer readable program instruction to personalize the electronic circuit system, thereby performing various aspects or operations.

[0083] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device create components for implementing the functions / actions specified in the flowchart and / or block diagram blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium, which may direct a computer, a programmable data processing device, and / or other device to operate in a particular manner, such that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in the flowchart and / or block diagram blocks.

[0084] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operating steps to be performed on the computer, other programmable apparatus, or other device, thereby producing a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in the flowchart and / or block diagram blocks.

[0085] The flowchart and block diagram in the figure illustrate the possible implementation architecture, function and operation of the system, method and computer-readable medium according to various embodiments. In this regard, each block in the flowchart or block diagram can represent a module, (multiple) microservices, segment or part of an instruction, which includes one or more executable instructions for implementing (multiple) specified logical functions. Compared with what is shown in the figure, the method, computer system and computer-readable medium may include more blocks, fewer blocks, different blocks or blocks arranged differently. In some alternative implementations, the functions described in the block may not appear in the order shown in the figure. For example, in fact, the two blocks shown in succession can be executed simultaneously or substantially simultaneously, or these blocks can sometimes be executed in the opposite order, depending on the functions involved. It will also be noted that each block illustrated in the block diagram and / or flowchart and the combination of blocks in the block diagram and / or flowchart can be implemented by a system based on special hardware, which performs a specified function or action or performs a combination of special hardware and computer instructions.

[0086] It is obvious that the system and / or method described herein can be implemented with different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code for implementing these systems and / or methods does not limit these implementations. Therefore, the operation and behavior of these systems and / or methods are described in this article without reference to specific software codes. It should be understood that software and hardware can be designed to implement these systems and / or methods based on the description herein.

Claims

1. A method for creating a lightweight photorealistic digital replica of a network for machine learning and training, the method comprising: generating a digital twin of a network, wherein the digital twin is calibrated based on receiving performance metric data from the network; training a machine learning model based on data generated from the digital twin; as well as The trained machine learning model is operated within the network.

2. The method of claim 1, wherein the network is based on an Open Radio Access Network (O-RAN).

3. The method according to claim 2, further comprising: The digital twin is operated within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

4. The method according to claim 2, further comprising: The digital twin is operated within a near real-time (near RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

5. The method according to claim 2, further comprising: The digital twin is operated in parallel with the O-RAN, wherein the digital twin is also operated outside of a non-real-time (non-RT) radio access network intelligent controller (RIC) framework and a near real-time (near-RT) radio access network intelligent controller (RIC) framework.

6. Method 1, wherein the step of generating a digital twin of the network further comprises: analyzing the performance of the machine learning model; as well as Based on the performance of the machine learning model, one or more network scenarios are generated via a radio access network (RAN) scenario generator.

7. The method according to claim 6, further comprising: The network is modeled based on network data received from the network and the one or more network scenarios generated from the RAN scenario generator.

8. The method according to claim 7, further comprising: monitoring performance of the modeled network; providing feedback to the RAN scenario generator based on the monitored performance of the modeled network; as well as The modeled network is optimized based on the feedback provided to the RAN scenario generator.

9. The method according to claim 8, wherein the digital twin includes an offline simulation module and a runtime simulation module.

10. The method according to claim 9, further comprising: generating a user equipment (UE) mobility pattern within the offline simulation module; simulating radio frequency (RF) propagation within the offline simulation module and generating an RF map representing at least in part power and interference at each location within a geographic area; as well as The UE mobility pattern and the generated RF map are loaded into an artificial intelligence (AI) or machine learning (ML) model being trained or tested at runtime to generate training or testing data.

11. An apparatus for creating a lightweight realistic digital replica of a network for machine learning and training, comprising: a memory storage device storing computer executable instructions; as well as a processor communicatively coupled to the memory storage device, wherein the processor is configured to execute the computer-executable instructions and cause the device to: generating a digital twin of a network, wherein the digital twin is calibrated based on receiving performance metric data from the network; training a machine learning model based on data generated from the digital twin; as well as The trained machine learning model is operated within the network.

12. The apparatus of claim 11, wherein the network is based on an Open Radio Access Network (O-RAN).

13. The apparatus of claim 12, wherein the computer executable instructions, when executed by the processor, further cause the apparatus to: The digital twin is operated within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

14. The apparatus of claim 12, wherein the computer executable instructions, when executed by the processor, further cause the apparatus to: The digital twin is operated within a near real-time (near RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for AI or ML model training and testing.

15. The apparatus of claim 12, wherein the computer executable instructions, when executed by the processor, further cause the apparatus to: The digital twin is operated in parallel with the O-RAN, wherein the digital twin is also operated outside of a non-real-time (non-RT) radio access network intelligent controller (RIC) framework and a near real-time (near-RT) radio access network intelligent controller (RIC) framework.

16. The apparatus of claim 111, wherein the step of generating a digital twin of the network, and wherein the computer executable instructions, when executed by the processor, further cause the apparatus to: analyzing the performance of the machine learning model; and Based on the performance of the machine learning model, one or more network scenarios are generated via a radio access network (RAN) scenario generator.

17. The apparatus of claim 16, wherein the computer executable instructions, when executed by the processor, further cause the apparatus to: The network is modeled based on network data received from the network and the one or more generated network scenarios from the RAN scenario generator.

18. The apparatus of claim 17, wherein the computer executable instructions, when executed by the processor, further cause the apparatus to: monitoring performance of the modeled network; providing feedback to the RAN scenario generator based on the monitored performance of the modelled network; and optimizing the modeled network based on the feedback provided to the RAN scenario generator; 19. The apparatus of claim 18, wherein the digital twin comprises an offline simulation module and a runtime simulation module.

20. A non-transitory computer-readable medium comprising computer-executable instructions for creating, by an apparatus, a lightweight lifelike digital replica of a network for machine learning and training, wherein the computer-executable instructions, when executed by at least one processor of the apparatus, cause the apparatus to: generating a digital twin of a network, wherein the digital twin is calibrated based on receiving performance metric data from the network; training a machine learning model based on data generated from the digital twin; as well as The trained machine learning model is operated within the network.

Images