Digital Twin for AI / ML Training and Testing

Abstract

A method for simulating a network for machine learning and training is provided. The method may include generating a simulated model (digital twin) of the network, where the simulated model is based on receiving network data from the network. The method may also include training a machine learning model based on the data received from the simulated model and operating the trained machine learning model within the network. In addition, the network may be based on an open radio access network (O-RAN). Further, the method may include operating the simulated model within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework. Also, the method may include operating the simulated model within a near-real-time (NRT) radio access network intelligent controller (RIC) framework. In addition, the method may include operating the simulated model in parallel with the O-RAN.

Description

[Technical Field]

[0001] The disclosure described herein relates to simulating an open radio access network (O-RAN) for machine learning and training. [Background technology]

[0002] Artificial intelligence (AI) and machine learning (ML)-based technologies for automation, management, orchestration, and optimization of the Radio Access Network (RAN) are critical elements for the foundation of the Open Radio Access Network (O-RAN) architecture. In particular, non-real-time (RT) and near-RT RAN Intelligent Controllers (RICs) are currently the two main hosts enabling RAN intelligence. However, there are many issues and challenges associated with traditional systems that need to be addressed across the RAN industry before any AI / ML-powered solution can be commercially deployed and generate real business value.

[0003] In particular, data availability for AI / ML model training is one of the major challenges facing the industry. Traditional AI / ML models are typically trained on real network data captured either through vendor-prepared or standardized key performance indicator (KPI)-exposed interfaces, which present the following challenges: limited data access, limited real-world sunny and rainy weather scenarios, and limited dynamic interactions between the AI / ML model and the network.

[0004] Furthermore, testing of rApps / xApps (which may be automated tools and applications) hosted by non-RT or quasi-RT RICs or any applications running on any virtual RAN platform is difficult because, when run on a live network, they may adversely affect the performance of the network itself. For example, any application that implements automatic antenna titling methods to reduce inter-cell interference or increase cell coverage may cause significant performance degradation on a live network if the parameters, configurations, or logic of those methods are not extensively tested on a simulation platform, and if that simulation platform does not adequately represent a real-world network. Summary of the Invention [Problem to be solved by the invention]

[0005] According to example embodiments, the disclosure described herein provides a novel architecture for deploying a “digital twin” module in an AI / ML training host in a non-RT RIC. Here, the digital twin, implemented through digital simulation and modeling, can represent a digital replica of the physical O-RAN network connected to the RIC. AI / ML models can be trained with training datasets generated from the digital twin, which can compensate for the limitations of actual data captured from the physical network, before deployment in rAppl / xApp. The digital twin module can be calibrated with physical network data to create an accurate replica of the network not only for the historical state when the physical network data was captured, but also for any future or unseen state for generating and building training scenarios. According to one or more example embodiments, the digital twin module can be deployed outside of a near-RT RIC, rApp, xApp, or RIC supporting both offline and online training of AI / ML models.

[0006] In another exemplary embodiment, one of the many applications of the present disclosure described herein is creating a lightweight digital replica (or virtual environment, model, or simulation) of a physical O-RAN network that is as realistic as the actual physical environment for efficient AI / ML model training and testing. This can be provided through advanced wireless network modeling and data-driven model calibration techniques. For example, billions of training and testing scenarios can be automatically generated from the digital twin module of the present disclosure described herein, which can significantly improve the performance and reliability of AI / ML solutions in RICs while significantly reducing costs and overcoming data availability challenges, among other benefits or technical improvements. Here, the methods and systems of the present disclosure described herein can help accelerate the maturation and commercial development of O-RAN and RAN intelligence technologies. Furthermore, the methods and systems of the present disclosure described herein can also be applied to and bring significant value to 4G, 5G, and 6G networks not based on O-RAN standards, as well as any other type of wireless network with intelligence, including, but not limited to, WiFi, Bluetooth, LoRa, V2X, and D2D.

[0007] In another example embodiment, a method for creating a lightweight, realistic digital replica of a network for machine learning and training is provided, the method including generating a digital twin of the network, the digital twin being calibrated based on receiving performance metric data from the network, training a machine learning model based on the data generated from the digital twin, and operating the trained machine learning model within the network.

[0008] Additionally, the method can include the network being based on an open radio access network (O-RAN).

[0009] Additionally, the method may include operating the digital twin within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing of AI or ML models.

[0010] The method may also include operating the digital twin within a near real-time (near-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing AI or ML models.

[0011] Further, the method may include operating the digital twin in parallel with the O-RAN, wherein the digital twin further operates outside of a non-real-time (non-RT) radio access network intelligent controller (RIC) framework and a near-real-time (near-RT) radio access network intelligent controller (RIC) framework.

[0012] Additionally, the method may include further comprising generating a digital twin of the network.

[0013] The method may also include analyzing performance of the machine learning model and generating one or more network scenarios via a radio access network (RAN) scenario generator based on the performance of the machine learning model.

[0014] Additionally, the method may include modeling the network based on the received network data from the network and the generated one or more network scenarios from the RAN scenario generator.

[0015] Additionally, the method may include monitoring performance of the modeled network and providing feedback to the RAN scenario generator based on the monitored performance of the modeled network.

[0016] The method may also include optimizing the modeled network based on the feedback provided to the RAN scenario generator.

[0017] Additionally, the method may include the digital twin comprising an offline simulation module and a runtime simulation module.

[0018] Further, the method may include generating user equipment (UE) mobility patterns in an offline simulation module, simulating radio frequency (RF) propagation in the offline simulation module to generate an RF map at least partially representing power and interference at each location in the geographic region, and loading the generated UE mobility patterns and RF map to generate training or test data for an artificial intelligence (AI) or machine learning (ML) model being trained or tested at run time.

[0019] In another exemplary embodiment, an apparatus for creating a lightweight, realistic digital replica of a network for machine learning and training is provided, the apparatus including: a memory storage that stores computer-executable instructions; and a processor communicatively coupled to the memory storage, the processor configured to execute the computer-executable instructions to cause the apparatus to generate a digital twin of the network, the digital twin being calibrated based on receiving performance metric data from the network, training a machine learning model based on the data generated from the digital twin, and operating the trained machine learning model within the network.

[0020] Additionally, the apparatus can include the network being based on an open radio access network (O-RAN).

[0021] Furthermore, the computer-executable instructions, when executed by the processor, further enable the device to operate the digital twin within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing AI or ML models.

[0022] Furthermore, the computer-executable instructions, when executed by the processor, further enable the device to operate the digital twin within a near-real-time (near-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing AI or ML models.

[0023] Additionally, the computer-executable instructions, when executed by the processor, further cause the device to operate the digital twin in parallel with the O-RAN, and the digital twin further operates outside of a non-real-time (non-RT) radio access network intelligent controller (RIC) framework and a near-real-time (near-RT) radio access network intelligent controller (RIC) framework.

[0024] Additionally, in the step of generating a digital twin of the network, the computer-executable instructions, when executed by the processor, may further cause the apparatus to analyze performance of the machine learning model and generate one or more network scenarios via a radio access network (RAN) scenario generator based on the performance of the machine learning model.

[0025] The computer-executable instructions, when executed by the processor, may also cause the apparatus to model the network based on the received network data from the network and the one or more network scenarios generated from the RAN scenario generator.

[0026] Furthermore, the computer-executable instructions, when executed by the processor, further cause the apparatus to monitor performance of the modeled network, provide feedback to the RAN scenario generator based on the monitored performance of the modeled network, and optimize the modeled network based on the provided feedback to the RAN scenario generator.

[0027] Additionally, the digital twin may include an offline simulation module and a runtime simulation module.

[0028] In another exemplary embodiment, a non-transitory computer-readable medium is provided that includes computer-executable instructions for creating a lightweight, realistic digital replica of a network for machine learning and training by an apparatus, the computer-executable instructions, when executed by at least one processor of the apparatus, causing the apparatus to: generate a digital twin of the network, the digital twin calibrated based on receiving performance metric data from the network; train a machine learning model based on data generated from the digital twin; and operate the trained machine learning model within the network.

[0029] The features, advantages, and significance of exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings, in which like reference numerals refer to like elements. [Brief explanation of the drawings]

[0030] [Figure 1]FIG. 1 is a diagram of a general system architecture of the disclosed network simulation and machine learning methods and systems described herein, according to one or more embodiments.

[0031] [Figure 2] FIG. 2 is a diagram of the process flow and various modules for the disclosed network simulation and machine learning methods and systems described herein, in accordance with one or more embodiments.

[0032] [Figure 3] FIG. 3 is another diagram of the process flow and various modules for the network simulation and machine learning methods and systems of the present disclosure described herein, in accordance with one or more embodiments.

[0033] [Figure 4] FIG. 4 is another diagram of the process flow and various modules for the network simulation and machine learning platform methods and systems of the present disclosure described herein, in accordance with one or more embodiments.

[0034] [Figure 5] FIG. 5 is another diagram of the process flow and various modules for the network simulation and machine learning methods and systems of the present disclosure described herein, in accordance with one or more embodiments.

[0035] [Figure 6] FIG. 6 is another diagram of the process flow and various modules for the network simulation and machine learning methods and systems of the present disclosure described herein, in accordance with one or more embodiments.

[0036] [Figure 7]FIG. 7 is another diagram of the process flow and various modules for the network simulation and machine learning methods and systems of the present disclosure described herein, in accordance with one or more embodiments.

[0037] [Figure 8] FIG. 8 is another diagram of the process flow and various modules for the network simulation and machine learning methods and systems of the present disclosure described herein, in accordance with one or more embodiments.

[0038] [Figure 9] FIG. 9 is another diagram of the process flow and various units / modules for the network simulation and machine learning methods and systems of the present disclosure described herein, in accordance with one or more embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0039] The following detailed description of the exemplary embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0040] The above disclosure provides illustration and description, but is not intended to be exhaustive or to limit implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). In addition, in the flowcharts and descriptions of operations provided below, it should be understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed (at least partially) concurrently, and the order of one or more operations may be rearranged.

[0041] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, and it will be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0042] Although particular combinations of features are recited in the claims and / or disclosed herein, these combinations are not intended to limit the disclosure of possible implementations. Indeed, many of these features can be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set.

[0043] No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and may be used interchangeably with "one or more." Where only one item is intended, the term "one" or similar language is used. Also, as used herein, terms such as "has," "have," "having," "include," and "including" are intended to be open-ended terms. Furthermore, the phrase "based on" is intended to mean "based at least in part on," unless specifically stated otherwise. Furthermore, phrases such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include A only, B only, or both A and B.

[0044] Throughout this specification, references to "one embodiment," "an embodiment," "a non-limiting exemplary embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with the illustrated embodiment is included in at least one embodiment of the solution. Thus, throughout this specification, the phrases "in one embodiment," "in an embodiment," "in one non-limiting exemplary embodiment," and similar language may, but do not necessarily, all refer to the same embodiment.

[0045] Furthermore, the described features, advantages, and characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that the present disclosure may be practiced without one or more of the specific features or advantages of a particular embodiment. In other cases, additional features and advantages may be recognized in particular embodiments that may not be present in all embodiments of the present disclosure.

[0046] In one implementation of the present disclosure described herein, a display page can include information residing in the memory of a computing device, and this information can be transmitted from the computing device to a database center and vice versa over a network. The information can be stored in memory at the computing device, in data storage at the edge of the network, or in a server at the database center. A computing device or a mobile device can accept non-transitory computer-readable media that can be stored in the mobile device's permanent or temporary memory or that can affect or initiate an action by the mobile device. Similarly, one or more servers can communicate with one or more mobile devices over a network and can transmit computer files residing in their memory. The network can include, for example, the Internet, a wireless communication network, or any other network for connecting one or more mobile devices to one or more servers.

[0047] Any discussion of computing or mobile devices may also apply to any type of networked device, including, but not limited to, mobile devices such as mobile phones (e.g., any "smartphone"), personal computers, server computers, or laptop computers, and telephones, personal digital assistants (PDAs), roaming devices such as network-attached roaming devices, wireless devices such as wireless email devices or other devices capable of wirelessly communicating with a computer network, or any other type of networked device that may communicate over a network and process electronic transactions. Any description of any mobile device mentioned may also apply to other devices, such as short-range ultra-high frequency (UHF) devices, near field communications (NFC), infrared (IR), and devices that include Wi-Fi capabilities, among others.

[0048] "Software," "application," "app," and "firmware" and similar phrases and terms may include any non-transitory computer-readable medium that stores a program that, when executed by a computer, causes the computer to perform a method, function, or control operation.

[0049] "Network" and similar phrases and terms may include one or more data links that enable the transport of electronic data between computer systems and / or modules. When information is transferred or provided to a computer over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless), the computer uses the connection as a computer-readable medium. Thus, by way of example and not limitation, a computer-readable medium may also include a network or data link that may be used to carry or store desired program code means in the form of computer-executable instructions or data structures and that may be accessed by a general-purpose or special-purpose computer.

[0050] Similar phrases and terms such as "portal" or "terminal" may include an intranet page, an internet page, locally residing software or application, a mobile device graphical user interface, or a digital presentation for a user. A portal may also be any graphical user interface for accessing the various modules, components, features, options, and / or attributes of the present disclosure described herein. For example, a portal may be a web page accessed with a web browser, a mobile device application, or any application or software residing on a computing device.

[0051] FIG. 1 illustrates a diagram of a general network architecture according to one or more embodiments. Referring to FIG. 1 , end users 110, network support team users 120, and administrator terminal / dashboard users 130 (collectively referred to herein as users 110, 120, and 130) can, according to one or more embodiments, bidirectionally communicate with a central server or application server 100 via a secure network. In addition, users 110, 120, and 130 may also, according to one or more embodiments, bidirectionally communicate directly with each other via the network system of the present disclosure described herein. Here, users 110 may be any type of customer of a network or communications service provider, a network service provider agent, or a vendor, such as users operating computing devices and user terminals A, B, and C, among others. Each of users 110 can communicate with server 100 via a respective terminal or portal, and server 110 can provide or automatically operate the network impact prediction engine system and method of the present disclosure described herein. Users 120 may include application development members or support agents of a network service provider for developing, integrating, and monitoring the network simulation and machine learning methods and systems of the present disclosure described herein, including assisting, scheduling / modifying network events, and providing support services to end users 110. Administrator terminal / dashboard users 130 may be any type of user with access privileges to access the dashboard or management portal of the present disclosure described herein, which can provide various user tools, GUI information, maps, open / closed / pending support tickets, graphs, and customer support options. It is contemplated within the scope of the present disclosure described herein that either of users 110 and 120 may also access the administrator terminal / dashboard 130 of the present disclosure described herein.

[0052] 1 , in accordance with one or more embodiments, the central server 100 of the present disclosure described herein can further bidirectionally communicate with a database / third-party server 140, which may also include a user. Here, the server 140 may include vendors and databases on which various captured, collected, or aggregated data, such as current, real-time, and past network-related historical and KPI data, may be stored and retrieved for network analysis, RCA, artificial intelligence (AI) processing, neural network models, machine learning, prediction, and simulation by the server 100. Additionally, the server 100 may include a digital twin module of the present disclosure described herein. However, within the scope of the present disclosure described herein, it is contemplated that the network simulation and machine learning methods and systems of the present disclosure described herein may include any type of general network architecture.

[0053] With further reference to FIG. 1 , one or more of the servers or terminals of elements 100-140 may include a personal computer (PC), a printed circuit board with a computing device, a minicomputer, a mainframe computer, a microcomputer, a telephone computing device, a wired / wireless computing device (e.g., a smartphone, a personal digital assistant (PDA)), a laptop, a tablet, a smart device, a wearable device, or any other similarly functional device.

[0054] 1, one or more servers, terminals, and users 100-140 may include a set of components such as a processor, memory, storage components, input components, output components, communication interfaces, and JSON UI rendering components. The set of components in a device may be communicatively coupled via a bus.

[0055] The bus may comprise one or more components that enable communication between one or more sets of components of the servers or terminals of elements 100-140. For example, the bus may be a communications bus, a crossover bar, a network, etc. The bus may be implemented using single or multiple (two or more) connections between one or more sets of components of the servers or terminals of elements 100-140. The disclosure is not limited in this respect.

[0056] One or more of the servers or terminals of elements 100-140 may include one or more processors. The one or more processors may be implemented in hardware, firmware, and / or a combination of hardware and software. For example, the one or more processors may include a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a general-purpose single-chip or multi-chip processor or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, or any conventional processor, controller, microcontroller, or state machine. The one or more processors may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In some embodiments, particular processes and methods may be performed by circuitry that is specific to a given function.

[0057] The one or more processors may control the overall operation of one or more of the servers or terminals of elements 100-140 and / or a set of one or more components of the servers or terminals of elements 100-140 (e.g., memory, storage components, input components, output components, communication interfaces, rendering components).

[0058] One or more of the servers or terminals of elements 100-140 may further comprise memory. In some embodiments, the memory may comprise random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic memory, optical memory, and / or another type of dynamic or static storage device. The memory may store information and / or instructions for use (e.g., execution) by the processor.

[0059] The storage component of one or more of the servers or terminals of elements 100-140 may store information and / or computer-readable instructions and / or code related to the operation and use of one or more of the servers or terminals of elements 100-140. For example, the storage component may include a hard disk (e.g., a magnetic disk, optical disk, magneto-optical disk, and / or solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a universal serial bus (USB) flash drive, a personal computer memory card international association (PCMCIA) card, a floppy disk, a cartridge, a magnetic tape, and / or another type of non-transitory computer-readable medium, along with a corresponding drive.

[0060] One or more of the servers or terminals of elements 100-140 may further comprise an input component. The input component may include one or more components that enable the server and one or more of the terminals 100-140 to receive information, such as via user input (e.g., a touchscreen, a keyboard, a keypad, a mouse, a stylus, a button, a switch, a microphone, a camera, etc.). Alternatively or additionally, the input component may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.).

[0061] The output components of any one or more of the servers or terminals of elements 100-140 may include one or more components that can provide output information from device 100 (e.g., a display, a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), a haptic feedback device, a speaker, etc.).

[0062] One or more of the servers or terminals of elements 100-140 may further comprise a communications interface. The communications interface may include a receiver component, a transmitter component, and / or a transceiver component. The communications interface may enable one or more of the servers or terminals of elements 100-140 to establish connections and / or transfer communications with other devices (e.g., a server, another device). The communications may be enabled via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communications interface may enable one or more of the servers or terminals of elements 100-140 to receive information from and / or provide information to another device. In some embodiments, the communication interface may provide for communication with another device over a network such as a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a private network, an ad hoc network, an intranet, the Internet, an optical fiber-based network, a cellular network (e.g., a fifth generation (5G) network, a sixth generation (6G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a telephone network (e.g., a public switched telephone network (PSTN)), etc., and / or a combination of these or other types of networks. Alternatively or additionally, the communication interface may enable communication with another device over a device-to-device (D2D) communication link such as FlashLinQ, WiMedia, Bluetooth, ZigBee, Wi-Fi, LTE, 5G, etc. In other embodiments, the communication interface may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, etc.It will be appreciated that other embodiments may be implemented in a variety of different architectures (e.g., bare metal architectures, any cloud-based or deployment architectures such as Kubernetes, Docker, OpenStack, etc.), without being limited thereto.

[0063] Figure 2 illustrates the process flow and various modules for the disclosed O-RAN modeling and training platform method and system described herein, in accordance with one or more example embodiments. In particular, Figure 2 illustrates the disclosed service, management, and orchestration (SMO) and non-RT RIC architecture and framework 200 described herein, which implements a digital twin module 202 (i.e., a simulated / replica model of the physical network) and a quasi-RT RIC framework for the physical O-RAN. In part, one of the goals of framework 200 is to improve the ML model training module 204A and testing process in a non-RT RIC framework. Here, the O-RAN central unit (O-CU) and O-RAN distributed units (O-DUs), which can be logical nodes, send network data, performance feedback for offline / online training to the ML training module / host 204 (via the O1 interface), send network data for ML inference to the rApp module 210 (via the O1 interface), send network data for calibration to the digital twin module 202 (via the O1 interface), send network data for online learning to the ML training module / host 222 (via the E2 interface), and send network data for ML inference to the ML inference module / host 220.

[0064] 2 , ML training module / host 204 can upload and download ML model data from rApp module 210, such as with respect to data from ML inference module / host 212 and ML training module / host 214. In particular, once a model has been trained and tested, it can be deployed within rApp module 210A for additional ML training via module 214 and ML model inference via module 212. Furthermore, ML training module / host 20A can bidirectionally communicate with ML model repository 206 to send and receive modeling data. Here, by using data from digital twin module 202 and collected physical network data from O-CU / O-DU module 250, the system can be used to train and test ML models within the actual physical network, either offline via ML training module / host 204 or online via ML model module 222.

[0065] It should be noted that actual network data from O-RAN interfaces is often limited and may not provide adequate training for models to address specific scenarios or cover all cases to ensure reliable operation driven by AI / ML techniques. Additionally, ML models may need to be tested and proven reliable in all possible scenarios before a real network issue occurs. However, as shown in FIG. 2 , the digital twin module 202 of the present disclosure provided within the non-RT RIC framework 200 described herein can improve the performance of the training and testing process by executing and generating all possible scenarios offline within the ML training module / host before the ML model is implemented within the actual physical network, thereby providing a mature, tested ML model for operation within the actual physical network. Here, the digital twin module 202 implemented by computer simulation can represent a digital replica of the physical O-RAN network as if it were real through advanced simulation and modeling.

[0066] Furthermore, rApps and xApps (Figures 2-6) can access the digital twin module 202 (and its functions) within the non-RT RIC framework and the quasi-RT RIC framework via the R1 and quasi-RT RIC API interfaces, respectively, and access AI / ML workflow-related services for training and testing AI / ML models. An rApp can directly access training data generated from the digital twin via the R1 interface for its own AI / ML model training, or load the AI / ML model onto a training host within the non-RT RIC framework for platform layer model training and testing, specifying via the R1 interface whether and when physical network data or digital twin data should be used. Within the non-RT RIC framework, xApp AI / ML model training occurs within the training host, and the xApp vendor or operator can specify via several standard interfaces whether and when physical or digital twin module data should be used for training and testing. xApp AI / ML model training can also occur within a quasi-RT RIC framework, where on the training host, the xApp can specify whether and when physical or digital twin module data should be used for training and testing via the quasi-RT RIC API interface. The xApp can also access training data generated from the digital twin via the quasi-RT RIC API interface for AI / ML model training in the xApp itself.

[0067] 2 , in one method of operation, the digital twin module 202 may be implemented on a timescale slower than real time. For example, in the 3GPP® 4G-LTE standard, a frame may have a duration of 10 milliseconds; when implementing a signal waveform on the digital twin module, the frame duration may last much longer than 10 milliseconds (e.g., 100 milliseconds, or 1 second or more, or any number that is not a multiple of 10 milliseconds). In practice, real-time processing may require numerous computing resources, multiple CPUs, and multiple threads, to enable LTE waveforms to be generated in real time. In contrast, the systems and methods of the present disclosure described herein can relax and scale real-time constraints and thus can be implemented with a limited number of CPUs, cores, threads, processes, or any other type of processing unit.

[0068] 3 illustrates an alternative embodiment of FIG. 2 in which the digital twin module 202 can be implemented within a quasi-RT RIC framework 260. In this embodiment, the O-CU / O-DU module 250 can send actual network data (via an E2 interface) to an ML training module / host 222 for training and testing, which is used by the xApp module 210 either offline or online, and the xApp module 210 can include an inference module 224 and an ML training module / host 226. The performance of the digital twin module 202 can be calibrated with actual data captured from the O-CU / O-DU module 250 (via an E2 interface) to ensure that the digital twin module behaves as closely as possible to an actual network deployment for reliable RIC AI / ML model training and testing. The xApp module 210 can also access the digital twin module 202 within the quasi-RT RIC framework 260 via a quasi-RT RIC API / SDK interface 232 for AI / ML model training and testing. The xApp module 210 can further directly access training data generated from the digital twin module 202 via the quasi-RT RIC API / SDK 232, or load AI / ML models (from the ML training module / host 222 or the ML model repository 206) onto a training host within the quasi-RT RIC framework for platform layer model training and testing, and specify whether and when physical network data or digital twin module data should be used via the quasi-RT RIC API / SDK 232. Here, other modules and methods such as those described with respect to FIG. 2 are incorporated herein with respect to FIG. 3.

[0069] 4 and 5 illustrate additional alternative embodiments of FIG. 2, in which the digital twin module 202 can be implemented in the application layer of the non-RT RIC 200 via the rApp module 210 for training and testing AI / ML models, as shown in FIG. 4, or in the application layer of the quasi-RT RIC framework 260 via the xApp module 230 for training and testing AI / ML models, as shown in FIG. 5. In either the FIG. 4 or FIG. 5 embodiment, AI / ML training and testing tasks are performed in the application layer rather than the non-RT or quasi-RT RIC framework / platform layer. This allows xApp or rApp vendors or third parties greater flexibility in selecting an implementation for AI / ML model training, rather than relying on non-RT RIC or quasi-RT RIC framework AI / ML training services provided via standard API interfaces (e.g., the R1 interface and the quasi-RT RIC API). Here, the FIG. 4 and FIG. 5 embodiments enable xApp and rApp vendors to provide AI / ML training and testing services based on the digital twin module 202 to other vendors and other applications via standard interfaces. Here, the other modules and methods described with respect to FIGS. 2-3 are incorporated herein with respect to FIGS.

[0070] FIG. 6 illustrates another alternative embodiment of FIG. 2 , in which the digital twin module 202 can be implemented outside the quasi-RT framework 260 and the non-RT RIC framework 200. In this embodiment, the digital twin module 202 can simulate an actual physical network environment and can also run in parallel with the actual physical network. From the perspective of the RIC framework 200 or 260, there can be no substantial difference between the physical network and the simulated network provided by the digital twin module 202. Here, the RIC framework 200 or 260 can communicate with both the digital twin module 202 and the physical network using the O-CU / O-DU module 250 (via standard O-RAN interfaces, O1, O2, and E2). Here, the embodiment of FIG. 6 can enable the digital twin module 202 to be provided through a third party or vendor instead of the RIC platform vendor. Furthermore, in this embodiment, interoperability issues and performance overhead of external interfaces (e.g., O1, O2, and E2) may need to be taken into account during online ML training, when several training scenarios are generated. Here, other modules and methods such as those described with respect to FIGS. 2-5 are incorporated herein with respect to FIG.

[0071] 7 illustrates the process flow and various modules of the digital twin module 202 of the present disclosure described herein, in accordance with some example embodiments. In particular, the digital twin module 202 can include a RAN scenario generator module 210 that can create and configure various network-related scenarios to the modeling module 300 for modeling and simulation. In particular, the module 300 can include a mobility / RF model module 302, a cloud model module 304, a RAN model module 306, and a traffic model module 308. The modeling module can also receive data from the O-RAN network 320 via an O-RAN interface 326 and send and receive data to and from the AI / ML model module 324. Furthermore, the AI / ML model module 324 can also receive data from the O-RAN network 320 via an O-RAN interface 322. Furthermore, the digital twin module 202 can also include a RAN analytics module 312 that includes an analytics engine module 314 (for performance feedback) that can receive data from the modeling module 300 and further send data as input to the RAN scenario generator module 310.

[0072] 7 , in one exemplary method of operation, the RAN scenario generator module 310 (which may be driven by AI / ML techniques) can automatically generate one or more test scenarios or network events (datasets) to configure parameters of the digital twin model or simulated O-RAN network to challenge the RIC AI / ML model module 324 during training and testing. The RAN scenario generator module 310 can also be trained and further evolved based on performance feedback from the RAN analytics module 312, which forms a generative adversarial network (GAN). The digital twin module 202 evolution process can run continuously in a loop, providing challenging network scenarios (e.g., down / inoperable nodes, network coverage issues, etc.) using the RIC AI / ML model being trained and tested. In an exemplary embodiment, the training and testing scenarios generated by the RAN scenario generator module 310 can automatically become more and more challenging as the performance of the RIC AI / ML model module 324 improves until a certain level of intelligence and reliability is achieved. The above-described methods can be applied at all stages of the AI / ML training and testing process in a RIC, including offline before the AI / ML model is deployed for operation, online after the AI / ML model is deployed but before control actions and guidance from the AI / ML model are provided to the network, and online after the AI / ML model is deployed and control actions and guidance are provided by the AI / ML model.

[0073] 8 illustrates an alternative embodiment to FIG. 7. Notably, the digital twin module 202 can also be accessed and operated in the cloud (or via an application server) via one or more third parties or vendors. Notably, the O-RAN network 320 can send network data to the modeling module 300, and the O-RAN network 320 can also receive information from the rApp / xApp module 330. In addition, the rApp / xApp module 330 can send and receive information to the modeling module 300 via an O-RAN interface 326. Furthermore, the rApp / xApp module 300 can send information to the RAN analytics module 312 via an internal interface 328. In one exemplary method of operation, the O1, O2, and E2 interfaces can be used to collect actual network data from the O-RAN network for training and testing AI / ML models used by the rApp / xApp module 330, either offline or online. The performance of the digital twin module 202 can be calibrated with actual data captured from the network via either standard O1, O2, and E2 interfaces or proprietary interfaces to ensure that the digital twin module 202 behaves as close as possible to a real network deployment for training and testing the reliable cloud RIC platform 240 and its AI / ML models. The digital twin module 202 can provide generative training data and interact with the AI / ML models in the cloud RIC platform 240 via either standard O1, O2, and E2 interfaces (O-RAN interface 326) or proprietary interfaces. It is also possible for the digital twin module 202 to be deployed within the cloud RIC platform 240 or the application layer. Note that other modules and methods, such as those described with respect to FIG. 7, are incorporated herein with respect to FIG. 8.

[0074] 9 , the digital twin module 202 system can include any one or more of the following units or modules. In some exemplary embodiments, there can be two different types of units: 1) “offline” units or modules that are implemented on a slower timescale relative to the real-time execution of the system, and 2) “runtime” units or modules that are implemented on the same or faster timescale relative to the real-time execution of the system. In one exemplary embodiment, the offline units or modules can include, but are not limited to, an RF grid generator module 410 and a user device / equipment (UE) mobility pattern generator module 402. Specifically, the RF grid generator module 410 can include a free-space path loss model module 412, a statistical fading model module 414, a ray tracing model module 416, and an AI / ML RF model module 418. The models can be flexibly selected based on the modeling accuracy requirements for training a particular type of AI / ML model for a particular use case. In another example embodiment, the runtime units or modules may include, but are not limited to, a mobility / RF model runtime module 402, a RAN model module 404, a core / traffic model module 406, and an O-cloud model module 408. However, within the scope of the disclosure described herein, it is contemplated that any unit that may belong to the digital twin module 202 system may be either an offline unit or a runtime unit.

[0075] As shown in FIG. 9 , the RF grid propagation generator module 410 and the user equipment (UE) mobility pattern behavior module 400 can be simulated by a mobility / RF model module that leverages advanced ray tracing or AI / ML from a ray tracing model module 416-based RF modeling technology from an RF model module 418 to realistically replicate the RF environment. Network protocol stack functions (i.e., physical layer, L2 and L3) are simulated in a RAN model that leverages existing RAN function implementation and simulation techniques. Traffic patterns and core behavior of upper layer applications are simulated by a traffic model or module 406. The cloud infrastructure of the O-RAN network is simulated by an O-cloud model or module 408. Here, the performance of the digital twin module 202 (i.e., the simulated O-RAN network) can be calibrated with actual data captured from open network interfaces, O1, O2, and A1, or any proprietary interface to ensure that the digital twin module 202 behaves as closely as possible to the training and testing of reliable RIC AI / ML models of actual network deployments.

[0076] With further reference to FIG. 9 , the runtime unit can directly interact with the RIC AI / ML unit during training and testing at relatively high speeds for large-scale interactive training and dataset generation. The behavior of the digital twin module model can be the same as the actual network for realistic reproduction and therefore may encounter the same computational complexity (e.g., implementation of the full L1, L2, and L3 stacks). However, to make the digital twin module lightweight, the clock speed for driving the model can be much slower than the network real-time requirements for training and testing purposes by relaxing its constraints on real-time constraints (e.g., on timescales). It is contemplated within the scope of the disclosure described herein that soft real-time, or faster than real-time, for large training dataset generation is also possible with a smaller amount of CPU and memory resources if hard real-time timing constraints are removed.

[0077] With further reference to FIG. 9 , it is contemplated that, within the scope of the present disclosure described herein, some behaviors of the actual physical network can be simplified and abstracted to further reduce the computational cost of the digital twin module 202 model. For example, RF environment techniques such as ray tracing (via module 416) or AI / ML-based RF models (via module 418) may involve high computational complexity. Such modeling calculations can be moved offline. As shown in FIG. 9 , the offline RF grid generator module 410 derives RF signal power and interference strength in the real-world environment based on ray tracing and AI / ML techniques, using limited CPU resources and at a much slower rate than their runtime counterparts. This offline process is acceptable because the RF large-scale environment does not change as frequently as the rest of the network that interacts with the RIC AI / ML model. The large RF environment typically changes when antennas and beam configurations are changed (e.g., antenna downtilt, azimuth angle, gain, beam pattern, etc.), which occurs at a relatively slow rate. Additionally, the UE mobility pattern calculations (via module 400) can also be performed at a slower speed, allowing for offline generation of UE movement trajectories for the runtime layer to read and play back. Furthermore, the RF grid and UE mobility pattern calculations can be performed using GPU or FPGA acceleration, which, with significant parallelism in the training and testing environment, can significantly improve speed without hard latency or real-time requirements. For some AI / ML training and testing scenarios where details of the RF environment are not required, the RF model can also be simplified using the simplest form of statistical-based RF modeling techniques (via module 414) or free-space path loss models (via module 412) to minimize complexity.

[0078] It should be understood that the specific order or hierarchy of blocks in the processes / flowcharts disclosed herein is an example of an exemplary approach. Based on design preferences, it should be understood that the specific order or hierarchy of blocks in the processes / flowcharts may be rearranged. Additionally, some blocks may be combined or omitted. The accompanying method claims present elements of the various blocks in an exemplary order and are not meant to be limited to the specific order or hierarchy presented.

[0079] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible level of technical detail. Furthermore, one or more of the components described above may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium (or media) having computer-readable program instructions for causing a processor to perform operations.

[0080] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction-execution device. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or ridge-in-groove structures with instructions recorded thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, should not be construed as a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through wires.

[0081] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0082] The computer-readable program code / instructions for carrying out operations may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform aspects or operations.

[0083] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the processor of the computer or other programmable data processing apparatus, generate means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that a computer-readable storage medium having instructions stored therein comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0084] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to generate a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0085] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, microservice(s), segment, or portion of instructions that includes one or more executable instructions for implementing the specified logical function(s). The methods, computer systems, and computer-readable media may include additional, fewer, different, or differently arranged blocks than those shown in the figures. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may actually be executed concurrently or substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs the specified functions or actions or executes a combination of special-purpose hardware and computer instructions.

[0086] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, and it will be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

Claims

1. 1. A method for creating a lightweight and realistic digital replica of a network for machine learning and training, comprising: generating a digital twin of a network, the digital twin being calibrated based on receiving performance metrics data from the network; training a machine learning model based on data generated from the digital twin; and operating the trained machine learning model within the network; A method comprising:

2. The method of claim 1 , wherein the network is based on an Open Radio Access Network (O-RAN).

3. operating the digital twin within a non-real-time (non-RT) Radio Access Network Intelligent Controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing AI or ML models; The method of claim 2 further comprising:

4. operating the digital twin within a near real-time (near-RT) Radio Access Network Intelligent Controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing AI or ML models; The method of claim 2 further comprising:

5. operating the digital twin in parallel with the O-RAN, wherein the digital twin further operates outside of a non-real-time (non-RT) Radio Access Network Intelligent Controller (RIC) framework and a near-real-time (near-RT) Radio Access Network Intelligent Controller (RIC) framework; The method of claim 2 further comprising:

6. generating a digital twin of the network, Analyzing the performance of the machine learning model; and generating one or more network scenarios via a radio access network (RAN) scenario generator based on the performance of the machine learning model; The method of claim 1 further comprising:

7. modeling the network based on the received network data from the network and the generated one or more network scenarios from the RAN scenario generator; The method of claim 6 further comprising:

8. monitoring the performance of the modeled network; providing feedback to the RAN scenario generator based on the monitored performance of the modeled network; and optimizing the modeled network based on the provided feedback to the RAN scenario generator; The method of claim 7 further comprising:

9. The method of claim 8 , wherein the digital twin comprises an offline simulation module and a runtime simulation module.

10. generating user equipment (UE) mobility patterns in the offline simulation module; simulating radio frequency (RF) propagation in the offline simulation module to generate an RF map at least partially representing power and interference at each location within a geographic region; and loading the generated UE mobility patterns and the RF maps to generate training or test data for an artificial intelligence (AI) or machine learning (ML) model being trained or tested at runtime; 10. The method of claim 9, further comprising:

11. 1. An apparatus for creating a lightweight and realistic digital replica of a network for machine learning and training, comprising: memory storage for storing computer-executable instructions; and a processor communicatively coupled to the memory storage; The processor executes the computer-executable instructions to cause the device to: generating a digital twin of a network, the digital twin being calibrated based on receiving performance metrics data from the network; training a machine learning model based on data generated from the digital twin; and operating the trained machine learning model within the network; 2. An apparatus configured to cause a

12. The apparatus of claim 11 , wherein the network is based on an Open Radio Access Network (O-RAN).

13. The computer-executable instructions, when executed by the processor, further cause the device to:

13. The apparatus of claim 12, wherein the digital twin operates within a non-real-time (non-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing AI or ML models.

14. The computer-executable instructions, when executed by the processor, further cause the device to:

13. The apparatus of claim 12, wherein the digital twin operates within a near real-time (near-RT) radio access network intelligent controller (RIC) framework as part of an artificial intelligence (AI) or machine learning (ML) workflow for training and testing AI or ML models.

15. The computer-executable instructions, when executed by the processor, further cause the device to: The apparatus of claim 12, wherein the digital twin operates in parallel with the O-RAN, and the digital twin further operates outside of a non-real-time (non-RT) radio access network intelligent controller (RIC) framework and a near-real-time (near-RT) radio access network intelligent controller (RIC) framework.

16. In the step of generating a digital twin of the network, the computer-executable instructions, when executed by the processor, further cause the device to: Analyzing the performance of the machine learning model; and generating one or more network scenarios via a radio access network (RAN) scenario generator based on the performance of the machine learning model; The apparatus of claim 11 , wherein the apparatus causes the following to be executed:

17. The computer-executable instructions, when executed by the processor, further cause the device to:

17. The apparatus of claim 16, further comprising: an apparatus configured to model the network based on the received network data from the network and the generated one or more network scenarios from the RAN scenario generator.

18. The computer-executable instructions, when executed by the processor, further cause the device to: monitoring the performance of the modeled network; providing feedback to the RAN scenario generator based on the monitored performance of the modeled network; and optimizing the modeled network based on the provided feedback to the RAN scenario generator; The apparatus of claim 17,

19. 20. The apparatus of claim 18, wherein the digital twin comprises an offline simulation module and a runtime simulation module.

20. 1. A non-transitory computer-readable medium comprising computer-executable instructions for creating a lightweight, realistic digital replica of a network for machine learning and training by an apparatus, the computer-executable instructions, when executed by at least one processor of the apparatus, causing the apparatus to: generating a digital twin of a network, the digital twin being calibrated based on receiving performance metrics data from the network; training a machine learning model based on data generated from the digital twin; and operating the trained machine learning model within the network; A non-transitory computer-readable medium for causing the execution of

Images