System and method for radio access network management using real-time video and RIS reconfigurable intelligent surface sensing data

Abstract

The present disclosure relates to telecommunications, radio sensing and computer vision, and more specifically to a system and method for integrating real-time radio and video sensing information to enable advanced Radio Access Network (RAN) management of an Open Radio Access Network (O-RAN), including dynamic channel prediction, adaptive beam management, and proactive handover techniques in high-frequency wireless communications.

Description

D E S C R I P T I O NSYSTEM AND METHOD FOR RADIO ACCESS NETWORK MANAGEMENT USING REAL-TIME VIDEO AND RIS RECONFIGURABLE INTELLIGENT SURFACE SENSING DATATECHNICAL FIELD

[0001] The present disclosure relates to telecommunications, radio sensing and computer vision, and more specifically to a system and method for integrating real-time radio and video sensing information to enable advanced Radio Access Network (RAN) management of an Open Radio Access Network (O-RAN), including dynamic channel prediction, adaptive beam management, and proactive handover techniques in high- frequency wireless communications. The disclosure includes a multi-agent communication structure to enable Integrated Sensing and Communications (ISAC) and improving Quality of Service (QoS) in next-generation wireless networks.BACKGROUND

[0002] Integrated Sensing and Communications (ISAC) is emerging as a key 6G trend, driven by the shift towards higher frequency bands and larger antenna arrays, including Large / Reconfigurable Intelligent Surfaces (LIS / RIS). This trend is bringing communications and sensing systems closer. This convergence is expected to enhance wireless networks with environmental sensing capabilities and increase resource usage efficiency [1], The integration of RIS into wireless environments is expected to improve the performance of both communications and sensing, particularly under non-line-of- sight conditions [2], ISAC is also creating an opportunity for integrating wireless communications and Computer Vision (CV) in a promising multimodal approach, unlocking the potential for high-resolution sensing. Applications range from device localisation to device-free environment imaging / mapping, including human sensing, with immense market potential in different verticals. CV enables accurate user / object tracking and environmental awareness, capabilities that complement radiocommunications-based sensing well. While the latter can be more computationally intensive, it deals better with obstructions or limited lighting conditions. This intersection of domains creates an opportunity for new Al-based heterogeneous and multimodal data fusion solutions, pivotal in enabling 6G communications. Addressing this interdisciplinary challenge requires advanced Research Infrastructures (Rl) and suitable tools. CONVERGE is a vision-radio Rl that bridges this gap by leveraging ISAC to facilitate a dual "View-to-Communicate, Communicate-to-View" approach [3],

[0003] A limited number of works address the integration of CV into the O-RAN architecture. In [4], a machine learning (ML) solution leverages visual data from video cameras at base stations (BS); by employing YOLOv3

[0011] , this solution uses bimodal data to proactively predict blockages and enable seamless handovers. In [5], CV addresses challenges in mmWave wireless systems; this approach predicts mmWave beams and blockages directly from Red, Green, and Blue (RGB) images and sub-6 GHz channels, removing the need for explicit channel knowledge. [5] conducts an evaluation using visual data to predict mmWave dynamic link blockages using two synthetic datasets generated with the ViWi framework

[0012] ; this prediction enables wireless networks to proactively manage beam switching and handovers. [6] introduces OpenRAN Gym, a framework for data-driven experimentation within the OpenRAN paradigm, enhancing closed-loop control; it supports the development, training, and testing of xApps, integrating service models with RAN nodes. [7] discusses the development of a gNB mounted on a mobile robotic platform; this solution offers wireless connectivity for User Equipments, UEs, and includes a novel an OnDemand Mobility Management Function (ODMMF) that monitors radio conditions and allows for real-time human control using video cameras on-board the mobile robotic platform. [8] proposes a mobile O-RAN with a gNB deployed on a mobile robotic platform capable of autonomous positioning; it also proposes a novel Mobility Management xApp that uses Signal-to-Noise Ratio (SNR) samples to position the mobile RAN, thus enhancing User Equipment, UE, link quality. [9],

[0010] use CV information to improve beam management techniques. FlexRIC

[0013] ,

[0014] is a flexible, modular, and programmable RAN Intelligent Controller (RIC) platform, used in O-RAN-based 5G and beyond. It is designed to enablenear real-time dynamic control and optimization of RAN functions by implementing custom control applications, known as xApps, that can be tailored to specific use cases. The usage of FlexRIC has been demonstrated in

[0015] , where interoperability tests between FlexRIC and an O-RAN Distributed Unit (O-DU) for slice management and radio resource management (RRM) were performed. FlexRIC was also used to successfully monitor and control the RAN in

[0016] ,

[0004] CONVERGE presents a vision-radio research infrastructure (Rl) by leveraging ISAC to facilitate a dual "View-to-Communicate, Communicate-to-View" approach [3], CONVERGE develops tools that are integrated in CONVERGE chambers, enabling the collection of experimental data from radio communications, radio sensing, and vision sensing. CONVERGE is aligned with the European Strategy Forum on Research Infrastructures (ESFRI) SLICES-RI

[0017] and will serve as an Rl that will provide the scientific community with open datasets of both experimental and simulated data, enabling research in 6G and beyond addressing various verticals, including telecommunications, automotive, manufacturing, media, and health. CONVERGE is based on OpenAirlnterface (OAI)

[0018] and O-RAN, using FlexRIC for its near-RT RIC.

[0005] These facts are disclosed in order to illustrate the technical problem addressed by the present disclosure.GENERAL DESCRI PTION

[0006] The present disclosure relates to telecommunications, radio sensing and computer vision, and more specifically to a system and method for integrating real-time radio and video sensing information to enable advanced Radio Access Network (RAN) management of an O-RAN, including dynamic channel prediction, adaptive beam management, and proactive handover techniques in high-frequency wireless communications. The disclosure includes a multi-agent communication structure to enable Integrated Sensing and Communications (ISAC), and improving Quality of Service (QoS) in next-generation wireless networks.

[0007] The proposed disclosure bridges a gap between telecommunications and computer vision / radio sensing by integrating radio and video sensing data to enhance real-time decision-making in radio access networks, for example 5G / 6G networks. Its low-latency, multi-agent architecture and integration with xApps enable significant QoS improvements, addressing key challenges in next-generation wireless systems. The xApps, running in the RAN Intelligent Controller (RIC) to control in near real-time the Radio Access Network (RAN), may play a relevant role as ISAC-based controllers. The disclosure also includes closing the sensing-control loop and expanding the system for outdoor deployments. The solution lies in the effective integration of Computer Vision (CV) sensing data with radio sensing in a multi-agent architecture to optimize real-time decisions in Radio Access Networks (RAN) like 5G / 6G. This integration enables xApps to proactively respond to dynamic changes in line-of-sight (LoS) conditions caused by obstacles, which was previously unachievable with traditional radio-only sensing methods.

[0008] The present disclosure in particular relates to a novel architecture capable of delivering real-time radio and video sensing information to xApps through a multi-agent approach. Since xApps target enabling the near real-time control of the RAN, for example by considering applications involving beam management from base stations or RISs towards mobile devices, it is therefore essential to experimentally evaluate the timescales involved in delivering sensing information from the agents to the xApp. It was performed an experimental evaluation under different dynamic constraints, such as the number of agents, message size, and message throughput, in order to assess the potential impact of the proposed architecture for real-time sensing applications. It was also considered a radio signal blockage use case to demonstrate the combination of radio and video sensing on an xApp. The main advantages are twofold: 1) a multiagent system capable of conveying radio and video sensing information to xApps in near realtime; 2) a video unit function capable of generating relevant video sensing / blockage messages in real time to be sent by an agent to the xApps.

[0009] An O-RAN, or Open RAN, can be defined as a disaggregated RAN functionality built using open interface specifications between elements. It can be implemented invendor-neutral hardware and software-defined technology based on open interfaces and community-developed standards. For the present disclosure, an O-RAN is exemplified as defined by the O-RAN Alliance (see further below).

[0010] A Reconfigurable intelligent surface (RIS) - also as known as an intelligent reflecting surface (IRS), and as a large intelligent surface (LIS) - can be defined as a programmable arrangement that can be used to control the propagation of electromagnetic (EM) waves by changing the electric and magnetic properties of a surface.

[0011] A RAN Intelligent Controller (RIC) enables intelligent radio resources management and optimization through software applications running on top of RIC, which are called xApps / rApps (see further below).

[0012] While a non-RT RIC manages events and resources with a response time of one second or more, a near-RT RIC manages events and resources requiring a faster response down to milliseconds (ms). Also, the non-RT RIC is deployed centrally, while the near-RT RIC can be deployed centrally or on the network edge (see further below).

[0013] An aspect of the present disclosure relates to a system for Radio Access Network, RAN, management, by means of real-time sensing data from at least one video source, for Integrated Sensing and Communications, ISAC, of an O-RAN, the system comprising: at least one xApp for sending RIC control requests from video sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent and a Video Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC, arranged to accept RIC subscription requests from the at least one xApp, and arranged to accept RIC control request from the at least one xApp; wherein the RAN Function E2 Agent is arranged to accept RIC subscription requests from the Near-RT RIC, and is arranged to accept RIC control requests from the Near- RT RIC;wherein the Near-RT RIC is further arranged to accept a Video sensing subscription request from the at least one xApp and, in response, send a corresponding Video sensing subscription request to the Video Function E2 Agent; wherein the Video Function E2 Agent is arranged to accept a Video sensing subscription request from the Near-RT RIC and, in response, send Video sensing data to the Near-RT RIC based on the at least one video source; wherein the Near-RT RIC is further arranged to accept Video sensing data from the Video Function E2 Agent and, in response, send the Video sensing data to the at least one xApp.

[0014] Another aspect of the present disclosure relates to a system for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, for Integrated Sensing and Communications, ISAC, of an O-RAN, the system comprising: at least one xApp for sending RIC control requests from RIS sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent and a RIS Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC, arranged to accept RIC subscription requests from the at least one xApp, and arranged to accept RIC control request from the at least one xApp; wherein the RAN Function E2 Agent is arranged to accept RIC subscription requests from the Near-RT RIC, and is arranged to accept RIC control requests from the Near- RT RIC; wherein the Near-RT RIC is further arranged to accept a RIS Sensing subscription request from the at least one xApp and, in response, send a corresponding RIS Sensing subscription request to the RIS Function E2 Agent; wherein the RIS Function E2 Agent is arranged to accept a RIS Sensing subscription request from the Near-RT RIC and, in response, send RIS sensing data to the Near- RT RIC based on the at least one RIS;wherein the Near-RT RIC is further arranged to accept RIS sensing data from the RIS Function E2 Agent and, in response, send the RIS sensing data to the at least one xApp.

[0015] Another aspect of the present disclosure relates to system for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, and from at least one video source, for Integrated Sensing and Communications, ISAC, of an O-RAN, the system comprising: at least one xApp for sending RIC control requests from video and / or RIS sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent; a Video Function E2 Agent and / or a RIS Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC, arranged to accept RIC subscription requests from the at least one xApp, and arranged to accept RIC control request from the at least one xApp; wherein the RAN Function E2 Agent is arranged to accept RIC subscription requests from the Near-RT RIC, and is arranged to accept RIC control requests from the Near- RT RIC; wherein the Near-RT RIC is further arranged to accept a Video sensing subscription request from the at least one xApp and, in response, send a corresponding Video sensing subscription request to the Video Function E2 Agent; wherein the Video Function E2 Agent is arranged to accept a Video sensing subscription request from the Near-RT RIC and, in response, send Video sensing data to the Near-RT RIC based on the at least one video source; wherein the Near-RT RIC is further arranged to accept Video sensing data from the Video Function E2 Agent and, in response, send the Video sensing data to the at least one xApp; wherein the Near-RT RIC is further arranged to accept a RIS Sensing subscription request from the at least one xApp and, in response, send a corresponding RIS Sensing subscription request to the RIS Function E2 Agent;wherein the RIS Function E2 Agent is arranged to accept a RIS Sensing subscription request from the Near-RT RIC and, in response, send RIS sensing data to the Near- RT RIC based on the at least one RIS; wherein the Near-RT RIC is further arranged to accept RIS sensing data from the RIS Function E2 Agent and, in response, send the RIS sensing data to the at least one xApp.

[0016] In an embodiment for better results, the at least one xApp comprises a first xApp for RIC Control and a second xApp for real-time sensing of video and / or RIS data, in particular wherein the at least one xApp comprises a first xApp for RIC control, a second xApp for real-time sensing of video data, and a third xApp for real-time sensing of RIS data.

[0017] In an embodiment for better results, the video sensing data is a video data stream.

[0018] In an embodiment for better results, the E2 Agents are arranged to communicate with the Near-RT RIC using a E2 interface.

[0019] In an embodiment for better results, the E2 interface is as defined by O-RAN.

[0020] In an embodiment for better results, the Near-RT RIC is arranged to receive E2 setup requests from the RAN Function E2 Agent, and is arranged to receive E42 setup requests and Video and / or RIS Setup requests from the at least one xApp.

[0021] In an embodiment for better results, the system is arranged such that: communications between the Near-RT RIC and the Video Function E2 Agent do not go through a gNB of the RAN, and / or communications between the Near-RT RIC and the RIS Function E2 Agent do not go through a gNB of the RAN; wherein the gNB is arranged for transmitting and receiving data between a RAN device and a core network of the RAN, in particular the gNB comprising a Radio Unit (O-RU), Distributed Unit (O-DU), and Centralized Unit (O-CU) of the O-RAN.

[0022] In an embodiment for better results, wherein the Video Function E2 Agent and / or the RIS Function E2 Agent is arranged for tracking, detecting, and / or predicting obstacle blockage of line-of-sight, LoS, between a gNB of the RAN and User Equipment, in particular the Video Function E2 Agent and / or the RIS Function E2 Agent is arranged for predicting object trajectory for tracking, detecting, and / or predicting obstacle blockage, or the at least one xApp is arranged for tracking, detecting, and / or predicting obstacle blockage of line-of-sight, LoS, between a gNB of the RAN and User Equipment, in particular the at least one xApp is arranged for predicting object trajectory for tracking, detecting, and / or predicting obstacle blockage, further in particular from a received video data stream from the Video Function E2 Agent.

[0023] In an embodiment for better results, wherein the tracked, detected, and / or predicted obstacle blockage of line-of-sight, LoS, between a gNB of the RAN and User Equipment is used for maintaining or improving connectivity between the gNB and the User Equipment.

[0024] In an embodiment for better results, wherein the Video Function E2 Agent and / or the RIS Function E2 Agent is arranged for sending one or more of the following: a E2 message when a blockage is predicted to obstruct the line-of-sight, LoS, between the gNB and the UE; a E2 message when a blockage is detected obstructing the line-of-sight, LoS, between the gNB and the UE; a E2 message when a blockage is no longer detected obstructing the line-of-sight, LoS, between the gNB and the UE; and timestamped E2 messages.

[0025] Another aspect of the present disclosure relates to a method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one video source, for Integrated Sensing and Communications, ISAC, of an O-RAN, using a system comprising: at least one xApp for sending RIC control requests from video sensing data;a plurality of E2 agents comprising a RAN Function E2 Agent and a Video Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC; the method comprising: the Near-RT RIC accepting RIC subscription requests from the at least one xApp, and accepting RIC control request from the at least one xApp; the RAN Function E2 Agent accepting RIC subscription requests from the Near-RT RIC, and accepting RIC control requests from the Near-RT RIC; wherein the method also comprises: the Near-RT RIC accepting a Video sensing subscription request from the at least one xApp and, in response, sending a corresponding Video sensing subscription request to the Video Function E2 Agent; the Video Function E2 Agent accepting a Video sensing subscription request from the Near-RT RIC and, in response, sending Video sensing data to the Near-RT RIC based on the at least one video source; the Near-RT RIC is further accepting Video sensing data from the Video Function E2 Agent and, in response, sending the Video sensing data to the at least one xApp.

[0026] Another aspect of the present disclosure relates to a method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, for Integrated Sensing and Communications, ISAC, of an O-RAN, using a system comprising: at least one xApp for sending RIC control requests from RIS sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent and a RIS Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC; the method comprising: the Near-RT RIC accepting RIC subscription requests from the at least one xApp, and accepting RIC control request from the at least one xApp; the RAN Function E2 Agent accepting RIC subscription requests from the Near-RT RIC, and accepting RIC control requests from the Near-RT RIC;wherein the method also comprises: the Near-RT RIC accepting a RIS Sensing subscription request from the at least one xApp and, in response, sending a corresponding RIS Sensing subscription request to the RIS Function E2 Agent; the RIS Function E2 Agent accepting a RIS Sensing subscription request from the Near-RT RIC and, in response, sending RIS sensing data to the Near-RT RIC based on the at least one RIS; the Near-RT RIC accepting RIS sensing data from the RIS Function E2 Agent and, in response, sending the RIS sensing data to the at least one xApp.

[0027] Another aspect of the present disclosure relates to a method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, and from at least one video source, for Integrated Sensing and Communications, ISAC, of an O-RAN, using a system comprising: at least one xApp for sending RIC control requests from video and / or RIS sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent; a Video Function E2 Agent and / or a RIS Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC; the method comprising: the Near-RT RIC accepting RIC subscription requests from the at least one xApp, and accepting RIC control request from the at least one xApp; the RAN Function E2 Agent accepting RIC subscription requests from the Near-RT RIC, and accepting RIC control requests from the Near-RT RIC; wherein the method also comprises: the Near-RT RIC accepting a Video sensing subscription request from the at least one xApp and, in response, sending a corresponding Video sensing subscription request to the Video Function E2 Agent; the Video Function E2 Agent accepting a Video sensing subscription request from the Near-RT RIC and, in response, sending Video sensing data to the Near-RT RIC based on the at least one video source;the Near-RT RIC is further accepting Video sensing data from the Video Function E2 Agent and, in response, sending the Video sensing data to the at least one xApp; the Near-RT RIC accepting a RIS Sensing subscription request from the at least one xApp and, in response, sending a corresponding RIS Sensing subscription request to the RIS Function E2 Agent; the RIS Function E2 Agent accepting a RIS Sensing subscription request from the Near-RT RIC and, in response, sending RIS sensing data to the Near-RT RIC based on the at least one RIS; the Near-RT RIC accepting RIS sensing data from the RIS Function E2 Agent and, in response, sending the RIS sensing data to the at least one xApp.

[0028] Another aspect of the present disclosure relates to Non-transitory computer- readable medium comprising computer program instructions for implementing a method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, and / or from at least one video source, for Integrated Sensing and Communications, ISAC, of an O-RAN, which when executed by a processor, cause the processor to carry out any of the methods of the present disclosure.

[0029] Further particular and preferred aspects are set out in the accompanying independent and dependent claims. Features of the dependent claims may be combined with features of the independent claims as appropriate, and in combinations other than those explicitly set out in the claims.

[0030] Where an apparatus feature is described as being operable to provide a function, it will be appreciated that this includes an apparatus feature which provides that function, or which is adapted or configured to provide that function.

[0031] Where elements are described as being connected or connectable, they may be directly connected. Where elements are described as being coupled or coupleable, they may be linked by one or more intervening or interposing elements.BRI EF DESCRIPTION OF TH E DRAWI NGS

[0032] The following figures provide preferred embodiments for illustrating the disclosure and should not be seen as limiting the scope of invention.

[0033] Figure 1: Sequence diagram of an xApp running on FlexRIC.

[0034] Figure 2: Sequence diagram of the disclosed vision-aided (video sensing) xApp running on FlexRIC.

[0035] Figure 3: Sequence diagram of the disclosed vision-aided (video streaming) xApp running on FlexRIC.

[0036] Figure 4: Sequence diagram of two vision-aided (video stream) xApps running on FlexRIC as disclosed.

[0037] Figure 5: Sequence diagram of the disclosed RIS-aided xApp running on FlexRIC.

[0038] Figure 6: Sequence diagram of the disclosed gNB-aided xApp running on FlexRIC.

[0039] Figure 7: Service-oriented architecture.

[0040] Figure 8: Service-oriented architecture with the disclosed multi-agent structure.

[0041] Figure 9: Detail of the disclosed multi-agent architecture.

[0042] Figure 10 A-C: Shows the performance of the proposed architecture under different scenarios.

[0043] Figure 11: Shows the SNR of the wireless link between the gNB and UE, along with synchronized Video Function messages.DETAILED DESCRI PTION

[0044] The present disclosure relates to telecommunications, radio sensing and computer vision, and more specifically to a system and method for integrating real-time radio and video sensing information to enable advanced Radio Access Network (RAN) management of an O-RAN, including dynamic channel prediction, adaptive beam management, and proactive handover techniques in high-frequency wireless communications. The disclosure includes a multi-agent communication structure toenable Integrated Sensing and Communications (ISAC) and improving Quality of Service (QoS) in next-generation wireless networks.

[0045] A preferred embodiment of the multi-agent approach of the present disclosure is based on the CONVERGE vision-radio Rl architecture. Fig. 8 introduces the key building blocks of the system and Fig. 9 decomposes these blocks, presents their control as virtual network functions, and identifies the interfaces.

[0046] A high-level architecture of a preferred embodiment comprises three main components: a Chamber, which facilitates physical experiments with base station / gNB, UE, and LIS equipment across various infrastructures; a Simulator, which supports scenario planning and testing via a Digital Twin; and a Core, which manages experiments, data, and ML models. This integrated setup enables both physical and simulated studies, allowing external users to perform real-time physical or offline virtual experiments utilizing the Chamber, Simulator, ML tools, and datasets for advanced research.

[0047] In an embodiment, the Chamber is equipped with a gNB, a UE, and a LIS, each featuring controls for placement (PCg, PCue, PCIis), radio communications (RCg, RCue, RCIis), radio sensing (RSg, RSue, RSlis), and video sensing (VSg, VSue, VSlis). The chamber also has a video sensing capability (VSc). The Core oversees operations, enabling user interaction for experiment setup, monitoring, and control. It includes the Application Function (CAF) for user interface, the Broker Function (CBF) for session orchestration, the Open Data Repository Function (CODRF) for data storage, and the Machine Learning Function (CMLF) for accessing ML tools. The Simulator, with its 3D Environment Modeller (3D-S), Vision Radio Simulator (VR-S), and Network Simulator (NET-S), creates a Digital Twin of the physical environment.

[0048] In an embodiment, the architecture adopts a service-oriented design aligned with the 5G Core network (Fig. 7 and 8), based on 3GPP standards

[0019] , leveraging 5G Core functionalities such as the Access and Mobility Management Function (AMF), the Session Management Function (SMF), and the User Plane Function (UPF), necessary for UE and gNB operations, while using the 5G New Radio (NR) specifications and facilitatingthe integration of new functions for operation. This adds functions to the control and sensing plane to perform a variety of tasks. The Video Function (CVF) receives and processes the video feeds from the video cameras in the Chamber. The control of the Chamber equipment and Simulator tools is performed using different functions (CUECF, CLISCF, CgNBCF, CTCF, CODRF, C3DSCF, CVRSCF, and CNETSCF), based on REpresentational State Transfer (REST) web services. More details on these functions can be found in

[0020] ,

[0049] In an embodiment, the proposed vision-aided gNB is based on the O-RAN architecture, as shown in Fig. 8. The gNB integrates the key components of O-RAN: the Radio Unit (O-RU), Distributed Unit (O-DU), and Centralized Unit (O-CU), following the 3GPP Fl interface and functional split 7.2

[0019] , Vision-sensing capabilities are added through a video unit (VU). The gNB interfaces with the Core through a web service-based interface for remote access and control (CgNB). The gNB also includes the Near Real- Time RIC forthe development and validation of xApps, which is used to improve network management by using both radio sensing and vision sensing data.

[0050] In an embodiment, the RIC is supported by a multi-agent architecture shown in Fig. 9. The RIC receives radio sensing data from the E2 agents on O-DU and O-CU through the ORAN E2 interface. It was extended this functionality through a new set of E2 agents (hereby also referred to as E2' agents) that will enable the RIC to gather video and sensing data from CVF and LIS, respectively. This novel E2 interface (hereby also referred as E2' interface) has a similar operation as the O-RAN E2 interface but optionally enables the connection to agents outside the RAN.

[0051] In an embodiment, the implementation of the proposed architecture which includes a testbed for delay measurements of sensing data and a novel vision-aided xApp integrated in a standalone 5G network. It was considered OpenAirlnterface (OAI)

[0018] and Mosaic5G's FlexRIC

[0013] for the Near-RT RIC due to its modular design.

[0052] In an embodiment, in order to evaluate the delay and the capability of receiving and processing data from multiple agents, a system was built composed of three virtual machines: one implementing the FlexRIC and two other machines implementing two E2'agents. The machines were based on Intel i7-8700 CPU, with 1 vCPU and 2 GB of RAM, running Ubuntu 24.04 and FlexRIC vl.0.0 and synchronized through Precision Time Protocol (PTP). A custom xApp was developed to accurately record delays. The E2 agents were enhanced by including a timestamp in every message, enabling precise latency comparisons on the xApp. Additionally, to control message sizes, an array of 8- bit unsigned integers was integrated into the existing message structure. Since the original setup supported a minimum interval of 1 ms between exchanged messages, adjustments were made to the timing mechanism to allow for shorter intervals. The CVF and FlexRIC exchange data via the new E2' interface, which is tailored to carry information, such as obstacle detection, through a novel set of messages generated by CVF, as detailed below. Abstract Syntax Notation One (ASN.l) for encoding and decoding these messages was used. ASNITools

[0021] was chosen for the CVF because it offers a simple API for handling ASN.l data structures. The connection between FlexRIC and xApp is performed through the E42 interface, which allows for: 1) Setup Request, 2) Setup Response, 3) Subscription Request, 4) Subscription Delete Request, and 5) E42 RIC Control Request messages.

[0053] In an embodiment, the CVF, OpenCV

[0022] is used to capture video frames and detect ArUco markers

[0023] , ArUco markers help identify UEs without the need to train a specific YOLO model to recognize such objects. Ultralytics YOLO

[0024] is employed for real-time object detection, known for its speed and accuracy. YOLO maintains continuous track of object identification and predicts future positions of obstacles across frames. OpenCV and Ultralytics YOLO together provide detection, tracking, and message exchange functionalities. This integrated approach allows the CVF to monitor and report obstacles. The YOLOv8n version was chosen for its reduced parameter count, making it less resource-intensive and well-suited to the solution of the present disclosure. This accelerates processing times and reduces CPU load, enhancing system efficiency and responsiveness. A pre-trained model was sufficient for the solution of the present disclosure, reducing the CPU requirements. A full HD camera, acting as the VU, was used at 30 frames per second (fps).

[0054] In an embodiment, the CVF processes video frames to detect and track obstacles, sending this information to the xApp through FlexRIC. This system interfaces with O-RAN xApps connected to it. The CVF generates three types of messages: Prior Blockage, Blockage, and Post Blockage messages.

[0055] In an embodiment, Prior Blockage messages are generated when an obstacle is predicted to obstruct the line-of-sight (LoS) between the gNB and the UE, based on the obstacle's current trajectory. To predict a blockage, CVF gathers the tracking history of the obstacle across several frames using YOLO and BoTSORT

[0025] algorithms. By establishing a tracking history and assuming a constant velocity for the obstacle's movement, the CVF calculates the obstacle's velocity to predict its future positions in upcoming video frames. This enables the system to assess potential interruptions of the LoS. Blockage messages are generated when a blockage of the LoS by an obstacle is detected. The system verifies whether the current position of the obstacle matches the last known position where the ArUco marker was detected and associated with the UE. If there is an overlap between the obstacle's most recent position and the UE's last recorded position, a Blockage message is generated. Post Blockage messages are generated when an obstacle no longer blocks the UE, based on the updated list of the status for each tracked obstacle. Post Blockage messages inform that the obstruction has been removed, allowing for adjustments in network management.

[0056] In an embodiment, the novel set of messages paves the way for enhancing the real-time responsiveness of the network and providing awareness regarding physical obstructions that may impact network performance.

[0057] In an embodiment, figure 1 - In this message sequence diagram, it can be observed the initialization of the xApp and the Near-RT RIC, the subscription of sensing information from the RAN and the RIC Control messages from the xApp to control the RAN.

[0058] In an embodiment, figure 2 - In this message sequence diagram, it can be observed the initialization of the xApp and the Near-RT RIC, the subscription of sensing information from the RAN by the xApp, the setup of the video sensing, and thesubscription of the video sensing by the xApp. Finally, the xApp sends control messages to the RIC in order to perform changes on the RAN.

[0059] In an embodiment, figure 3 - In this message sequence diagram, it can be observed the initialization of the xApp and the Near-RT RIC, the subscription of sensing information from the RAN by the xApp, the setup of the video sensing, and the subscription of the video stream by the xApp. Finally, the xApp sends control messages to the RIC in order to perform changes on the RAN.

[0060] In an embodiment, figure 4 - In this message sequence diagram, it can be observed the initialization of the xApp and the Near-RT RIC, the subscription of sensing information from the RAN by the xApps, the setup of the video sensing, and the subscription of the video stream by two different xApps. Finally, one of the xApps sends control messages to the RIC in order to perform changes on the RAN.

[0061] In an embodiment, figure 5 - In this message sequence diagram, it can be observed the initialization of the xApp and the Near-RT RIC, the subscription of sensing information from the RAN by the xApp, the setup of the RIS, the placement control of the RIS, and the subscription of the RIS sensing by the xApp. Finally, the xApp sends control messages to the RIC in order to perform changes on the RAN.

[0062] In an embodiment, figure 11 - In this message sequence diagram, it can be observed the initialization of the xApp and the Near-RT RIC, the subscription of sensing information from the RAN by the xApp, the setup of the gNB placement, and the subscription of the gNB sensing by the xApp. Finally, the xApp sends control messages to the RIC in order to perform changes on the RAN.

[0063] In an embodiment, figure 7 - This figure shows the service-oriented architecture, where it can be observed the UE, LIS and gNB inside a chamber and also a video function that is responsible for collecting and processing the video feeds from the cameras. The Core is composed of a set of functions that support the equipment and network components. The simulator is composed of three blocks that simulate different physical and network-level aspects.

[0064] In an embodiment, figure 8 shows the service-oriented architecture, where it can be observed the disclosed multi-modal, multi-equipment control and data acquisition to the xApps via the E2 and E2' Agents through the Near-RT RIC Framework.

[0065] In an embodiment, figure 9 - This figure shows a typical implementation of the disclosed system, including a UE, a Video Function, a LIS and a gNB. It can be observed that the multi-modal, multi-equipment control and data acquisition to the xApps via the E2 and E2' Agents through the Near-RT RIC Framework. The interfaces Cuevf, Clisvf, cgnb, Clis, Cvg and Cue are according to an embodiment of the disclosure as specified on CONVERGE project.

[0066] In an embodiment, the OAI 5G Core Network, FlexRIC, the CVF, and the gNB were deployed on an Acer Aspire A715-74G laptop with 16 GB of RAM and a GeForce GTX 1050 (3 GB) GPU. The UE was deployed on an HP EliteBook 840 laptop with a 4-core Intel CPU and 8 GB of RAM. The OAI 5G Core Network was deployed using Docker containers, requiring a 4-core CPU and 16 GB of RAM

[0018] , FlexRIC is not resource-intensive and has similar requirements to the OAI 5G Core Network.

[0067] In an embodiment, the gNB and UE were implemented using Ettus USRP B210 Software-Defined Radios (SDRs) due to their cost-effectiveness and popularity within the community. A carrier frequency of 3.6 GHz was employed for the 5G RAN and two W5084K dipole antennas were attached to each SDR.

[0068] In order to assess the adequacy of the proposed architecture for real-time sensing applications, it was first evaluated how the message delay is affected by the number of agents, message size, and message rate. Then, it was defined a use case testing scenario to assess the functionality of the entire system and implemented a vision-aided xApp, which processes the sensing information gathered from the CVF and the SNR values collected from the O-DU via the E2 interface.

[0069] In message delay evaluation, it was assessed the performance of the proposed architecture under different scenarios, measuring the message delay since its generation by an E2' agent outside the gNB until its reception by an xApp. Fig. 10A shows the latency considering message sizes from 16 to 2048 byte and 1 to 6 agents, generatinga rate (throughput) of 1000 sensing message / s per agent. It was observed that the average latency is smaller than 1 ms and that the number of supported agents depends on the message size: 6 agents are supported simultaneously when 16-byte messages are generated, while only 1 agent is supported when 2048-byte messages are generated. If at least 2 agents are required, the sensing messages should not exceed 1024 byte, considering the rate of 1000 message / s per agent. Fig. 10B shows the latency as the number of agents increases under different message throughput, considering 16-byte messages. It was observed that the average delay is under 0.5 ms and that the number of agents depends on the message throughput. For 500 message / s per agent, 9 agents are supported. Fig. 10C shows the latency with the increase of the message throughput for different number of agents, considering 16-byte messages. For the selected setup, it can be concluded that the number of supported agents decreases with the increase of the throughput of sensing messages, with a limit of 6 agents for one message generated per millisecond per agent, and two agents for a message generated per agent each 200 microsecond. The observed limitations may be caused by the FlexRIC implementation and its system of interrupts required to handle message concurrency, which may have prevented us from testing the usage of more agents under certain message size and message throughput conditions, and may have provided some outliers under heavy load. Improvements in the implementation and more computing power would probably improve the latency values. Even though, the obtained results confirm the support for a relevant number of simultaneous agents introducing low delay, appropriate for the near real-time RAN decisions envisaged for xApps.

[0070] To experimentally validate the combination of radio and video sensing in the RAN under a representative use case testing scenario, it was developed a simple vision- aided xApp. It is able to receive and process messages related to environmental conditions, detected by the CVF and made available through FlexRIC, while integrating decoded CVF messages with SNR measurements collected from the E2 interface. The xApp was deployed alongside the FlexRIC to minimize latency between the two components.

[0071] In an embodiment, fig. 11 shows the SNR of the wireless link between the gNB and UE, along with synchronized Video Function messages, received over time by the vision-aided xApp. A blockage prediction use case testing scenario was designed to evaluate the impact of blockages on the LoS between the gNB and the UE. By maintaining fixed positions for both the gNB and the UE, it was introduced an obstacle to assess its effect on signal quality. This approach also aimed to validate the accuracy of the messages sent by CVF regarding the presence of blockages. The results from this scenario serve as a baseline for evaluating the benefits of integrating CV solutions into 5G and 6G networks. The experiment demonstrated the vision-aided xApp's ability to receive messages from the CVF along with the associated SNR values for the wireless link established between the gNB and the UE. Fig. 11 shows the SNR variation over time, synchronized with each message received from the CVF. SNR values are collected by the vision-aided xApp every 10 ms, with 20 samples gathered over each 200 ms interval, aligning with the CVF video processing rate of 5 fps. With this, each SNR value corresponds to a message received from the CVF. Fig. 11 is divided into three distinct periods, separated by transition lines. In the first period, the obstacle moves towards the UE and Prior Blockage messages are received by the xApp. The SNR is high during this period because there is no LoS obstruction between the gNB and the UE. The first transition is labelled LoS lost, marking the beginning of a blockage period.

[0072] In this period, the obstacle stops in front of the UE and the first Blockage message is received by the xApp, indicating the beginning of an obstruction. As expected, the average SNR decreases during this period. The second transition is labelled Return of LoS, for which a Post Blockage message indicates the end of the blockage caused by the obstacle and the return to adequate SNR conditions. In the third period, the obstacle moves away from the UE, no longer blocking the LoS. As expected, the average SNR values are close to those associated with the Prior Blockage messages, confirming the removal of the blockage. This graph shows that radio sensing and video sensing provide accurate results by leveraging the combination of multimodal information on the xApp for faster RAN decisions, as the CV algorithm can detect the blockage before it actually occurs, anticipating the decision by 500 ms.

[0073] The radio channels envisaged for the high frequency 6G radio links will be much affected by the loss of LoS between radio transmitters and receivers caused by obstacles that can be easily traced by video cameras. In this disclosure, it was presented a multiagent architecture that integrates radio and video sensing data to enhance the capabilities of 5G RAN intelligent controllers. By adding CV sensing data to RAN, it was enabling real-time intelligent controllers running xApps to make better and faster decisions.

[0074] The experimental results obtained validate the effectiveness of the proposed architecture, which was based on O-RAN and FlexRIC, showing that the system can maintain an average sensing delay under 1 ms even with multiple agents involved.

[0075] Moreover, the ability of xApps to utilize both radio and video data for network optimization decisions highlights the potential of ISAC in enhancing the Quality of Service (QoS) in next-generation wireless networks. Future work includes closing the sensing and control loop and extending the testbed for outdoor scenarios.

[0076] The features used in the present document can be implemented using the definitions in specification 3GPP TS 23.501 V16.20.0 (2024-06), available at https: / / www.3gpp.org / specifications-technologies, and also as defined in ETSI TS 103 982 V8.0.0 (2024-01), available at https: / / www.etsi.org / standards, referencing O-RAN Alliance Technical Specifications of WG3: Near real-time RIC and E2 Interface Workgroup, available at https: / / specifications.o-ran.org / , as retrieved on 03.12.2024, namely:O-RAN Use Cases and Requirements 7.0, O-RAN. WG3.TS.UCR-R004-v07.00, October 2024, Technical Specification, R004. This document details the functional and non-functional requirements on Near-RT RIC and E2 interface from the O-RAN use cases under O-RAN WG3.O-RAN E2 General Aspects and Principles (E2GAP) 6.0, 0-RAN.WG3.E2GAP-R004- v06.00, October 2024, Technical Specification, R004. This document (E2GAP) together with (E2AP) describes the general architecture of Near-RT RIC and the main functions and procedures supported over the E2 interface, and the E2 application protocol of Near-RT RIC.O-RAN E2 Application Protocol (E2AP) 6.0, 0-RAN.WG3.E2AP-R004-v06.00, October 2024, Technical Specification, R004. This document (E2AP) together with (E2GAP) describes the general architecture of Near-RT RIC and the main functions and procedures supported over the E2 interface, and the E2 application protocol of Near-RT RIC.O-RAN E2 Service Model (E2SM) 6.0, 0-RAN.WG3.E2SM-R004-v06.00, October 2024, Technical Specification, R004. This document serves as the overall framework for the set of specialized E2SMs, each of which is dedicated to a specific RAN function. This document also includes the common information elements used across the specialized E2SMs.O-RAN E2 Service Model (E2SM) KPM 5.0, 0-RAN.WG3.E2SM-KPM-R003-v05.00, June 2024, Technical Specification, R003. This specification defines the functions and protocols of the E2 interface, which connects the O-RAN Near-RT RIC to the underlying O-RAN nodes (i.e. E2 nodes or E2 agents).O-RAN E2 Service Model (E2SM), RAN Control 6.0, 0-RAN.WG3.E2SM-RC-R003- v06.00, June 2024, Technical Specification, R003. This document specifies the capabilities exposed over E2 interface to enable efficient control of RAN, including radio beam control, radio access control, connected node mobility, etc.O-RAN E2 Service Model (E2SM) Cell Configuration and Control 4.0, O- RAN.WG3.E2SM-CCC-R003-v04.00, June 2024, Technical Specification, R003. This document aims at exposing configuration and control related processes on a celllevel basis.O-RAN Near-RT RIC Architecture 6.0, 0-RAN.WG3.RICARCH-R003-v06.00, June 2024, Technical Specification, R003. The O-RAN Near-RT RIC Architecture document specifies the Near-RT RIC internal architecture and functionalities, and the stage 2 definitions of Near-RT RIC APIs. This version includes new Near-RT RIC API procedures for E2 subscription audit, API registration, xApp data sharing and AI / ML support.O-RAN Near-RT RIC APIs specification 2.0, 0-RAN.WG3.RICAPI-R003-v02.00, June 2024, Technical Specification, R003. This technical specification defines thesignalling and data transport protocols for Near-RT RIC APIs. The protocols are developed in accordance with the functionalities and interactions stated in O- RAN.WG3.RICARCH specification.O-RAN Use Cases and Requirements 5.0, 0-RAN.WG3.UCR-R003-v05.00, February 2024, Technical Specification, R003. This document details the functional and nonfunctional requirements on Near-RT RIC and E2 interface from the O-RAN in O-RAN WG3.O-RAN 01 Interface Specification for Near Real Time RAN Intelligent Controller 1.0, O-RAN. WG3.01-lnterface-for-Near-RT-RIC-R003-v01.00, March 2023, R003. This specification defines O-RAN 0AM interface functions and protocols for the O-RAN 01 interface for the Near RT RIC. The document studies the functions conveyed over the interface, including management functions, procedures, operations, and corresponding solutions, and identifies existing standards and industry work that can serve as a basis for an O-RAN implementation including a Near-RT RIC.

[0077] This work was supported by the CONVERGE project which has received funding under the European Union's Horizon Europe research and innovation programme under Grant Agreement No 101094831, including top-up funding by UK Research and Innovation (UKRI) under the UK government's Horizon Europe funding guarantee.

[0078] The term "comprising" whenever used in this document is intended to indicate the presence of stated features, integers, steps, components, but not to preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0079] Flow diagrams of particular embodiments of the presently disclosed methods are depicted in figures. The flow diagrams illustrate the functional information one of ordinary skill in the art requires to perform said methods required in accordance with the present disclosure.

[0080] It will be appreciated by those of ordinary skill in the art that unless otherwise indicated herein, the particular sequence of steps described is illustrative only and can be varied without departing from the disclosure. Thus, unless otherwise stated the stepsdescribed are so unordered meaning that, when possible, the steps can be performed in any convenient or desirable order.

[0081] It is to be appreciated that certain embodiments of the disclosure as described herein may be incorporated as code (e.g., a software algorithm or program) residing in firmware and / or on computer useable medium having control logic for enabling execution on a computer system having a computer processor, such as any of the servers described herein. Such a computer system typically includes memory storage configured to provide output from execution of the code which configures a processor in accordance with the execution. The code can be arranged as firmware or software, and can be organized as a set of modules, including the various modules and algorithms described herein, such as discrete code modules, function calls, procedure calls or objects in an object-oriented programming environment. If implemented using modules, the code can comprise a single module or a plurality of modules that operate in cooperation with one another to configure the machine in which it is executed to perform the associated functions, as described herein.

[0082] Furthermore, it is to be understood that the invention encompasses all variations, combinations, and permutations in which one or more limitations, elements, clauses, descriptive terms, etc., from one or more of the claims orfrom relevant portions of the description is introduced into another claim. For example, any claim that is dependent on another claim can be modified to include one or more.

[0083] The disclosure should not be seen in any way restricted to the embodiments described and a person with ordinary skill in the art will foresee many possibilities to modifications thereof. The above-described embodiments are combinable. The following claims further set out particular embodiments of the disclosure.

[0084] REFERENCES[1] W. Saad, M. Bennis, and M. Chen, "A vision of 6G wireless systems: Applications, trends, technologies, and open research problems," IEEE network, vol. 34, no. 3, pp. 134-142, 2019.[2] F. Liu, Y. Cui, C. Masouros, J. Xu, T. X. Han, Y. C. Eldar, and S. Buzzi, "Integrated sensing and communications: Toward dual-functional wireless networks for 6G and beyond," IEEE journal on selected areas in communications, vol. 40, no. 6, pp. 1728-1767, 2022.[3] F. B. Teixeira, M. Ricardo, A. Coelho, H. P. Oliveira, P. Viana, N. Paulino, H. Fontes, P. Marques, R. Campos, and L. M. Pessoa, "CONVERGE: A vision-radio research infrastructure towards 6G and beyond," in 2024 EuCNC&6G Summit, 2024, pp. 1015-1020.[4] G. Charan, M. Alrabeiah, and A. Alkhateeb, "Vision-aided 6G wireless communications: Blockage prediction and proactive handoff," IEEE Tran, on Vehicular Technology, vol. 70, pp.10 193-10 208, Oct 2021.[5] M. Alrabeiah, A. Hredzak, and A. Alkhateeb, "Millimeter wave base stations with cameras: Vision-aided beam and blockage prediction," in IEEE 91st VTC2020-Spring, May 2020, pp. 1-5.[6] L. Bonati, M. Polese, S. D'Oro, S. Basagni, and T. Melodia, "Intelligent closed-loop ran control with xapps in openran gym," in European Wireless 2022; 27th European Wireless Conference, 2022, pp. 1-6.[7] D. Maia, A. Coelho, and M. Ricardo, "Obstacle-aware on-demand 5g network using a mobile robotic platform," in WiMob 2022, 2022, pp. 470-473.[8] G. Queiros, P. Correia, A. Coelho, and M. Ricardo, "Autonomous ' control and positioning of a mobile radio access node employing the o-ran architecture," in 19th Wireless On- Demand Network Systems and Services Conference (WONS), 2024, pp. 25-28.[9] S. Kim, Y. Ahn, D. Park, and B. Shim, "Vomtc: Vision objects for millimeter and terahertz communications," IEEE Transactions on Cognitive Communications and Networking, pp. 1- 1, 2024.

[0010] Y. Ahn, J. Kim, S. Kim, K. Shim, J. Kim, S. Kim, and B. Shim, "Toward intelligent millimeter and terahertz communication for 6g: Computer vision-aided beamforming," IEEE Wireless Communications, vol. 30, no. 5, pp. 179-186, 2022.

[0011] A. Farhadi and J. Redmon, "Yolov3: An incremental improvement," in Computer Vision and Pattern Recognition, vol. 1804. Berlin / Heidelberg, Germany: Springer, 2018.

[0012] M. Alrabeiah, A. Hredzak, Z. Liu, and A. Alkhateeb, "Viwi: A deep learning dataset framework for vision-aided wireless communications," in 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), 2020, pp. 1-5.

[0013] M. Project, "Flexric: Flexible ran intelligent controller," https: / / gitlab.eurecom.fr / mosaic5g / flexric, 2024.

[0014] R. Schmidt, M. Irazabal, and N. Nikaein, "Flexric: An sdk for nextgeneration sd-rans," in Proceedings of the 17th International Conference on emerging Networking Experiments and Technologies, 2021, pp. 411- 425.

[0015] F. A. Bimo, R.-G. Cheng, C.-C. Tseng, C.-R. Chiang, C.-H. Huang, and X.-W. Lin, "Design and implementation of next-generation research platforms," in 2023 IEEE Globecom Workshops, 2023, pp. 1777-1782.

[0016] R. Ferreira, J. Fonseca, J. Silva, M. Tendulkar, P. Duarte, M. Araujo, ' R. Barbosa, B. Mendes, and A. Goes, "Enhancing network performance based on 5G network function and slice load analysis," in 2023 IEEE 24th WoWMoM. IEEE, 2023, pp. 340-342.

[0017] S. Fdida, N. Makris, T. Korakis, R. Bruno, A. Passarella, P. Andreou, B. Belter, C. Crettaz, W. Dabbous, Y. Demchenko et al., "Slices, a scientific instrument for the networking community," Computer Communications, vol. 193, pp. 189-203, 2022.

[0018] "Openairinterface — 5G software alliance for democratising wireless innovation," https: / / 0penairinterface.0rg / / , accessed on: August 20, 2024.

[0019] "3GPP ts 38.215, 5G nr physical layer measurements," https: / / www.etsi.org / deliver / etsi_ts / 138200_138299 / 138215 / 16.02.00_60 / ts_138215vl6 0200p.pdf, accessed on: August 20, 2024.

[0020] "CONVERGE project," https: / / converge-project.eu, accessed on: August 20, 2024.

[0021] E. Moqvist and Contributors, "asnltools: Asn.l parser and serializer," https: / / github.com / eerimoq / asnltools, 2024.

[0022] I. Culjak, D. Abram, T. Pribanic, H. Dzapo, and M. Cifrek, "A brief introduction to opencv," in 2012 Proceedings of the 35th International Convention MIPRO, 2012, pp. 1725- 1730.

[0023] S. Garrido-Jurado, R. Munoz-Salinas, F. Madrid-Cuevas, and M. Martin-Jimenez, "Automatic generation and detection of highly reliable fiducial ' markers under occlusion," Pattern Recognition, vol. 47, no. 6, pp. 2280- 2292, 2014.

[0024] U. Team, "Ultralytics documentation," 2024, accessed on: August 20, 2024. [Online], Available: https: / / docs.ultralytics.com.

[0025] N. Aharon, R. Orfaig, and B.-Z. Bobrovsky, "Bot-sort: Robust associations multipedestrian tracking," arXiv preprint arXiv:2206.14651, 2022.

Claims

C L A I M S1. System for Radio Access Network, RAN, management, by means of real-time sensing data from at least one video source, for Integrated Sensing and Communications, ISAC, of an O-RAN, the system comprising: at least one xApp configured to generate RIC control requests from video sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent and a Video Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC, arranged to accept RIC subscription requests from the at least one xApp, and arranged to accept RIC control requests from the at least one xApp; wherein the RAN Function E2 Agent is arranged to accept RIC subscription requests from the Near-RT RIC, and is arranged to accept RIC control requests from the Near- RT RIC; wherein the Near-RT RIC is further arranged to accept a Video sensing subscription request from the at least one xApp and, in response, send a corresponding Video sensing subscription request to the Video Function E2 Agent; wherein the Video Function E2 Agent is arranged to accept a Video sensing subscription request from the Near-RT RIC and, in response, send Video sensing data to the Near-RT RIC based on the at least one video source; wherein the Near-RT RIC is further arranged to accept Video sensing data from the Video Function E2 Agent and, in response, send the Video sensing data to the at least one xApp.

2. System for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, for Integrated Sensing and Communications, ISAC, of an O-RAN, the system comprising: at least one xApp configured to generate RIC control requests from RIS sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent and a RIS Function E2 Agent;a Near Real-time RAN Intelligent Controller, Near-RT RIC, arranged to accept RIC subscription requests from the at least one xApp, and arranged to accept RIC control requests from the at least one xApp; wherein the RAN Function E2 Agent is arranged to accept RIC subscription requests from the Near-RT RIC, and is arranged to accept RIC control requests from the Near- RT RIC; wherein the Near-RT RIC is further arranged to accept a RIS sensing subscription request from the at least one xApp and, in response, send a corresponding RIS sensing subscription request to the RIS Function E2 Agent; wherein the RIS Function E2 Agent is arranged to accept a RIS sensing subscription request from the Near-RT RIC and, in response, send RIS sensing data to the Near- RT RIC based on the at least one RIS; wherein the Near-RT RIC is further arranged to accept RIS sensing data from the RIS Function E2 Agent and, in response, send the RIS sensing data to the at least one xApp.

3. System for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, and from at least one video source, for Integrated Sensing and Communications, ISAC, of an O- RAN, the system comprising: at least one xApp configured to generate RIC control requests from video and / or RIS sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent; a Video Function E2 Agent and / or a RIS Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC, arranged to accept RIC subscription requests from the at least one xApp, and arranged to accept RIC control requests from the at least one xApp; wherein the RAN Function E2 Agent is arranged to accept RIC subscription requests from the Near-RT RIC, and is arranged to accept RIC control requests from the Near- RT RIC;wherein the Near-RT RIC is further arranged to accept a Video sensing subscription request from the at least one xApp and, in response, send a corresponding Video sensing subscription request to the Video Function E2 Agent; wherein the Video Function E2 Agent is arranged to accept a Video sensing subscription request from the Near-RT RIC and, in response, send Video sensing data to the Near-RT RIC based on the at least one video source; wherein the Near-RT RIC is further arranged to accept Video sensing data from the Video Function E2 Agent and, in response, send the Video sensing data to the at least one xApp; wherein the Near-RT RIC is further arranged to accept a RIS sensing subscription request from the at least one xApp and, in response, send a corresponding RIS sensing subscription request to the RIS Function E2 Agent; wherein the RIS Function E2 Agent is arranged to accept a RIS sensing subscription request from the Near-RT RIC and, in response, send RIS sensing data to the Near- RT RIC based on the at least one RIS; wherein the Near-RT RIC is further arranged to accept RIS sensing data from the RIS Function E2 Agent and, in response, send the RIS sensing data to the at least one xApp.

4. System according to any of the previous claims, wherein the at least one xApp comprises a first xApp for RIC Control and a second xApp for real-time sensing of video and / or RIS data, in particular wherein the at least one xApp comprises a first xApp for RIC control, a second xApp for real-time sensing of video data, and a third xApp for real-time sensing of RIS data.

5. System according to any of the previous claims, wherein the video sensing data is a video data stream.

6. System according to any of the previous claims, wherein the E2 Agents are arranged to communicate with the Near-RT RIC using an E2 interface.

7. System according to the previous claim, wherein the E2 interface is as defined by O-RAN.

8. System according to any of the previous claims, wherein the Near-RT RIC is arranged to receive E2 setup requests from the RAN Function E2 Agent, and is arranged to receive E42 setup requests and Video and / or RIS Setup requests from the at least one xApp.

9. System according to any of the previous claims, wherein the system is arranged such that: communications between the Near-RT RIC and the Video Function E2 Agent do not go through a gNB of the RAN, and / or communications between the Near-RT RIC and the RIS Function E2 Agent do not go through a gNB of the RAN; wherein the gNB is arranged for transmitting and receiving data between a RAN device and a core network of the RAN, in particular the gNB comprising a Radio Unit (O-RU), Distributed Unit (O-DU), and Centralized Unit (O-CU) of the O-RAN.

10. System according to any of the previous claims, wherein the Video Function E2 Agent and / or the RIS Function E2 Agent is arranged for tracking, detecting, and / or predicting obstacle blockage of line-of-sight, LoS, between a gNB of the RAN and User Equipment, in particular the Video Function E2 Agent and / or the RIS Function E2 Agent is arranged for predicting object trajectory for tracking, detecting, and / or predicting obstacle blockage, or the at least one xApp is arranged for tracking, detecting, and / or predicting obstacle blockage of line-of-sight, LoS, between a gNB of the RAN and User Equipment, in particular the at least one xApp is arranged for predicting object trajectory for tracking, detecting, and / or predicting obstacle blockage, further in particular from a received video data stream from the Video Function E2 Agent.

11. System according to any of the previous claims, wherein the Video Function E2Agent and / or the RIS Function E2 Agent is arranged for sending one or more of the following: a E2 message when a blockage is predicted to obstruct the line-of-sight, LoS, between the gNB and the UE; a E2 message when a blockage is detected obstructing the line-of-sight, LoS, between the gNB and the UE; a E2 message when a blockage is no longer detected obstructing the line-of-sight, LoS, between the gNB and the UE; and timestamped E2 messages.

12. Method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one video source, for Integrated Sensing and Communications, ISAC, of an O-RAN, using a system comprising: at least one xApp for sending RIC control requests from Video sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent and a Video Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC; the method comprising: the Near-RT RIC accepting RIC subscription requests from the at least one xApp, and accepting RIC control request from the at least one xApp; the RAN Function E2 Agent accepting RIC subscription requests from the Near-RT RIC, and accepting RIC control requests from the Near-RT RIC; wherein the method also comprises: the Near-RT RIC accepting a Video sensing subscription request from the at least one xApp and, in response, sending a corresponding Video sensing subscription request to the Video Function E2 Agent; the Video Function E2 Agent accepting a Video sensing subscription request from the Near-RT RIC and, in response, sending Video sensing data to the Near-RT RIC based on the at least one video source;the Near-RT RIC is further accepting Video sensing data from the Video Function E2 Agent and, in response, sending the Video sensing data to the at least one xApp.

13. Method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, for Integrated Sensing and Communications, ISAC, of an O-RAN, using a system comprising: at least one xApp configured to generate RIC control requests from RIS sensing data; a plurality of E2 agents comprising a RAN Function E2 Agent and a RIS Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC; the method comprising: the Near-RT RIC accepting RIC subscription requests from the at least one xApp, and accepting RIC control requests from the at least one xApp; the RAN Function E2 Agent accepting RIC subscription requests from the Near-RT RIC, and accepting RIC control requests from the Near-RT RIC; wherein the method also comprises: the Near-RT RIC accepting a RIS sensing subscription request from the at least one xApp and, in response, sending a corresponding RIS sensing subscription request to the RIS Function E2 Agent; the RIS Function E2 Agent accepting a RIS sensing subscription request from the Near-RT RIC and, in response, sending RIS sensing data to the Near-RT RIC based on the at least one RIS; the Near-RT RIC accepting RIS sensing data from the RIS Function E2 Agent and, in response, sending the RIS sensing data to the at least one xApp.

14. Method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, and from at least one video source, for Integrated Sensing and Communications, ISAC, of an O- RAN, using a system comprising: at least one xApp configured to generate RIC control requests from video and / or RIS sensing data;a plurality of E2 agents comprising a RAN Function E2 Agent; a Video Function E2 Agent and / or a RIS Function E2 Agent; a Near Real-time RAN Intelligent Controller, Near-RT RIC; the method comprising: the Near-RT RIC accepting RIC subscription requests from the at least one xApp, and accepting RIC control requests from the at least one xApp; the RAN Function E2 Agent accepting RIC subscription requests from the Near-RT RIC, and accepting RIC control requests from the Near-RT RIC; wherein the method also comprises: the Near-RT RIC accepting a Video sensing subscription request from the at least one xApp and, in response, sending a corresponding Video sensing subscription request to the Video Function E2 Agent; the Video Function E2 Agent accepting a Video sensing subscription request from the Near-RT RIC and, in response, sending Video sensing data to the Near-RT RIC based on the at least one video source; the Near-RT RIC is further accepting Video sensing data from the Video Function E2 Agent and, in response, sending the Video sensing data to the at least one xApp; the Near-RT RIC accepting a RIS sensing subscription request from the at least one xApp and, in response, sending a corresponding RIS sensing subscription request to the RIS Function E2 Agent; the RIS Function E2 Agent accepting a RIS Sensing subscription request from the Near-RT RIC and, in response, sending RIS sensing data to the Near-RT RIC based on the at least one RIS; the Near-RT RIC accepting RIS sensing data from the RIS Function E2 Agent and, in response, sending the RIS sensing data to the at least one xApp.

15. Non-transitory computer-readable medium comprising computer program instructions for implementing a method for Radio Access Network, RAN, management, by means of real-time sensing data from at least one reconfigurable intelligent surface, RIS, and / or from at least one video source, for IntegratedSensing and Communications, ISAC, of an O-RAN, which when executed by a processor, cause the processor to carry out the method of any of the claims 12-14.

Images