System and method of selecting modules of transformer based model

The system dynamically selects transformer-based model modules using an unsupervised classification mechanism, addressing inefficiencies in conventional models by enhancing adaptability, reducing overhead, and ensuring consistent performance in wireless communication systems.

WO2026238219A1PCT designated stage Publication Date: 2026-11-19HUAWEI TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2025/062837
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-12
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Conventional transformer-based models in wireless communication systems face inefficiencies due to high computational and memory requirements, extensive retraining, and inconsistent performance across tasks, limiting their scalability and real-time processing capabilities.

Method used

A system and method for dynamically selecting task-specific processing modules within a transformer-based model using an unsupervised classification mechanism, leveraging an embedding layer and classifier to generate a score map for module selection, reducing computational overhead and inference latency.

Benefits of technology

Enables adaptive task specialization with reduced computational and memory overhead, lower latency, and consistent performance across diverse communication tasks, maintaining compatibility with existing AI-driven systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2025062837_19112026_PF_FP_ABST
    Figure EP2025062837_19112026_PF_FP_ABST
Patent Text Reader

Abstract

A method of selecting modules of transformer-based model implemented in node of data communications network, comprising steps of receiving a plurality N of input data samples at transformer-based model implemented in node of data communications network. The method further comprises modifying each of N data samples according to an embedding layer of the transformer- based model, then receiving modified data samples at classifier of transformer based model, and generating, by classifier, score map using received modified data samples, where the score map has N entries, wherein each of N entries is vector representing a likelihood of corresponding data sample belonging to specific module of plurality of processing modules of transformer based model, where each module consists of plurality of data arrays. The method comprises generating stack of plurality of processing modules in accordance with the generated score map, and injecting stack of processing modules into transformer-based model.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] SYSTEM AND METHOD OF SELECTING MODULES OF TRANSFORMER BASED MODEL

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to the field of wireless communication systems and more specifically, to a system and method of selecting modules of a transformer-based model implemented in a node of a data communications network. BACKGROUND

[0004] In modem Artificial Intelligence (Al)-driven wireless communication systems, a transformer-based model has emerged as a powerful tool for channel estimation, beam prediction, and positioning tasks. The transformer-based model leverages selfattention mechanisms to extract complex patterns from large datasets, enabling improved signal processing and network optimization. An AI / Machine Learning (ML) integration is becoming essential in next-generation wireless networks, particularly within 3GPP standardization efforts, driving the need for efficient transformer-based architectures. However, despite several advantages, conventional transformer models require vast amounts of labelled data and significant computational resources for training and specialization, posing a major challenge to real-world deployment, especially in resource-constrained environments.

[0005] Conventional methods for enhancing transformer-based model performance across multiple tasks rely on three main strategies: (i) introducing additional processing stages to extract more expressive features, (ii) expanding training datasets to improve generalization, and (iii) increasing model complexity by adding new weights in the form of more transformer blocks or attention heads. Although conventional methods improve transformer-based models' adaptability, several drawbacks arise. First, finetuning with specialized datasets requires extensive parameter modifications, which is risky and could lead to model degradation. Second, increasing transformer model parameters leads to higher memory consumption and computational overhead, limiting scalability. Finally, inference latency rises as model complexity grows, making real-time processing challenging in wireless communication systems where low-latency responses are indispensable.

[0006] Furthermore, existing solutions for multi-task adaptation in transformer-based models often require explicit supervision, relying on predefined labels or auxiliary inputs to guide task specialization. The dependency on labelled data creates additional challenges in scenarios where obtaining ground-truth labels is difficult or costly. Alternative techniques, such as knowledge distillation, transfer knowledge from large models to task-specific models, but increase memory requirements due to the need to store a pre-trained transformer model. Similarly, architectural modifications designed to enhance multi-task performance often require fundamental changes to the transformer-based model, increasing implementation complexity and limiting compatibility with existing ML pipelines. Consequently, conventional methods lack the ability to provide an efficient, scalable, and automated approach to specialize the Transformer architecture across diverse communication tasks. The Al-driven wireless communication systems face challenges related to inefficient transformer-based model specialization, increased computational and memory requirements due to extensive re-training and larger model sizes, and inconsistent performance across multiple tasks.

[0007] Therefore, in light of the foregoing discussion, there exists a need for an improved transformer-based architecture that enables automatic task specialization, minimizes computational and memory overhead, reduces inference latency, and ensures consistent performance across multiple wireless communication tasks while maintaining compatibility with existing Al-driven communication systems.SUMMARY

[0008] The present disclosure provides a system and method of selecting modules of a transformer-based model implemented in a node of a data communications network. The present disclosure provides a solution to the existing problem of an inefficient transformer-based model specialization, increased computational and memory requirements due to extensive retraining and larger model sizes, and inconsistent performance across multiple tasks in wireless communication systems. An aim of the present disclosure is to provide a solution that overcomes at least partially the problems encountered in the prior art and provides an improved transformer-based architecture that enables automatic task specialization, minimizes computational and memory overhead, reduces inference latency, and ensures consistent performance across multiple wireless communication tasks while maintaining compatibility with existing Al-driven communication systems. The present disclosure introduces an unsupervised classification mechanism that dynamically selects task-specific processing modules of the transformer-based model from a predefined bank of processing modules. This approach allows the transformer-based models to adapt to different tasks in real-time while maintaining computational efficiency and low latency.

[0009] The object of the present disclosure is achieved by the solutions provided in the enclosed independent claims. Advantageous implementations of the present disclosure are further defined in the dependent claims.

[0010] In one aspect, the present disclosure provides a method of selecting modules of a transformer -based model implemented in a node of a data communications network, comprising steps of receiving a plurality N of input data samples at the transformerbased model implemented in the node of the data communications network. The method further comprises modifying each of the N data samples according to an embedding layer of the transformer-based model. The method further comprises receiving the modified data samples at a classifier of the transformer based model and generating, by the classifier, a score map using the received modified data samples, where the score map has N entries, where each of the N entries is a vector representing a likelihood of a corresponding data sample belonging to a specific module of a plurality of processing modules of the transformer based model, where each module consists of a plurality of data arrays. The method further comprises generating a stack of the plurality of processing modules in accordance with the generated score map, and injecting the stack of processing modules into the transformer-based model.

[0011] The disclosed method for selecting modules of the transformer-based model enhances multi-task adaptability in Al-driven communication systems. The method operates within the transformer-based model deployed in the node of the data communications network, optimizing task-specific processing without requiring additional supervision. Specifically, the method comprises receiving the plurality of N input data samples at the transformer-based model, modifying each input sample through the embedding layer, and passing the transformed data into an unsupervised classifier. The classifier generates the score map, where each entry represents the likelihood of a corresponding input sample being associated with a specific processing module from the predefined bank of modules. The selected modules, structured as data arrays, are then compiled into the stack of processing modules and injected into the transformer-based model to refine the feature map before passing it through the transformer blocks. By leveraging the dynamic module selection mechanism, the transformer-based model autonomously adapts to diverse communication tasks, reducing computational overhead and optimizing inference efficiency. The comprehensive approach enables improved adaptability, lower latency, and enhanced task specialization in wireless communication applications, surpassing traditional transformer architectures that rely on static configurations and task-specific re-training.

[0012] In an implementation form, the embedding layer creates an output feature map according to the input samples. By automating feature extraction through the embedding layer, the method reduces preprocessing overhead, improves classification accuracy, and enables real-time adaptability.In a further implementation form, the plurality of processing modules is a bank of positional encodings. By using the bank of positional encodings, the method dynamically assigns the most suitable encoding to each input sample, ensuring that the transformer-based model processes the input data samples in a way that preserves task-specific dependencies and optimizes feature representation.

[0013] In a further implementation form, the receiving step includes receiving the bank of positional encodings related to the data samples. By incorporating the bank of positional encodings, the method enables the transformer-based model to dynamically select the most appropriate encoding for each input data sample, improving adaptability to different communication tasks. In a further implementation form, different positional encoding sequences are attributed to different input classes of the input data samples. By dynamically selecting positional encodings tailored to input classes, the method enables the model to automatically specialize in multiple tasks without requiring explicit task labels or re-training. This results in lower computational overhead, improved accuracy, and faster inference times, making it particularly beneficial for real-time wireless communication systems.

[0014] In a further implementation form, in a training phase, a loss function of the transformer-based model includes a regularization term to force a prediction to create evenly distributed data subsets for each element in each of the positional encoding sequences. By enforcing an even distribution of data subsets, the method ensures that the model learns from a diverse range of encodings, making it more adaptable to varying communication tasks and reducing the risk of biased decision-making.

[0015] In a further implementation form, the node of the data communications network is a terminal device. By deploying the transformer-based model directly within the terminal device, the method enables on-device processing, reducing reliance on centralized computing resources and improving real-time adaptability.

[0016] In a further implementation form, the terminal device generates the score map. By generating the score map locally, the terminal device enables real-time module selection, improves response time, and enhances adaptability, ensuring enhanced performance in Al-driven wireless communication tasks.

[0017] In a further implementation form, the terminal device transmits an output of the embedding layer to a Base Station (BS). By transmitting the embedding layer output instead of raw data, the method reduces processing overhead at the BS, minimizes network bandwidth usage, and enhances real-time adaptability, enabling more efficient Al-driven wireless communication within a communication system.

[0018] In a further implementation form, the terminal device transmits an index of the score map to the BS. By transmitting only the index, the method minimizes data transmission requirements, enabling the BS to retrieve the required processing module while keeping communication efficient.

[0019] In a further implementation form, the terminal device generates a stack of the plurality of processing modules and combines them with the output of the embedding layer. By combining the processing module stack with the embedding layer output, this ensures that input feature maps are task-optimized, leading to higher accuracy and improved generalization across different communication scenarios.

[0020] In a further implementation form, the terminal device sends the combined signals to a BS. By allowing the terminal device to pre-process data, apply task-specific processing modules, and transmit enhanced feature representations, the method offloads some computational tasks from the BS, enabling faster response times, improved resource allocation, and more efficient network operations.In a further implementation form, a system comprising means adapted for carrying out all the steps of the method according to any preceding method claim.

[0021] The system manifests all the advantages and technical effects of the method of the present disclosure.

[0022] In a further implementation form, a computer program comprising instructions for carrying out all the steps of the method according to any preceding method claim, when said computer program is executed on a computer system.

[0023] It is to be appreciated that all the aforementioned implementation forms can be combined.

[0024] It has to be noted that all devices, elements, circuitry, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application, as well as the functionalities described to be performed by the various entities, are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof. It will be appreciated that features of the present disclosure are susceptible to being combined in various combinations without departing from the scope of the present disclosure as defined by the appended claims.

[0025] Additional aspects, advantages, features, and objects of the present disclosure would be made apparent from the drawings and the detailed description of the illustrative implementations construed in conjunction with the appended claims that follow.

[0026] BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The summary above, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the present disclosure, exemplary constructions of the disclosure are shown in the drawings. However, the present disclosure is not limited to specific methods and instrumentalities disclosed herein. Moreover, those skilled in the art will understand that the drawings are not to scale. Wherever possible, like elements have been indicated by identical numbers.

[0028] Embodiments of the present disclosure will now be described, by way of example only, with reference to the following diagrams wherein:

[0029] FIG. 1 illustrates a network environment of a communication system comprising a first node and a second node, in accordance with an embodiment of the present disclosure;

[0030] FIG. 2A is a block diagram that illustrates various exemplary components of a second node, in accordance with an embodiment of the present disclosure;

[0031] FIG. 2B is a block diagram that illustrates various components of a transformer-based model, in accordance with an embodiment of the present disclosure;

[0032] FIG. 3 is a flowchart of a method of selecting modules of the transformer-based model, in accordance with an embodiment of the present disclosure;

[0033] FIG. 4 illustrates selection of modules of a transformer-based model, in accordance with an embodiment of the present disclosure;

[0034] FIG. 5 is a flowchart that depicts an exemplary implementation scenario of a transformer-based model in a communication system utilizing a bank of positional encodings, in accordance with an embodiment of the present disclosure; andFIG. 6 is a flowchart that depicts an exemplary implementation scenario of a transformer-based model in a communication system utilizing a bank of positional encodings, in accordance with another embodiment of the present disclosure.

[0035] In the accompanying drawings, an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent. A non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.

[0036] DETAILED DESCRIPTION OF EMBODIMENTS

[0037] The following detailed description illustrates embodiments of the present disclosure and ways in which they can be implemented. Although some modes of carrying out the present disclosure have been disclosed, those skilled in the art would recognize that other embodiments for carrying out or practicing the present disclosure are also possible.

[0038] FIG. 1 illustrates a network environment of a system comprising a first node and a second node, in accordance with an embodiment of the present disclosure. With reference to FIG. 1, there is shown a system 100 comprising a first node 102 and a second node 104, communicating through a communication network 106. Furthermore, the second node 104 comprises a transformer-based model 108.

[0039] The system 100 may include suitable logic, circuitry, and interfaces configured to facilitate adaptive data processing and communication between multiple communication nodes. The system 100 includes at least a communication node 1 (such as the first node 102) and a communication node 2 (such as the second node 104), to transmit data, receive data, and also includes processing modules. The system 100 may be implemented in different network environments, including wireless, satellite, and loT-based communication networks, to support diverse applications. The system 100 may further integrate intelligent processing techniques, such as classification and transformer-based models, to enhance adaptability and optimize performance under varying communication conditions.

[0040] The first node 102 may include suitable logic, circuitry, and / or interfaces configured to transmit data to the second node 104 within the system 100 while enabling the transformer-based model 108 to perform processing for an adaptive task. The first node 102 may be implemented as, but is not limited to, a base station, an Intemet-of- Things controller, a satellite communication node, or any networked communication apparatus capable of generating and sending structured data for further processing.

[0041] The second node 104 (also referred to as a terminal device) may include suitable logic, circuitry, and / or interfaces configured to receive and process a plurality of input data samples from the first node 102 within the system 100. The second node 104 may be implemented as, but is not limited to, a chip, a chipset, a User Equipment (UE), an loT device, a satellite communication node, or any networked communication apparatus capable of executing the transformer-based model 108 for adaptive tasks. Upon receiving the plurality of input data samples from the first node 102, the second node 104 modifies the input data sample using an embedding layer before processing it through an unsupervised classifier.

[0042] The communication network 106 includes a medium, such as a communication channel, through which the first node 102 potentially communicates with the second node 104. Examples of the communication network 106 may include, but are not limited to, a cellular network (e.g., a 5G, or 5G NR network, such as sub 6 GHz, cmWave, or mmWave communication network), a cloud network, a Local Area Network (LAN), a vehicle-to-network ( V2N) network, a Metropolitan Area Network (MAN), and / or the Internet.The transformer-based model 108 may be referred to as a machine learning-based processing model specifically configured to enhance the task adaptation within the system 100, enabling efficient and automated data transformation. The transformerbased model 108 features the embedding layer that modifies the plurality of input data samples before passing them to the unsupervised classifier, which generates the score map to determine the most suitable processing module from a predefined bank of modules. The transformer-based model 108 works in conjunction with the embedding layer, the classifier, and the transformer blocks to enable real-time task specialization, ensuring improved performance in applications, such as Channel State Information (CSI) estimation, beam prediction, and signal classification, while maintaining adaptability and efficiency across diverse wireless communication tasks.

[0043] FIG. 2A is a block diagram that illustrates various exemplary components of a second node, in accordance with an embodiment of the present disclosure. FIG. 2A is described in conjunction with elements from FIG. 1. With reference to FIG. 2A, there is shown a block diagram 200A that illustrates various exemplary components of the second node 104. The second node 104 includes a memory 202, the transformer-based model 108, a network interface 204, and a processor 206.

[0044] The memory 202 may include suitable logic, circuitry, and / or interfaces that is configured to store machine code and / or instructions executable by the processor 206. Examples of implementation of the memory 202 may include, but are not limited to, an Electrically Erasable Programmable Read-Only Memory (EEPROM), Random Access Memory (RAM), Read Only Memory (ROM), Hard Disk Drive (HDD), Flash memory, a Secure Digital (SD) card, Solid-State Drive (SSD), a computer readable storage medium, and / or CPU cache memory. The memory 202 may store an operating system and / or a computer program product to operate the second node 104. The computer readable storage medium for providing a non-transient memory may include, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.

[0045] The network interface 204 may be referred to as a communication interface specifically designed to enable data exchange between the first node 102 and the second node 104. The network interface 204 facilitates the transmission and reception of data packets between the first node 102 and the second node 104, ensuring seamless connectivity for transformer-based processing tasks. The network interface 204 features suitable logic, circuitry, and / or interfaces to support wired or wireless communication protocols, allowing efficient data flow within the system 100. The network interface 204 is connected to the transformer-based model 108, where it assists in transmitting input data samples to be processed and in receiving modified output data for further analysis or decision-making.

[0046] The processor 206 may include suitable logic, circuitry, and / or interfaces that is configured to execute instructions stored in the memory 202. Examples of the processor 206 may include, but are not limited to an integrated circuit, a co-processor, a microprocessor, a microcontroller, a complex instruction set computing (CISC) processor, an application-specific integrated circuit (ASIC) processor, a reduced instruction set (RISC) processor, a very long instruction word (VLIW) processor, a central processing unit (CPU), a state machine, a data processing unit, and other processors or circuits. Moreover, the processor 206 may refer to one or more individual processors, processing devices, a processing unit that is part of a machine.

[0047] FIG. 2B is a block diagram that illustrates various exemplary components of a transformer-based model 108, in accordance with an embodiment of the present disclosure. FIG. 2B is described in conjunction with elements from FIGs. 1 and 2A. With reference to FIG. 2B, there is shown a block diagram 200B that illustrates various exemplary components of the transformerbased model 108. The transformer-based model 108 includes a plurality N of input data samples 208, an embedding layer 210, a classifier 212, a score map 214, a plurality of processing modules 216, a stack of plurality of processing modules 218 and a plurality of transformer blocks 220. Furthermore, each of the plurality of processing modules 216 includes a plurality of data arrays 222. The plurality N of input data samples 208 are provided as an input to the transformer-based model 108.The plurality N of input data samples 208 may be referred to as a collection of N distinct input data instances received by the transformer-based model 108 within the communication system (i.e., the system 100). The plurality N of input data samples 208 serve as the fundamental units of information that undergo transformation through various processing layers, enabling intelligent classification and module selection. The plurality N of input data samples 208 represents structured data instances, such as wireless signals, encoded transmissions, feature vectors, or measurement reports, transmitted from the first node 102 to the second node 104 for processing. Each data sample within the plurality N of input data samples 208 is first modified by the embedding layer 210 of the transformer-based model 108, ensuring that the input data sample is transformed into a suitable representation for further analysis.

[0048] The embedding layer 210 may be referred to as a feature transformation layer specifically designed to convert the plurality N of input data samples 208 into a structured representation suitable for processing within the transformer-based model 108. The embedding layer 210 extracts meaningful representations from the plurality N of input data samples 208, ensuring that relevant features are preserved while reducing dimensional complexity. The embedding layer 210 is connected to the network interface 204, where it receives the plurality N of input data samples 208 and applies transformation techniques, such as vector encoding, normalization, or positional encoding to enhance interpretability of the transformer-based model 108.

[0049] The classifier 212 may be referred to as a decision-making component specifically designed to analyze the plurality N of input data samples 208 and determine the most suitable processing module for each input data sample within the transformer-based model 108. The classifier 212 employs a Multi-Layer Perceptron (MLP) with two hidden layers. The classifier 212 maps an input data sample to a corresponding score vector, which represents the selection probabilities for the plurality of processing modules 216. The first hidden layer transforms the input data sample into an intermediate representation, followed by a second hidden layer that refines the mapping to the module selection space.

[0050] The score map 214 may be referred to as a structured output representation generated by the classifier 212 to determine the most suitable processing module for each input data sample of the plurality N of input data samples 208. The score map 214 comprises of N entries, where each entry corresponds to an input data sample and is represented as a vector indicating the likelihood of the sample belonging to a specific processing module from the plurality of processing modules 216. In an implementation scenario, the score map 214 may be generated using a MLP classifier with softmax-based activation, ensuring that only the most relevant processing module is selected for each input data sample.

[0051] The plurality of processing modules 216 may include multiple computational units configured to perform specialized processing tasks. The plurality of processing modules 216 may be responsible for executing different stages of data processing, such as feature extraction, classification, encoding, and transformation, to enhance the functionality of the system 100. Each processing module within the plurality of processing modules 216 may be dynamically selected based on system requirements, enabling adaptive processing tailored to specific tasks. The plurality of processing modules 216 may operate in parallel or sequentially, depending on the processing workflow, to enhance efficiency and resource utilization.

[0052] The stack of plurality of processing modules 218 may be referred to as a dynamically selected set of processing modules that are retrieved from the plurality of processing modules 216, based on an input classification mechanism. Each processing module within the stack of plurality of processing modules 218 is designed to perform a specific function, such as feature transformation, signal processing, or encoding, and is selected based on the score map 214. The selection process is governed by a sparse matrix representation, where only the most relevant module is activated per input data sample, ensuring efficient resource utilization.

[0053] Each of the plurality of transformer blocks 220 may be referred to as a fundamental building unit of the transformer-based model 108, comprising multi-head self-attention, a Feed-Forward Neural Network (FFN), layer normalization, and residual connections. The self-attention mechanism enables the model to weigh the importance of different words in a sequence, whilethe FFN refines the representation for better contextual understanding. The layer normalization and residual connections lead to stabilizing training and improving gradient flow, allowing deeper networks to be trained efficiently.

[0054] The plurality of data arrays 222 may include multiple structured data sets that store and organize information for processing. Each data array may represent a collection of values, parameters, or feature representations used in various computational tasks, such as classification, encoding, or signal transformation. The plurality of data arrays 222 may be dynamically updated based on input data samples received from sensors, measurements, or processed outputs from other system components (such as the score map 214). The plurality of data arrays 222 may facilitate efficient data handling, allowing the system 100 to access, modify, and transmit relevant information for adaptive processing. Additionally, the plurality of data arrays 222 may support parallel data processing by enabling multiple computations to occur simultaneously across different modules.

[0055] In operation, there is provided the system 100 of selecting modules of the transformer-based model 108 implemented in a node of a data communications network. In an implementation scenario, the transformer-based model 108 may be implemented at the second node 104 of the system 100 (as shown in FIG. 1). In another implementation scenario, a few components (e.g., the embedding layer 210, the classifier 212, the score map 214, the plurality of processing modules 216, and the stack of plurality of processing modules 218) of the transformer-based model 108 may be implemented at the second node 104 (e.g., a User Equipment (UE)) and few components (e.g., the plurality of transformer blocks 220) may be implemented at the first node 102 (e.g., a Base Station (BS)) of the system 100.

[0056] The system 100 is configured to receive the plurality N of input data samples 208 at the transformer-based model 108 implemented in the node of the data communications network. The plurality N of input data samples 208 serve as the basis for adaptive feature extraction, allowing the transformer-based model 108 to dynamically adjust its processing strategy for different tasks, such as Channel State Information (CSI) estimation, beam prediction, and signal classification. Further, the system 100 is configured to modify each of the N data samples according to the embedding layer 210 of the transformer-based model 108. Each of the plurality N of input data samples 208 is passed through the embedding layer 210, where each data sample is transformed into a structured feature vector. The transformation ensures that each of the N data samples is properly formatted before being analyzed by the classifier 212, which subsequently determines the most appropriate processing module for each input data sample. Further, the system 100 is configured to receive the modified data samples at the classifier 212 of the transformer-based model 108. The modified data samples by the embedding layer 210 are then received by the classifier 212, which applies a MLP architecture with hidden layers and a softmax activation function to generate the score map 214. The system 100 is further configured to generate the score map 214 using the received modified data samples at the classifier 212, where the score map 214 has N entries, where each of the N entries is a vector representing a likelihood of a corresponding data sample belonging to a specific module of the plurality of processing modules 216 of the transformer-based model 108, where each module consists of the plurality of data arrays 222. The classifier 212 generates the score map 214, where each N entry represents a probability distribution over the available processing modules in the plurality of processing modules 216. The module (or processing module) with the highest likelihood is selected for each input data sample. The selected processing modules consist of structured data arrays of the plurality of data arrays 222, which modify the output of the embedding layer 210 before passing the data through the plurality of transformer blocks 220. Further, the system 100 is configured to generate the stack of the plurality of processing modules 218 in accordance with the generated score map 214 and injecting the stack of plurality of processing modules 218 into the transformer-based model 108. The stack of plurality of processing modules 218 is compiled, ensuring that each processing module is assigned to the specific input data sample. The generated stack of plurality of processing modules 218 is then injected into the transformer-based model 108, passing the data through the plurality of transformer blocks 220 for inference or training.

[0057] In accordance with an embodiment, the embedding layer 210 creates an output feature map according to the input data samples. The embedding layer 210 is responsible for transforming raw input data (i.e., the plurality N of input data samples 208) into astructured representation that can be effectively processed by the transformer-based model 108. The embedding layer 210 applies vector encoding to map categorical or numerical inputs into a high-dimensional space. Once processed, the embedding layer 210 generates the output feature map, which captures the vital characteristics of the plurality N of input data samples 208. The resulting output feature map serves as an intermediate representation that preserves desired characteristics of the plurality N of input data samples 208 while preparing them for further processing.

[0058] In accordance with an embodiment, the plurality of processing modules 216 is a bank of positional encodings. In an implementation scenario of the system 100, the plurality of processing modules 216 may be used as the bank of positional encodings. The positional encoding refers to a mathematical technique used to incorporate sequence order information into the input data samples for processing by the transformer-based model 108. Each processing module corresponds to a unique positional encoding, which is used to modify the output of embedding layer 210 based on the characteristics of the plurality N of input data samples 208. The positional encodings aid the transformer-based model 108 in capturing sequential and spatial relationships in data by providing contextual information about each input data sample. The corresponding positional encodings from the bank of positional encodings are selected for each data sample and compiled in a similar manner as the stack of plurality of processing modules 218 is complied, which is then injected into the transformer-based model 108 to enhance the prediction accuracy of the model. The positional encodings modify the feature representation before the modified data samples is passed through the plurality of transformer blocks 220, ensuring enhanced feature alignment and improved task-specific processing. By using the bank of positional encodings, the system 100 dynamically assigns the most suitable encoding to each input data sample, ensuring that the transformer-based model 108 processes data in a way that preserves task-specific dependencies and feature representation.

[0059] In accordance with an embodiment, the system 100 is configured to receive the bank of positional encodings related to the input data samples. The positional encodings provide required contextual information about the input data samples, allowing the transformer-based model 108 to incorporate spatial, temporal, or frequency-domain relationships into the feature extraction process. The bank of positional encodings is configured as a structured repository of predefined positional encodings. By leveraging the predefined structure of the bank of positional encodings, the embedding layer 210 effectively integrates sequence-dependent information into the input representation. The structured assignment of positional encodings ensures that variations in input sequence arrangements do not disrupt the learning process within the transformer-based model 108. The positional encodings refine the output of embedding layer 210, ensuring that the transformer-based model 108 effectively captures task-specific structural relationships before processing the data through the plurality of transformer blocks 220. In accordance with an embodiment, different positional encoding sequences are attributed to different input classes of the input data samples. The plurality of processing modules 216, structured as the bank of positional encodings, is designed to associate distinct encoding sequences with specific input classes, ensuring that each category of input data samples receives a unique transformation within the embedding layer 210. The classifier 212 within the transformer-based model 108 evaluates the plurality N of input data samples 208 and assigns appropriate positional encoding sequences based on predefined classification criteria. The selected positional encoding sequences are then compiled as the stack 218 of plurality of positional encodings and injected into the transformer-based model 108 to modify the embedding output before passing through the plurality of transformer blocks 220. By dynamically associating different positional encoding sequences with corresponding input classes, the system 100 improves context-awareness in the transformer-based model 108, ensuring that input data samples are processed with positional encoding schemes that enhance their underlying structure and relationships.

[0060] In accordance with an embodiment, in a training phase, a loss function of the transformer-based model 108 includes a regularization term to force a prediction to create evenly distributed data subsets for each element in each of the positional encoding sequences. The regularization term is designed to balance the classification process, ensuring that the transformerbased model 108 does not overfit to specific positional encodings but instead learns to distribute input data samples uniformly across available encoding options. To ensure balanced training, the loss function of the transformer-based model 108incorporates the regularization term, which penalizes scenarios where data subsets are unevenly distributed across positional encoding sequences. This encourages the transformer-based model 108 to create evenly distributed data subsets, ensuring that each positional encoding receives a proportionate share of training examples. The transformer-based model 108 is then trained using this loss function, allowing it to develop a more generalized representation of positional encodings. By enforcing balanced distribution during training of the transformer-based model 108, the system 100 enhances the ability of the transformer-based model 108 to adapt dynamically, improves accuracy across multiple tasks, and ensures robust and unbiased feature extraction in the system 100.

[0061] In accordance with an embodiment, the node of the data communications network is a terminal device. In an implementation scenario, the second node 104 of the system 100 may be the terminal device. Examples of the terminal device my include, but are not limited to, a User Equipment (UE), a chip, a chipset, an loT device, a mobile communication device, or any networked endpoint capable of transmitting and receiving data. The terminal device is equipped with the transformer-based model 108, which processes incoming input data samples and dynamically selects processing modules or positional encodings to enhance task-specific performance. By deploying the transformer-based model 108 directly within the terminal device, the system 100 enables on-device processing, reducing reliance on centralized computing resources and improving real-time adaptability. In accordance with an embodiment, the terminal device generates the score map 214. The score map 214 contains N entries, each representing the probability distribution of the input data sample (or the plurality N of input data samples 208) across different processing modules within the plurality of processing modules 216. By generating the score map 214 locally, the terminal device enables real-time module selection, improves response time, and enhances adaptability, ensuring enhanced performance in Al-driven wireless communication tasks, such as CSI estimation, beam prediction, and signal classification within the system 100.

[0062] In accordance with an embodiment, the terminal device transmits an output of the embedding layer 210 to a Base Station (BS). In the implementation scenario, where the few components (e.g., the embedding layer 210, the classifier 212, the score map 214, the plurality of processing modules 216, and the stack of plurality of processing modules 218) of the transformer-based model 108 are implemented at the second node 104 (e.g., a User Equipment (UE)) and few components (e.g., the plurality of transformer blocks 220) are implemented at the first node 102 (e.g., a BS) of the system 100, the second node 104 (i.e., the terminal device) transmits an output of the embedding layer 210 to the first node 102 (i.e., the BS). An exemplary implementation scenario is shown and will be described, for example, in conjunction with FIGs. 5 and 6.

[0063] Upon receiving the output of the embedding layer 210, the BS processes the data further using the classifier 212, which generates the score map 214 to determine the most relevant processing module from the plurality of processing modules 216 or positional encoding sequence from the bank of positional encodings. The BS then compiles the stack of plurality of processing modules 218 (or the stack of positional encoding) and injects them into the transformer-based model 108, and performs final inference. The output of the embedding layer 210 comprises structured feature maps that enhance the interpretability of raw data before being processed by Al-driven models at the BS. By offloading the intermediate representation instead of raw data, the terminal device enables more efficient network communication, reducing bandwidth consumption and optimizing processing performance at the BS.

[0064] In accordance with an embodiment, the terminal device transmits an index of the score map 214 to the BS. Instead of transmitting the score map 214 as a whole, the terminal device extracts an index representing the selected processing module and transmits it to the BS over the communication network 106. Upon receiving the index, the BS retrieves the corresponding processing module, compiles the stack of plurality of processing modules 218, and injects it into the transformer-based model 108 for final inference. By transmitting the index of the score map 214, the system 100 minimizes network load, reduces latency, and enhances real-time adaptability, ensuring efficient and scalable Al-driven communication within the system 100.In accordance with an embodiment, the terminal device generates the stack of the plurality of processing modules 218 and combines them with the output of the embedding layer 210. The terminal device selects the corresponding processing modules from the plurality of processing modules 216 and compiles them into the stack of plurality of processing modules 218. The terminal device then combines the stack of plurality of processing modules 218 with the output of the embedding layer 210, modifying the feature representations before passing them through the plurality of transformer blocks 220 for further processing. By enabling the terminal device to generate and apply processing module stacks, the system 100 enhances realtime Al-driven decision-making, reduces reliance on network infrastructure, and improves adaptability in Al-based wireless communication systems, and ensures enhanced performance.

[0065] In accordance with an embodiment, the terminal device sends the combined signals to the BS. The terminal device first receives the plurality of N input data samples 208, processes them through the embedding layer 210, and classifies them using the classifier 212, which generates the score map 214. Based on the score map 214, the terminal device selects the most suitable processing module (or positional encoding) for each input data sample from the plurality of processing modules 216 (or the bank of positional encodings), compiles them into the stack of plurality of processing modules 218 (or the stack of selected positional encodings), and combines them with the output of the embedding layer 210 and sends them to the BS. The combination of these signals enables the BS to reconstruct the structured data representation, apply the appropriate positional encoding, and enhance subsequent processing tasks. The structured transmission of combined signals eliminates the requirement for redundant signalling overhead, leading to reduced latency and enhanced spectrum efficiency. By leveraging the classifier 212 output along with the transformed data, the BS gains a more comprehensive understanding of the input characteristics, allowing for improved resource allocation, adaptive communication strategies, and more accurate feature extraction for downstream processing.

[0066] FIG. 3 is a flowchart of a method of selecting modules of a transformer-based model, in accordance with an embodiment of the present disclosure. FIG. 3 is described in conjunction with elements from FIGs. 1, 2A, and 2B. With reference to FIG. 3, there is shown a method 300 of selecting modules of the transformer-based model 108 that includes steps 302 to 312. The second node 104 comprising the transformer-based model 108 is configured to execute the method 300.

[0067] Referring to FIG. 3, at step 302, the method 300 comprises receiving the plurality N of input data samples 208 at the transformer-based model 108 implemented in the node of the data communications network. The step 302 involves the reception of the plurality N of input data samples 208 at the transformer-based model 108 from the first node 102 within the system 100. The reception of the plurality N of input data samples 208 is executed using a standardized data exchange protocol. The plurality N of input data samples 208 may include structured communication signals, network status measurements, or other relevant data formats required for Al-driven processing. In an implementation scenario, the node of the data communications network is a terminal device.

[0068] At step 304, the method 300 comprises modifying each of the N data samples according to the embedding layer 210 of the transformer-based model 108. The step 304 involves processing the N input data samples through the embedding layer 210, which transforms raw input data into structured feature representations. By modifying the plurality N of input data samples 208, the method 300 provides an efficient feature extraction, improved classification accuracy, and enhanced adaptability in Al-driven communication tasks within the system 100. The embedding layer 210 creates an output feature map according to the input data samples. The output feature map captures the most relevant characteristics of the input data samples, ensuring that each data sample is represented in a form suitable for further classification and module selection.

[0069] At step 306, the method 300 comprises receiving the modified data samples at the classifier 212 of the transformer-based model 108. The step 306 involves transferring the N modified data samples from the embedding layer 210 to the classifier 212, where each data samples undergoes further processing to determine the most suitable processing module from the plurality of processing modules 216 for each input data samples.In an implementation scenario, the plurality of processing modules 216 is the bank of positional encodings. The positional encodings aid the plurality of transformer blocks 220 to capture sequential and spatial relationships in data by providing contextual information about each input data sample. The positional encodings modify the feature representation before the data is passed through the plurality of transformer blocks 220, ensuring optimal feature alignment and improved task-specific processing.

[0070] In accordance with an embodiment, the receiving step includes receiving the bank of positional encodings related to the input data samples. The first node 102 transmits data samples along with the corresponding bank of positional encodings, ensuring that the receiving node (i.e., the second node 104) has the required encoding structures to optimize feature representation. By receiving the bank of positional encodings, the method 300 allows the transformer-based model 108 to dynamically adapt its encoding selection based on input characteristics.

[0071] Moreover, different positional encoding sequences are attributed to different input classes of the input data samples. The transformer-based model 108 utilizes the bank of positional encodings, where each positional encoding sequence is tailored to a specific input class to enhance the ability of transformer-based model 108 to capture task-relevant relationships within the input data samples.

[0072] In accordance with an embodiment, in a training phase, the loss function of the transformer-based model 108 includes a regularization term to force a prediction to create evenly distributed data subsets for each element in each of the positional encoding sequences. By adding the loss function when the distribution of assigned encodings deviates from an even spread, thereby encouraging the transformer-based model 108 to explore and utilize a broader range of positional encodings. As a result, the training process leads to a more diverse and generalized feature representation, improving the model’s adaptability across different input data samples.

[0073] At step 308, the method 300 comprises generating, by the classifier 212, the score map 214 using the received modified data samples, where the score map 214 has N entries, where each of the N entries is a vector representing a likelihood of a corresponding data sample belonging to a specific module of the plurality of processing modules 216 of the transformer-based model 108, where each module consists of the plurality of data arrays 222. The step 308 involves generating the score map 214 based on the classifier 212 analysis of the structured feature representations, allowing the transformer-based model 108 to dynamically select the most appropriate processing module for each input data sample. Each processing module comprises of the plurality of data arrays 222, which are responsible for refining and enhancing the feature transformation process.

[0074] At step 310, the method 300 comprises generating the stack of the plurality of processing modules 218 in accordance with the generated score map. The step 310 involves selecting and organizing the processing modules based on the likelihood values assigned in the score map 214, ensuring that each input data sample is paired with the most appropriate processing module and thereafter, generating the stack of the plurality of processing modules 218.

[0075] At step 312, the method 300 comprises injecting the stack of processing modules 218 into the transformer-based model 108. The step 312 involves integrating the stack of processing modules 218, generated in step 310, into the transformer-based model 108 to refine the feature transformation process. Each processing module in the stack of plurality of processing modules 218, consisting of the plurality of data arrays 222, is applied to the corresponding input data sample based on the classification results from the score map 214.

[0076] The steps 302 to 312 are only illustrative, and other alternatives can also be provided where one or more steps are added, or one or more steps are provided in a different sequence without departing from the scope of the claims herein.

[0077] There is provided a computer program product comprising instructions for performing the method 300 when executed by one or more processors (e.g., the processor 206) in the second node 104 of the system 100. The computer program is implemented as an algorithm, embedded in a software stored in the non-transitory computer-readable storage medium having programinstructions stored thereon, the program instructions being executable by the one or more processors in the computer system to execute the method 300. The non-transitory computer-readable storage means may include, but are not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Examples of implementation of computer-readable storage medium, but are not limited to, an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Random Access Memory (RAM), a Read Only Memory (ROM), a Elard Disk Drive (EIDD), a Flash memory, a Secure Digital (SD) card, a Solid-State Drive (SSD), a computer-readable storage medium, and / or a CPU cache memory.

[0078] FIG. 4 illustrates selection of modules of a transformer-based model, in accordance with an embodiment of the present disclosure. FIG. 4 is described in conjunction with elements from FIGs. 1, 2A, 2B, and 3. With reference to FIG. 4, there is shown a block diagram 400 that illustrates selection of modules of the transformer-based model 108. The block diagram 400 represents the processing of the plurality N of input data samples 208 through a sequence of steps to select and apply the most relevant processing modules dynamically. There is further shown the plurality N of input data samples 208, the embedding layer 210, the classifier 212, the score map 214, the plurality of processing modules 216, a module selector 402, a module stack 404, the stack of the plurality of processing modules 218, and the plurality of transformer blocks 220.

[0079] The module selector 402 may be referred to as a computational component responsible for dynamically selecting the most suitable processing module for each input data sample within the plurality of processing modules 216. The module selector 402 operates by analyzing the score map 214, where each entry in the score map 214 corresponds to the input data sample and represents the likelihood of the sample belonging to a specific processing module from a predefined set of modules (i.e., the plurality of processing modules 216). The module selector 402 utilizes an unsupervised selection mechanism to automate the assignment of modules, ensuring adaptive feature transformation based on the input characteristics.

[0080] The module stack 404 may be referred to as a dynamically structured arrangement of the plurality of processing modules 216. The module stack 404 is formed by selecting and organizing the most relevant processing modules from the predefined bank of modules according to the likelihood values assigned to each input sample in the score map 214. The module stack 404 is injected into the plurality of transformer blocks 220, where it enables task-specific feature adjustments for improved inference and training performance.

[0081] Referring to the FIG. 4, the plurality N of input data samples 208 labelled from “Input 0” to “Input N” represents a batch of N input data samples received from the communication Node 1 (such as the first node 102). The plurality N of input data samples 208 pass through the embedding layer 210, the embedding layer 210 modifies each data sample to create an output feature map. The output of embedding layer 210 follows two paths, one path to the classifier 212 and second path directly towards the plurality of transformer blocks 220. The classifier 212 generates the score map 214 (represented by H in the FIG. 4) where each entry indicates the likelihood of the input data sample belonging to a specific processing module. The module selector 402 which includes the plurality of processing modules 216 (shown in the dotted box in FIG. 4 as Module M, Module 1 , Module 2, Module 3, Module 4) is configured to use the score map 214 to determine a suitable processing module corresponding to each input data sample. In an implementation scenario, the plurality of processing modules 216 may represent a bank of positional encodings. Furthermore, the module stack 404 then assembles the selected processing modules based on the results of the score map 214, as shown in the dashed box where specific modules are matched to specific inputs (e.g., Module 3 to Input 0, Module 4 to Input 1, etc.) and generates the stack of the plurality of processing modules 218. Thereafter, the stack of the plurality of processing modules 218 is combined with the output of the embedding layer 210. The combined operation, represented by a circle with a plus sign, combines the output from the embedding layer 210 with the selected module stack (represented by Af , ), and this combined data is sent to the plurality of transformer blocks 220 for inference or continued training of the transformer-based model 108.In an exemplary scenario, consider a multi-node communication network where the first node 102 is the BS and the second node 104 is the UE. The first node 102 transmits pilot signals, such as CSI-RS or SSB beams, to the second node 104, which receives the signals and processes them to estimate the wireless channel characteristics. The received data samples are passed through the embedding layer 210 at the second node 104 and subsequently classified using the transformer-based model 108 equipped with adaptive processing modules. The classifier 212 generates the score map 214 that assigns each input data sample to the most suitable processing module based on the signal conditions and network environment. The stack of the plurality of processing modules 218 is then dynamically assembled and injected into the plurality of transformer blocks 220 for further refinement. The plurality of transformer blocks 220 processes the modified data, enhancing feature extraction and improving the accuracy of CSI estimation. This approach enhances beam prediction accuracy and ensures enhanced resource allocation, ultimately leading to improved network efficiency and higher spectral utilization in dynamic communication environments. FIG. 5 is a flowchart that depicts an exemplary implementation scenario of a transformer-based model in a communication system utilizing a bank of positional encodings, in accordance with an embodiment of the present disclosure. FIG. 5 is described in conjunction with elements from FIGs. 1, 2A, 2B, 3, and 4. With reference to FIG. 5, there is shown a flowchart 500 that depicts an exemplary implementation scenario of the transformer-based model 108 in a communication system utilizing the Bank of Positional Encodings (BPE). There is further shown a Positional Encoding (PE) selector 502. The flowchart 500 includes a series of operations from 504 to 520 during the communication between the first node 102 (i.e., the BS) and the second node 104 (i.e., the UE).

[0082] At operation 504, the first node 102 (i.e., the BS) is configured to design the BPE. The BPE comprises multiple encoding schemes, each designed to enhance data representation in different communication scenarios.

[0083] At operation 506, the first node 102 (i.e., the BS) is configured to transmit either the whole BPE or only a few BPE parameters to the second node 104 (i.e., the UE).

[0084] At operation 508, the first node 102 (i.e., the BS) is configured to transmit data (i.e., the plurality N of input data samples) to the second node 104 (i.e., the UE). The transmitted data may comprise either pilot signals or reference signals used for various wireless communication tasks, such as CSI estimation or beam prediction.

[0085] At operation 510, upon receiving the transmitted data from the first node 102 (i.e., the BS), the second node 104 (i.e., the UE) is configured to perform measurements and preprocessing on the received data. The preprocessing is carried out using the embedding layer 210, which extracts useful features from the received data. The preprocessing step ensures that relevant information is passed to the classifier 212 for positional encoding selection.

[0086] < " >

[0087]

[0088] > At operation 514, the PE selector 502 processes the sparse matrix H to generate the stack of the plurality of positional encodings selected from the BPE. The PE selector 502 chooses the PE with highest probability within the BPE.

[0089] The selected set of PEs for an input batch of size N is computed as shown in Equation (2).

[0090]

[0091] where i E [0, N — 1] and j E [0, M — 1] . The stack of the plurality of positional encodings has N elements, one for each input data sample. The generated stack of plurality of positional encodings is then introduced to modify the output of the embedding layer 210 via aggregation.

[0092] At operation 516, the stack of the plurality of positional encodings is combined with the output of the embedding layer 210. At operation 518, the second node 104 (i.e., theUE) is configured to transmit the combined output of embedding layer 210 and the selected PE stack back to the first node 102 (i.e., the BS). The output signal includes frequency components unique to the selected PE, making it distinguishable and optimized for further processing.

[0093] At operation 520, the first node 102 (i.e., the BS) receives the combined output from the second node 104 (i.e., the UE) and processes it using the plurality of transformer blocks 220. The plurality of transformer blocks 220 uses the selected PEs to perform inference tasks, such as the CSI prediction, beam selection, or signal classification. The use of dynamically assigned PEs enhances the accuracy of feature extraction and improves the efficiency of the system 100.

[0094] FIG. 6 is a flowchart that depicts an exemplary implementation scenario of a transformer-based model in a communication system utilizing a bank of positional encodings, in accordance with another embodiment of the present disclosure. FIG. 6 is described in conjunction with elements from FIGs. 1, 2A, 2B, 3, 4 and 5. With reference to FIG. 6, there is shown a flowchart 600 that depicts an exemplary implementation scenario of the transformer-based model 108 in a communication system utilizing the BPE. The flowchart 600 includes a series of operations from 602 to 616 during the communication between the first node 102 (i.e., the BS) and the second node 104 (i.e., the UE). The second node 104 (i.e., the UE) processes received data from the first node 102 (i.e., the BS) using the output of embedding layer 210 and the classifier 212. The output of the classifier 212, which is a sparse score matrix, is transmitted back to the first node 102 (i.e., the BS) along with the embedding layer 210 output. The first node 102 (i.e., the BS) then reconstructs the stack of positional encodings and combines them with the received output from the embedding layer 210 before passing it through the plurality of transformer blocks 220.

[0095] At operation 602, the process begins with the first node 102 (i.e., the BS) designing the BPE, which comprises various positional encoding modules that can be selected based on input characteristics. Furthermore, the first node 102 (i.e., theBSjis configured to transmit the designed BPE (either the whole BPE or a few parameters of the BPE) to the second node 104 (i.e., the UE). At operation 604, the first node 102 (i.e., the BS) transmits data (e.g., pilot signals, control information, or raw features) to the second node 104 (i.e., the UE).

[0096] At operation 606, upon receiving the data from the first node 102 (i.e., the BS), the second node 104 (i.e., the UE) extracts relevant features from the received data and passes the extracted relevant features to the embedding layer 210. The embedding layer 210 transforms the extracted relevant features into a feature space suitable for classification.

[0097] At operation 608, the processed data is sent to the classifier 212, which generates the score map 214 which is used to assign a suitable set of PEs to each input data sample. The classifier 212 is implemented as a multi-layer perceptron (MLP) with two hidden layers. The classifier makes use of a standard MLP with one hidden layer as represented by Equation (3), with N size of mini-batch and g number of groups in the BPE.

[0098] C = Softmax(MLP(XEmb)), C E RNxs (3)The first hidden layer transforms the input as — > R™ with intermediate dimension m < d. The second hidden layer maps the intermediate representation to the output space f2: W" — > RM. The MLP uses the ReLU activation. The final score map H is generated using the so Umax function to highlight the most suitable PE, followed by Top-1 selection to ensure a single PE is chosen for each input, as represented by the Equation (1).

[0099] The output of the classifier 212, H is a compressed sparse matrix with non-negative values, where only the highest softmax score is retained per input sample.

[0100] At operation 610, the second node 104 (i.e., the UE) transmits the score map 214 (sparse matrix H ) to the PE selector 502, for selecting the suitable PE from the BPE for each input data sample.

[0101] At operation 612, the second node 104 (i.e., the UE) transmits the embedding layer 210 output to the first node 102 (i.e., the BS).

[0102] At operation 614, the first node 102 (i.e., the BS) combines the stack of PEs (reconstructed from the output of the classifier 212 and the output of the embedding layer 210 received from the second node 104 (i.e., the UE). This combination enhances the extracted features, making the signal more robust for further processing.

[0103] At operation 616, the final combined signal is passed through the plurality of transformer blocks 220 at the first node 102 (i.e., the BS). The plurality of transformer blocks 220 utilizes the assigned PEs to perform inference tasks, such as the channel state estimation, beam selection, and signal classification.

[0104] The present disclosure introduces an efficient and adaptive approach for module selection in the transformer-based model 108 implemented at the second node 104 of the system 100. Unlike conventional transformer-based architectures that rely on fixed positional encodings or manual retraining for task specialization, the disclosed system and method 300 leverages the plurality of processing modules 216 and the classifier 212 to dynamically select and apply task-specific feature transformations. The adaptability enables real-time specialization for different input samples, ensuring an enhanced feature extraction and classification without increasing computational overhead. By generating the score map 214 to determine the most relevant processing module for each input data samples, the method 300 enhances task efficiency, eliminates the requirement for explicit supervision, and optimizes resource allocation. Additionally, by injecting the stack of plurality of processing modules 218 into the transformer-based model 108, the system 100 ensures seamless integration of specialized transformations, leading to improved accuracy, reduced inference latency, and enhanced adaptability to changing network conditions. These innovations make the system 100 and the method 300 particularly beneficial for wireless communication applications, such as CSI estimation, beam prediction, and adaptive signal classification, where prior solutions were limited by static processing architectures and high computational costs.

[0105] Modifications to embodiments of the present disclosure described in the foregoing are possible without departing from the scope of the present disclosure as defined by the accompanying claims. Expressions such as "including", "comprising", "incorporating", "have", "is" used to describe and claim the present disclosure are intended to be construed in a non-exclusive manner, namely allowing for items, components or elements not explicitly described also to be present. Reference to the singular is also to be construed to relate to the plural. The word "exemplary" is used herein to mean "serving as an example, instance or illustration". Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features from other embodiments. The word "optionally" is used herein to mean "is provided in some embodiments and not provided in other embodiments". It is appreciated that certain features of the present disclosure, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the present disclosure, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable combination or as suitable in any other described embodiment of the disclosure.

Claims

CLAIMS1. A method (300) of selecting modules of a transformer-based model (108) implemented in a node of a data communications network, comprising steps of:receiving a plurality N of input data samples (208) at the transformer-based model (108) implemented in the node of the data communications network;modifying each of the N data samples according to an embedding layer (210) of the transformer-based model (108); receiving the modified data samples at a classifier (212) of the transformer-based model (108);generating, by the classifier (212), a score map (214) using the received modified data samples, wherein the score map (214) has N entries, wherein each of the N entries is a vector representing a likelihood of a corresponding data sample belonging to a specific module of a plurality of processing modules (216) of the transformer-based model (108), wherein each module consists of a plurality of data arrays (222);generating a stack of the plurality of processing modules (218) in accordance with the generated score map; and injecting the stack of the plurality of processing modules (218) into the transformer-based model (108).

2. The method (300) of claim 1, wherein the embedding layer (210) creates an output feature map according to the input samples.

3. The method (300) of claim 1, wherein the plurality of processing modules (216) are a bank of positional encodings.

4. The method (300) of claim 1 , wherein the receiving step includes receiving a bank of positional encodings related to the data samples.

5. The method (300) of claim 1, wherein different positional encoding sequences are attributed to different input classes of the input data samples.

6. The method (300) of claim 5, wherein, in a training phase, a loss function of the transformer-based model (108) includes a regularization term to force a prediction to create evenly distributed data subsets for each element in each of the positional encoding sequences.

7. The method (300) of claim 1 , wherein the node of the data communications network is a terminal device.

8. The method (300) of claim 7, wherein the terminal device generates the score map (214).

9. The method (300) of claim 8, wherein the terminal device transmits an output of the embedder layer (210) to a Base Station, BS.

10. The method (300) of claim 8, wherein the terminal device transmits an index of the score map (214) to the BS.

11. The method (300) of claim 8, wherein the terminal device generates a stack of the plurality of processing modules (218) and combines them with the output of the embedding layer (210).

12. The method (300) of claim 11, wherein the terminal device sends the combined signals to the BS.

13. A system (100) comprising means adapted for carrying out all the steps of the method (300) according to any preceding method claim.

14. A computer program comprising instructions for carrying out all the steps of the method (300) according to any preceding method claim, when said computer program is executed on a computer system.