Big model distributed collaborative inference method and system based on 5g-a endogenous intelligent network element

Abstract

The present application belongs to the technical field of mobile communication and artificial intelligence, and specifically relates to a large model distributed collaborative inference method and system based on 5G-A endogenous intelligent network element, which is constructed on a 5G-A network composed of a user equipment (UE), a 5G base station, a session management function (SMF) and an AI inference network element (AIF), and completes adaptive distributed parallel mode switching scheduling through inference service initialization and heterogeneous network element fragmentation deployment, control plane QoS flow guidance and session establishment, AIF network element prequel layer relay calculation, intelligent N3 link tensor serialization transmission, base station subsequent layer calculation and Token autoregressive generation. The present application is strictly based on the 5G-Advanced network endogenous intelligent architecture design, the large language model is split by layer, and is deeply integrated into the 5G wireless access network and the core network user plane, so that low-delay, high-robust and privacy-protected edge collaborative inference is realized.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of mobile communication and artificial intelligence, specifically relating to a large-scale distributed collaborative reasoning method and system based on 5G-A endogenous intelligent network elements. Background Technology

[0002] With the explosive growth of large language models based on the Transformer architecture, achieving low-latency, high-privacy large model inference at the network edge has become a core requirement for scenarios such as intelligent connected vehicles, industrial control, and edge-side intelligent assistants. Current edge inference and distributed large model deployments mainly exhibit the following typical architectures:

[0003] 1. High-performance cluster deployment on the cloud: The terminal device forwards the request to the remote cloud GPU cluster through the wide area network. After the cloud computing is completed, the token sequence is returned to the terminal.

[0004] 2. Pure On-device Local Inference: Local single-machine inference is performed on mobile phones, personal computers, or isolated edge devices by compressing the model through techniques such as model quantization and pruning.

[0005] 3. External Mobile Edge Computing (MEC): An independent MEC server is mounted outside the 5G core network to carry inference traffic through a standard IP channel.

[0006] The shortcomings and pain points of existing technologies:

[0007] 1. Cloud-based deployments suffer from high network latency and pose serious privacy risks: Long-distance WAN links involve multi-hop routing, resulting in significant network latency and jitter, severely impacting the first-word response time (TTFT) of interactive large-scale models. Furthermore, user-inputted prompts contain a large amount of sensitive information, and uploading them to external public clouds can easily lead to data privacy leaks.

[0008] 2. Hardware bottlenecks on edge devices prevent them from supporting large parameter models: The memory capacity and computing power of edge standalone devices and mobile terminals are extremely limited. When faced with models with 7B, 13B or even larger parameter counts, there are problems such as "not being able to fit" or extremely slow inference speed.

[0009] In summary, 5G's inherent communication network elements suffer from idle computing power and a lack of intrinsic AI inference architecture: In the existing 5G-A network architecture, network elements such as base stations and user plane functions deploy a large amount of hardware computing power (such as high-performance general-purpose CPUs, DSPs, or FPGAs), but this computing power is completely confined to native communication functions (such as signal processing, protocol parsing, and data routing and forwarding). Due to the lack of intrinsic inference architecture design, network elements cannot parse the characteristics of inference tasks, resulting in the isolation between network infrastructure and computing infrastructure, and the huge amount of idle computing power at the edge cannot be effectively utilized. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention provides a large-scale distributed collaborative reasoning method and system based on 5G-A endogenous intelligent network elements.

[0011] On one hand, this invention provides a large-scale distributed collaborative reasoning method based on 5G-A endogenous intelligent network elements, comprising:

[0012] S1. System initialization and sharded deployment:

[0013] Based on the 5G-A network architecture, an endogenous distributed inference system is constructed, consisting of User Equipment (UE), base station gNB, Session Management Function (SMF), and AI Inference Network Element (AIF) integrated into the UPF. The scheduling module collects the memory, computing power, and N3 link bandwidth resources of gNB and AIF, and vertically divides the large language model with a total of L layers into a preorder K layer and a postorder LK layer. The parameters of the preorder K layer model are loaded into the AIF, and the parameters of the postorder LK layer model are loaded into gNB. A lightweight inference engine is built in.

[0014] S2. Control plane inference flow orientation guidance:

[0015] The UE initiates a large model inference session and uploads a Prompt data stream; the SMF identifies the data stream as a dedicated QoS stream for AI inference and sends N4 control rules to the UPF. The AIF intercepts the inference data packets, while regular service traffic is still transmitted through the UPF's native forwarding channel.

[0016] S3, AIF performs pre-model layer calculations:

[0017] AIF parses the UE input message to obtain the Token sequence, and uses continuous batch processing to complete the forward propagation of the pre-K layers in the Prefill stage, generating an intermediate activation tensor with dimensions [b, s, h]; where b is the batch size, s is the sequence length, and h is the model hidden layer dimension;

[0018] S4, Intelligent N3 Interface Tensor Encapsulation Transmission:

[0019] The N3 interface between the gNB and the UPF is extended with GTP-U. Inference metadata is embedded in the GTP-U extension header. The inference metadata includes the model ID, hierarchical index K, batch size b, and micro-batch index. The intermediate activation tensor is serialized and encapsulated into an extended GTP-U message, which is then transmitted to the target gNB via the N3 link.

[0020] S5, Base Station relay completes remaining inference and Token generation:

[0021] The gNB parses the GTP-U extended header to extract inference metadata, deserializes the intermediate activation tensor and binds it to the corresponding user inference context; the lightweight inference engine performs forward computation of the subsequent LK layer, generates a single-step token via the decoding layer; the token is returned to the UE through the 5G downlink air interface, and the context is retained to perform Decode iteration in a loop until the end symbol is generated;

[0022] S6, Adaptive Distributed Parallel Mode Switching Scheduling:

[0023] The scheduling module monitors the N3 link bandwidth B(t) and link jitter J(t) in real time, and dynamically switches between pipelined parallel (PP) mode and tensor parallel (TP) mode based on the communication quantization formula.

[0024] When link bandwidth is insufficient or jitter exceeds the limit, PP mode is enabled, and the communication volume per iteration is VPP. total =b×s×h×d, and simultaneously enable micro-batch scheduling to achieve overlap of computation and communication timing, where d is the length of a single data type byte;

[0025] When gNB and AIF are co-located and link bandwidth B(t) ≥ B thresh And the jitter J(t) ≤ J thresh When switching to TP mode, TP single-layer communication VTP layer =4bshd, total traffic VTP total =4LTP bshd Among them, B thresh J represents the bandwidth threshold. thresh This indicates the jitter threshold.

[0026] On the other hand, the present invention provides a large-scale distributed collaborative reasoning system based on 5G-A endogenous intelligent network elements, comprising:

[0027] User equipment (UE), as the inference request initiator, transmits and receives inference input prompt and generated token via 5G air interface;

[0028] The base station gNB has a built-in lightweight inference engine that stores and executes post-layering of large models. It also integrates the N3 extended message unpacking module to complete tensor deserialization and post-inference computation.

[0029] The Session Management Element (SMF), a control plane policy scheduling unit, identifies AI inference QoS flows and issues N4 rules to achieve targeted interception of inference traffic;

[0030] The AI ​​inference network element AIF is integrated and deployed on the UPF server. It has a built-in high-performance inference engine, tensor serialization module, and context cache module, which can carry out the pre-sequence hierarchical calculation of large models and generate intermediate activation tensors.

[0031] Upgrade the intelligent N3 interface, expand the GTP-U message header, support embedded inference metadata, and serve as a high-dimensional tensor transmission channel between AIF and gNB;

[0032] The adaptive scheduling module, deployed on the AIF or SMF side, collects N3 link bandwidth and jitter indicators in real time, dynamically switches between TP and PP parallel modes based on the communication quantization formula, and issues micro-batch scheduling instructions.

[0033] The beneficial effects of this invention are:

[0034] 1. Construct an inherently intelligent network architecture to activate idle computing power in network elements: Breaking away from the traditional model of deploying large models in the cloud or using external MECs, this approach deeply integrates large models into the 5G Radio Access Network (RAN) and User Plane Network Elements (UPF) by dividing them into layers. Without disrupting or modifying the 3GPP standard 5G session framework, this achieves the inherent transformation of network infrastructure into "intelligent computing nodes," fully exploring and pooling physically adjacent and idle computing resources in the gNB and core network UPF.

[0035] 2. Enables cross-network element pipelined relay computing to ensure data privacy and security: The large model is vertically divided into a pre-processor layer and a subsequent layer. The first half of the model computation is performed by a new user plane network element integrating AI inference capabilities. Then, the intermediate activation tensor is seamlessly and efficiently routed and transmitted to the base station through the upgraded and expanded intelligent N3 data interface. The base station completes the remaining model layer computation and token generation. This ensures that the entire autoregressive inference lifecycle is completely kept within the local edge network domain, fundamentally reducing the risk of external leakage of user sensitive prompts and generated content.

[0036] 3. An adaptive collaborative scheduling strategy based on quantitative communication theory is provided: Addressing the characteristics of 5G-A edge backhaul networks—limited bandwidth, variable channels, and inherent air interface jitter—a single-step autoregressive iterative quantitative calculation model for communication throughput is built into the system scheduling. Through theoretical formula comparison (proving that TP's communication throughput is linearly proportional to the number of layers and is much greater than PP's), intelligent dynamic degradation and on-demand selection of the distributed parallel mode are achieved. Micro-batch scheduling hides communication latency, ensuring extremely high latency robustness of the system under limited network resources.

[0037] 4. Achieving a fully intrinsic closed loop for edge data, providing communication-level privacy and security: Traditional solutions require high-frequency uploading of users' original sensitive prompts and complete autoregressive context to external public clouds, posing a serious risk of privacy leakage. This invention, through an intrinsic network element architecture, enables the entire lifecycle of user inference requests to be completed within the local 5G-A LAN. Traffic exposed to external networks is zero, achieving communication-level hard isolation of privacy.

[0038] 5. Pooling Network Element Computing Power to Achieve Zero Hardware Upgrade Commercial Costs: This invention utilizes underutilized idle computing resources in existing base station servers and core network general-purpose servers for virtualization, pooling, and relay computing. It eliminates the need for additional investment in expensive dedicated AI computing cards at the edge or reconstruction of the physical network. Through purely software-level intrinsic extensions of standard network element functions, it enables distributed existing network devices to collaboratively support edge inference services involving billions of models with extremely low energy efficiency, demonstrating significant industrial application and commercial value. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of edge network element model segmentation and reasoning in this invention;

[0040] Figure 2 This is a diagram showing the 5G-A intrinsic intelligent network architecture and the logical structure of each control plane / user plane network element of the present invention;

[0041] Figure 3 This is the end-to-end timing flow diagram of cross-network element collaborative inference in this invention;

[0042] Figure 4 This is a performance comparison chart of edge distributed inference and simulated cloud distributed inference of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] A large-scale distributed collaborative reasoning method based on 5G-A endogenous intelligent network elements includes:

[0045] Cross-network element distributed LLM collaborative inference:

[0046] like Figure 1 As shown, this embodiment is built on a 5G-A network consisting of User Equipment (UE), 5G base stations, Session Management Function (SMF), and AI Inference Network Element (AIF). The system pre-loads a large language model network layer with a total of L layers, which are vertically segmented by layer.

[0047] The specific implementation process of this embodiment includes the following core steps, such as: Figure 2 As shown, the 5G-A intrinsic intelligent network architecture and the logical structure diagram of each control plane / user plane network element demonstrate how the large model inference flow is dynamically intercepted and redirected by the N4 rules issued by the control plane network element SMF. This allows the AIF network element, integrated with a general-purpose server, to take over the execution of the large model's pre-processing layer computation using technologies such as continuous batch processing. Furthermore, the upgraded and extended intelligent N3 interface seamlessly routes the intermediate activation tensor to the lightweight AI inference runtime inherent in the 5G base station. The base station then takes over to complete the large model's post-processing layer forward computation and the local integrated closed-loop generation of the final token.

[0048] Inference service initialization and heterogeneous network element sharding deployment:

[0049] AIF dynamically obtains the hardware resource status of each network element node, including available video memory, computing power, and current network link bandwidth.

[0050] Based on network element resource constraints, the LLM model with L layers is vertically divided into two parts: the pre-sequence layer (layers 1 to K) and the subsequent layer (layers K+1 to L).

[0051] The scheduling module uses the task binding interface of the Ray distributed framework to force the model parameters of the preceding layers 1 to K, including weight matrices and bias terms, to be loaded locally in the AIF network element integrated in the UPF; and to load the model parameters of the subsequent layers K+1 to L locally in the lightweight inference engine of the 5G base station cluster.

[0052] Control plane QoS flow guidance and session establishment:

[0053] User equipment initiates an intelligent inference application session for a large language model through a standard 5G wireless access link, such as sending a data stream containing user prompts.

[0054] The control plane network element SMF dynamically identifies that the data flow belongs to the preset AI inference QoS flow, and issues the corresponding N4 control plane rules to the user plane according to the characteristics of the flow.

[0055] According to the N4 rule, the AIF network element automatically intercepts inference application data packets destined for or originating from the UE and redirects them to the inference pipeline built into the AIF, while non-inference regular data packets in the network continue to be forwarded quickly through the traditional data channel of the UPF.

[0056] AIF network element's preceding layer relay calculation:

[0057] The AIF network element receives IP packets from the standard user plane, parses them, and converts them into a sequence of input tokens that can be recognized by the large model.

[0058] AIF network elements utilize locally loaded pre-order model parameters from layers 1 to K for forward propagation calculation (Prefill phase), while simultaneously managing the context of concurrent users using continuous batch processing technology.

[0059] After the Kth layer of computation, AIF generates an intermediate activation tensor at the stage boundary, with a basic dimension size of [b,s,h] under multiple concurrent and single autoregressive iteration steps.

[0060] Tensor serialization transmission of the intelligent N3 link:

[0061] The AIF network element serializes and encapsulates the generated intermediate activation tensor.

[0062] The upgraded N3 interface uses the GTP-U protocol to extend the header, embedding inference metadata, including model ID, layer split point index K, current batch size b, micro-batch index, etc., into the GTP-U header.

[0063] The encapsulated tensor data packet is sent to the target base station via the N3 backhaul link.

[0064] Subsequent layer computations of the base station and autoregressive token generation, such as... Figure 3As shown, the control plane SMF issues rules to the user plane AIF network element through the N4 interface and accurately captures and intercepts the inference flow initiated by the UE. Subsequently, the AIF network element initiates continuous batch processing and executes locally hosted pre-order model layer calculations. After generating a high-dimensional activation tensor at the stage boundary, it is encapsulated in an extended header along with inference metadata and transmitted through the N3 link. Finally, the front-end 5G base station receives and deserializes the tensor, restores the corresponding user session context, takes over, executes the remaining post-order model layer calculations, and generates a new token. This token is then directly fed back to the UE for presentation through the standard downlink air interface, and the state is automatically switched to the Decode iteration mode with a constant input length of 1 until the end symbol is generated.

[0065] The base station receives the tensor data packet through the upgraded N3 interface. The hardware-level unpacking module reads the inference metadata in the GTP-U extended header and accurately maps the deserialized high-dimensional activation tensor to the inference runtime context of the user session.

[0066] The base station's lightweight inference engine takes over the computation and performs the remaining forward propagation of the model from layer K+1 to layer L.

[0067] After the final language model decoding head completes its calculations, the base station generates the next new token for the current prediction.

[0068] The base station returns the newly generated token directly to the user equipment through the 5G downlink standard user plane channel, and at the same time synchronizes or retains the state of the token for the next autoregressive iterative Decode stage until the end symbol is output.

[0069] Adaptive selection of distributed parallel strategies in heterogeneous network environments:

[0070] This embodiment demonstrates a specific implementation method for dynamically switching distributed inference modes when faced with varying 5G backhaul network quality. The AIF inference engine periodically executes the following algorithm steps:

[0071] Link quality monitoring: The scheduling module measures the available bandwidth B(t) and latency jitter J(t) of the N3 backhaul link between the base station gNB and the core network UPF-AIF in real time.

[0072] Theoretical Quantization Calculation of Traffic Volume: If the current concurrent batch size is b, the input sequence length is s, the hidden layer dimension is h, the data type bytes are d, and the system chooses tensor parallelism and the number of deployment layers is... The system then uses the formula to quantitatively calculate the unidirectional core communication volume generated in a single autoregressive iteration across the entire network:

[0073]

[0074] If the system selects pipelined parallelism (PP), then the boundary communication volume is quantitatively calculated according to the formula:

[0075]

[0076] To adapt to the variable channel quality and limited backhaul bandwidth among 5G-A edge network elements, this invention designs an adaptive distributed collaborative inference strategy and conducts a rigorous theoretical and quantitative analysis of the communication volume of two distributed modes, tensor parallelism and pipeline parallelism, under a single autoregressive iteration:

[0077] Let the batch size of concurrent user requests be b, the length of the current input sequence be s (the length of the Prompt Token in the Prefill stage of the first character generation, and a constant value of 1 in the Decode stage of subsequent Token generation), the hidden layer dimension of the LLM model be h, the total number of network layers be L, and the number of bytes of data type be d (d=2 bytes in FP16 precision).

[0078] Traffic analysis in Tensor Parallel (TP) mode:

[0079] If the model is split across two nodes, gNB and AIF, according to the operator dimension (TP degree is 2), and following the Megatron-LM style partitioning, a single-layer Transformer contains two core modules: multi-head self-attention and a feedforward network. Specifically, the Q, K, V projection matrices of MHA are partitioned by column, and the output projection matrix is ​​partitioned by row; the first fully connected layer of FFN is partitioned by column, and the second fully connected layer is partitioned by row. In one round of forward propagation, the row partitioning operator must call an All-Reduce operation between the two nodes to recover the complete activation values; a single layer requires two All-Reduce synchronizations. In a two-node interconnected network, the amount of unidirectional communication data generated by a standard All-Reduce operation is equal to the size of the transmission matrix (b×s×h×d). Therefore, the formula for the amount of unidirectional communication data generated between network elements by a single-layer Transformer is:

[0080]

[0081] If all network elements share an LTP layer model that uses tensor parallel deployment, then the total core unidirectional communication volume VTP_total generated by a single forward autoregressive iteration among multiple network elements satisfies:

[0082]

[0083] Conclusion Analysis: Communication Volume and Model Layer Number in TP Mode It is linearly proportional. Due to the air interface jitter and inherent transmission delay in the 5G-A backhaul network, the high-frequency (twice per layer) All-Reduce synchronization can easily cause the computing threads to be frequently blocked and waiting, which places extremely high and demanding requirements on the backhaul network bandwidth.

[0084] (2) Traffic analysis of pipelined parallel (PP) mode:

[0085] In pipelined parallel mode, the model is vertically partitioned into layers. In the pipelined architecture designed in this scheme, the preceding network element runs the first K layers, and the subsequent network element runs the remaining L−K layers. Inter-node intermediate activation value matrix transmission occurs only once at the stage boundary (i.e., the boundary between the Kth layer and the K+1th layer), with matrix dimensions [b, s, h]. Therefore, the formula for the total unidirectional pipelined parallel communication volume of a single forward autoregressive iteration step across heterogeneous network elements is:

[0086]

[0087] Conclusion Analysis: The core communication of PP mode is completely independent of the number of model layers L, and is only related to the activation value scale of the layer group boundary.

[0088] (3) Adaptive scheduling strategy:

[0089] Based on the above analysis, the communication volume ratio between TP and PP under the same concurrency and sequence scale can be derived as follows:

[0090]

[0091] because yes of The pipelined parallelism decouples network communication to the layer group boundary, significantly reducing the bandwidth overhead of the N3 interface in the backhaul network. This invention establishes adaptive control based on this quantization formula: the scheduling module monitors the bandwidth and jitter of the N3 link in real time. When network bandwidth is limited or jitter is high, the system automatically switches to pipelined parallel mode and uses micro-batch pipeline scheduling to overlap computation and communication, hiding communication latency. TP mode is only activated when network elements are co-located and have ultra-high bandwidth interconnection, fundamentally ensuring the system's latency robustness in the 5G-A edge environment.

[0092] Adaptive handover decision:

[0093] PP mode activation: When the network environment is detected to be a typical edge backhaul scenario, i.e., bandwidth is limited or jitter is high, the communication volume ratio between the two modes is increased. The system automatically switches to or remains in pipelined parallel mode. At this point, the system further activates a micro-batch scheduling strategy, causing the computation time of the gNB to overlap with the tensor transfer time of the N3 interface on the time axis, thereby hiding communication latency.

[0094] TP mode activation: When it is detected that the base station and UPF are co-located or have a dedicated fiber optic interconnection with ultra-high bandwidth and extremely low latency, the following conditions are met. and At this time, the system automatically switches to tensor parallel mode, splitting each operator to be computed simultaneously at both ends and frequently calling All-Reduce to fully utilize the parallel computing power of multiple nodes. Among them, B thresh J represents the bandwidth threshold. thresh This indicates the jitter threshold.

[0095] The AIF network element is deployed on the 5G core network UPF general server. While fully retaining the UPF data packet routing and forwarding capabilities, it also inherently integrates a large model inference runtime. The AIF has a built-in context management module for parsing N3 extended header metadata and mapping the received serialized tensors to the corresponding inference session context.

[0096] In the tensor parallel TP mode, the single-layer Transformer multi-head attention (MHA) and feedforward network (FFN) of the 5G-A network architecture adopt the Megatron-LM partitioning strategy; the QKV projection of MHA is partitioned by column and the output matrix is ​​partitioned by row; the first layer of FFN is fully connected by column partitioning and the second layer is fully connected by row partitioning; a single layer completes two All-Reduce synchronization operations to restore the complete activation value.

[0097] The communication throughput in PP mode is only related to the dimension of the layer boundary activation tensor and is independent of the total number of layers L; the total communication throughput in TP mode is related to the number of parallel layers L. TP The ratio of TP to PP communication traffic is linearly proportional, with R being the ratio of TP to PP communication traffic. comm =4L TP , where R comm L represents the communication volume ratio between the two modes. TP The number of deployment layers.

[0098] The model's hierarchical pipeline can be expanded in multiple levels; the large model is divided into three or more segments, which are deployed on UEs, multiple base stations, and multiple levels of UPF-AIF network elements to form a multi-level pipeline collaborative inference architecture.

[0099] A large-scale distributed collaborative reasoning system based on 5G-A endogenous intelligent network elements includes: This architecture, without disrupting the 3GPP standard 5G session framework and control plane interaction process, divides the computational responsibilities of the large language model into layers and deploys them on various heterogeneous network elements within the 5G-A network.

[0100] User Equipment (UE): As the initiator of the inference request, it receives the generated token through the standard 5G user plane. The entire inference lifecycle is presented to the UE as a standard IP application session. All sensitive prompts and autoregressive context intermediate states are retained within the local edge domain to avoid exposure to the external public cloud.

[0101] 5G base station (gNB): Serves as the front-end executor for collaborative inference. The gNB natively integrates a lightweight inference engine, which is responsible for admission control of user requests, lightweight preprocessing (including prompt word parsing and tokenization / embedding calculation), and is responsible for executing the first few layers of forward propagation of the large language model to generate intermediate activation tensors.

[0102] 5G Session Management Function (SMF): Responsible for policy orchestration and intelligent flow guidance on the control plane. By identifying specific Quality of Service flows belonging to large model inference tasks, SMF dynamically issues N4 control rules to the user plane, accurately guiding target inference traffic to AI inference function network elements, while performing traditional routing on regular non-inference data packets in the network, ensuring zero disruption to existing network infrastructure.

[0103] Upgraded N3 Data Plane Interface: The traditional N3 interface between gNB and UPF only performs pure IP datagram tunneling encapsulation. This invention extends and upgrades the N3 interface protocol, making it an intelligent link capable of carrying high-dimensional tensors and inference metadata. By embedding inference metadata such as model ID, layer index, and micro-batch index in the GTP-U extended header, seamless and efficient serialized transmission of intermediate activation tensors across network elements is achieved.

[0104] The entire inference process is closed-loop within the 5G-A local edge domain. The user's original prompt, inference intermediate tensors, and context key-value cache do not flow out of the local area network, and no data is uploaded to the external public cloud.

[0105] The adaptive scheduling module sets a bandwidth threshold B. thresh Jitter threshold J thresh When the link metrics exceed the threshold, the pipeline parallel (PP) mode is automatically locked, and the tensor parallel (TP) mode is only enabled when the link metrics are better than the threshold.

[0106] This invention proposes a novel intelligent large-scale model inference network element—AIF—based on UPF integration. This network element connects the network data forwarding plane and the intelligent computing plane. Its key design features are as follows:

[0107] Establishing an intrinsic computing power anchor point: Unlike traditional MEC solutions that externally mount distributed servers, AIF is directly deployed and resides on the general-purpose server of the user plane network element in the 5G core network. As the AI ​​computing anchor point and backend executor on the edge side, AIF retains the high-throughput packet routing and forwarding capabilities of the standard UPF while intrinsically integrating a high-performance large-model inference engine.

[0108] Tensor Pipeline and Inference Context Management: AIF manages the latter half of the large language model's model fragmentation and completes the final autoregressive decoding. It can intercept pipelined inference metadata contained in the GTP-U extension header from the upgraded N3 interface in real time. By parsing the model ID, micro-batch index, and sequence state in the metadata, AIF dynamically and accurately maps the received serialized intermediate activation tensors to the correct inference runtime context, completing subsequent model layer forward computation.

[0109] In this embodiment, the reasoning method of the present invention is simulated experimentally:

[0110] A performance comparison of edge distributed inference and simulated cloud distributed inference, such as Figure 4 As shown in Figure (d), under the same model scale, the word generation latency (TPOT) at the edge and the cloud are almost identical, confirming that the computing power of the cloud and the edge is equivalent, thus pinpointing the performance bottleneck precisely to the network transmission link. In a concurrent environment of 32, in Figure (a), for the 1.5B model, the cloud latency is 4.2 times that at the edge, and for the 7B model, the Mean TTFT at the cloud is 4.7 times that at the edge, indicating that the inference latency at the edge is much lower than that at the cloud, greatly optimizing the user experience. Figure (c) shows that TTFTJitter reflects the instability of the cloud network link. The jitter of the 7B model in the cloud environment is as high as 317.90 ms, while that at the edge is only 95.79 ms. In Figure (b), the edge long-tail latency of the 1.5B model is reduced by 63.9% compared to the cloud, and the edge long-tail latency of the 7B model is reduced by 79.72% compared to the cloud. The stress curves in Figures (e) and (f) show that between concurrency levels of 14 and 32, the TTFT increases slowly with increasing concurrency, demonstrating good scheduling capabilities. When concurrency exceeds 32, latency for both Edge and Cloud experiences an exponential spike. This "inflection point" signifies that GPU computing power or memory bandwidth has reached its physical limit. In 5G edge network element capacity planning, 32 is the "critical concurrency capacity" for this configuration.

[0111] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A large-scale distributed collaborative reasoning method based on 5G-A endogenous intelligent network elements, characterized in that, include: S1. System initialization and sharded deployment: Based on the 5G-A network architecture, an endogenous distributed inference system is constructed, consisting of User Equipment (UE), base station gNB, Session Management Function (SMF), and AI Inference Network Element (AIF) integrated into the UPF. The scheduling module collects the memory, computing power, and N3 link bandwidth resources of gNB and AIF, and vertically divides the large language model with a total of L layers into a preorder K layer and a postorder LK layer. The parameters of the preorder K layer model are loaded into the AIF, and the parameters of the postorder LK layer model are loaded into gNB. A lightweight inference engine is built in. S2. Control plane inference flow orientation guidance: The UE initiates a large model inference session and uploads a Prompt data stream; the SMF identifies the data stream as a dedicated QoS stream for AI inference and sends N4 control rules to the UPF. The AIF intercepts the inference data packets, while regular service traffic is still transmitted through the UPF's native forwarding channel. S3, AIF performs pre-model layer calculations: AIF parses the UE input message to obtain the Token sequence, and uses continuous batch processing to complete the forward propagation of the pre-K layers in the Prefill stage, generating an intermediate activation tensor with dimensions [b, s, h]; where b is the batch size, s is the sequence length, and h is the model hidden layer dimension; S4, Intelligent N3 Interface Tensor Encapsulation Transmission: The N3 interface between the gNB and the UPF is extended with GTP-U. Inference metadata is embedded in the GTP-U extension header. The inference metadata includes the model ID, hierarchical index K, batch size b, and micro-batch index. The intermediate activation tensor is serialized and encapsulated into an extended GTP-U message, which is then transmitted to the target gNB via the N3 link. S5, Base Station relay completes remaining inference and Token generation: The gNB parses the GTP-U extended header to extract inference metadata, deserializes the intermediate activation tensor and binds it to the corresponding user inference context; the lightweight inference engine performs forward computation of the subsequent LK layer, generates a single-step token via the decoding layer; the token is returned to the UE through the 5G downlink air interface, and the context is retained to perform Decode iteration in a loop until the end symbol is generated; S6, Adaptive Distributed Parallel Mode Switching Scheduling: The scheduling module monitors the N3 link bandwidth B(t) and link jitter J(t) in real time, and dynamically switches between pipelined parallel (PP) mode and tensor parallel (TP) mode based on the communication quantization formula. When link bandwidth is insufficient or jitter exceeds the limit, PP mode is activated, and the communication volume V per iteration is reduced. PP-total =b×s×h×d, and simultaneously enable micro-batch scheduling to achieve overlap of computation and communication timing, where d is the length of a single data type byte; When gNB and AIF are co-located and link bandwidth B(t) ≥ B thresh And the jitter J(t) ≤ J thresh When switching to TP mode, the TP single-layer communication volume V TP-layer =4bshd, total communication volume V TP-total =4L TP-bshd Among them, B thresh J represents the bandwidth threshold. thresh L represents the jitter threshold. TP Indicates the number of model layers.

2. The large-model distributed collaborative reasoning method based on 5G-A endogenous intelligent network elements according to claim 1, characterized in that, The AIF network element is deployed on the 5G core network UPF general server. While fully retaining the UPF data packet routing and forwarding capabilities, it also inherently integrates a large model inference runtime. The AIF has a built-in context management module for parsing N3 extended header metadata and mapping the received serialized tensors to the corresponding inference session context.

3. The large-model distributed collaborative reasoning method based on 5G-A endogenous intelligent network elements according to claim 1, characterized in that, In the tensor parallel TP mode, the single-layer Transformer multi-head attention (MHA) and feedforward network (FFN) of the 5G-A network architecture adopt the Megatron-LM partitioning strategy; the QKV projection of MHA is partitioned by column and the output matrix is ​​partitioned by row; the first layer of FFN is fully connected by column partitioning and the second layer is fully connected by row partitioning; a single layer completes two All-Reduce synchronization operations to restore the complete activation value.

4. The large-model distributed collaborative reasoning method based on 5G-A endogenous intelligent network elements according to claim 1, characterized in that, The communication throughput in PP mode is only related to the dimension of the layer boundary activation tensor and is independent of the total number of layers L; the total communication throughput in TP mode is related to the number of parallel layers L. TP The ratio of TP to PP communication traffic is linearly proportional, with R being the ratio of TP to PP communication traffic. comm =4L TP , where R comm L represents the communication volume ratio between the two modes. TP The number of deployment layers.

5. The large-model distributed collaborative reasoning method based on 5G-A endogenous intelligent network elements according to claim 1, characterized in that, The model's hierarchical pipeline can be expanded in multiple levels; the large model is divided into three or more segments, which are deployed on UEs, multiple base stations, and multiple levels of UPF-AIF network elements to form a multi-level pipeline collaborative inference architecture.

6. A large-scale distributed collaborative reasoning system based on 5G-A endogenous intelligent network elements, used to implement the collaborative reasoning method as described in any one of claims 1 to 5, characterized in that, include: User equipment (UE), as the inference request initiator, transmits and receives inference input prompt and generated token via 5G air interface; The base station gNB has a built-in lightweight inference engine that stores and executes post-layering of large models. It also integrates the N3 extended message unpacking module to complete tensor deserialization and post-inference computation. The Session Management Element (SMF), a control plane policy scheduling unit, identifies AI inference QoS flows and issues N4 rules to achieve targeted interception of inference traffic; The AI ​​inference network element AIF is integrated and deployed on the UPF server. It has a built-in high-performance inference engine, tensor serialization module, and context cache module, which can carry out the pre-sequence hierarchical calculation of large models and generate intermediate activation tensors. Upgrade the intelligent N3 interface, expand the GTP-U message header, support embedded inference metadata, and serve as a high-dimensional tensor transmission channel between AIF and gNB; The adaptive scheduling module, deployed on the AIF or SMF side, collects N3 link bandwidth and jitter indicators in real time, dynamically switches between TP and PP parallel modes based on the communication quantization formula, and issues micro-batch scheduling instructions.

7. A large-model distributed collaborative reasoning system based on 5G-A endogenous intelligent network elements according to claim 6, characterized in that, The AIF network element hardware is compatible with any one or more heterogeneous computing acceleration cards such as GPU, NPU, FPGA, and ASIC, and is used to accelerate the inference computation of the pre-sequence layer of large models.

8. A large-model distributed collaborative reasoning system based on 5G-A endogenous intelligent network elements according to claim 6, characterized in that, The entire inference process is closed-loop within the 5G-A local edge domain. The user's original prompt, inference intermediate tensors, and context key-value cache do not flow out of the local area network, and no data is uploaded to the external public cloud.

9. A large-model distributed collaborative reasoning system based on 5G-A endogenous intelligent network elements according to claim 6, characterized in that, The adaptive scheduling module sets a bandwidth threshold B. thresh Jitter threshold J thresh When the link metrics exceed the threshold, the pipeline parallel (PP) mode is automatically locked, and the tensor parallel (TP) mode is only enabled when the link metrics are better than the threshold.

Images