Data processing method, device and system

Abstract

The embodiment of the invention provides a data processing method, device and system, and relates to the technical field of communication. The method comprises the steps that under the condition that a target model reasoning task is triggered, an edge node obtains a service segment identifier (SID) list corresponding to a target model; the hierarchical reasoning node corresponding to the Kth SID in the SID list performs hierarchical processing on the to-be-processed data to obtain intermediate data, and sends an IPv6-based service segment routing SRv6 message to the hierarchical reasoning node corresponding to the (K + 1) th SID in the SID list, and the SRv6 message comprises the SID list and the intermediate data; k is a positive integer from 1 to N-1 until the hierarchical reasoning node corresponding to the last SID in the SID list completes the reasoning task of the target model. Therefore, each hierarchical reasoning node performs cooperative reasoning, the resource utilization rate is improved, the bandwidth consumption and the transmission cost are reduced, the cross-domain transmission frequency is reduced, and the delay is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technology, and in particular to a data processing method, device, and system. Background Art

[0002] In the existing technology, models are often deployed in a centralized mode in the cloud or centralized data center, so that the data to be processed is inferred in the cloud or centralized data center, simplifying the resource management of model inference.

[0003] However, this method requires transmitting the original data to the cloud or centralized data center for processing, resulting in poor real-time data transmission, high bandwidth resource consumption, overloaded cloud computing power, and unbalanced resource utilization. Summary of the Invention

[0004] This application provides a data processing method, device, and system that, when a target model inference task is triggered, can obtain a list of SIDs corresponding to the target model. Based on the multiple hierarchical inference nodes indicated in the SID list, the data to be processed is hierarchically processed to obtain intermediate data, until the hierarchical inference node corresponding to the last SID in the SID list completes the target model inference task. Thus, the hierarchical inference nodes collaborate in inference, improving resource utilization, reducing bandwidth consumption and transmission costs, reducing the number of cross-domain transmissions, and reducing latency.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a data processing method is provided, the method comprising:

[0007] When the target model inference task is triggered, the edge node obtains the service segment identification SID list corresponding to the target model; the SID list includes N SIDs, N is a positive integer, and the SID list is used to indicate multiple hierarchical inference nodes of the target model; the edge node is the hierarchical inference node corresponding to the first SID in the SID list; the hierarchical inference node corresponding to the Kth SID in the SID list performs hierarchical processing on the processed data to obtain intermediate data, and sends an IPv6-based service segment routing SRv6 message to the hierarchical inference node corresponding to the K+1th SID in the SID list, and the SRv6 message includes the SID list and the intermediate data; until the hierarchical inference node corresponding to the last SID in the SID list completes the inference task of the target model, K is a positive integer from 1 to N-1.

[0008] In a possible implementation of the first aspect, a model shard of the target model is deployed in the hierarchical inference node, the SID includes a model hierarchy identifier of the target model, and the hierarchical inference node corresponding to the Kth SID performs hierarchical processing on the data to be processed through the model shard corresponding to the model hierarchy identifier to obtain intermediate data.

[0009] In a possible implementation of the first aspect, a model shard of the target model is deployed in the hierarchical inference node, the SID includes the computing function type of the target model, and the hierarchical inference node corresponding to the Kth SID performs hierarchical processing of the computing function type on the data to be processed to obtain intermediate data.

[0010] In a possible implementation manner of the first aspect, the computing function type of the target model includes: any one or more combinations of preprocessing, model reasoning, or result aggregation.

[0011] In a possible implementation of the first aspect, the SID includes a model identifier of the target model, a hierarchical inference node corresponding to the Kth SID, and hierarchical processing is performed on the data to be processed through the model shards of the target model corresponding to the model identifier to obtain intermediate data.

[0012] In a possible implementation of the first aspect, the SID includes a data compression algorithm identifier, where the data compression algorithm identifier is used to indicate a data compression method of the SRv6 message.

[0013] In a possible implementation of the first aspect, the method further includes: the hierarchical inference node corresponding to the Kth SID encapsulates a service segment routing header (SRH) of the SRv6 message, where the SRH includes a remaining number of hops and a SID list; wherein the remaining number of hops is used to indicate the number of inference nodes that have not performed data processing, and the remaining number of hops is a positive integer; and the next hop is the K+1th SID in the SID list.

[0014] In a possible implementation manner of the first aspect, the method further includes: the hierarchical reasoning node corresponding to the Kth SID encapsulates the payload of the SRv6 message, where the payload includes intermediate data obtained by the hierarchical reasoning node corresponding to the Kth SID performing hierarchical processing on the data to be processed.

[0015] In a possible implementation of the first aspect, the method also includes: the SRv6 controller obtains status information of each candidate node; the status information includes the computing load and network link status of each candidate node; the SRv6 controller determines multiple hierarchical inference nodes corresponding to the target model based on the status information, the multiple hierarchical inference nodes are used for collaborative reasoning, the target model includes multiple model shards, and the multiple model shards are respectively deployed to the corresponding hierarchical inference nodes based on the computing capabilities of the multiple hierarchical inference nodes; the SRv6 controller generates an SID list based on the multiple hierarchical inference nodes.

[0016] In a possible implementation of the first aspect, determining, based on the state information, multiple hierarchical reasoning nodes corresponding to the target model includes:

[0017] Determine multiple reasoning nodes according to formula 1;

[0018] min(α(T_MAX-T_E2E)+β(1-max(U_i))+γ(B_saved)) Formula 1;

[0019] Among them, α, β, γ are weight coefficients, T_E2E is the delay between nodes in the network link state, U_i is the resource utilization of the node in the computing load, and B_saved is the bandwidth saved by data compression.

[0020] In a possible implementation of the first aspect, the method further includes: when the computational load of the hierarchical reasoning node corresponding to the Kth SID is greater than or equal to a first threshold, or the inference time of the hierarchical reasoning node corresponding to the Kth SID is greater than or equal to a second threshold, the SRv6 controller determines a new hierarchical reasoning node and replaces the hierarchical reasoning node corresponding to the Kth SID in the SID list with the new hierarchical reasoning node.

[0021] In a possible implementation of the first aspect, the hierarchical reasoning nodes of the target model include edge nodes, regional nodes and cloud nodes, and the method includes: the edge node preprocesses the data to be processed and / or performs low-level model reasoning, and sends a first SRv6 message to the regional node; the first SRv6 message includes a SID list and data after low-level model reasoning; the regional node decapsulates the first SRv6 message and performs middle-level model reasoning, and sends a second SRv6 message to the cloud node; the second SRv6 message includes a SID list and data after middle-level model reasoning; the cloud node decapsulates the second SRv6 message and performs high-level model reasoning, and completes the reasoning task of the target model.

[0022] In a possible implementation of the first aspect, the SID includes a first field, a second field, a third field and a fourth field; the first field is used to indicate the address information of the hierarchical reasoning node; the second field is used to indicate the computing function type identifier of the target model and / or the model identifier of the target model; the third field is used to indicate the model hierarchy identifier of the target model; and the fourth field is used to indicate additional parameters for model reasoning.

[0023] In a second aspect, a communication system is provided, including: an SRv6 controller and a hierarchical reasoning node, the hierarchical reasoning node including an edge node, a regional node and a cloud node, the SRv6 controller and the hierarchical reasoning node are used to execute the method in the above-mentioned first aspect and any possible implementation method of the first aspect.

[0024] In a third aspect, a communication device is provided, which includes: at least one communication unit and a processing unit; the communication unit and the processing unit are used to execute the method executed by any device in any of the above aspects and any possible implementation of the aspect.

[0025] In a fourth aspect, a communication device is provided, comprising: at least one processor and an interface circuit, the interface circuit being used to receive signals from other communication devices outside the communication device and transmit them to the processor or to send signals from the processor to other communication devices outside the communication device, the processor being used to implement the method executed by any device in any of the above aspects and any possible implementation of that aspect through logic circuits or execution code instructions.

[0026] Optionally, the communication device further comprises: a memory for storing program instructions, wherein the processor is coupled to the memory via an interface.

[0027] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program or instruction, and the computer program or instruction is configured to execute a method executed by any device in any of the above aspects and any possible implementation of the aspect.

[0028] In the sixth aspect, a chip is provided, comprising: an interface circuit and a logic circuit, the interface circuit being used to receive signals from other chips outside the chip and transmit them to the logic circuit, or to send signals from the logic circuit to other chips outside the chip, and the logic circuit being used to implement the method executed by any device in any of the above aspects and any possible implementation methods of that aspect.

[0029] In the seventh aspect, a computer program product is provided, which includes: a computer program or instructions, which, when the computer program or instructions are run on a computer, enables the computer to execute any method executed by any device in any of the above aspects and any possible implementation of the aspect.

[0030] In summary, when the target model inference task is triggered, the edge node obtains the SID list corresponding to the target model, and based on the hierarchical inference node corresponding to the Kth SID in the SID list, performs hierarchical processing on the data to be processed, obtains intermediate data, and sends an SRv6 message including the intermediate data and the SID list to the hierarchical inference node corresponding to the K+1th SID in the SID list, so that the hierarchical inference node corresponding to the K+1th SID continues to perform model inference until the hierarchical inference node corresponding to the last SID in the SID list completes the inference task of the target model. Thus, multiple hierarchical inference nodes can collaborate to jointly perform the inference of the target model, improve resource utilization, reduce the load pressure of the central node, and only need to transmit intermediate data between adjacent nodes, without transmitting all the data to be processed, thereby reducing the bandwidth consumption and transmission cost of data transmission. In addition, hierarchical reasoning is combined with SID, and the address information of each hierarchical reasoning node in the model reasoning link is directly carried in the SID of the SRv6 message. The hierarchical reasoning node corresponding to the Kth SID can obtain the information of the next hierarchical reasoning node based on the SID, without having to resend the request to obtain the information of the next hierarchical reasoning node, reducing the number of cross-domain data transmissions, reducing latency, and improving real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a system architecture provided in an embodiment of the present application;

[0032] Figure 2 A signaling interaction diagram of a data processing method provided in an embodiment of the present application;

[0033] Figure 3 A signaling interaction diagram of another data processing method provided in an embodiment of the present application;

[0034] Figure 4 A schematic structural diagram of a communication device provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of the structure of another communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of the embodiments. The following terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. "Multiple" can be understood as "at least two"; "multiple" can be understood as "at least two".

[0037] In the embodiments of the present application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0038] In the embodiments of this application, "and / or" is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0039] In the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "set", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0040] Model reasoning is typically deployed in a centralized fashion in the cloud or a centralized data center, enabling centralized model reasoning and simplifying resource management. While centralized reasoning can simplify resource management, it still presents the following issues:

[0041] First, traditional centralized reasoning requires uploading all raw data (such as video streams and sensor data) to the cloud. This consumes a large amount of data transmission bandwidth from edge nodes to the cloud. Furthermore, the raw data contains a large amount of redundant information (for example, background frames), while actual reasoning may only focus on key frames or local features, resulting in redundant data transmission. Furthermore, model reasoning may be divided into multiple stages, and all raw data must be uploaded during each stage of reasoning. Intermediate results during the reasoning process cannot be uploaded, resulting in high consumption of data transmission bandwidth resources and overloaded cloud computing power.

[0042] Second, model inference latency is high and real-time performance is poor. Raw data must be transmitted across a wide area network (WAN) to the cloud, resulting in long round-trip times. Furthermore, when processing data centrally in the cloud, high concurrency can easily lead to resource contention, resulting in task queuing (for example, GPU instance scheduling delays).

[0043] Third, resource utilization is unbalanced. Edge devices are solely responsible for data collection, underutilizing computing resources. Data is processed centrally in the cloud, placing heavy load on the cloud and requiring frequent capacity expansion, which increases hardware and resource costs.

[0044] Based on this, an embodiment of the present application provides a data processing method. When a target model reasoning task is triggered, the edge node obtains the SID list corresponding to the target model, and based on the hierarchical reasoning node corresponding to the Kth SID in the SID list, performs hierarchical processing on the processed data to obtain intermediate data, and sends an SRv6 message including the intermediate data and the SID list to the hierarchical reasoning node corresponding to the K+1th SID in the SID list, so that the hierarchical reasoning node corresponding to the K+1th SID continues to perform model reasoning until the hierarchical reasoning node corresponding to the last SID in the SID list completes the reasoning task of the target model.

[0045] This data processing method shards the centralized deployment model and deploys it on different nodes, such as edge nodes, regional nodes, and cloud nodes. Based on the segment routing v6 (SRv6) protocol of the IPv6 forwarding plane, it generates a service segment identifier (SID). Different nodes process the model data in stages based on the SID indication and send the processing results to the next node. The nodes collaborate with each other to improve resource utilization, reduce bandwidth consumption and transmission costs, reduce the number of cross-domain transmissions, reduce latency, and improve real-time performance.

[0046] In addition, there are other problems with related technologies:

[0047] Fourth, service rigidity and poor scalability. Traditional centralized reasoning uses a static service chain approach with fixed transmission nodes. It cannot dynamically adjust the computing node path and is difficult to adapt to model changes or node failures.

[0048] Taking the above problems into consideration, the data processing method provided in the embodiment of the present application can comprehensively consider the status information of each node, use a multi-objective optimization model to generate the optimal reasoning strategy, determine the hierarchical reasoning nodes of the optimal path, and dynamically adjust the hierarchical reasoning nodes in real time when the hierarchical reasoning nodes fail or resources are insufficient, thereby improving the efficiency of model reasoning.

[0049] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the schematic flow chart provided in the present application mainly uses the controller and different nodes (for example, edge nodes, regional nodes, and cloud nodes) as examples of the execution subjects of the interactive schematic to illustrate the method, but the present application does not limit the execution subjects of the interactive schematic. For example, the nodes in the schematic flow chart may also be devices, apparatuses, chips, chip systems, or processors that support the node to implement the method, or may be logic modules or software that can implement all or part of the functions of the method.

[0050] See also Figure 1 , Figure 1 A schematic diagram of a system architecture provided in an embodiment of the present application. Figure 1 As shown, the system provided by the embodiment of the present application may include: an SRv6 controller 10, an edge node 20, a regional node 30 and a cloud node 40.

[0051] The SRv6 controller 10 is the core control component of the SRv6 network protocol, responsible for network path calculation, policy issuance, and network status management. The SRv6 controller 10 can include a model sharding management module, a network status monitoring module, a reinforcement learning path decision engine module, and a SID mapping table.

[0052] In some embodiments, the model shard management module is used to manage and deploy model shards. For example, the SRv6 controller 10 can shard the model through the model shard management module and deploy the model shards to each node, such as edge nodes 20, regional nodes 30, and cloud nodes 40, based on the node computing power and the computing power requirements of the model shards.

[0053] In some embodiments, the network status monitoring module is used to monitor the status of the network and each node, obtain node load (for example, CPU or GPU resource utilization), link delay, bandwidth, and computing task queue length information.

[0054] In some embodiments, the reinforcement learning path decision engine module is used to select the model inference node of the optimal path. Exemplarily, the SRv6 controller 10 can input the status information obtained by the network status monitoring module into the reinforcement learning path decision engine module, and output the optimal node path through the learning model.

[0055] In some embodiments, the SID mapping table is used to store the mapping relationship between model shards and SIDs. For example, if model shard 1 is deployed on edge node 20 and has SID SID 1, and model shard 2 is deployed on regional node 30 and has SID SID 2, the SID mapping table may store that model shard 1 corresponds to SID 1, and model shard 2 corresponds to SID 2.

[0056] The edge node 20 is deployed at the edge of the network, near the data source or terminal. In some embodiments, the edge node 20 can be an industrial gateway, an edge server, a router (integrated computing module), an on-board computing unit, etc. The edge node 20 can also be other devices, which are not limited in the embodiments of the present application.

[0057] The edge node 20 includes a model preprocessing module, a low-level model reasoning module, and an encapsulation / decapsulation function module.

[0058] In some embodiments, the edge node 20 may pre-process the raw data through a pre-processing module, for example, normalizing the image.

[0059] In some embodiments, the edge node 20 may perform model inference on the data processed by the preprocessing module through a low-level model inference module to obtain intermediate data.

[0060] In some embodiments, the edge node 20 may encapsulate data into an SRv6 message through an encapsulation function module, and the edge node 20 may decode the SRv6 message through a decapsulation function module.

[0061] The regional node 30 is located between the edge node 20 and the cloud node 30, and generally includes a mid-level model reasoning module, a data cache module, and an SRv6 forwarding or processing function module.

[0062] In some embodiments, the regional node 30 may perform mid-level model inference on the data inferred by the edge node 20 through a mid-level model inference module to obtain intermediate data.

[0063] In some embodiments, the regional node 30 may cache the inferred data through a data cache module, for example, cache the intermediate feature map through the data cache module.

[0064] In some embodiments, the regional node 30 may use the SRv6 forwarding function module to send the SRv6 message data to the next node, such as the cloud node 40. The regional node 30 may use the SRv6 processing function module to process and update the message data.

[0065] The cloud node 40 is typically deployed in a core data center and has massive computing and storage resources. The cloud node 40 may include a high-level model inference module, a result aggregation module, and an SRv6 termination processing module.

[0066] In some embodiments, the cloud node 40 may perform high-level model reasoning on the intermediate data through a high-level model reasoning module. For example, the cloud node 40 may perform high-level model reasoning on the data after reasoning on the regional node 30 through a high-level model reasoning module.

[0067] In some embodiments, the cloud node 40 may aggregate the results of the inference data through a result aggregation module to obtain prediction data.

[0068] In some embodiments, the cloud node 40 may process the prediction data through the SRv6 termination processing function module and return the final result.

[0069] Below, the embodiment of the present application will be described with Figure 1 Taking the SRv6 controller 10, edge node 20, regional node 30 and cloud node 40 of the shown architecture as an example, the data processing method provided by the embodiment of the present application is described in detail in combination with the accompanying drawings and application scenarios. The method is executed by the SRv6 controller, edge node, regional node and cloud node. Among them, the SRv6 controller can be Figure 1 The SRv6 controller 10 in the SRv6 controller 10 or the device in the SRv6 controller 10. The edge node can be Figure 1 The edge node 20 in the edge node 20 or the device or apparatus in the edge node 20. The regional node may be Figure 1 The cloud node can be a regional node 30 or a device or apparatus in the regional node 30. Figure 1 The cloud node 40 or the device or apparatus in the cloud node 40. To simplify the description, the method is explained by taking the execution of the method by the SRv6 controller, the edge node, the regional node and the cloud node as an example.

[0070] See also Figure 2 , Figure 2 This is a signaling interaction diagram of a data processing method provided in an embodiment of the present application. Figure 2 As shown, the data processing method provided in the embodiment of the present application may include:

[0071] S101. When a target model inference task is triggered, the edge node obtains a service segment identifier (SID) list corresponding to the target model.

[0072] Among them, triggering the target model inference task is used to instruct the edge node to start data processing on the data to be processed. In some embodiments, the target model inference task can be triggered when the client initiates an inference request to the edge node. In other embodiments, the target model inference task can be triggered when the client sends the data to be processed to the edge node. The embodiments of the present application do not limit the conditions for triggering the target model inference task.

[0073] The SID list is used to indicate multiple hierarchical inference nodes of the target model, facilitating forwarding of the data to be processed to the multiple hierarchical inference nodes for model inference. The data to be processed can be the raw data collected by the client or processed data from the raw data. This embodiment of the application does not limit the data to be processed.

[0074] As mentioned above, the SID list is generated by the SRv6 controller.

[0075] In some embodiments, the SRv6 controller obtains status information of each candidate node, determines multiple hierarchical reasoning nodes corresponding to the target model based on the status information of each candidate node, and generates a SID list based on the multiple hierarchical reasoning nodes.

[0076] Among them, each candidate node is a candidate node for target model deployment, and the candidate node includes multiple hierarchical reasoning nodes.

[0077] In some embodiments, the status information includes the computational load and network link status of each candidate node.

[0078] The computational load of each candidate node is used to indicate the computational reasoning capability of the target model. The computational load of each candidate node includes the resource utilization of the node or the memory occupancy information of the node. The computational load of each candidate node may also include other content, which is not limited in the embodiments of the present application.

[0079] The network link status of each candidate node is used to indicate the communication status information between each candidate node. The network link status of each candidate node can include any one or more combinations of latency, bandwidth, or packet loss rate. The network link status of each candidate node can also include other content, which is not limited in the embodiments of the present application.

[0080] In summary, the SRv6 controller can determine multiple hierarchical inference nodes corresponding to the target model based on the status information of each candidate node, thereby facilitating the inference of the data to be processed on the target model shards of multiple hierarchical inference nodes, improving the resource utilization of multiple hierarchical inference nodes, and reducing the load on the central node.

[0081] In some embodiments, the state information may include the latency of the candidate node and the resource utilization of the candidate node. The SRv6 controller may determine the multiple hierarchical reasoning nodes corresponding to the target model through the target optimization algorithm of Formula 1.

[0082] min(α(T_MAX-T_E2E)+β(1-max(U_i))+γ(B_saved)) Formula 1;

[0083] Where α, β, and γ are weight coefficients that can be dynamically adjusted based on the candidate node status information. T_MAX is the maximum latency of the candidate node, T_E2E is the latency between nodes in the network link state, U_i is the resource utilization of the node in the computational load, and B_saved is the bandwidth saved by data compression.

[0084] In other embodiments, the SRv6 controller may further perform iterative calculations, as shown in Formula 2, to better determine the multiple hierarchical inference nodes corresponding to the target model.

[0085] min((α(T_MAX-T_E2E)+β(1-max(U_i))+γ(B_saved))-Reward) Formula 2;

[0086] Among them, the reward function Reward can be expressed as:

[0087] Reward=-(α′(T_MAX-T_E2E)+β′(1-max(U_i))+γ′(B_saved)) Formula 3;

[0088] Among them, α′, β′, γ′ are the weight coefficients of the previous iteration.

[0089] In summary, the SRv6 controller obtains the status information of each candidate node and, based on this information, uses Equation 1 or Equation 2 to determine the optimal multiple hierarchical inference nodes for the target model, thereby reducing latency. Furthermore, by enabling multiple hierarchical inference nodes to collaboratively infer the data to be processed, the resource load on the central node is reduced, improving resource utilization.

[0090] The target model includes multiple model shards, and the multiple model shards are deployed on corresponding hierarchical inference nodes based on the computing capabilities of multiple hierarchical inference nodes.

[0091] For example, the target model is ResNet-50, which consists of five main stages: Stage 0 to Stage 4. Each stage has gradually decreasing feature map resolution and increasing computational complexity. Stage 0 is the initial convolutional layer. Stages 1-4 are stacked residual blocks. The multiple hierarchical inference nodes determined by the SRv6 controller are edge camera nodes, edge gateway servers, regional data centers, and cloud AI servers.

[0092] Based on the computing power of the above four nodes, the model is divided into the following four parts:

[0093] The edge camera node has low computing power and serves as a preprocessing module to perform video decoding and image frame normalization. The edge gateway server has medium computing power and serves as a low-level feature extraction group to infer Stage 0 and Stage 1. The regional data center has high computing power and serves as a mid-level feature extraction group to infer Stage 2. The cloud AI server has extremely high computing power and serves as a high-level inference and classification group to infer Stage 3, Stage 4, and the classification head.

[0094] In related technologies, the target model performs preprocessing and Stage 0-4 inference on a single node, as shown below:

[0095] [Preprocessing] → [Stage0] → [Stage1] → [Stage2] → [Stage3] → [Stage4] → [Classification Header].

[0096] In the embodiment of the present application, the target model is split into multiple parts and processed on multiple hierarchical inference nodes, as shown below:

[0097] Edge camera node: [Preprocessing] → Edge gateway server: [Stage0-Stage1] → Regional data center: [Stage2] → Cloud AI server: [Stage3-Stage4] and [Classification head].

[0098] In summary, the target model can be divided into multiple model shards and deployed to multiple hierarchical inference nodes respectively, so that multiple hierarchical inference nodes can perform reasoning at different stages and reduce the resource load of target model reasoning.

[0099] After determining the multiple hierarchical reasoning nodes, the SRv6 controller generates a SID list based on the multiple hierarchical reasoning nodes.

[0100] The SID list includes N SIDs, where N is a positive integer and one SID corresponds to one hierarchical reasoning node. The edge node is the hierarchical reasoning node corresponding to the first SID in the SID list.

[0101] Exemplarily, the SID list may be segment list = [SID_Edge, SID_Region, SID_Cloud], which is used to indicate three hierarchical reasoning nodes of the target model, namely, edge nodes, regional nodes, and cloud nodes.

[0102] In some embodiments, the SID includes a first field, a second field, a third field, and a fourth field.

[0103] The first field may also be called a locator field, which is used to indicate the address information of the hierarchical reasoning node; wherein the address information of the hierarchical reasoning node is used to uniquely indicate the address of the hierarchical reasoning node. For example, 2001:db8:edge1 is used to indicate the address of the edge node.

[0104] The second field may also be called a function field, which is used to indicate the computing function type identifier of the target model and / or the model identifier of the target model.

[0105] The computing function type of the target model is used to indicate how the hierarchical reasoning node processes the data to be processed. In some embodiments, the computing function type of the target model may include any one or more combinations of preprocessing, model reasoning, or result aggregation. In other embodiments, the computing function type of the target model may also include post-processing. The embodiments of the present application do not limit the content of the computing function type of the target model.

[0106] In the SID, the target model's computing function type is typically indicated by its identifier. For example, 1 represents preprocessing, 2 represents model inference, and 3 represents result aggregation. This embodiment of the application does not limit the method for indicating the target model's computing function type.

[0107] Therefore, the hierarchical reasoning node can perform corresponding computing function processing on the data to be processed based on the computing function type identifier of the target model indicated by the SID.

[0108] The target model's model identifier uniquely identifies the target model. A hierarchical inference node can determine the target model based on the target model identifier and process the data to be processed. For example, the target model identifier can be ResNet-50, indicating a deep convolutional neural network model.

[0109] The third field may also be called a layer field, which is used to indicate the model layer identifier of the target model.

[0110] The model level identifier of the target model is used to uniquely indicate the model shard of the target model. For example, the target model is ResNet-50, which is grouped into 5 model shards according to the convolutional layer, and each model shard corresponds to a model level identifier of the target model. For example, the model level identifier corresponding to model shard 1 is stage1, the model level identifier corresponding to model shard 2 is stage2, the model level identifier corresponding to model shard 3 is stage3, the model level identifier corresponding to model shard 4 is stage4, and the model level identifier corresponding to model shard 5 is stage5.

[0111] The fourth field may also be referred to as the param field, which is used to indicate additional parameters for target model inference.

[0112] In some embodiments, the additional parameters of the target model inference include a data compression algorithm identifier, which is used to indicate the data compression method of the SRv6 message.

[0113] For example, the additional parameter may be Zstd, which indicates that the data to be processed is compressed using the Zstd method. The additional parameters for target model inference may also include other parameter information, which is not limited in the present embodiment.

[0114] It should be understood that the above-mentioned first field, second field, third field and fourth field are merely exemplary descriptions, and the embodiments of the present application do not limit the names or contents of the first field, second field, third field and fourth field.

[0115] In addition, the SID may include the above-mentioned first field, second field, third field and fourth field, or it may only include any one or more combinations of the first field, second field, third field and fourth field. The embodiment of the present application does not limit this.

[0116] In a specific implementation, as shown in Table 1, the SID is illustrated by taking the SID including the first field, the second field, and the third field as an example.

[0117] Table 1

[0118] node SID Example Functional Description Edge camera nodes 2001:db8:cam1::1:Preprocess Video decoding and frame normalization Edge Gateway Server 2001:db8:edge1::2:Stage0-1 Perform Stage 0-Stage 1 reasoning Regional Data Center 2001:db8:region1::2:Stage2 Perform Stage 2 inference Cloud Server 2001:db8:cloud::2:Stage3-4 Execute Stage 3-Stage 4 and classification

[0119] In summary, the edge node can obtain the service segment identifier SID list corresponding to the target model. The SID list contains multiple SIDs. Each SID includes the address of the hierarchical reasoning node, the computing function type identifier of the target model, the model identifier of the target model, the model level identifier of the target model, or any one or more combinations of additional parameters for target model reasoning, so that the data to be processed can be inferred according to the corresponding function on the corresponding hierarchical reasoning node.

[0120] S102. The hierarchical reasoning node corresponding to the Kth SID in the SID list performs hierarchical processing on the data to be processed to obtain intermediate data, and sends an SRv6 message to the hierarchical reasoning node corresponding to the K+1th SID in the SID list. The SRv6 message includes the SID list and the intermediate data, until the hierarchical reasoning node corresponding to the last SID in the SID list completes the reasoning task of the target model, where K is a positive integer from 1 to N-1.

[0121] The hierarchical inference node is where the model shards of the target model are deployed. The hierarchical inference node corresponding to the Kth SID in the SID list can perform hierarchical processing on the data to be processed based on the indication information in the Kth SID to obtain intermediate data.

[0122] In some embodiments, the SID includes a model hierarchy identifier of the target model, and the hierarchical reasoning node corresponding to the Kth SID performs hierarchical processing on the data to be processed through the model shard corresponding to the model hierarchy identifier to obtain intermediate data.

[0123] Exemplarily, the model level is identified as stage1, and the hierarchical reasoning node corresponding to the Kth SID performs hierarchical processing on the data to be processed through the model shard 1 corresponding to stage1 to obtain intermediate data.

[0124] Exemplarily, the model level is identified as stage2, and the hierarchical reasoning node corresponding to the Kth SID performs hierarchical processing on the data to be processed through the model shard 2 corresponding to stage1 to obtain intermediate data.

[0125] In summary, the hierarchical reasoning node corresponding to the K-th SID can find the corresponding model shard through the model level identifier indicated in the SID, and then perform reasoning on the data to be processed.

[0126] In other embodiments, the SID includes a computing function type of a target model, and the hierarchical inference node corresponding to the Kth SID performs hierarchical processing of the computing function type on the data to be processed to obtain intermediate data.

[0127] Exemplarily, the computing function type is preprocessing, and the hierarchical reasoning node corresponding to the K-th SID preprocesses the data to be processed to obtain intermediate data.

[0128] Exemplarily, the computing function type is model reasoning, and the hierarchical reasoning node corresponding to the K-th SID performs model reasoning on the data to be processed to obtain intermediate data.

[0129] Exemplarily, the computing function type is result aggregation, and the hierarchical reasoning node corresponding to the K-th SID performs result aggregation on the data to be processed to obtain intermediate data.

[0130] In summary, the hierarchical reasoning node corresponding to the Kth SID can infer the computing function type of the processed data through the computing function type indicated in the SID, without having to implement the inference of all computing function types on one inference node, thereby reducing the computing load pressure of the inference node.

[0131] In other embodiments, the SID includes a model identifier of a target model, a hierarchical inference node corresponding to the Kth SID, and hierarchical processing is performed on the data to be processed through the model shards of the target model corresponding to the model identifier to obtain intermediate data.

[0132] For example, assume that the hierarchical inference node corresponding to the Kth SID deploys model shards of different models, for example, model shard 1 of the ResNet50 model and model shard 1 of the natural language processing pre-trained (bidirectional encoder representations from transformers, BERT) model. The hierarchical inference node corresponding to the Kth SID obtains the corresponding model shard 1 based on the model identifier ResNet50 of the target model indicated in the SID, and performs hierarchical processing on the data to be processed to obtain intermediate data.

[0133] In summary, the hierarchical reasoning node corresponding to the Kth SID can determine the correct model shard through the model identifier of the target model indicated in the SID, and perform reasoning on the data to be processed, thereby ensuring that the data to be processed is reasoned on the model shard of the correct target model.

[0134] After the hierarchical reasoning node corresponding to the Kth SID completes reasoning, it sends an SRv6 message to the hierarchical reasoning node corresponding to the K+1th SID in the SID list. The SRv6 message includes the SID list and intermediate data.

[0135] In some embodiments, the hierarchical reasoning node corresponding to the Kth SID encapsulates a service segment routing header (SRH) of the SRv6 message, where the SRH includes the remaining number of hops and a SID list.

[0136] The remaining hop count is used to indicate the number of hierarchical reasoning nodes that have not yet processed data. The remaining hop count is a positive integer, and the next hop is the K+1th SID in the SID list.

[0137] For example, the number of hierarchical reasoning nodes is 3. After the first hierarchical reasoning node completes reasoning, the number of remaining hops is 2, and the next hop is indicated as the next SID in the SID list, that is, the second SID. After the second hierarchical reasoning node completes reasoning, the number of remaining hops is 1, and the next hop is indicated as the next SID in the SID list, that is, the third SID. After the third hierarchical reasoning node completes reasoning, the number of remaining hops is 0.

[0138] In other embodiments, the SRH of the SRv6 message may further include a flag bit for indicating a compression flag or the urgency of data processing. The embodiments of the present application do not limit the content of the SRH of the SRv6 message.

[0139] For example, the format of an SRv6 message is as follows:

[0140]

[0141] Next Header indicates the next header type. For example, 0x2B indicates that the next header type is SRv6. Segments Left indicates the number of remaining hops. Segments Left decreases dynamically until it reaches 0, indicating the last hierarchical inference node in the SID list. Last Entry indicates the last entry in the SID list. The index of the last SID is the total number of hops minus 1. For example, the index of SID_cloud is 3 minus 1, meaning the index of SID_cloud is 2.

[0142] In summary, the SRH of the SRv6 message can include the remaining hop count and SID list, which are used to indicate the number of hierarchical reasoning nodes that have not processed data and the information of the next hierarchical reasoning node to be processed, so as to facilitate sending the data to be processed to different hierarchical reasoning nodes in sequence for model reasoning, thereby ensuring the integrity of model reasoning.

[0143] In some embodiments, the hierarchical reasoning node corresponding to the Kth SID encapsulates a payload of the SRv6 message, where the payload includes intermediate data obtained by the hierarchical reasoning node corresponding to the Kth SID performing hierarchical processing on the data to be processed.

[0144] In summary, after the hierarchical inference node corresponding to the Kth SID completes the inference, it will encapsulate the intermediate data obtained by inference into an SRv6 message, so that the hierarchical inference node corresponding to the K+1th SID can continue to perform model inference on the intermediate data until the inference task is completed.

[0145] In addition, SRv6 messages not only indicate hierarchical reasoning node information through SRv6 messages, but also include intermediate data obtained from hierarchical processing, which integrates computing and the network, breaking the isolation between the network and computing in the traditional model reasoning process.

[0146] Below, the data processing method proposed in the embodiment of the present application is described by taking the hierarchical reasoning nodes of the target model including edge nodes, regional nodes and cloud nodes as an example.

[0147] See also Figure 3 , Figure 3 This is a signaling interaction diagram of another data processing method provided in an embodiment of the present application. Figure 3 As shown, the data processing method provided in the embodiment of the present application may include:

[0148] S201: The edge node preprocesses the data to be processed and / or performs low-level model inference, and sends a first SRv6 message to the regional node.

[0149] The first SRv6 message includes a SID list and data after low-level model inference.

[0150] There may be one or more edge nodes, and the embodiment of the present application does not limit the number of edge nodes.

[0151] In some embodiments, the edge nodes may be edge camera nodes and edge gateway servers.

[0152] The edge camera node pre-processes the data to be processed. Exemplarily, the edge camera node decodes the data to be processed and normalizes it into a 224×224 red, green, blue (RGB) image.

[0153] The edge camera node encapsulates the preprocessed data into an SRv6 message. The message list indicates the next hop to forward to the edge gateway server. For example, the SID list includes [SID_edge1, SID_regionB, SID_cloud]. In this case, the next hop is SID_edge1, and the number of remaining hops is 3.

[0154] After receiving the SRv6 message, the edge gateway server unpacks the SRv6 message and performs low-level model reasoning on the preprocessed data in the SRv6 message. For example, stage 1-stage 1 reasoning is performed on the preprocessed data in the SRv6 message.

[0155] The edge gateway server encapsulates the data inferred by the low-level model into the first SRv6 message. The message list is used to indicate the next hop forwarding to the regional node. For example, the SID list is [SID_edge1, SID_regionB, SID_cloud], the next hop points to SID_regionB, and the number of remaining hops is 2. The SID list can also be [SID_regionB, SID_cloud]. This embodiment of the application does not limit the content of the SID list.

[0156] S202. The regional node decapsulates the first SRv6 message and performs mid-level model inference, and sends a second SRv6 message to the cloud node.

[0157] The second SRv6 message includes the SID list and the data after the middle-level model inference.

[0158] There may be one or more regional nodes, and the embodiment of the present application does not limit the number of regional nodes.

[0159] In some embodiments, a regional node may be a regional data center.

[0160] After receiving the first SRv6 message, the regional data center unpacks the first SRv6 message and performs middle-level model reasoning on the data after low-level model reasoning in the first SRv6 message. For example, stage 2 reasoning is performed on the data after low-level model reasoning in the first SRv6 message.

[0161] The regional data center encapsulates the data after mid-level model inference into a second SRv6 message. The message list is used to indicate the next hop forwarding to the cloud node. For example, the SID list is [SID_edge1, SID_regionB, SID_cloud], the next hop is SID_cloud, and the number of remaining hops is 1. The SID list can also be [SID_cloud]. This embodiment of the application does not limit the content of the SID list.

[0162] S203. The cloud node decapsulates the second SRv6 message and performs high-level model reasoning, and completes the reasoning task of the target model.

[0163] Among them, there can be one cloud node or multiple cloud nodes, and the embodiment of the present application does not limit the number of cloud nodes.

[0164] In some embodiments, the cloud node may be a cloud server.

[0165] After receiving the second SRv6 message, the cloud server unpacks the second SRv6 message and performs high-level model reasoning on the data after the middle-level model reasoning in the second SRv6 message. For example, stage3-stage4 reasoning is performed on the data after the middle-level model reasoning in the second SRv6 message.

[0166] The cloud server post-processes or aggregates the data after high-level model inference, generates the final inference result, and returns the inference result to the client.

[0167] In summary, the target model can execute the reasoning of the sharded model on multiple hierarchical inference nodes, reducing the inference load pressure of the central node, making full use of the resources of edge nodes, regional nodes or cloud nodes, and improving resource utilization.

[0168] The above describes the process of multiple hierarchical reasoning nodes reasoning about the data to be processed. However, during the reasoning process, the hierarchical reasoning nodes may fail or have insufficient resources. The following describes the situation where a hierarchical reasoning node may fail or have insufficient resources.

[0169] In some embodiments, when the computational load of the hierarchical reasoning node corresponding to the Kth SID is greater than or equal to a first threshold, or the inference duration of the hierarchical reasoning node corresponding to the Kth SID is greater than or equal to a second threshold, the SRv6 controller determines a new hierarchical reasoning node and replaces the hierarchical reasoning node corresponding to the Kth SID in the SID list with the new hierarchical reasoning node.

[0170] Among them, the first threshold is a predefined computing load threshold. The first threshold can be a value predefined by the SRv6 controller or a value predefined by the hierarchical reasoning node. The embodiment of the present application does not limit the predefined subject of the first threshold.

[0171] Exemplarily, the first threshold can be 80%. When the computational load of the hierarchical reasoning node corresponding to the Kth SID is greater than or equal to 80%, the SRv6 controller determines a new hierarchical reasoning node and replaces the hierarchical reasoning node corresponding to the Kth SID in the SID list with the new hierarchical reasoning node.

[0172] Among them, the second threshold is a predefined inference time threshold. The second threshold can be a value predefined by the SRv6 controller or a value predefined by the hierarchical inference node. The embodiment of the present application does not limit the predefined subject of the second threshold.

[0173] Exemplarily, the second threshold can be 100 milliseconds. When the inference time of the hierarchical inference node corresponding to the Kth SID is greater than or equal to 100 milliseconds, the SRv6 controller determines a new hierarchical inference node and replaces the hierarchical inference node corresponding to the Kth SID in the SID list with the new hierarchical inference node.

[0174] In summary, when the computational load of the hierarchical inference node corresponding to the Kth SID is large or the inference time is long, the SRv6 controller determines a new hierarchical inference node and replaces the hierarchical inference node corresponding to the Kth SID in the SID list with the new hierarchical inference node, thereby avoiding the target model inference delay caused by a certain hierarchical inference node and reducing the target model inference delay.

[0175] It should be understood that Figures 1 to 3 The flowcharts or scenario diagrams shown are only for ease of understanding and are not intended to limit the embodiments of the present application to the examples shown in the diagrams. In fact, those skilled in the art will Figures 1 to 3 The examples in can be equivalently transformed to obtain more implementation methods.

[0176] Combined with the above Figures 1 to 3 , describes in detail the data processing method provided by the embodiment of the present application. Figure 4 and Figure 5It should be understood that the communication device of the present invention can execute the various data processing methods of the aforementioned embodiments of the present invention, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0177] In the above embodiments, the SRv6 controller and the hierarchical reasoning node can perform some or all of the steps in each embodiment. These steps or operations are only examples, and the embodiments of the present application can also perform other operations or variations of various operations. In addition, the various steps can be performed in different orders presented in the embodiments, and it is possible that not all operations in the embodiments of the present application need to be performed. Moreover, the size of the sequence number of each step does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0178] Illustratively, an embodiment of the present application further provides a communication device.

[0179] See also Figure 4 , Figure 4 A schematic structural diagram of a communication device provided in an embodiment of the present application.

[0180] The communication device 500 may include a communication module 520. The communication module 520 may implement corresponding communication functions, which may be internal communication functions of the communication device 500 or communication functions between the communication device 500 and other devices. Optionally, the communication module 520 may also be referred to as a communication interface or a transceiver module. Optionally, the communication device 500 also includes a processing module 510. The processing module 510 may implement corresponding processing functions.

[0181] Optionally, the communication device 500 further includes a storage module, which can be used to store instructions and / or data; the processing module 510 can read the instructions and / or data in the storage module to enable the communication device 500 to implement the aforementioned method embodiment.

[0182] In one possible design, the communication device 500 may correspond to the SRv6 controller and hierarchical reasoning node in the above method embodiments, or components (such as circuits, chips, or chip systems) configured in the SRv6 controller and hierarchical reasoning node. The communication device 500 can be used to execute the steps or processes performed by the SRv6 controller and hierarchical reasoning node in any of the above method embodiments.

[0183] Illustratively, an embodiment of the present application further provides a communication device.

[0184] See also Figure 5 , Figure 5A schematic diagram of the structure of another communication device provided in an embodiment of the present application.

[0185] like Figure 5 As shown, the communication device 500 can be a chip, chip system, or processor that implements the above method, such as an SRv6 controller or a hierarchical inference node. The communication device 500 can be used to implement the method described in the above method embodiment. For details, please refer to the description of the above method embodiment.

[0186] The communication device 500 may include one or more processors 610, which may also be referred to as processing units or processing modules, and may implement certain control functions. The processor 610 may be a general-purpose processor or a dedicated processor, such as a baseband processor or a central processing unit. The baseband processor may be used to process communication protocols and communication data, while the central processing unit may be used to control the communication device 500 (e.g., base station, baseband chip, user, user chip), execute software programs, and process software program data.

[0187] In an optional design, the processor 610 may also store instructions and / or data, and the instructions and / or data may be executed by the processor 610 so that the communication device 500 executes the method described in the above method embodiment.

[0188] In another optional design, the communication device 500 may include a communication interface 620 for implementing receiving and transmitting functions. For example, the communication interface 620 may be a transceiver circuit, an interface, an interface circuit, or a transceiver. The transceiver circuit, interface, interface circuit, or transceiver for implementing the receiving and transmitting functions may be separate or integrated. The transceiver circuit, interface, interface circuit, or transceiver may be used for reading and writing code / data, or the transceiver circuit, interface, interface circuit, or transceiver may be used for transmitting or delivering signals.

[0189] Optionally, the communication device 500 may include one or more memories 630, which may store instructions. The instructions may be executed on the processor 610, causing the communication device 500 to perform the method described in the above method embodiment. Optionally, the memory 630 may also store data. Optionally, the processor 610 may also store instructions and / or data. The processor 610 and memory 630 may be provided separately or integrated together.

[0190] It should be understood that, in one possible design, each step in the method embodiment provided in the present application can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0191] In one implementation, the communication device 500 may correspond to the SRv6 controller and hierarchical reasoning node in the above-mentioned method embodiment, and may be used to execute the various steps and / or processes performed by the SRv6 controller and hierarchical reasoning node in the above-mentioned method embodiment. The processor 610 may be used to execute instructions stored in the memory 630, and when the processor 610 executes the instructions stored in the memory, the processor 610 is used to execute the various steps and / or processes of the above-mentioned method embodiment corresponding to the SRv6 controller and hierarchical reasoning node.

[0192] It should be understood that the above-mentioned processing device may be one or more chips. For example, the processing device may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0193] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0194] Based on the methods provided in the embodiments of the present application, the present application also provides a chip system, which includes one or more processors configured to retrieve and execute instructions stored in a memory, thereby executing the methods of the embodiments of the present application. The chip system can be composed of a chip or can include a chip and other discrete devices.

[0195] Among them, the chip system may include an input circuit or interface for sending information or data, and an output circuit or interface for receiving information or data.

[0196] According to the method provided in the embodiment of the present application, the present application also provides a communication system, which includes the aforementioned SRv6 controller and hierarchical reasoning node.

[0197] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, which, when the computer program code is run on a computer, enables the computer to execute the various steps or processes performed by the SRv6 controller and the hierarchical reasoning node in any of the aforementioned method embodiments.

[0198] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable storage medium, which stores program code. When the program code runs on a computer, the computer executes the various steps or processes performed by the SRv6 controller and the hierarchical reasoning node in any of the aforementioned method embodiments.

[0199] The computer-readable storage medium may be the aforementioned volatile memory or non-volatile memory, or may include both volatile memory and non-volatile memory.

[0200] In the embodiments of this application, each term and English abbreviation is provided for convenience of description and shall not constitute any limitation to this application. This application does not exclude the possibility of defining other terms that can achieve the same or similar functions in existing or future agreements.

[0201] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part.

[0202] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0203] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0204] In short, the above is only a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A data processing method, characterized in that: The method comprises: When a target model inference task is triggered, the edge node obtains a service segment identifier (SID) list corresponding to the target model; the SID list includes N SIDs, where N is a positive integer, and the SID list is used to indicate multiple hierarchical inference nodes of the target model; the edge node is the hierarchical inference node corresponding to the first SID in the SID list; The hierarchical reasoning node corresponding to the Kth SID in the SID list performs hierarchical processing on the processed data to obtain intermediate data, and sends an IPv6-based service segment routing SRv6 message to the hierarchical reasoning node corresponding to the K+1th SID in the SID list, wherein the SRv6 message includes the SID list and the intermediate data; until the hierarchical reasoning node corresponding to the last SID in the SID list completes the reasoning task of the target model, where K is a positive integer from 1 to N-1.

2. The method according to claim 1, characterized in that The model shard of the target model is deployed in the hierarchical inference node, the SID includes the model level identifier of the target model, and the hierarchical inference node corresponding to the Kth SID in the SID list performs hierarchical processing on the data to be processed to obtain intermediate data, including: The hierarchical reasoning node corresponding to the K-th SID performs hierarchical processing on the data to be processed by identifying the model slice corresponding to the model hierarchy to obtain intermediate data.

3. The method according to claim 1, characterized in that The model shard of the target model is deployed in the hierarchical inference node, the SID includes the computing function type of the target model, and the hierarchical inference node corresponding to the Kth SID in the SID list performs hierarchical processing on the data to be processed to obtain intermediate data, including: The hierarchical reasoning node corresponding to the K-th SID performs hierarchical processing of the computing function type on the data to be processed to obtain intermediate data.

4. The method according to claim 3, characterized in that The computing function type of the target model includes: any one or more combinations of preprocessing, model reasoning or result aggregation.

5. The method according to claim 1, wherein The SID includes the model identifier of the target model. The hierarchical inference node corresponding to the Kth SID in the SID list performs hierarchical processing on the data to be processed to obtain intermediate data, including: The hierarchical inference node corresponding to the K-th SID performs hierarchical processing on the data to be processed through the model slice of the target model corresponding to the model identifier to obtain intermediate data.

6. The method according to claim 1, characterized in that The SID includes a data compression algorithm identifier, and the data compression algorithm identifier is used to indicate a data compression method of the SRv6 message.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The hierarchical inference node corresponding to the K-th SID encapsulates the service segment routing header SRH of the SRv6 message, where the SRH includes a remaining number of hops and the SID list; wherein the remaining number of hops is used to indicate the number of inference nodes that have not performed data processing, and the remaining number of hops is a positive integer; and the next hop is the K+1-th SID in the SID list.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: The SRv6 controller obtains status information of each candidate node; the status information includes the computing load and network link status of each candidate node; The SRv6 controller determines, based on the state information, a plurality of hierarchical inference nodes corresponding to the target model, the plurality of hierarchical inference nodes being used for collaborative inference, the target model including a plurality of model shards, the plurality of model shards being respectively deployed to corresponding hierarchical inference nodes based on computing capabilities of the plurality of hierarchical inference nodes; The SRv6 controller generates the SID list based on the multiple hierarchical inference nodes.

9. The method according to claim 1, characterized in that The method further comprises: When the computational load of the hierarchical reasoning node corresponding to the K-th SID is greater than or equal to the first threshold, or the inference time of the hierarchical reasoning node corresponding to the K-th SID is greater than or equal to the second threshold, the SRv6 controller determines a new hierarchical reasoning node and replaces the hierarchical reasoning node corresponding to the K-th SID in the SID list with the new hierarchical reasoning node.

10. The method according to claim 1, characterized in that The hierarchical reasoning nodes of the target model include edge nodes, regional nodes, and cloud nodes. The hierarchical reasoning node corresponding to the Kth SID in the SID list performs hierarchical processing on the processed data to obtain intermediate data, and sends an IPv6-based service segment routing SRv6 message to the hierarchical reasoning node corresponding to the K+1th SID in the SID list. The SRv6 message includes the SID list and the intermediate data, until the hierarchical reasoning node corresponding to the last SID in the SID list completes the reasoning task of the target model, where K is a positive integer from 1 to N-1, including: The edge node preprocesses the data to be processed and / or performs low-level model inference on the data, and sends a first SRv6 message to the regional node; the first SRv6 message includes the SID list and the data after the low-level model inference; The regional node decapsulates the first SRv6 message and performs mid-level model inference, and sends a second SRv6 message to the cloud node; the second SRv6 message includes the SID list and the data after the mid-level model inference; The cloud node decapsulates the second SRv6 message and performs high-level model reasoning, and completes the reasoning task of the target model.

11. A communication system, characterized in that: include: An SRv6 controller and a hierarchical reasoning node, wherein the hierarchical reasoning node includes an edge node, a regional node, and a cloud node, and the SRv6 controller and the hierarchical reasoning node are used to execute the method according to any one of claims 1 to 10.

12. A communication device, characterized in that: include: At least one processor and an interface circuit, the interface circuit being used to receive signals from other communication devices outside the communication device and transmit them to the processor or to send signals from the processor to other communication devices outside the communication device, the processor being used to implement the method performed by the SRv6 controller as described in any one of claims 1 to 10 through logic circuits or execution of code instructions, or being used to implement the method performed by the hierarchical reasoning node as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The method comprises a computer program or an instruction, which, when executed on a computer, causes the computer to execute the method executed by the SRv6 controller as claimed in any one of claims 1 to 10, or causes the computer to execute the method executed by the hierarchical reasoning node as claimed in any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product includes: a computer program or instructions, which, when executed on a computer, causes the computer to execute the method executed by the SRv6 controller as claimed in any one of claims 1 to 10, or causes the computer to execute the method executed by the hierarchical reasoning node as claimed in any one of claims 1 to 10.

Images