Method and apparatus for splitting transformer model in mobile edge environment

Abstract

The application discloses a kind of splitting method and device of Transformer model in mobile edge environment, the method includes: selecting the number of mobile edge devices not more than the layer number of Transformer model encoder as equipment group, obtains the computing capacity of equipment, transmission power, the distance between equipment, channel bandwidth, channel noise are combined to calculate the D2D communication time between equipment;Optimization problem is solved by using branch and bound method to obtain the optimal transmission path;Based on the optimal transmission path obtained, the Transformer model is placed according to layer on the corresponding mobile edge device.The application improves the communication efficiency and the overall computing performance of system, and reduces the operation complexity of splitting and arranging network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed computing, and particularly relates to a Transformer model segmentation method and device in a mobile edge environment. BACKGROUND

[0002] The breakthrough of deep learning promotes the development of artificial intelligence driven fields such as computer vision and natural language processing. The success of deep neural networks is due to training large-scale data sets on large-scale models. However, the increasing size of models and data makes it impossible for a single machine device to complete the training of a deep neural network model. Therefore, using cloud computing to complete the training is a natural choice. Under the traditional cloud computing framework, the data source needs to unload the data generated by itself to the cloud for unified use of large-scale distributed machine learning schemes (such as data parallelism) for computation. However, due to the large amount of data generated by various sensors and Internet of Things devices every hour (such as more than 1TB), transmitting these data to the cloud causes a large amount of communication overhead; in addition, the unloading of data will also bring privacy and security problems, such as malicious servers, hacker attacks, etc. In order to solve these two problems, researchers have proposed a machine learning paradigm based on split learning (also known as split learning), which divides the machine learning model into multiple parts and places them on different devices (clients and servers) for different propagation training, without the need for users to transmit raw data to the cloud, thereby avoiding the generation of communication overhead and unloading security risks.

[0003] The Transformer model is a deep learning model that uses a self-attention mechanism that calculates the weight of each part of the input data and sums it up with the weight. Early Transformers were mainly applied in the field of natural language processing. Similar to recurrent neural networks, Transformers aim to process sequential data (such as text translation). However, unlike recurrent neural networks that recursively process data, Transformers process all data simultaneously, so Transformers not only solve the "forgetting" drawback in recurrent neural networks, but also can be parallelized to speed up model training. Current Transformer models can be applied not only in the field of natural language processing, but also in the field of computer vision. Figure 1As shown, a general Transformer model mainly contains two parts, namely an encoder module and a decoder module. In natural language processing (such as translation tasks), the Transformer model needs to use the encoder module and the decoder module at the same time, while for classification tasks (such as text classification and picture classification), the Transformer model uses the encoder module. The present application is applied to classification tasks and only uses the encoder module. The encoder module of the Transformer model is composed of several isomorphic encoders, and the output of each encoder is the input of the next encoder. For the input of the Transformer model, an embedding layer is used to map to a low-dimensional space and add position information to the input. The encoder mainly contains two parts, namely a multi-head attention module and a feed-forward neural network, and in addition, a residual connection and layer normalization method are used after each module to accelerate the convergence efficiency of the model.

[0004] Generally, split learning divides a complete model into two parts, namely a client network and a server network. A batch of edge devices and a server cooperatively train the entire model, and each edge device updates the client network and the server network using the data generated by itself in turn, and sends the updated model to the next edge device. This not only guarantees the privacy of the data of the edge device itself, but also reduces the computational and storage overhead of the edge device itself, and is an efficient distributed machine learning paradigm. However, this kind of split learning will bring intolerable communication overhead, especially the communication overhead between the edge device and the server, which not only needs a long communication time, consumes a large amount of energy of itself, but also produces high economic cost. Multi-hop split learning can solve the problem of expensive communication overhead. However, due to the high heterogeneity of edge devices, their computing and storage capabilities are quite different, which leads to the fact that randomly splitting the network and arranging it to a group of edge devices may seriously reduce the training efficiency of the model. Although the encoders of the Transformer model are isomorphic, in order to train a Transformer model most efficiently using multi-hop split learning, the splitting and arranging part is an NP-hard problem, and cannot be solved using a polynomial-time algorithm. SUMMARY

[0005] The present application proposes a split method and device for a Transformer model in a mobile edge environment, based on the multi-hop split learning paradigm, which improves the communication efficiency and the overall computing performance of the system while protecting the privacy of the data of the edge device, and reduces the computational complexity of splitting and arranging the network.

[0006] Technical scheme: A split method for a Transformer model in a mobile edge environment, comprising the following steps:

[0007] The number of mobile edge devices not greater than the number of layers of the Transformer model encoder is selected as the device group, the computing capacity and the transmission power of the devices are obtained, and the D2D communication time between the devices is calculated based on the distance between the devices, the channel bandwidth and the channel noise;

[0008] The most powerful edge device holds the most Transformer layers, and the remaining devices hold one Transformer layer as the segmentation basis. Based on the D2D communication time between the devices, an optimization problem model is established to minimize the computing time of the Transformer model, and a branch and bound algorithm is used to solve the established optimization problem to obtain the optimal transmission path.

[0009] Based on the obtained optimal transmission path, the Transformer model is segmented and placed on the corresponding mobile edge device according to the layers.

[0010] Preferably, the calculation method of the D2D communication time is as follows:

[0011]

[0012] Where t ij represents the transmission time from device i to device j, A represents the data amount of transmission, s ij represents the data transmission rate from device i to device j, W is the bandwidth of the channel, a is the path loss index, N o is the noise power, p i represents the transmission power of device i, d ij represents the physical distance between device i and device j.

[0013] Preferably, the established optimization problem is as follows:

[0014]

[0015] The constraint condition is:

[0016]

[0017] Where N is the number of mobile edge devices in the device group, x ij is an indication variable, taking the value of 1 representing that device i transmits data to device j, and taking the value of 0 representing that device i does not transmit data to device j, w i represents the time required by device i to calculate each layer of the Transformer model, and z i is an integer that limits a connected component.

[0018] Preferably, the optimal transmission path obtained by using the branch and bound algorithm comprises: obtaining a suboptimal solution as an upper bound of the search algorithm using a genetic algorithm, generating a priority queue and adding a starting node to the queue; then when the queue is not empty and the lower bound of the head node is less than the global upper bound, the following operations are performed: taking out the head element of the queue, traversing the remaining nodes, if the lower bound obtained by adding the traversed node to the path is less than or equal to the current upper bound, adding the node to the priority queue, if a leaf node is searched and the total time consumption of the path is less than the current global upper bound, updating the upper bound.

[0019] Preferably, the placing of the Transformer model by layer on the corresponding mobile edge device comprises: deploying the Transformer model in the edge device in turn according to the transmission path, wherein the edge device with the strongest computing power holds the most Transformer layers, i.e. the number of network layers minus the number of devices plus 1, and all the remaining devices hold one layer of network model.

[0020] The application also provides a device for splitting a Transformer model in a mobile edge environment, comprising:

[0021] The communication time determination module is configured to select the number of mobile edge devices not greater than the number of encoder layers of the Transformer model as a device group, obtain the computing power and transmission power of the devices, and calculate the D2D communication time between the devices in combination with the distance, channel bandwidth and channel noise between the devices;

[0022] The path solving module is configured to hold the most Transformer layers in the edge device with the strongest computing power, and hold one layer of Transformer layer in the remaining devices as the splitting basis, establish an optimization problem model based on the D2D communication time between the devices and taking the minimization of the computing time of the Transformer model as the target, and solve the established optimization problem to obtain the optimal transmission path by using the branch and bound algorithm.

[0023] The model splitting module is configured to place the Transformer model by layer on the corresponding mobile edge device based on the obtained optimal transmission path.

[0024] The application also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs are executed by the processor to implement the steps of the Transformer model splitting method in a mobile edge environment as described above.

[0025] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the Transformer model splitting method in a mobile edge environment.

[0026] The application further provides a mobile edge computing system, which comprises a plurality of mobile edge devices, the mobile edge devices are assigned split Transformer models based on optimal transmission paths, the number of mobile edge devices is not greater than the number of layers of the Transformer model, and the optimal transmission paths are obtained based on the following method.

[0027] The number of mobile edge devices that is not greater than the number of layers of the Transformer model encoder is selected as a device group, the computing capacity and the sending power of the devices are obtained, the D2D communication time between the devices is calculated in combination with the distance between the devices, the channel bandwidth and the channel noise, the most Transformer layers are held by the edge device with the strongest computing capacity, and one layer of Transformer layers is held by the remaining devices, the splitting basis is established, an optimization problem model is established based on the D2D communication time between the devices and the target of minimizing the computing time of the Transformer model, and the branch and bound algorithm is used to solve the optimization problem to obtain the optimal transmission path.

[0028] Compared with the prior art, the application has the following advantages and beneficial effects: based on the characteristics of the Transformer model, the application proposes an efficient training method based on the multi-hop splitting learning paradigm, under certain conditions of network topology, an approximate ratio of 2 network splitting and arrangement strategy can be obtained in polynomial time, under other conditions, a search strategy based on the branch and bound method is proposed, and the time complexity of the brute force search optimal splitting and arrangement is greatly reduced. The application solves the problem of how to split the network in multi-hop splitting learning, greatly improves the resource utilization rate and the speed of Transformer model training. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is an architecture diagram of a general Transformer model;

[0030] Figure 2a and Figure 2b are two basic architecture diagrams of basic splitting learning;

[0031] Figure 3 is a basic architecture diagram of multi-hop splitting learning;

[0032] Figure 4 is a flowchart of the Transformer model splitting method of the application. DETAILED DESCRIPTION

[0033] The technical solutions of the present application are further described below with reference to the accompanying drawings.

[0034] The basic split learning has two forms, which are original split learning and U-shaped split learning, and their architectures are shown in Figure 2a and Figure 2b In the original split learning, the training process is as follows:

[0035] 1) The client cuts the model into two parts, the former part is placed on the client side, and the latter part is placed on the server side;

[0036] 2) The client performs forward propagation until the cut layer, and transmits the intermediate data to the server side;

[0037] 3) The server side accepts the intermediate data and continues to perform forward propagation, obtains the calculation result, performs back propagation until the cut layer and updates the network parameters, and transmits the intermediate data to the client;

[0038] 4) The client accepts the intermediate data and continues to perform back propagation and update the network parameters, and after the calculation is completed, transmits the network parameters to the next client;

[0039] 5) Repeat 2) to 4) until the model converges.

[0040] The purpose of U-shaped split learning is to protect the features of original data while protecting label privacy, and the network model is divided into three parts, and the last part is placed back on the client side for calculation to achieve the purpose of protecting label privacy.

[0041] The above split learning is based on the client-server architecture, which not only produces intolerable communication delay, wastes its own energy, but also produces expensive economic cost. In addition, there are some variants of split learning, such as multi-hop split learning, which is a decentralized architecture, and its architecture is shown in Figure 3 Under this architecture, the network is divided into multiple parts and placed in nearby edge devices, using the Device to Device (D2D) communication mode to reduce the load on the service base station, and under the premise of protecting data privacy, not only reduces the communication cost but also saves economic cost. Its training process is as follows:

[0042] 1) The network is placed in n different devices according to the specified split method;

[0043] 2) Device 1 to device n perform forward propagation in turn;

[0044] 3) Device n to device 1 perform back propagation and update parameters in turn;

[0045] 3) Repeat 2) to 3) until the model converges.

[0046] The present application is based on a multi-layer Transformer model, the purpose is to find an optimal segmentation scheme and find an optimal placement scheme for the cut sub-model, so that the training speed of the whole network is the fastest. The present application proposes an algorithm to solve this problem, named M-SLT algorithm. This algorithm relies on a decentralized architecture, in which the Transformer model is divided into multiple sub-models and placed on a set of edge devices, and the intermediate results are exchanged through D2D links. Compared with the client-server architecture, M-SLT improves the communication efficiency, because D2D communication usually has lower delay and higher data rate than communication with edge server through cellular link. In addition, by fully exploiting the computing power of edge devices, segmented learning can be carried out in a resource-saving and flexible manner. The core of this algorithm is to find the arrangement of Transformer model subnetworks on edge devices, through this algorithm, the optimal arrangement is obtained to obtain the optimal model training speed, after obtaining the optimal arrangement, the model is placed on the edge device in order, among them, the device with the strongest computing power arranges the most network layers, the remaining devices arrange one layer, for example, three devices get the arrangement of 2, 1, 3, there are five layers of network, the computing power of device 1 is the strongest, then device 2 arranges the first layer of network, device 1 arranges the second to fourth layer of network, and device 3 arranges the fifth layer of network.

[0047] In the present application, the problem is modeled as follows:

[0048] Using represents the mobile edge device within the device group, where device 1 owns the original training data, in the mobile edge environment, when device i transmits its own intermediate result to the remaining nodes, its transmission power is set to p i . For any two devices i and j, use d ij to represent the physical distance between them. Therefore, the data transmission rate of device i to device j can be derived using Shannon formula:

[0049]

[0050] Where W is the bandwidth of the channel, a is the path loss exponent, N o is the noise power. Due to the inherent computing characteristics of the Transformer model, the data volume of the intermediate result between each two layers is a constant, represented by A, so the transmission time t ij from device i to device j is A / s ij . Therefore, the core of this problem is to find an arrangement (where device 1 should be the starting point) so that the edge devices under this arrangement can efficiently train, when the arrangement is found, the number of network layers is arranged on the edge devices in turn.

[0051] To establish this optimization problem, an indicator variable x ij is defined, which is 1 when device i transmits data to device j (otherwise x ij is 0). According to the previous assumption, the indicator variable must satisfy the following constraints:

[0052]

[0053]

[0054] However, these two constraints cannot guarantee that there is only one connected component in the network transmission topology. Therefore, the following constraint needs to be introduced:

[0055]

[0056] z i is a virtual variable of device i, which can guarantee that the path formed by the modeling contains only one connected component (i.e., no sub-loop). This can be proven by contradiction. ① If device i transmits to device j, then x ij = 1, and according to the condition, z j ≥ z i + 1, that is, the value of z is consistently increasing along the loop. ② Assuming that there is a sub-loop that satisfies the above constraint, there will be a sub-loop that does not include the starting device (1), for example, 2->3->2, then z2≥z3+1≥z2+2 is contradictory. Therefore, the modeling does not include sub-loops.

[0057] w i is used to represent the time required by device i to calculate each layer of the Transformer model, so the following optimization problem can be established:

[0058]

[0059] which satisfies the following constraints:

[0060]

[0061] It can be proven that the Traveling Salesman Problem can be reduced to this optimization problem in polynomial time, and each city in the Traveling Salesman Problem can correspond to an edge device in the optimization problem, both of which can be abstracted as finding the shortest Hamiltonian path in a graph. Therefore, this optimization problem is an NP-hard problem, which can only be solved by approximate solutions or brute-force search algorithms.

[0062] Referring to Figure 4The main process of the method is as follows: first, the communication time between each device in the device group is determined according to the signal transmission power of each device in the device group, the distance between each two devices, the channel bandwidth, the channel noise and the like. The branch and bound algorithm is used to calculate the transmission path. Finally, the network model is segmented, distributed and trained according to the solved path.

[0063] Specifically:

[0064] The device group is abstracted as a complete graph G=(V, E), wherein the devices are nodes V in the graph, and the communication time and the calculation time are edges E in the graph. An adjacency matrix C is used to store the complete graph. In the search algorithm, the following definitions will be used:

[0065] The determined path U: for the current device searched, the determined path U=(r1, r2,...r k ) is obtained. U represents an order, and r k represents the kth device in the order;

[0066] The path U1 without the starting point: in the path U, the path U1 without r1 is obtained;

[0067] The path U2 without the ending point: in the path U, the path U2 without r k is obtained;

[0068] The upper bound ub: an initial transmission order is obtained, and the total time consumed by the path is used as the upper bound of the search space. When a leaf node is searched, if the total time consumed by the path is less than the upper bound, the upper bound is updated.

[0069] The lower bound lb: for the search path U of the current search position, the lower bound is updated according to the following formula. If the possible lower bound generated by the current search has exceeded the lower bound, the pruning is directly performed. If the lower bounds of all remaining unsearched positions exceed the upper bound, the result of the upper bound is directly returned.

[0070]

[0071] c is the adjacency matrix of the transmission time, c[r1] represents the r1th row of the matrix c, and c[:][r k ] represents the r k th column of the matrix c.

[0072] The following algorithm flow is executed:

[0073] a) an initial upper bound ub is obtained using a genetic algorithm, a priority queue Q is generated, and the starting point is added to the queue;

[0074] b) when the lower bound of the head point of the queue Q is less than the upper bound, the following loop is executed:

[0075] (1) take out the head element p of the queue;

[0076] (2) traverse the remaining nodes, if the node is added to the lower bound of the path p is less than or equal to the current upper bound, then it is added to the priority queue Q, if the current search to the leaf node, and the total time of the path is less than the current upper bound, then update the upper bound.

[0077] After determining the transmission path, the Transformer model is deployed in the edge device in turn according to the transmission path, wherein the edge device with the strongest computing power holds the most Transformer layers (i.e. the number of network layers minus the number of devices plus 1), and all the remaining devices hold one layer of network model.

[0078] The application also provides a Transformer model segmentation device in a mobile edge environment, comprising:

[0079] The communication time determination module is configured to select the number of mobile edge devices not greater than the number of Transformer model encoder layers as a device group, and obtain the computing power and transmission power of the devices, and calculate the D2D communication time between the devices in combination with the distance, channel bandwidth and channel noise between the devices;

[0080] The path solving module is configured to take the most Transformer layers held by the edge device with the strongest computing power and one layer of Transformer layer held by the remaining devices as the segmentation basis, establish an optimization problem model based on the D2D communication time between the devices and taking the minimization of the computing time of the Transformer model as the target, and solve the established optimization problem to obtain the optimal transmission path by using the branch and bound algorithm;

[0081] The model segmentation module is configured to segment and place the Transformer model according to the obtained optimal transmission path.

[0082] It should be understood that the Transformer model segmentation device in a mobile edge environment in the embodiments of the application can realize all the technical solutions in the above method embodiments, and the functions of each functional module can be realized according to the methods in the above method embodiments, and the specific implementation process can be referred to the related description in the above embodiments, which will not be described here.

[0083] The application also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the program is executed by the processor to realize the steps of the Transformer model segmentation method in a mobile edge environment as described above.

[0084] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the method for splitting a Transformer model in a mobile edge environment.

[0085] The application further provides a mobile edge computing system, which comprises a plurality of mobile edge devices, the mobile edge devices are assigned split Transformer models based on optimal transmission paths, the number of mobile edge devices is not greater than the number of layers of the Transformer model, and the optimal transmission paths are obtained based on the following method: selecting a number of mobile edge devices that is not greater than the number of layers of the Transformer model encoder as a device group, obtaining the computing capacity and transmission power of the devices, and calculating the D2D communication time between the devices in combination with the distance between the devices, the channel bandwidth and the channel noise; the edge device with the strongest computing capacity holds the most Transformer layers, and the remaining devices hold one layer of Transformer layer as the splitting basis, an optimization problem model is established based on the D2D communication time between the devices and taking the minimization of the computing time of the Transformer model as the target, and the branch and bound algorithm is used to solve the optimization problem to obtain the optimal transmission path.

[0086] The application models the optimization problem of splitting and learning a Transformer model using multi-hop in a mobile edge environment, proposes a splitting method (M-SLT) for assisting training of a Transformer model in a mobile edge environment, greatly reduces the time complexity of brute-force search by pruning in the search space, and finally obtains the optimal solution. The application is based on a decentralized architecture, divides the Transformer model into a plurality of sub-models, and respectively places the sub-models on a group of edge devices, and the intermediate results are exchanged through D2D links. Compared with the client-server architecture, the M-SLT improves the communication efficiency, because the D2D communication usually has lower delay and higher data rate than the communication with the edge server through the cellular link.

Claims

1. A method for splitting a Transformer model in a mobile edge environment, the method comprising: The method comprises the following steps: selecting a number of mobile edge devices not greater than a number of layers of a Transformer model encoder as a device group, obtaining computing capability and sending power of the devices, and calculating D2D communication time between the devices in combination with distances between the devices, channel bandwidths, and channel noises; taking the number of Transformer layers held by the edge device with the strongest computing capability as the most as a segmentation basis, establishing an optimization problem model based on the D2D communication time between the devices and taking minimization of computing time of the Transformer model as a target, and solving the established optimization problem by using a branch and bound algorithm to obtain an optimal transmission path; segmenting and placing the Transformer model by layer on the corresponding mobile edge devices based on the obtained optimal transmission path; The calculation method of the D2D communication time is as follows: ； wherein, denotes the transmission time from device i to device j, denotes the amount of data transmitted, denotes the data transmission rate from device i to device j, is the bandwidth of the channel, is the path loss exponent, is the noise power, denotes the transmission power of device i, denotes the physical distance between device i and device j; The established optimization problem is as follows: ； The constraint condition is as follows: ； where N is the number of mobile edge devices in the device group, is an indicator variable, taking value 1 if device i transmits data to device j, and 0 otherwise, denotes the time taken by device i to compute each layer of the Transformer model, is an integer that limits one connected component, denotes a dummy variable for device i; The optimal transmission path obtained by using the branch and bound algorithm comprises: obtaining a suboptimal solution by using a genetic algorithm as an upper bound of a search algorithm, generating a priority queue and adding a starting node to the queue; then, when the queue is not empty and a lower bound of a head node is less than a global upper bound, the following operations are performed: taking out a head element of the queue, traversing remaining nodes, if a node traversed is added to a path to obtain a lower bound less than or equal to a current upper bound, the node is added to the priority queue, if a leaf node is searched and total time consumption of the path is less than the current global upper bound, the upper bound is updated.

2. The method of claim 1, wherein, Segmenting and placing the Transformer model by layer on the corresponding mobile edge devices comprises: deploying the Transformer model in the edge devices in sequence according to the transmission path, wherein the edge device with the strongest computing capability holds the most Transformer layers, that is, a number of network layers minus a number of devices plus 1, and all the other devices hold one layer of the network model. 3.A device for splitting a Transformer model in a mobile edge environment, characterized in that, The method comprises: The communication time determination module is configured to select a number of mobile edge devices not greater than a number of layers of a Transformer model encoder as a device group, obtain computing capability and sending power of the devices, and calculate D2D communication time between the devices in combination with distances between the devices, channel bandwidths, and channel noises; The calculation method of the D2D communication time is as follows: ； wherein, denotes the transmission time from device i to device j, denotes the amount of data transmitted, denotes the data transmission rate from device i to device j, is the bandwidth of the channel, is the path loss exponent, is the noise power, denotes the transmission power of device i, denotes the physical distance between device i and device j; The path solving module is configured to take the number of Transformer layers held by the edge device with the strongest computing capability as the most as a segmentation basis, establish an optimization problem model based on the D2D communication time between the devices and take minimization of computing time of the Transformer model as a target, and solve the established optimization problem by using a branch and bound algorithm to obtain an optimal transmission path; wherein the established optimization problem is as follows: ； The constraint condition is as follows: ； where N is the number of mobile edge devices in the device group, is an indicator variable, taking value 1 if device i transmits data to device j, and 0 otherwise, denotes the time taken by device i to compute each layer of the Transformer model, is an integer that limits one connected component, denotes a dummy variable for device i; The optimal transmission path is obtained by using a branch and bound algorithm, including: using a genetic algorithm to obtain a suboptimal solution as an upper bound of the search algorithm, generating a priority queue and adding a starting node to the queue; then when the queue is not empty and the lower bound of the head node is less than the global upper bound, the following operations are performed: taking out the head element, traversing the remaining nodes, if the lower bound obtained by adding the traversed node to the path is less than or equal to the current upper bound, adding the node to the priority queue, if a leaf node is searched and the total time consumption of the path is less than the current global upper bound, updating the upper bound; The model segmentation module is configured to segment and place the Transformer model by layer on the corresponding mobile edge device based on the obtained optimal transmission path.

4. A computer device, comprising: Comprise: One or more processors; Memory; And One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs are executed by the processor to implement the steps of the Transformer model segmentation method in the mobile edge environment as claimed in any one of claims 1-2.

5. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the Transformer model segmentation method in the mobile edge environment as claimed in any one of claims 1-2.

6. A mobile edge computing system comprising a number of mobile edge devices, characterized in that, The mobile edge device is assigned a segmented Transformer model based on the optimal transmission path, wherein the number of mobile edge devices is not greater than the number of Transformer model layers, and the optimal transmission path is obtained based on the following method: Select the number of mobile edge devices not greater than the number of Transformer model encoder layers as a device group, obtain the computing capacity and transmission power of the devices, and calculate the D2D communication time between the devices based on the distance between the devices, channel bandwidth and channel noise; The edge device with the strongest computing capacity holds the most Transformer layers, and the remaining devices hold one layer of Transformer layer as the segmentation basis, an optimization problem model is established based on the D2D communication time between the devices and the objective of minimizing the computing time of the Transformer model, and a branch and bound algorithm is used to solve the optimization problem to obtain the optimal transmission path; The calculation method of D2D communication time is as follows: ； wherein, denotes the transmission time from device i to device j, denotes the amount of data transmitted, denotes the data transmission rate from device i to device j, is the bandwidth of the channel, is the path loss exponent, is the noise power, denotes the transmission power of device i, denotes the physical distance between device i and device j; The modeling optimization problem is as follows: ； The constraint condition is: ； where N is the number of mobile edge devices in the device group, is an indicator variable, taking value 1 if device i transmits data to device j, and 0 otherwise, denotes the time device i takes to compute each layer of the Transformer model, is an integer that limits one connected component, denotes a dummy variable for device i; The optimal transmission path is obtained by using a branch and bound algorithm, including: using a genetic algorithm to obtain a suboptimal solution as an upper bound of the search algorithm, generating a priority queue and adding a starting node to the queue; then when the queue is not empty and the lower bound of the head node is less than the global upper bound, the following operations are performed: taking out the head element, traversing the remaining nodes, if the lower bound obtained by adding the traversed node to the path is less than or equal to the current upper bound, adding the node to the priority queue, if a leaf node is searched and the total time consumption of the path is less than the current global upper bound, updating the upper bound.

Images