Neural network deployment method, apparatus, device, and medium

Abstract

The application discloses a neural network deployment method and device, equipment and a computer readable storage medium, and the method comprises the following steps: according to a preset training strategy, training a self-encoder neural network corresponding to a terminal device and an edge device and a downstream task neural network corresponding to a cloud computing center; according to a preset fixed unloading strategy and an adaptive unloading strategy, determining a first unloading node between the edge device and the cloud computing center and a second unloading node between the terminal device and the edge device through the trained self-encoder neural network and the downstream task neural network; unloading the trained downstream task neural network at the first unloading node, and unloading the trained self-encoder neural network at the second unloading node. The application can improve the network service quality on the basis of improving the neural network deployment efficiency.

Description

[0001] Neural network deployment methods, devices, equipment and media Technical Field

[0002] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and computer-readable storage medium for deploying a neural network. Background Technology

[0003] In the traditional centralized cloud model, raw data generated by terminal devices is uploaded to the cloud computing center for processing. The massive amount of data generated by these terminal devices puts enormous pressure on network bandwidth, thus reducing service quality. To address this, the edge computing paradigm is introduced. This allows edge devices, which possess certain computing and storage capabilities on the terminal side, to migrate some deep neural network models running in the cloud computing center. This enables edge intelligence, alleviating the load on the cloud computing center and reducing network bandwidth pressure.

[0004] However, since there are usually long communication links between the cloud and the edge, network instability can easily lead to increased latency and reduced efficiency in the deployment of neural network model training or inference, resulting in a decrease in network service quality.

[0005] Therefore, it is essential to figure out how to offload and schedule computing tasks at the terminal, edge, and cloud to improve the deployment efficiency of neural networks and the quality of network services. Summary of the Invention

[0006] The main objective of this invention is to provide a neural network deployment method, apparatus, device, and computer-readable storage medium, which aims to deploy a neural network system including terminal devices, edge devices, and cloud computing centers, thereby improving network service quality while enhancing neural network deployment efficiency.

[0007] To achieve the above objectives, the present invention provides a neural network deployment method, which is applied to a neural network deployment system, the neural network deployment system comprising: terminal devices, edge devices, and a cloud computing center;

[0008] The neural network deployment method includes:

[0009] According to the preset training strategy, the autoencoder neural networks corresponding to the terminal devices and the edge devices, as well as the downstream task neural networks corresponding to the cloud computing center, are trained.

[0010] Based on the preset fixed offloading strategy and adaptive offloading strategy, the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device are determined respectively.

[0011] The trained downstream task neural network is unloaded at the first unloading node, and the trained autoencoder neural network is unloaded at the second unloading node for neural network deployment.

[0012] Optionally, before the step of training the autoencoder neural networks corresponding to the terminal device and the edge device, and the downstream task neural network corresponding to the cloud computing center according to a preset training strategy, the method further includes:

[0013] Construct autoencoder neural networks corresponding to the terminal device and the edge device, and construct downstream task neural networks corresponding to the cloud computing center.

[0014] Optionally, the step of constructing the autoencoder neural network of the terminal device and the edge device, and constructing the downstream task neural network of the cloud computing center, includes:

[0015] Determine the computing power of the neural network running in the terminal device, the edge device, and the cloud computing center;

[0016] The target computation time, target compressed data volume, and task execution capability of the autoencoder neural network are obtained, and the expected running time of the autoencoder neural network is obtained based on the computational capability.

[0017] Based on the target computation time, the target compressed data volume, the task execution capability, and the expected running time, the model evaluation index of the autoencoder neural network is determined.

[0018] Multiple initial neural networks are obtained by sampling models from a predefined model search space;

[0019] Based on the model evaluation metrics, the plurality of initial neural networks are screened to obtain the autoencoder neural network;

[0020] Based on the hidden layer features of the autoencoder neural network, a downstream task neural network corresponding to the cloud computing center is constructed.

[0021] Optionally, before the step of training the autoencoder neural network and the downstream task neural network according to a preset training strategy, the method further includes:

[0022] The preset training strategy is determined based on the data type of the dataset collected by the terminal device, wherein the data type includes: private dataset and public dataset, the preset training strategy corresponding to the private dataset includes a first training strategy, and the preset training strategy corresponding to the public dataset includes a second training strategy.

[0023] When the data type of the dataset is a privacy dataset, the step of training the autoencoder neural network and the downstream task neural network according to a preset training strategy includes:

[0024] According to the first training strategy, the autoencoder neural networks corresponding to the terminal device and the edge device are trained. The encoder weights of the trained autoencoder neural network are frozen, and the hidden layer features of the trained autoencoder neural network are obtained. Based on the hidden layer features, the downstream task neural network of the cloud computing center is trained in the cloud; or,

[0025] According to the first training strategy, the autoencoder neural network and the downstream task neural network are trained simultaneously.

[0026] Optionally, when the dataset is a public dataset, the step of training the autoencoder neural network and the downstream task neural network according to a preset training strategy includes:

[0027] According to the second training strategy, the autoencoder neural network is trained in the cloud, the encoder weights of the trained autoencoder neural network are frozen, and the downstream task neural network is trained in the cloud based on the hidden layer features of the trained autoencoder neural network; or,

[0028] According to the second training strategy, the autoencoder neural network and the downstream task neural network are concatenated to obtain the overall neural network, and the overall neural network is trained in the cloud.

[0029] Optionally, the step of determining the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device respectively, based on a preset fixed offloading strategy and an adaptive offloading strategy, through a trained autoencoder neural network and a downstream task neural network, includes:

[0030] According to the fixed offloading strategy, the first offloading node between the edge device and the cloud computing center is determined by the trained autoencoder neural network and the downstream task neural network.

[0031] The adaptive offloading strategy is used to cluster multiple terminal devices according to their computing performance to obtain a preset number of terminal device groups, wherein each terminal device group is matched with a corresponding edge device.

[0032] The following data are obtained: the first and second running times of the autoencoder neural network on the edge device and the terminal device group; the third running time of the downstream task neural network on the cloud computing center; the first communication duration between the terminal device and the edge device; the second communication duration between the edge device and the cloud computing center; and the result return time of the cloud computing center.

[0033] Based on the first running time, the second running time, the third running time, the first communication time, the second communication time, and the result return time, an average forward propagation time metric is constructed between the edge device and the cloud computing center.

[0034] Based on the average time taken for a single forward propagation, a second offloading node is determined between the terminal device and the edge device.

[0035] Optionally, after the step of clustering multiple terminal devices into a preset number of terminal device groups based on the computing performance of the terminal devices using the adaptive offloading strategy, the method further includes:

[0036] Based on the network communication quality of the terminal devices in the terminal device group, the matching relationship between the terminal devices and the edge devices is adjusted to obtain a new terminal device clustering result. Based on the new terminal device clustering result, the step of determining the offloading node of the terminal devices and the edge devices through the adaptive offloading strategy is executed.

[0037] To achieve the above objectives, the present invention also provides a neural network deployment apparatus, the neural network deployment apparatus comprising:

[0038] The training module is used to train the autoencoder neural networks corresponding to the terminal devices and the edge devices, as well as the downstream task neural networks corresponding to the cloud computing center, according to a preset training strategy.

[0039] The determination module is used to determine, based on a preset fixed offloading strategy and an adaptive offloading strategy, a trained autoencoder neural network and a downstream task neural network, a first offloading node between the edge device and the cloud computing center, and a second offloading node between the terminal device and the edge device, respectively.

[0040] An unloading module is used to unload the trained downstream task neural network at the first unloading node and the trained autoencoder neural network at the second unloading node, so as to deploy the neural network deployment system.

[0041] To achieve the above objectives, the present invention also provides an apparatus comprising a memory, a processor, and a neural network deployment program stored in the memory and executable on the processor, wherein the neural network deployment program, when executed by the processor, implements the steps of the neural network deployment method as described above.

[0042] Furthermore, to achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a neural network deployment program, which, when executed by a processor, implements the steps of the neural network deployment method as described above.

[0043] To achieve the above objectives, the present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the neural network deployment method as described above.

[0044] This invention provides a neural network deployment method, apparatus, device, computer-readable storage medium, and computer program product. The method involves training the autoencoder neural networks corresponding to the terminal device and the edge device, as well as the downstream task neural network corresponding to the cloud computing center, according to a preset training strategy. Based on preset fixed offloading and adaptive offloading strategies, a first offloading node between the edge device and the cloud computing center and a second offloading node between the terminal device and the edge device are determined. The trained downstream task neural network is offloaded at the first offloading node, and the trained autoencoder neural network is offloaded at the second offloading node to deploy the neural network.

[0045] Compared to existing edge device-based neural network deployment methods, this invention trains the autoencoder neural networks corresponding to terminal devices and edge devices, as well as the downstream task neural network corresponding to the cloud computing center. Furthermore, using the trained neural networks and based on preset fixed and adaptive offloading strategies, a first offloading node between the edge devices and the cloud computing center, and a second offloading node between the terminal devices and the edge devices are determined. Then, the trained downstream task neural network and autoencoder neural network are offloaded at the first and second offloading nodes, respectively. Therefore, this invention fully utilizes the resource heterogeneity between different devices by training the neural networks of the terminal devices, edge devices, and cloud computing center separately, thereby reducing communication overhead and load between devices and ensuring data privacy. In addition, by using two offloading strategies—fixed and adaptive—to determine the neural network offloading nodes and offload the neural network at the corresponding nodes, the load on the edge devices is reduced, thus improving the deployment efficiency of the neural network system and significantly enhancing network communication quality. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention;

[0047] Figure 2 This is a first flowchart illustrating an embodiment of the neural network deployment method of the present invention;

[0048] Figure 3 This is a second flowchart illustrating an embodiment of the neural network deployment method of the present invention;

[0049] Figure 4-1 This is a schematic diagram of the first training of an autoencoder neural network according to an embodiment of the neural network deployment method of the present invention;

[0050] Figure 4-2 This is a schematic diagram of the second training of an autoencoder neural network according to an embodiment of the neural network deployment method of the present invention;

[0051] Figure 4-3 This is a schematic diagram of the third training of an autoencoder neural network according to an embodiment of the neural network deployment method of the present invention;

[0052] Figure 4-4 This is a schematic diagram of the fourth training step of the autoencoder neural network according to an embodiment of the neural network deployment method of the present invention;

[0053] Figure 5 This is a schematic diagram of neural network unloading training in an embodiment of the neural network deployment method of the present invention;

[0054] Figure 6 This is a schematic diagram of the functional modules of an embodiment of the neural network deployment device of the present invention.

[0055] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0057] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0058] The device in this embodiment of the invention can be a smartphone, tablet computer, computer, server or other network device, etc. The device in this embodiment can be used to achieve efficient deployment of neural networks.

[0059] like Figure 1As shown, the device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0060] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the neural network deployment device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0061] like Figure 1 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a neural network deployment program. The operating system is a program that manages and controls the device's hardware and software resources, supporting the operation of the neural network deployment program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the neural network deployment program stored in the memory 1005 and perform the following operations:

[0062] According to the preset training strategy, the autoencoder neural networks corresponding to the terminal devices and the edge devices, as well as the downstream task neural networks corresponding to the cloud computing center, are trained.

[0063] Based on the preset fixed offloading strategy and adaptive offloading strategy, the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device are determined respectively.

[0064] The trained downstream task neural network is unloaded at the first unloading node, and the trained autoencoder neural network is unloaded at the second unloading node for neural network deployment.

[0065] Furthermore, before the step of training the autoencoder neural networks corresponding to the terminal device and the edge device, as well as the downstream task neural network corresponding to the cloud computing center, according to the preset training strategy, the processor 1001 can call the neural network deployment program stored in the memory 1005 and perform the following operations:

[0066] Construct autoencoder neural networks corresponding to the terminal device and the edge device, and construct downstream task neural networks corresponding to the cloud computing center.

[0067] Furthermore, the processor 1001 can be used to invoke the neural network deployment program stored in the memory 1005 and perform the following operations:

[0068] Determine the computing power of the neural network running in the terminal device, the edge device, and the cloud computing center;

[0069] The target computation time, target compressed data volume, and task execution capability of the autoencoder neural network are obtained, and the expected running time of the autoencoder neural network is obtained based on the computational capability.

[0070] Based on the target computation time, the target compressed data volume, the task execution capability, and the expected running time, the model evaluation index of the autoencoder neural network is determined.

[0071] Multiple initial neural networks are obtained by sampling models from a predefined model search space;

[0072] Based on the model evaluation metrics, the plurality of initial neural networks are screened to obtain the autoencoder neural network;

[0073] Based on the hidden layer features of the autoencoder neural network, a downstream task neural network corresponding to the cloud computing center is constructed.

[0074] Furthermore, before the step of training the autoencoder neural network and the downstream task neural network according to the preset training strategy, the processor 1001 may call the neural network deployment program stored in the memory 1005 and perform the following operations:

[0075] The preset training strategy is determined based on the data type of the dataset collected by the terminal device, wherein the data type includes: private dataset and public dataset, the preset training strategy corresponding to the private dataset includes a first training strategy, and the preset training strategy corresponding to the public dataset includes a second training strategy.

[0076] The processor 1001 can be used to call the neural network deployment program stored in the memory 1005 and perform the following operations:

[0077] According to the first training strategy, the autoencoder neural networks corresponding to the terminal device and the edge device are trained. The encoder weights of the trained autoencoder neural network are frozen, and the hidden layer features of the trained autoencoder neural network are obtained. Based on the hidden layer features, the downstream task neural network of the cloud computing center is trained in the cloud; or,

[0078] According to the first training strategy, the autoencoder neural network and the downstream task neural network are trained simultaneously.

[0079] Furthermore, the processor 1001 can be used to invoke the neural network deployment program stored in the memory 1005 and perform the following operations:

[0080] According to the second training strategy, the autoencoder neural network is trained in the cloud, the encoder weights of the trained autoencoder neural network are frozen, and the downstream task neural network is trained in the cloud based on the hidden layer features of the trained autoencoder neural network; or,

[0081] According to the second training strategy, the autoencoder neural network and the downstream task neural network are concatenated to obtain the overall neural network, and the overall neural network is trained in the cloud.

[0082] Furthermore, the processor 1001 can be used to invoke the neural network deployment program stored in the memory 1005 and perform the following operations:

[0083] According to the fixed offloading strategy, the first offloading node between the edge device and the cloud computing center is determined by the trained autoencoder neural network and the downstream task neural network.

[0084] The adaptive offloading strategy is used to cluster multiple terminal devices according to their computing performance to obtain a preset number of terminal device groups, wherein each terminal device group is matched with a corresponding edge device.

[0085] The following data are obtained: the first and second running times of the autoencoder neural network on the edge device and the terminal device group; the third running time of the downstream task neural network on the cloud computing center; the first communication duration between the terminal device and the edge device; the second communication duration between the edge device and the cloud computing center; and the result return time of the cloud computing center.

[0086] Based on the first running time, the second running time, the third running time, the first communication time, the second communication time, and the result return time, an average forward propagation time metric is constructed between the edge device and the cloud computing center.

[0087] Based on the average time taken for a single forward propagation, a second offloading node is determined between the terminal device and the edge device.

[0088] Furthermore, the processor 1001 can be used to invoke the neural network deployment program stored in the memory 1005 and perform the following operations:

[0089] Based on the network communication quality of the terminal devices in the terminal device group, the matching relationship between the terminal devices and the edge devices is adjusted to obtain a new terminal device clustering result. Based on the new terminal device clustering result, the step of determining the offloading node of the terminal devices and the edge devices through the adaptive offloading strategy is executed.

[0090] Because edge devices can be deployed across a wide range, some deep neural network models that run on the central cloud can be migrated to edge devices, thereby enabling edge intelligence and alleviating the load on the central cloud and network bandwidth pressure. Therefore, edge computing has been widely used in computationally intensive tasks in edge-cloud environments.

[0091] However, edge intelligence also has some problems that cannot be ignored: there are usually long communication links between the cloud and the edge, which can easily lead to increased latency and reduced efficiency in the deployment of neural network model training or inference when the network is unstable.

[0092] Based on the above problems, and considering that edge devices are resource-constrained and dynamically changing, and that terminal devices also have different communication and computing capabilities, exhibiting strong resource heterogeneity, this invention aims to offload and schedule computing tasks on edge nodes to achieve the design and deployment of multi-strategy neural network models.

[0093] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the neural network deployment method of the present invention.

[0094] This invention provides an embodiment of a neural network deployment method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0095] The neural network deployment method of the present invention is applied to a neural network deployment system, which includes multiple terminal devices (e.g., the terminal devices may be data acquisition devices of the Internet of Things), multiple edge devices, and multiple cloud computing centers.

[0096] Based on this, the neural network deployment method in this embodiment includes the following steps:

[0097] Step S10: According to the preset training strategy, train the autoencoder neural network corresponding to the terminal device and the edge device, as well as the downstream task neural network corresponding to the cloud computing center.

[0098] It should be noted that, in this embodiment, autoencoder neural networks corresponding to terminal devices and edge devices can be pre-built, and downstream task neural networks corresponding to the cloud computing center can be built.

[0099] Based on this, after constructing the autoencoder neural network in terminal devices and edge devices, as well as the downstream task neural network in the cloud computing center, the aforementioned autoencoder neural network and downstream task neural network will be further trained in order to further deploy the trained autoencoder neural network and downstream task neural network.

[0100] Step S20: Based on the preset fixed offloading strategy and adaptive offloading strategy, the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device are determined respectively through the trained autoencoder neural network and downstream task neural network.

[0101] Step S30: Unload the trained downstream task neural network at the first unloading node and unload the trained autoencoder neural network at the second unloading node to deploy the neural network.

[0102] In this embodiment, as Figure 3 As shown, after the above-mentioned autoencoder neural network and downstream task neural network are trained, the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device will be determined according to the preset fixed offloading strategy and adaptive offloading strategy, and through the trained autoencoder neural network and downstream task neural network.

[0103] Furthermore, at the first unloading node, the trained downstream task neural network is unloaded, allowing it to be moved to the cloud computing center. Simultaneously, at the second unloading node, the trained autoencoder neural network is unloaded, enabling it to be moved to the corresponding terminal and edge devices.

[0104] As can be seen, in this embodiment, an unloading strategy is provided through the above unloading operation, which includes a fixed unloading strategy and an adaptive unloading strategy. Through the above two unloading strategies, the resource information of the device can be used to enable the terminal to perform clustering, so that the neural network model can be efficiently unloaded in terminal, edge and cloud environments, thereby reducing the overall inference latency of the neural network model. Moreover, the terminal device only needs to store a part of the neural network model.

[0105] Furthermore, prior to step S10 above, the following may also be included:

[0106] Step S40: Construct autoencoder neural networks in the terminal device and the edge device respectively, and construct downstream task neural networks in the cloud computing center.

[0107] In this embodiment, as Figure 3 As shown, the autoencoder neural network in the aforementioned terminal devices and edge devices is first constructed. At the same time, the downstream task neural network of the cloud computing center also needs to be constructed.

[0108] By constructing autoencoder neural networks and downstream task neural networks on terminal devices, edge devices, and multiple cloud computing centers respectively, the neural network model can make full use of heterogeneous resources at the end (terminal device), edge (edge ​​device), and cloud (cloud computing center) levels. This further reduces the inference latency of the neural network model, reduces the communication overhead between the edge server and the cloud computing center, and meets data privacy requirements.

[0109] Specifically, for example, in this embodiment, the number of terminals is a Defined as The number of edge devices is b Defined as Cloud computing center is defined as 。k Layered neural network model is defined as The amount of data output by each layer is l If in i and j If the neural network is unloaded at the point of application, the model running on the terminal will be... The model for edge device operation is The model that the cloud computing center operates on is That is, the complete neural network model is Among them, the autoencoder neural network is The downstream task neural network is .

[0110] In this embodiment, an autoencoder neural network is first constructed in the terminal device and the edge device. Simultaneously, a downstream task neural network in the cloud computing center is also constructed. After constructing the autoencoder neural network in the terminal device and the edge device, and the downstream task neural network in the cloud computing center, they are further trained. After training, based on preset fixed and adaptive offloading strategies, and using the trained autoencoder neural network and downstream task neural network, offloading nodes between the terminal device and the edge device, and between the edge device and the cloud computing center, are determined to achieve neural network deployment.

[0111] Compared to existing edge device-based neural network deployment methods, this invention first deploys neural networks on terminal devices, edge devices, and a cloud computing center separately. The neural networks are then trained. Furthermore, based on preset fixed and adaptive offloading strategies, a first offloading node between the terminal devices and edge devices, and a second offloading node between the edge devices and the cloud computing center, are determined using the trained neural networks. The trained autoencoder neural network and downstream task neural networks are then offloaded at the first and second offloading nodes, respectively. Therefore, this invention, by constructing neural networks separately on terminal devices, edge devices, and the cloud computing center, fully utilizes the resource heterogeneity between different devices, reduces communication overhead and load, improves communication quality, and ensures data privacy between devices. In addition, by using two offloading strategies—fixed and adaptive—to determine the neural network offloading nodes, efficient offloading of the neural network based on device performance is achieved, improving the deployment efficiency of the neural network.

[0112] Furthermore, based on the first embodiment of the neural network deployment method of the present invention, a second embodiment of the neural network deployment method of the present invention is proposed.

[0113] In this embodiment, step S40 above, "constructing the autoencoder neural networks in the terminal device and the edge device respectively, and constructing the downstream task neural network in the cloud computing center", may include:

[0114] Step S401: Determine the computing power of the terminal device, the edge device, and the cloud computing center;

[0115] Step S402: Obtain the target computation time, target compressed data volume, and task execution capability of the autoencoder neural network, and based on the computational capability, obtain the expected running time of the autoencoder neural network;

[0116] Step S403: Based on the target computation time, the target compressed data volume, the task execution capability, and the expected running time, determine the model evaluation index of the autoencoder neural network;

[0117] Step S404: Sample models from a preset model search space to obtain multiple initial neural networks;

[0118] Step S405: Based on the model evaluation index, the plurality of initial neural networks are screened to obtain the autoencoder neural network;

[0119] Step S406: Construct the downstream task neural network in the cloud computing center based on the hidden layer features of the autoencoder neural network.

[0120] In this embodiment, the specific process of constructing the autoencoder neural network corresponding to the terminal device, the edge terminal, and the downstream task neural network of the cloud computing center includes:

[0121] 1) Calculate the computing power (i.e., the computing power in this embodiment) that can be provided by each device in the terminal, edge server, and cloud computing center. Define the devices. The number of floating-point calculations per second is The memory swap bandwidth is Neural network model Its computational complexity is The memory swap volume is Obtain the equipment d The calculated strength is ,Model m The calculated strength is Then the equipment d The computing power required to run a neural network model is ;

[0122] 2) Design an autoencoder neural network. Based on data privacy requirements, the autoencoder neural network is deployed on terminal devices and edge devices (such as edge servers), and the computing power required by the encoder portion matches the average computing power that the terminal devices and edge devices can provide collaboratively. Specifically, define the target computation time. t target and target compressed data volume l target and the encoder's task execution capabilities. Task ( M AE The encoder's partial runtime (i.e., the expected runtime in this embodiment):

[0123]

[0124] in, Avg ( P T )and Avg ( P E ) represent the average computing power of terminal devices and edge devices, respectively.

[0125] 3) Calculate the time consumption based on the target defined above. t target Target compressed data volume l target Expected runtime and task execution capability Task ( M AE ), and combined with preset parameters Determine the model evaluation metrics for autoencoder neural networks:

[0126]

[0127] 4) Design initial neural networks by setting different convolution schemes, pooling schemes, fully connected schemes, activation function schemes, residual schemes, and concatenation schemes as a preset model search space. Then, model sampling is performed from this model search space to obtain multiple initial neural networks. These initial neural networks are then quickly trained with a small number of samples (datasets from the terminal device). Finally, the initial neural networks are selected based on model evaluation metrics to obtain an autoencoder neural network. It is understood that the computing power required by the obtained autoencoder neural network is matched with the average computing power that can be provided by the terminal device and edge device in collaboration.

[0128] 5) Based on the hidden layer features of the constructed autoencoder neural network, design a downstream task neural network deployed in a cloud computing center. The computing power required by this model should match the computing power that the cloud computing center can provide. Define the target computation time of this downstream task neural network. t target Set preset parameters At this point, the model evaluation metrics for the downstream task neural network are:

[0129]

[0130] in, t (M down The time taken to run the downstream task neural network is denoted by ). Furthermore, based on this model evaluation metric, the steps for sampling models from the preset model search space and ultimately obtaining the downstream task neural network are consistent with the aforementioned autoencoder neural network construction method, and will not be elaborated upon here.

[0131] Furthermore, before step S10, "training the autoencoder neural networks corresponding to the terminal device and the edge device, and the downstream task neural network corresponding to the cloud computing center according to a preset training strategy," the following may also be included:

[0132] Step S50: Determine the preset training strategy based on the data type of the dataset collected by the terminal device, wherein the data type includes: private dataset and public dataset, the preset training strategy corresponding to the private dataset includes a first training strategy, and the preset training strategy corresponding to the public dataset includes a second training strategy;

[0133] It should be noted that, in order to ensure the user's data privacy requirements, the corresponding training strategy needs to be determined in advance. The dataset in this embodiment can be a dataset collected by the terminal device. In this embodiment, according to the user's data privacy requirements, the dataset can be divided into private datasets and public datasets. Then, depending on the different dataset types, the first training strategy or the second training strategy in the preset training strategy can be used to train the constructed autoencoder neural network and the downstream task neural network.

[0134] Based on this, step S10 above, "training the autoencoder neural network and the downstream task neural network according to a preset training strategy," may include:

[0135] Step S101: According to the first training strategy, train the autoencoder neural networks corresponding to the terminal device and the edge device; freeze the encoder weights of the trained autoencoder neural network; obtain the hidden layer features of the trained autoencoder neural network; and train the downstream task neural network of the cloud computing center in the cloud based on the hidden layer features; or,

[0136] Step S102: According to the first training strategy, the autoencoder neural network and the downstream task neural network are trained simultaneously.

[0137] Based on the above description, when the dataset collected through the terminal device is a privacy dataset, in order to ensure data security, in this embodiment, the first training strategy can be used to train the constructed autoencoder network and the downstream task neural network.

[0138] Specifically, for example, such as Figures 4-1 to 4-2As shown in this embodiment, when the data type of the dataset is specifically a privacy dataset, it is also necessary to consider whether to train the autoencoder neural network independently. It can be understood that if there are privacy requirements for the data (i.e., the data type of the dataset is privacy-sensitive data), then the dataset can only be accessed by terminal devices or edge devices. In this case, there are two training schemes:

[0139] 1) When an autoencoder needs to be trained independently. For example... Figure 4-1 As shown, the first training strategy at this time can be as follows: complete the training of the autoencoder neural network in the terminal and edge environment, freeze the encoder weights of the trained autoencoder neural network, then obtain the hidden layer features of the trained autoencoder neural network, send the hidden features to the cloud computing center, and then train the downstream task neural network based on the hidden features.

[0140] 2) Alternatively, when it is not necessary to train the autoencoder independently, such as Figure 4-2 As shown, the first training strategy at this time can be as follows: train the encoder neural network and the downstream task neural network simultaneously on the edge device and the cloud computing center, respectively.

[0141] Furthermore, in step S10 above, "training the autoencoder neural networks corresponding to the terminal device and the edge device, as well as the downstream task neural network corresponding to the cloud computing center, according to a preset training strategy" may further include:

[0142] Step S103: According to the second training strategy, train the autoencoder neural network in the cloud, freeze the encoder weights of the trained autoencoder neural network, and train the downstream task neural network in the cloud based on the hidden layer features of the trained autoencoder neural network; or,

[0143] Step S104: According to the second training strategy, the autoencoder neural network and the downstream task neural network are spliced ​​together to obtain the overall neural network, and the overall neural network is trained in the cloud.

[0144] Based on the above description, in this embodiment, if there is no requirement for data privacy, the second training strategy described above can be used to train the encoder neural network and the downstream task neural network.

[0145] Specifically, such as Figure 4-3 and Figure 4-4 As shown, the first step is to send the dataset collected through the terminal device to the cloud.

[0146] 1) When it is necessary to train the autoencoder independently, such as Figure 4-3As shown, the second training strategy at this point can specifically be as follows: The autoencoder neural network is trained in the cloud, and then the encoder weights are frozen after training. Furthermore, the downstream task neural network is connected to the hidden layer output of the trained autoencoder neural network, and the hidden layer features of the trained autoencoder neural network are used to train the downstream task neural network in the cloud.

[0147] 2) Alternatively, when it is not necessary to train the autoencoder independently, such as Figure 4-4 As shown, the second training strategy at this time can be as follows: the autoencoder neural network is spliced ​​with the downstream task neural network to obtain a complete overall neural network, and then the overall neural network is directly trained in the cloud.

[0148] In one embodiment, in incremental training mode, the update strategy for encoder weights running on terminal devices and edge devices can be selected as no update, partial update, or full update to meet different task requirements and have better generalization.

[0149] Further, in step S20 above, "based on preset fixed offloading strategies and adaptive offloading strategies, the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device are determined respectively through the trained autoencoder neural network and downstream task neural network," may include:

[0150] Step S201: According to the fixed offloading strategy, the first offloading node between the edge device and the cloud computing center is determined through the trained autoencoder neural network and the downstream task neural network.

[0151] Step S202: Based on the adaptive offloading strategy, the second offloading node of the terminal device and the edge device is determined through the trained autoencoder neural network and the downstream task neural network.

[0152] Step S203: Using the adaptive offloading strategy, based on the computing performance of the terminal devices, multiple terminal devices are clustered to obtain a preset number of groups of terminal devices, wherein each group of terminal devices has a matching edge device.

[0153] Step S204: Obtain the first and second running times of the autoencoder neural network run by the edge device and the terminal device group, the third running time of the downstream task neural network run by the cloud computing center, the first communication duration between the terminal device and the edge device, the second communication duration between the edge device and the cloud computing center, and the result return time of the cloud computing center.

[0154] Step S205: Based on the first running time, the second running time, the third running time, the first communication time, the second communication time, and the result return time, construct the average time consumption of a single forward propagation between the edge device and the cloud computing center;

[0155] Step S206: Based on the average time taken for a single forward propagation, determine the offloading node between the terminal device and the edge device.

[0156] It should be noted that, in this embodiment, in order to efficiently offload the neural network at the terminal device, edge device, and cloud computing center, and reduce the overall latency of neural network model inference deployment, a preset fixed offloading strategy and an adaptive offloading strategy are adopted respectively. The first offloading node of the edge device and the cloud computing center and the second offloading node of the terminal device and the edge device are determined, so that the trained downstream task neural network is offloaded at the first offloading node, and the trained autoencoder neural network is offloaded at the second offloading node.

[0157] Specifically, for example, the offloading node between edge devices and the cloud computing center can be directly implemented using the aforementioned fixed offloading strategy, such as... Figure 5 As shown, it can be uninstalled at the node. j The trained autoencoder neural network is offloaded to the terminal device and edge server, while the trained downstream task neural network is offloaded to the cloud computing center. However, for the offloading node between the terminal device and the edge device... i Determining the optimal parameters requires comprehensive consideration of factors such as the computing performance of the terminal devices and the resource heterogeneity between the terminal devices and edge devices.

[0158] Specifically, for example:

[0159] 1) Considering the communication quality between terminal devices and edge devices, as well as between edge devices and the cloud computing center, and to achieve maximum computing efficiency based on device computing power, one edge device can be allocated to each type of terminal device. For example, to group heterogeneous terminal devices with similar computing performance into one category, the number of clusters (i.e., the preset number of categories in this embodiment) is defined as the number of edge devices, b. Furthermore, the range for the number of terminals in each cluster is set as follows: This involves considering the computing power of each terminal device, and arranging the terminal devices in ascending order of computing power. The edge devices are arranged in descending order of computing power. Based on this, it will be divided into the above order. b Group and have That is, the terminal In clustering and pair(K,E) One-to-one correspondence.

[0160] 2) For a complete forward propagation (in this embodiment, "forward" is defined as the propagation direction from the edge device to the cloud computing center). Specifically, define... B For network bandwidth, t d This is for network latency. Furthermore, based on the aforementioned clustered groups of terminal devices and their matching edge devices, in the clustering... k Next n The runtime of the autoencoder neural network running on each terminal device (i.e., the first runtime): The communication time between the terminal device and the corresponding edge device (i.e., the first communication time): The runtime of the edge device running its autoencoder neural network (i.e., the second runtime): The communication time between the edge device and the cloud computing center (i.e., the second communication duration): The runtime of the downstream task neural network in the cloud computing center (i.e., the third runtime): And the result return time: (The result return time in this embodiment can be understood as the time required for the cloud computing center to return the task calculation result to the task initiating device, such as the terminal device). At this time, if the terminal device does not run the autoencoder neural network (this can be understood as the autoencoder neural network not performing segmentation, i.e., the second offload point is canceled, and the autoencoder neural network runs completely at the edge. However, in other cases, the autoencoder neural network is segmented at the offload point, with the first part running at the terminal and the second part running at the edge), then the communication time between the terminal device and the edge device is: ,in, Data This refers to data collected from terminal devices that needs to be predicted. Furthermore, assuming no task congestion occurs on edge devices, the optimal method for selecting the terminal device clustering and matching it with the corresponding edge devices for each type of terminal device is as follows: And the optimal offload point between terminal devices and edge devices in the class. The average time of a single forward propagation can be used as an evaluation metric:

[0161]

[0162] in, That is, by solving... This allows us to select the optimal terminal clustering and edge device matching method, as well as the optimal offloading node between the terminal device and the edge device.

[0163] Furthermore, after step S203 above, "using the adaptive offloading strategy to cluster multiple terminal devices according to the computing performance of the terminal devices to obtain a preset number of groups of terminal devices," it may further include:

[0164] Step S207: Based on the network communication quality of each terminal device in each group of terminal devices, adjust the matching relationship between each terminal device and the edge device to obtain a new clustering result of the terminal devices. Based on the new clustering result of the terminal devices, execute the step of determining the offloading node of the terminal device and the edge device through the adaptive offloading strategy.

[0165] It should be noted that, in this embodiment, after clustering the terminal devices and determining the edge devices that match each type of terminal device, the matching method between the terminal devices and edge devices in the class can be adjusted in order to further and continuously track the subsequent operation of the neural network system, so as to improve the system service quality.

[0166] Specifically, for example, starting from terminal device 1 and proceeding to terminal device a, network communication quality is considered sequentially to determine whether to replace the currently assigned edge device for each terminal device. For example, terminal device... T r Compare the currently allocated terminal communication bandwidth Communication bandwidth with adjacent terminals At this point, if:

[0167]

[0168] And the performance changes meet Furthermore, the number of terminals handled by the three edge devices after the transfer still meets the requirements. Then the terminal device T r Introducing new edge devices middle.

[0169] In one embodiment, after the edge-cloud neural network system has been running for a certain period of time, if the health status of the terminal device and the edge device or the computing power allocated to the model is... P If changes occur and do not recover within a certain period of time, clustering can be optimized in whole or in part. K Matching relationship between clustering terminals and edge devices and encoder unload point This achieves a new balance. Additionally, copies of the downstream task neural network models from the cloud computing center can be stored on edge devices to enable timely switching to local inference when the cloud computing center becomes unavailable, ensuring service stability.

[0170] This invention enables neural networks to fully utilize heterogeneous resources across edge and cloud environments, reducing model inference latency, minimizing communication overhead between edge servers and the central cloud, and meeting data privacy requirements. Furthermore, by collecting resource information from each device, such as computing power and communication bandwidth, a suitable neural network model for device operation is designed. An autoencoder's hidden layer compresses the original data, reducing communication load and mitigating communication quality degradation caused by network fluctuations. Simultaneously, different training and deployment strategies are employed to meet varying data privacy requirements, alleviating cloud center computing pressure and making the invention universally applicable to various neural network system deployment scenarios. A two-stage offloading strategy utilizes device resource information to cluster terminal devices, allowing the neural network model to be efficiently offloaded across terminal, edge, and cloud environments, resulting in a reduction in overall neural network inference latency. Moreover, terminal devices only need to store a portion of the neural network model. By continuously tracking the system's subsequent operation, a copy of the downstream model from the central cloud is stored on the edge server, allowing for local inference when necessary, and dynamically adjusting the offloading status to achieve optimal inference.

[0171] Furthermore, embodiments of the present invention also propose a neural network deployment device, referring to... Figure 6 The neural network deployment device includes:

[0172] The neural network deployment device includes:

[0173] The training module is used to train the autoencoder neural networks corresponding to the terminal devices and the edge devices, as well as the downstream task neural networks corresponding to the cloud computing center, according to a preset training strategy.

[0174] The determination module is used to determine, based on a preset fixed offloading strategy and an adaptive offloading strategy, a trained autoencoder neural network and a downstream task neural network, a first offloading node between the edge device and the cloud computing center, and a second offloading node between the terminal device and the edge device, respectively.

[0175] The unloading module is used to unload the trained downstream task neural network at the first unloading node and the trained autoencoder neural network at the second unloading node for neural network deployment.

[0176] The extended content of the specific implementation of the neural network deployment device of the present invention is basically the same as the embodiments of the neural network deployment method described above, and will not be repeated here.

[0177] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a neural network deployment program, which, when executed by a processor, implements the steps of the neural network deployment method described below.

[0178] The various embodiments of the neural network deployment device and computer-readable storage medium of the present invention can be referred to the various embodiments of the neural network deployment method of the present invention, and will not be repeated here.

[0179] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0180] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0181] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a device (which may be a smartphone, computer, server, etc.) to execute the methods described in the various embodiments of the present invention.

[0182] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for deploying a neural network, characterized in that, The neural network deployment method is applied to a neural network deployment system, which includes: terminal devices, edge devices, and a cloud computing center; The neural network deployment method includes: According to the preset training strategy, the autoencoder neural networks corresponding to the terminal devices and the edge devices, as well as the downstream task neural networks corresponding to the cloud computing center, are trained. Based on the preset fixed offloading strategy and adaptive offloading strategy, the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device are determined respectively. The trained downstream task neural network is unloaded at the first unloading node, and the trained autoencoder neural network is unloaded at the second unloading node for neural network deployment. Before the step of training the autoencoder neural networks corresponding to the terminal device and the edge device, and the downstream task neural network corresponding to the cloud computing center according to a preset training strategy, the method further includes: Construct autoencoder neural networks corresponding to the terminal device and the edge device, and construct downstream task neural networks corresponding to the cloud computing center; The steps of constructing the autoencoder neural network of the terminal device and the edge device, and constructing the downstream task neural network of the cloud computing center, include: Determine the computing power of the neural network running in the terminal device, the edge device, and the cloud computing center; The target computation time, target compressed data volume, and task execution capability of the autoencoder neural network are obtained, and the expected running time of the autoencoder neural network is obtained based on the computational capability. Based on the target computation time, the target compressed data volume, the task execution capability, and the expected running time, the model evaluation index of the autoencoder neural network is determined. Multiple initial neural networks are obtained by sampling models from a predefined model search space; Based on the model evaluation metrics, the plurality of initial neural networks are screened to obtain the autoencoder neural network; Based on the hidden layer features of the autoencoder neural network, a downstream task neural network corresponding to the cloud computing center is constructed.

2. The neural network deployment method as described in claim 1, characterized in that, Before the step of training the autoencoder neural networks corresponding to the terminal device and the edge device, and the downstream task neural network corresponding to the cloud computing center according to a preset training strategy, the method further includes: The preset training strategy is determined based on the data type of the dataset collected by the terminal device, wherein the data type includes: private dataset and public dataset, the preset training strategy corresponding to the private dataset includes a first training strategy, and the preset training strategy corresponding to the public dataset includes a second training strategy. When the data type of the dataset is a privacy dataset, the step of training the autoencoder neural networks corresponding to the terminal device and the edge device, as well as the downstream task neural network corresponding to the cloud computing center, according to a preset training strategy, includes: According to the first training strategy, the autoencoder neural networks corresponding to the terminal device and the edge device are trained. The encoder weights of the trained autoencoder neural network are frozen, and the hidden layer features of the trained autoencoder neural network are obtained. Based on the hidden layer features, the downstream task neural network of the cloud computing center is trained in the cloud; or, According to the first training strategy, the autoencoder neural network and the downstream task neural network are trained simultaneously.

3. The neural network deployment method as described in claim 2, characterized in that, When the dataset is a public dataset, the step of training the autoencoder neural network and the downstream task neural network according to a preset training strategy includes: According to the second training strategy, the autoencoder neural network is trained in the cloud, the encoder weights of the trained autoencoder neural network are frozen, and the downstream task neural network is trained in the cloud based on the hidden layer features of the trained autoencoder neural network; or, According to the second training strategy, the autoencoder neural network and the downstream task neural network are concatenated to obtain the overall neural network, and the overall neural network is trained in the cloud.

4. The neural network deployment method as described in claim 1, characterized in that, The step of determining the first offloading node between the edge device and the cloud computing center and the second offloading node between the terminal device and the edge device, respectively, based on a preset fixed offloading strategy and an adaptive offloading strategy, using a trained autoencoder neural network and a downstream task neural network, includes: According to the fixed offloading strategy, the first offloading node between the edge device and the cloud computing center is determined by the trained autoencoder neural network and the downstream task neural network. The adaptive offloading strategy is used to cluster multiple terminal devices according to their computing performance to obtain a preset number of terminal device groups, wherein each terminal device group is matched with a corresponding edge device. The following data are obtained: the first and second running times of the autoencoder neural network on the edge device and the terminal device group; the third running time of the downstream task neural network on the cloud computing center; the first communication duration between the terminal device and the edge device; the second communication duration between the edge device and the cloud computing center; and the result return time of the cloud computing center. Based on the first running time, the second running time, the third running time, the first communication duration, the second communication duration, and the result return time, an average forward propagation time metric is constructed between the edge device and the cloud computing center. Based on the average time taken for a single forward propagation, a second offloading node is determined between the terminal device and the edge device.

5. The neural network deployment method as described in claim 4, characterized in that, After the step of clustering multiple terminal devices into a preset number of terminal device groups based on the computing performance of the terminal devices using the adaptive offloading strategy, the method further includes: Based on the network communication quality of the terminal devices in the terminal device group, the matching relationship between the terminal devices and the edge devices is adjusted to obtain a new terminal device clustering result. Based on the new terminal device clustering result, the step of determining the offloading node of the terminal devices and the edge devices through the adaptive offloading strategy is executed.

6. A neural network deployment device, characterized in that, The neural network deployment device includes: The training module is used to train the autoencoder neural networks corresponding to the terminal devices and edge devices, as well as the downstream task neural networks corresponding to the cloud computing center, according to a preset training strategy. The determination module is used to determine, according to a preset fixed offloading strategy and an adaptive offloading strategy, a first offloading node between the edge device and the cloud computing center and a second offloading node between the terminal device and the edge device, respectively. The unloading module is used to unload the trained downstream task neural network at the first unloading node and the trained autoencoder neural network at the second unloading node for neural network deployment. The training module is also used for: Construct autoencoder neural networks corresponding to the terminal device and the edge device, and construct downstream task neural networks corresponding to the cloud computing center; The training module is also used for: Determine the computing power of the neural network running in the terminal device, the edge device, and the cloud computing center; The target computation time, target compressed data volume, and task execution capability of the autoencoder neural network are obtained, and the expected running time of the autoencoder neural network is obtained based on the computational capability. Based on the target computation time, the target compressed data volume, the task execution capability, and the expected running time, the model evaluation index of the autoencoder neural network is determined. Multiple initial neural networks are obtained by sampling models from a predefined model search space; Based on the model evaluation metrics, the plurality of initial neural networks are screened to obtain the autoencoder neural network; Based on the hidden layer features of the autoencoder neural network, a downstream task neural network corresponding to the cloud computing center is constructed.

7. A device, characterized in that, The device includes a memory, a processor, and a neural network deployment program stored in the memory and executable on the processor, wherein the neural network deployment program, when executed by the processor, implements the steps of the neural network deployment method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a neural network deployment program, which, when executed by a processor, implements the steps of the neural network deployment method as described in any one of claims 1 to 5.

Images