Embedding Layer Latency-Aware Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network model training methods result in high latency when a model trained on one device is executed on another device with different computing capabilities, due to varying execution latencies of network layers across devices.
Innovation Solution
A network model training method that includes constructing an initial model with an embedding layer based on preset network layer latency information, which accounts for different device types, and adjusting model parameters to minimize latency on other device types by using target latency data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the network model is trained on a server with high computing capability, then the training speed is improved, but the latency during operation on terminal apparatus with different computing capabilities increases
Solution Approach 1:
The patent applies preliminary action by constructing an embedding layer in advance that stores preset network layer latency information for multiple device types. During model training, this pre-prepared latency data is directly queried and used to calculate loss values, avoiding the need to perform actual latency measurements on target devices during training. This preliminary preparation of latency information enables the model to be optimized for multiple device types without sacrificing training efficiency.
Solution Approach 2:
The patent uses copying by creating virtual representations of different device types through the embedding layer. Instead of physically training models on multiple actual devices, the system copies latency characteristics of different device types into the embedding layer, allowing the training process to simulate and optimize for multiple device environments using a single training server. This virtual copying enables cross-device optimization without requiring multiple physical training platforms.
2Adaptability or versatility
If the network model is trained on a specified apparatus, then the training flexibility is improved, but the execution latency on other apparatus types increases
Solution Approach 1:
The patent implements universality by designing an embedding layer that serves multiple functions simultaneously: it stores latency information for multiple device types, provides latency data for loss calculation, and enables the model trained on one device type to be optimized for multiple other device types. This multi-functional embedding structure allows a single training process to produce models that perform well across diverse device platforms, achieving both training flexibility and low execution latency on various apparatus types.
Solution Approach 2:
The embedding layer acts as an intermediary between the training process and multiple target device types. Instead of directly training on each device type or making assumptions about device characteristics, the embedding layer mediates by providing accurate latency information that bridges the gap between the training environment and various deployment environments. This intermediary structure enables flexible training on specified apparatus while ensuring optimal performance on other apparatus types.
Data Source
AI summary
This disclosure discloses a network model training method and apparatus, an electronic apparatus and a computer-readable storage medium. The method includes: acquiring training data and inputting the training data into an initial model to obtain output data, wherein the initial model includes an embedding layer, the embedding layer is constructed based on preset network layer latency information, the preset network layer latency information includes network layer types and at least two types of latency data corresponding to each network layer type, and each type of latency data corresponds to different device types; inputting a current device type and a target network layer type of each target network layer in the initial model into the embedding layer to obtain target latency data corresponding to other device type; calculating a target loss value based on the target latency data, the training data and the output data, and adjusting parameters of the initial model based on the target loss value; and obtaining a target model based on the initial model in response to a training completion condition is satisfied. By means of the method, the target model has a minimum latency when running on a device corresponding to the other device type.