Retraining-free quantization field adaptive method for edge device

Abstract

A retraining-free quantization field self-adaption method suitable for edge equipment comprises the steps that the optimal quantization bit width is determined through multi-quantization bit width search, a domain discriminator is constructed to reduce the feature distribution difference between a source domain and a target domain, and a subnet with the largest number of parameters and the largest quantization bit width in a quantization network is used for generating a pseudo tag. And carrying out preliminary training on the quantization network, and evaluating the convergence speed and training difficulty of each quantization subnetwork. In the subnet training stage, the subnets are grouped according to a pre-training result, a differentiated training strategy is implemented, and the subnets with slow convergence speed obtain more training resources. A knowledge distillation technology is adopted, and knowledge is transmitted from a subnet group with a high convergence speed to a subnet group with a low convergence speed, so that a learning process is accelerated and performance improvement is realized; the method effectively improves the reasoning speed of the model, reduces the reasoning overhead, improves the classification accuracy on the premise of reducing the calculation amount, and provides an efficient solution for the deployment of a deep learning model on edge equipment.

Description

Technical Field

[0001] The present invention relates to deep learning and edge computing, and in particular to a quantization domain adaptation method for edge devices without retraining. Background Art

[0002] Existing deep learning has made great progress in computer vision tasks. However, in practical applications, these models are difficult to deploy directly to edge devices due to the limited and dynamically changing computing resources of edge devices. In addition, due to the problem of domain drift in actual use, these models have weak actual performance capabilities.

[0003] Unsupervised Domain Adaptation (UDA) aims to improve the generalization ability of the model by aligning the shared representation space of the source and target domains and reducing the domain shift between the training data and the edge device environment. In order to further improve the target domain performance, researchers have tried to combine the neural architecture search (NAS) technology. SlimDA enhances the cross-domain generalization ability by integrating the weight sharing module into SymNet and sampling subnets from the model library using the Sandwich rule, which is suitable for ensemble learning tasks. Recently, AnyDA proposed a subnet architecture that supports three configurable dimensions of depth, width and resolution, which can customize subnets according to different edge device requirements. In addition, AnyDA introduced a bootstrap recursive distillation method that combines self-supervised learning with knowledge distillation to maximize the information content of the target data as a regularization method. However, these methods rely on full-precision networks and ignore the problem of numerical precision redundancy, thereby increasing computational and memory requirements and are not suitable for resource-limited edge devices. How to apply existing quantization techniques, such as LSQ and LSQ+, to numerous unsupervised domain adaptation subnets remains a research issue.

[0004] The main shortcomings of the existing technologies are: 1. In full-precision unsupervised domain adaptation, some methods configure multiple dimensions of the subnets to obtain different subnets with different computing budgets, so that retraining is not required; some methods use ensemble learning to improve the classification accuracy of all subnets; however, these methods do not consider the network's weight precision redundancy and the different effects of different quantization bit widths on different subnets. 2. In quantized unsupervised domain adaptation, existing methods use multiple quantization bit widths to quantize all subnets, and use self-boosting distillation to improve the classification accuracy of quantized subnets, but this method only uses pseudo-label technology to solve the problem of no labels in the target domain, and does not consider that different quantized subnets require different training times.

[0005] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0006] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide a quantization domain adaptation method for edge devices without retraining.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for quantization domain adaptation without retraining for edge devices, characterized by comprising the following steps:

[0009] Step 1: Quantization network construction: Perform multi-quantization bit width search to adapt to edge devices with different computing resources, and determine the optimal quantization bit width for each quantization subnet in the unsupervised domain adaptation network; build a domain discriminator for the quantization network to distinguish the features of the source domain and the target domain, so as to guide the training of the feature extraction quantization network through the feedback of the domain discriminator, so that the feature distribution of the source domain and the target domain is closer; use the pseudo-label method to generate pseudo-labels for the unlabeled target domain data through classification prediction of the subnet with the largest number of parameters and the largest quantization bit width;

[0010] Step 2: Pre-training: Perform preliminary training on the quantized network to evaluate the convergence speed and training difficulty of each quantized subnet; use the test data set to evaluate the quantized subnet, obtain the classification accuracy of each quantized subnet, and determine the convergence speed and training difficulty of the quantized subnet based on the classification accuracy;

[0011] Step 3, subnet training: According to the convergence speed and training difficulty obtained in the pre-training stage, the quantized subnets are grouped to implement differentiated training strategies; a sampling training method that is inversely proportional to the subnet convergence speed is adopted to allocate more training resources to subnets with slower convergence; knowledge distillation is implemented, and the subnet group with fast convergence speed is used as the teacher model to transfer knowledge to the subnet group with slow convergence speed as the student model, thereby accelerating the latter's learning process.

[0012] Furthermore, the domain discriminator uses a multi-layer perceptron (MLP) network to distinguish the features of the source domain and the target domain to achieve a binary classification task.

[0013] Furthermore, the output of the feature layer of each quantization subnet is input into the domain discriminator to discriminate whether the input feature comes from the source domain or the target domain; the discrimination loss is calculated and fed back for the result of the domain discriminator; gradient descent is used for the domain classification layer, and gradient reversal descent is used for the feature extraction layer.

[0014] Furthermore, the pseudo labeling method comprises:

[0015] Use the subnet with the most parameters and the largest quantization bit width among all pre-trained quantized networks to classify and predict the target domain data;

[0016] Select predictions whose classification probability exceeds a preset threshold as pseudo labels;

[0017] The pseudo labels are used to perform supervised training on unlabeled data of the target domain.

[0018] Furthermore, during subnet training, the calculation and optimization process of classification loss includes:

[0019] The features extracted by the quantization subnet are input into the classification layer to predict the classification category of the data;

[0020] For labeled source domain data, the cross entropy loss of the classification layer is directly calculated based on its true label;

[0021] For unlabeled target domain data, the classification prediction of the subnetwork with the most parameters and the largest quantization bit width among all quantized networks is used as pseudo-label for supervision to calculate the loss of the classification layer;

[0022] The classification losses of the source domain and the target domain are weighted and summed to optimize the parameters of the quantization network; preferably, the weight of the source domain loss is 0.5 and the weight of the target domain loss is 1.0.

[0023] Furthermore, for the unlabeled target domain data, the following steps are performed:

[0024] Use the subnet with the most parameters and the largest quantization bit width among all quantized networks to classify and predict the target domain data and obtain the classification probability distribution;

[0025] Determine the maximum classification probability of each target domain data sample as a candidate pseudo label;

[0026] If the maximum probability of the candidate pseudo-label reaches or exceeds the preset confidence threshold, the predicted category is used as the pseudo-label;

[0027] The pseudo-labels are used as supervisory signals to train the unlabeled data of the target domain.

[0028] Furthermore, during subnet training, a group training and knowledge distillation mechanism based on convergence speed is adopted, including:

[0029] Pre-train the quantized subnetwork to evaluate its classification accuracy, thereby determining the convergence speed of each subnetwork;

[0030] The quantized subnetworks are grouped according to the classification accuracy to achieve clustering of the subnetwork convergence speed;

[0031] Apply a sampling strategy that is inversely proportional to the subnet convergence speed for training to increase the training intensity of the subnet with slow convergence speed;

[0032] Implement group knowledge distillation and use subnets with fast convergence speed to transfer knowledge to subnets with slow convergence speed to promote the improvement of overall network performance.

[0033] Furthermore, in step one, the appropriate subnetwork dimensions, including the input resolution, width, and quantization bit width of the network, are determined through preliminary experiments to optimize the subnetwork configuration.

[0034] Furthermore, in step three, quantization-aware training (LSQ) is applied to enable the network to learn the appropriate quantization step size during training to reduce the error in the quantization process.

[0035] A computer program product includes a computer program, which, when executed by a processor, implements the quantization domain adaptation method for edge devices without retraining.

[0036] The present invention has the following beneficial effects:

[0037] The present invention proposes a quantization training method for unsupervised domain adaptation suitable for edge devices, which can achieve effective migration of quantization subnets between source domains and target domains without retraining. By introducing a domain discriminator, the present invention can effectively solve the domain offset problem that may occur between different domains of quantization subnets, thereby enhancing the generalization ability of the model in the target domain. In addition, by using the pseudo-labels generated by the subnet with the most parameters and the largest quantization bit width among all quantization networks, the present invention solves the problem of lack of labels in the target domain in unsupervised domain adaptation, so that the quantization network can be effectively trained on unlabeled data. By evaluating and classifying the convergence speed of the subnet, the present invention realizes differentiated training of subnets with fast convergence speed and subnets with slow convergence speed. Through the knowledge distillation technology, the fast-converging subnet can guide the learning of the slow-converging subnet, thereby improving the overall training efficiency and balancing the classification accuracy and reasoning efficiency. This method not only improves the inference speed of the model and reduces the overhead of model inference, but also improves the classification accuracy of the model while reducing the computing resource consumption of edge devices, reduces the requirements for downloading and storage, and achieves classification capability guarantee under low bit width, providing an efficient and practical solution for the deployment of deep learning models on edge devices.

[0038] Compared with the conventional technology, the embodiments of the present invention have the following significant advantages:

[0039] 1. The present invention is a training method for unsupervised domain adaptation of edge devices without retraining quantization. A domain discriminator is used to solve the domain offset problem between the source domain and the target domain of the quantization subnet, and unsupervised domain adaptation is extended to the quantization subnet that does not require retraining, thereby improving the inference speed of the model and reducing the model inference overhead.

[0040] 2. By evaluating and classifying the convergence speed of the subnets, we can use the subnets with fast convergence speed to distill knowledge from the subnets with slow convergence speed, and increase the training frequency of the subnets with slow convergence speed by a probability inversely proportional to the convergence speed, thus ensuring that all quantized subnets can get a reasonable number of training times. This enables all subnets to achieve a balance between classification accuracy and inference efficiency.

[0041] 3. The method of the present invention can perform inference with less edge device computing resources, use a method that does not require retraining, reduce related download and storage overhead, and improve the classification accuracy of the model while reducing the amount of calculation.

[0042] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a quantization method for unsupervised domain adaptation tasks without retraining according to an embodiment of the present invention;

[0044] Figure 2 It is an algorithm framework diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0046] In practical applications, due to the limited and dynamically changing computing resources of edge devices, it is difficult to deploy deep learning models directly to edge devices. In addition, due to the problem of domain drift in actual use, the actual performance of the model is weak. In order to solve these problems, the present invention further quantizes the unsupervised domain adaptation network. In this process, the present invention encounters the following technical problems: 1. Because of the multiple quantization bit widths, the number of subnets increases exponentially. How to train these subnets efficiently. 2. For quantized subnets, how to make them have the same classification accuracy as the full-precision network.

[0047] Application 1: Under different working conditions of different edge devices, networks with different computing amounts are required. At the same time, in order to improve the efficiency of reasoning, the network is often quantized. The present invention solves this problem through quantization technology that does not require retraining. However, when deploying quantized networks in edge devices, there will be a domain offset problem. The present invention adds a domain discrimination network to the quantized network, and the resulting quantized network has good domain adaptability.

[0048] Application 2: When encountering many quantized subnets composed of shared parameters, how to train them efficiently is a problem. The present invention obtains the convergence difficulty of each subnet through preliminary experiments, and trains each subnet using a sampling frequency inversely proportional to the difficulty. At the same time, the present invention also clusters the convergence speed, and uses the subnets with fast convergence to perform knowledge distillation on the subnets with slow convergence. This can train many subnets efficiently and achieve a balance between classification accuracy and efficiency.

[0049] In order to reduce precision redundancy, the present invention proposes to apply quantization to unsupervised domain adaptation, and use domain discriminator, group distillation and inverse sampling training method on the network, so that the classification ability of all networks can be guaranteed at low bit width.

[0050] See also Figure 1 and Figure 2 , an embodiment of the present invention provides a quantization domain adaptation method for edge devices without retraining, comprising the following steps:

[0051] Step 1, quantization network construction: perform multi-quantization bit width search to adapt to edge devices with different computing resources, and determine the optimal quantization bit width for each quantization subnet in the unsupervised domain adaptation network; build a domain discriminator for the quantization network to distinguish the features of the source domain and the target domain, so as to guide the training of the feature extraction quantization network through the feedback of the domain discriminator, so that the feature distribution of the source domain and the target domain is closer; use the pseudo-label method to generate reliable pseudo-labels for the unlabeled target domain data through the classification prediction of the quantization subnet with the most parameters and the largest quantization bit width, so as to solve the label missing problem of the target domain data in the quantization network training. In this step, the appropriate subnet dimensions can be determined through preliminary experiments, including the input resolution, width and quantization bit width of the network, to optimize the subnet configuration.

[0052] Step 2: Pre-training: Perform preliminary training on the quantized network to evaluate the convergence speed and training difficulty of each quantized subnet; use the test data set to evaluate the quantized subnet, obtain the classification accuracy of each quantized subnet, and determine the convergence speed and training difficulty of the quantized subnet based on the classification accuracy, providing data support for the formulation of subsequent training strategies.

[0053] Step 3, quantized subnet training: According to the convergence speed and training difficulty obtained in the pre-training stage, the quantized subnets are grouped to implement differentiated training strategies; among them, a sampling training method that is inversely proportional to the subnet convergence speed is adopted to allocate more training resources to the subnets with slower convergence to optimize the overall training efficiency; knowledge distillation is implemented, and the subnet group with fast convergence speed is used as the teacher model to transfer knowledge to the subnet group with slow convergence speed as the student model to accelerate the latter's learning process and realize information sharing and performance improvement among the subnets. In this step, the quantization-aware training method (LSQ) can be applied to enable the network to learn the appropriate quantization step size during the training process to reduce the error in the quantization process.

[0054] The quantization domain adaptation method for edge devices without retraining in the present invention can effectively solve the challenge of deploying high-precision deep learning models in edge devices under resource-constrained conditions, while overcoming the domain drift problem and achieving effective adaptation of the model in new domains without retraining the entire network. By combining quantization technology and domain adaptation strategy, the present invention provides a deep learning model deployment solution that can run efficiently on edge devices and has good generalization ability.

[0055] In some embodiments, the domain discriminator uses a multi-layer perceptron (MLP) network to distinguish features of the source domain and the target domain to achieve a binary classification task.

[0056] The output of the feature layer of each quantized subnet is input into the domain discriminator to discriminate whether the input feature comes from the source domain or the target domain; the discriminant loss is calculated and fed back for the result of the domain discriminator; gradient descent is used for the domain classification layer, and gradient reversal descent is used for the feature extraction layer.

[0057] In some embodiments, the pseudo-labeling method includes: using the subnet with the most parameters and the largest quantization bit width among all quantization networks to classify and predict the target domain data; selecting predictions whose classification probabilities exceed a preset threshold as pseudo-labels; and using the pseudo-labels to perform supervised training on unlabeled data in the target domain.

[0058] During subnet training, the calculation and optimization process of classification loss includes: inputting the features extracted by the quantization subnet into the classification layer to predict the classification category of the data; for labeled source domain data, directly calculating the cross entropy loss of the classification layer according to its true label; for unlabeled target domain data, using the classification prediction of the subnet with the most parameters and the largest quantization bit width in all quantization networks as pseudo-labels for supervision, and calculating the loss of the classification layer; performing weighted summation of the classification losses of the source domain and the target domain to optimize the parameters of the quantization network; preferably, the weight of the source domain loss is 0.5, and the weight of the target domain loss is 1.0.

[0059] For unlabeled target domain data, the following steps are performed: use the subnet with the most parameters and the largest quantization bit width among all quantization networks to classify and predict the target domain data to obtain a classification probability distribution; determine the maximum classification probability of each target domain data sample as a candidate pseudo-label; if the maximum probability of the candidate pseudo-label reaches or exceeds a preset confidence threshold, use the predicted category as a pseudo-label; use the pseudo-label as a supervisory signal to train the unlabeled data of the target domain.

[0060] In some embodiments, when training subnets, a convergence speed-based group training and knowledge distillation mechanism is adopted, specifically including: pre-training quantized subnets to evaluate their classification accuracy, so as to determine the convergence speed of each subnet; grouping the quantized subnets according to the classification accuracy to achieve clustering of the subnet convergence speed; applying a sampling strategy that is inversely proportional to the subnet convergence speed for training to enhance the training intensity of subnets with slow convergence speeds; implementing group knowledge distillation, using subnets with fast convergence speeds to transfer knowledge to subnets with slow convergence speeds, so as to promote the improvement of overall network performance.

[0061] The specific embodiments of the present invention are further described below.

[0062] Figure 1 Flow chart of a quantization method for unsupervised domain adaptation tasks without retraining according to an embodiment of the present invention. Figure 1 As shown, a quantization domain adaptation method for edge devices without retraining includes:

[0063] Unsupervised domain adaptation of quantized networks: Use multiple quantization bit widths to simultaneously search for quantization bit widths for unsupervised domain adaptation networks that do not require retraining, and use domain discriminators and pseudo-labeling techniques to perform unsupervised domain adaptation on different quantized subnets. This strategy can obtain different computing quantum networks with different quantization bit widths, and can solve the label-free problem and domain adaptation problem of unsupervised domain adaptation.

[0064] Subnet training method: First, some pre-training is performed to obtain the convergence speed and training difficulty of each quantized subnet, and then a training sampling method inversely proportional to the convergence speed is used for different subnets. In addition, the subnets are grouped according to the difficulty of convergence, and the subnet group with fast convergence is used to perform knowledge distillation on the subnet group with slow convergence, thereby accelerating the overall convergence speed.

[0065] In the unsupervised domain adaptation of the quantized network, the present invention uses the domain discriminator and pseudo-label technology to solve the domain drift and unlabeled problems of the target domain. The following steps are performed once in each training:

[0066] S1. Select appropriate dimensions to configure the subnet; use a 3-layer MLP to build a domain discriminator for binary classification tasks; input the output of the feature layer of each quantized subnet into the domain discriminator to determine whether the input feature comes from the source domain or the target domain; calculate the discriminant loss for the result of the domain discriminator and feed it back; use gradient descent for the domain classification layer and gradient reversal descent for the feature extraction layer.

[0067] S2. Use the subnet features obtained in step S1 to pass through the classification layer to obtain the classification category; for the labeled source domain, directly use the classification loss; for the unlabeled target domain, use the classification category of the subnet with the most parameters and the largest quantization bit width in all quantized networks as a pseudo-label for supervision; obtain the classification loss for gradient backpropagation;

[0068] S3. For the losses obtained in step S1 and step S2, weight coefficients of 0.5 and 1.0 are preferably used for weighted summation.

[0069] Step S1 specifically includes:

[0070] The appropriate subnet dimensions are obtained through preliminary experiments, and the network input resolution, width, and quantization bit width are finally selected to configure the subnet. In the network that does not need to be retrained, the quantization-aware training method LSQ is used for training, which can learn the appropriate quantization step size for different subnets, thereby reducing the quantization error.

[0071] Use a multi-layer perceptron to build a domain discriminator, and use the output of the shared network feature layer as the input of the domain discriminator to determine whether the feature comes from the source domain or the target domain. Perform a binary classification task, use gradient descent in the domain classification layer, and use reverse gradient descent in the feature extraction layer to solve the problem of domain feature drift.

[0072] Step S2 and step S3 specifically include:

[0073] The output of the subnetwork feature layer in step S1 is used as the input of the category classification layer to obtain the classification category; for the labeled source domain data, the cross entropy loss of the classification is directly calculated; for the unlabeled target domain, the maximum value of the classification probability obtained by the maximum network is used as the candidate value. If the maximum probability reaches the set threshold, the prediction is used as the pseudo-label as the supervision signal of the target domain data, thereby training the unlabeled data of the target domain; for the two types of losses obtained in step S2, weighted summation is performed with coefficients of 0.5 and 1.0.

[0074] In the subnet training method, a sampling training method inversely proportional to the subnet convergence speed and a group distillation method obtained therefrom are used. The specific steps are as follows:

[0075] T1: Pre-train for 10 rounds and use the test data to obtain the classification accuracy of each quantized subnet;

[0076] T2: Arrange the classification accuracy from high to low, and divide the arrangement into several groups continuously;

[0077] T3: Use classification accuracy to represent the convergence speed of the subnet, and use a sampling method inversely proportional to it for training;

[0078] T4: For the different groups obtained in T2, use the subnet group with fast convergence speed to perform knowledge distillation on the subnet group with slow convergence speed. Steps T1 and T2 are to cluster the many subnets according to the convergence speed of training, so that corresponding knowledge distillation can be performed according to the difference in convergence speed.

[0079] Step T3, in order to further strengthen the training of the subnet with slow convergence speed, the training probability of the subnet with slow convergence speed is increased, so that each subnet can achieve a balance between classification accuracy and computational complexity.

[0080] Step T4 uses clustering to perform knowledge distillation. By implementing targeted knowledge transfer to subnets in different groups, the performance of the entire quantization network is optimized to ensure that all subnets can effectively learn and adapt to the data in the target domain.

[0081] Figure 2 The algorithm framework of the embodiment of the present invention is shown.

[0082] In one example, the pseudo code for the training algorithm is as follows:

[0083]

[0084] In general, the present invention provides a training method for unsupervised domain adaptation of edge devices without retraining quantization. The method first performs pre-training to understand the convergence speed and training difficulty of different quantized subnets. On this basis, a training sampling strategy based on the inverse of the convergence speed is designed for different subnets. That is, for subnets with slower convergence, more training resources are allocated, and for subnets with faster convergence, the number of training times is reduced. In this way, training resources can be effectively utilized to improve the overall training efficiency. In addition, based on the convergence difficulty of the subnet, the quantized subnets are divided into different groups. By means of knowledge distillation, the subnet group with faster convergence is used to transfer knowledge to the subnet group with slower convergence, so that the subnet with slower convergence can accelerate learning. This intra-group distillation not only speeds up the overall convergence speed, but also realizes information sharing and improvement among the subnets. Finally, the strategy obtains a quantized network model that does not require retraining, which can be effectively applied to unsupervised domain adaptation tasks and solves the challenges of domain adaptation under unlabeled data. Through this method, the model not only achieves efficient convergence, but also has strong domain adaptability, providing a flexible and efficient quantitative solution for practical applications.

[0085] The important features of the present invention are:

[0086] 1. From the perspective of model quantization, the present invention transforms the unsupervised domain adaptation network into a quantized network that does not require retraining. By using domain discriminators for different quantized subnetworks, the domain adaptation capability of the quantized model is improved.

[0087] 2. From the perspective of efficient training, the present invention proposes how to efficiently train all subnets in a large number of subnets. By pre-training the model, the convergence speed of each subnet is obtained, and then sampling is performed inversely proportional to the speed, so that the network with slow convergence speed can get more training, and finally the balance between accuracy and efficiency of all subnets is achieved.

[0088] 3. Starting from how to comprehensively improve the classification accuracy of subnets, the present invention proposes to group all subnets according to their classification accuracy, and then perform knowledge distillation on the group with low classification accuracy by the group with high classification accuracy, so that all subnets can be gradually trained.

[0089] The main technical advantages of the present invention are as follows:

[0090] 1. The present invention uses a domain discriminator for the quantized subnet from the perspective of solving the domain drift between different domains without retraining, thereby solving the problem of domain feature drift caused by different sources. In order to solve the problem of no label in the target domain in unsupervised domain adaptation, the present invention uses the reliable classification prediction obtained by the subnet with the most parameters and the largest quantization bit width in all quantized networks as a pseudo-label to progress the target domain, thereby completing the unsupervised domain adaptation task of the quantized network.

[0091] 2. The present invention classifies subnets according to the convergence speeds of different subnets, uses subnets with fast convergence speeds to perform knowledge distillation on subnets with slow convergence speeds, and simultaneously performs sampling training on subnets according to the inverse ratio of the convergence speeds, so that subnets with slow convergence speeds get more training opportunities, thereby enabling each subnet to achieve a balance between classification accuracy and reasoning efficiency.

[0092] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0093] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0094] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.

[0095] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0096] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0097] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0098] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0099] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0100] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or makes a difference to the prior art. The contribution can be in the form of software products. The computer software product is stored in a A storage medium includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: a mobile storage device, ROM, RAM, a magnetic disk, or an optical disk, etc., various media that can store program codes.

[0101] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0102] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0103] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0104] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A quantization domain adaptation method for edge devices without retraining, characterized in that: The following steps are involved: Step 1: Quantization network construction: Perform multi-quantization bit width search to adapt to edge devices with different computing resources, and determine the optimal quantization bit width for each quantization subnetwork in the unsupervised domain adaptation network; build a domain discriminator for the quantization network to distinguish the features of the source domain and the target domain, so as to guide the training of the feature extraction quantization network through the feedback of the domain discriminator, so that the feature distribution of the source domain and the target domain is closer; Use the pseudo-label method to generate pseudo-labels for unlabeled target domain data through classification prediction of the subnetwork with the most parameters and the largest quantization bit width among all quantized networks; Step 2: Pre-training: Perform preliminary training on the quantized network to evaluate the convergence speed and training difficulty of each quantized sub-network; Use the test data set to evaluate the quantized subnet, obtain the classification accuracy of each quantized subnet, and determine the convergence speed and training difficulty of the quantized subnet based on the classification accuracy; Step 3: Subnet training: According to the convergence speed and training difficulty obtained in the pre-training stage, the quantized subnets are grouped to implement differentiated training strategies. Among them, a sampling training method that is inversely proportional to the subnet convergence speed is adopted to allocate more training resources to the subnets with slower convergence. Implement knowledge distillation, use the subnet group with fast convergence speed as the teacher model, transfer knowledge to the subnet group with slow convergence speed as the student model, and accelerate the latter's learning process.

2. The method for quantization domain adaptation without retraining for edge devices according to claim 1, characterized in that: The domain discriminator uses a multi-layer perceptron (MLP) network to distinguish the features of the source domain and the target domain to achieve a binary classification task.

3. The method for quantization domain adaptation without retraining for edge devices according to claim 1 or 2, characterized in that: The output of the feature layer of each quantization subnet is input into the domain discriminator to discriminate whether the input feature comes from the source domain or the target domain; the discrimination loss is calculated for the result of the domain discriminator and then fed back; Gradient descent is used for the domain classification layer, and gradient reversal descent is used for the feature extraction layer.

4. The method for quantization domain adaptation without retraining for edge devices according to any one of claims 1 to 3, characterized in that: The pseudo labeling method comprises: The subnet with the most parameters and the largest quantization bit width among all quantized networks is used to classify and predict the target domain data; Select predictions whose classification probability exceeds a preset threshold as pseudo labels; The pseudo labels are used to perform supervised training on unlabeled data of the target domain.

5. The method for quantization domain adaptation without retraining for edge devices according to any one of claims 1 to 4, characterized in that: When training the subnet, the calculation and optimization process of classification loss includes: The features extracted by the quantization subnet are input into the classification layer to predict the classification category of the data; For labeled source domain data, the cross entropy loss of the classification layer is directly calculated based on its true label; For unlabeled target domain data, the classification prediction of the subnetwork with the most parameters and the largest quantization bit width among all quantized networks is used as pseudo-label for supervision to calculate the loss of the classification layer; The classification losses of the source domain and the target domain are weighted and summed to optimize the parameters of the quantization network; preferably, the weight of the source domain loss is 0.5 and the weight of the target domain loss is 1.

0.

6. The method for quantization domain adaptation without retraining for edge devices according to claim 5, characterized in that: For unlabeled target domain data, perform the following steps: Use the subnet with the most parameters and the largest quantization bit width among all quantized networks to classify and predict the target domain data and obtain the classification probability distribution; Determine the maximum classification probability of each target domain data sample as a candidate pseudo label; If the maximum probability of the candidate pseudo-label reaches or exceeds the preset confidence threshold, the predicted category is used as the pseudo-label; The pseudo-labels are used as supervisory signals to train the unlabeled data of the target domain.

7. The method for quantization domain adaptation without retraining for edge devices according to any one of claims 1 to 6, characterized in that: When training the subnet, a group training and knowledge distillation mechanism based on convergence speed is adopted, including: Pre-train the quantized subnetwork to evaluate its classification accuracy, thereby determining the convergence speed of each subnetwork; The quantized subnetworks are grouped according to the classification accuracy to achieve clustering of the subnetwork convergence speed; Apply a sampling strategy that is inversely proportional to the subnet convergence speed for training to increase the training intensity of the subnet with slow convergence speed; Implement group knowledge distillation and use subnets with fast convergence speed to transfer knowledge to subnets with slow convergence speed to promote the improvement of overall network performance.

8. The method for quantization domain adaptation without retraining for edge devices according to any one of claims 1 to 7, characterized in that: In step 1, the appropriate subnetwork dimensions, including the network's input resolution, width, and quantization bit width, are determined through preliminary experiments to optimize the subnetwork configuration.

9. The method for quantization domain adaptation without retraining for edge devices according to any one of claims 1 to 8, characterized in that: In step three, quantization-aware training (LSQ) is applied to enable the network to learn the appropriate quantization step size during training to reduce the error in the quantization process.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for quantization domain adaptation without retraining for edge devices as described in any one of claims 1 to 9 is implemented.

Images