Terminal equipment deep learning network lightweight adaptation method and device

Abstract

The invention provides a terminal equipment deep learning network lightweight adaptation method and device, and the method comprises the steps: constructing a lightweight action space deployed for target terminal equipment according to the hardware resource constraint of the target terminal equipment and the network state of an original deep learning network, and selecting an initial adaptation strategy, the light-weight action space is composed of various light-weight actions; dynamically coupling the lightweight action space, constructing a total optimization target in combination with the optimization target of each lightweight action, and iteratively optimizing the adaptation strategy; and carrying out lightweight processing on the original deep learning network according to the optimized adaptation strategy to obtain a lightweight deep learning network meeting the hardware resource constraint of the target terminal equipment. According to the method, self-adaptive network weight reduction is carried out for different terminal devices, and the adaptation capability on a resource-limited terminal is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent terminal and deep learning network lightweight technology, and particularly relates to a terminal device deep learning network lightweight adaptation method and device. BACKGROUND

[0002] As an important application platform of deep learning network, terminal devices (such as mobile devices and embedded devices) need lightweight technology support to adapt to terminal devices due to the constraints of storage, computing resources and energy consumption. The existing technology reduces the network size and reduces the computing demand by pruning, low-rank decomposition, parameter quantization and knowledge distillation, providing a basis for terminal deployment. However, the resource conditions of terminal devices such as storage and power consumption differ significantly, and the existing methods use fixed lightweight strategies, which are difficult to dynamically adapt to different devices or load scenarios, resulting in limited deployment flexibility. Moreover, existing research focuses on improving the performance of single technologies such as pruning, quantization and distillation, without considering the coupling effect between multiple technologies, and lacks a dynamic adjustment mechanism for terminal device hardware resources. SUMMARY

[0003] To overcome the shortcomings of the prior art, the present application provides a terminal device deep learning network lightweight adaptation method and device based on dynamic coupling, which adaptively lightens the network for different terminal devices and effectively improves the adaptation capability on resource-constrained terminals.

[0004] In one aspect, the present application provides a terminal device deep learning network lightweight adaptation method based on dynamic coupling, comprising:

[0005] According to the hardware resource constraints of the target terminal device and the network state of the original deep learning network, a lightweight action space for deployment on the target terminal device is constructed and an initial adaptation strategy is selected, the lightweight action space being composed of multiple lightweight actions;

[0006] The lightweight action space is dynamically coupled, and a total optimization target is constructed in combination with the optimization target of each lightweight action, and the adaptation strategy is iteratively optimized;

[0007] The original deep learning network is lightened according to the optimized adaptation strategy, and a lightweight deep learning network that meets the hardware resource constraints of the target terminal device is obtained.

[0008] In one embodiment of the present application, the hardware resource constraints include the upper limit of acceptable delay, the upper limit of acceptable network parameters and the upper limit of acceptable network computing amount of the target terminal device, and the network state includes the actual network parameter amount, the computing amount, the inference delay and the accuracy of the network on the target terminal device.

[0009] In an embodiment of the present application, the lightweight action space for the target terminal device deployment is constructed by:

[0010] defining a plurality of lightweight actions for the target terminal device deployment and an optimization target corresponding to each lightweight action, the lightweight actions including network compression operations, quantization operations, network architecture search operations;

[0011] constructing the lightweight action space for the target terminal device according to the defined lightweight actions.

[0012] In an embodiment of the present application, the network compression operation adopts knowledge distillation, and the computational load of the target terminal device is reduced by minimizing the prediction result loss of the teacher network and the student network. The first loss function is constructed as:

[0013] L comp =α·L CE (O S ,Y)+(1-α)·L KL (O S / T,O T / T);

[0014] wherein L CE is a cross-entropy loss for the student network to learn the real label Y; L KL is a KL divergence loss for the student network to learn the soft label of the teacher network; T is a temperature parameter for adjusting the smoothing degree of the soft label; and a is a weight for balancing the cross-entropy loss and the KL divergence loss.

[0015] The optimization target of the network compression operation is to minimize the first loss function.

[0016] In an embodiment of the present application, the quantization operation adopts adaptive post-training quantization, and the computational efficiency is optimized by automatically assigning bit widths of different network layers. The second loss function is constructed as:

[0017] L qn =∑ i C(G i ,b i ),ΔE(b i})≤∈

[0018] wherein C(G i ,b i ) is the network computation cost, G i is the i-th layer network, b i is the bit width of the i-th layer network, ΔE({b i}) is the network accuracy loss, and ∈ is the maximum allowed accuracy loss.

[0019] The optimization objective of the quantization operation is to minimize the second loss function.

[0020] In one embodiment of the present invention, the network architecture search operation includes searching for the number of network layers, channel width, and / or connection mode, so as to adjust the local micro-topology of the network to adapt to the hardware resource constraints of the target terminal device.

[0021] The search space S is represented as: S = {G = (V, E, H) | G}; where G represents a candidate network structure adapted to the target terminal device; V = {v1, v2, ..., v n} is a set of nodes in a network, where each node v i Represents a network layer; E = {e ij} is a join set, where e ij H represents the connection method from layer i to layer j; H is the hardware resource constraint of the target terminal device.

[0022] The performance of the candidate network structures selected during the current search is evaluated, and the first evaluation score R(G) is obtained. t ), G t It is the candidate network structure selected at the current time t, and the optimization objective of the network architecture search operation is to maximize the first evaluation score.

[0023] In one embodiment of the present invention, dynamically coupling the lightweight action space includes:

[0024] Model the coupling relationship between different lightweight actions and construct a third loss function, expressed as:

[0025] L couple =-α cq ·CE×QE-α cn ·CE×NE-α qn ·QE×NE,

[0026] Where CE is the performance improvement brought by the network compression operation, QE is the storage and inference latency optimization brought by the quantization operation, NE is the performance and complexity balance improvement brought by the network architecture search operation, and α cq α cn α qn These are the corresponding coupling weight coefficients;

[0027] By adjusting the coupling weight coefficients to collaboratively optimize each lightweight action, the optimization contribution of each lightweight action to terminal resources is balanced, so that the lightweight action space adapts to the hardware resource constraints of the target terminal device. The optimization objective of the lightweight action space is to minimize the third loss function.

[0028] In an embodiment of the present application, the adaptive strategy is iteratively optimized by constructing a total optimization target according to the optimization target of each lightweight action, and the adaptive strategy comprises:

[0029] The dynamic selection strategy of lightweight actions is constructed as A t ~ π(A t |s' t ), wherein π is the adaptive strategy, s' t is the hardware resource constraint of the target terminal device at time t, the network state, and the fusion vector of each lightweight action, s' t = concat(s t , H, CE, QE, NE), s t is the network state;

[0030] The network state at the current time is evaluated to construct a second evaluation score F(s t ), and the optimization target of the network state is to maximize the second evaluation score to improve the adaptation effect of the network on the target terminal device;

[0031] The optimization target of the network compression operation, the optimization target of the quantization operation, the optimization target of the network architecture search operation, the optimization target of the lightweight action space, and the optimization target of the network state are weighted to construct a total optimization target function, and the total optimization target function is:

[0032]

[0033] According to the total optimization target function, the optimal adaptive strategy that meets the preset constraint condition in the iteration process is selected according to the dynamic selection strategy representation.

[0034] In an embodiment of the present application, the adaptive strategy is iteratively optimized by using a reinforcement learning-based method, an evolutionary algorithm-based method, or a gradient optimization-based method.

[0035] Another aspect of the present application also provides a terminal device deep learning network lightweight adaptation device based on dynamic coupling, comprising:

[0036] An initial adaptation module is configured to construct a lightweight action space for deployment of a target terminal device and select an initial adaptive strategy according to the hardware resource constraint of the target terminal device and the network state of an original deep learning network, wherein the lightweight action space is composed of multiple lightweight actions;

[0037] A strategy optimization module is configured to dynamically couple the lightweight action space, construct a total optimization target according to the optimization target of each lightweight action, and iteratively optimize the adaptive strategy;

[0038] A lightweight adaptation module is configured to perform lightweight processing on the original deep learning network according to the optimized adaptation strategy, so as to obtain a lightweight deep learning network satisfying the hardware resource constraint of the target terminal device.

[0039] From the above scheme, the advantages of the present application are as follows:

[0040] The terminal device deep learning network lightweight adaptation method provided by the present application realizes high-performance network deployment under the condition of limited computing resources of different terminal devices. According to the hardware resource constraint of the target terminal device and the network state of the original deep learning network, a lightweight action space for deployment of the target terminal device is constructed and an initial adaptation strategy is selected. The lightweight action space is dynamically coupled, and a total optimization target is constructed in combination with the optimization target of each lightweight action itself, and the adaptation strategy is iteratively optimized. The original deep learning network is processed according to the optimized adaptation strategy, so as to obtain a lightweight deep learning network satisfying the hardware resource constraint of the target terminal device. This method solves the conflict problem between different lightweight actions, and realizes adaptive lightweight for diversified terminal devices. The adaptation strategy can be dynamically adjusted according to the change of the hardware resource constraint of the target terminal device, which not only realizes high adaptability and low resource consumption between network lightweight and terminal deployment, but also significantly improves the deployment stability, inference speed and precision maintaining ability of the network in the resource limited environment, providing support for efficient network operation in the terminal. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 Fig. 1 shows a general flowchart of a terminal device deep learning network lightweight adaptation method provided by an embodiment of the present application;

[0042] Figure 2 Fig. 2 shows a structure diagram of a terminal device deep learning network lightweight adaptation device provided by an embodiment of the present application.

[0043] Among them, the reference signs are:

[0044] 300: terminal device deep learning network lightweight adaptation device;

[0045] 310: initial adaptation module;

[0046] 320: strategy optimization module;

[0047] 330: lightweight adaptation module. DETAILED DESCRIPTION

[0048] It is to be noted that the relative terms such as first and second and the like in the present disclosure are only used to differentiate one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0049] Without further limitation, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0050] As mentioned before, diversified terminals have different limitations in storage, calculation, power consumption and other resources. The terminal device deep learning network adaptation method of the prior art only optimizes single action, has a lightweight bottleneck, and lacks self-adaptive ability to terminal devices with resource differences. To this end, the present application provides a terminal device deep learning network lightweight adaptation method based on dynamic coupling to realize adaptive adaptation for different terminal devices.

[0051] In particular, with reference to Figure 1 , it is shown Figure 1 that the total flowchart of the terminal device deep learning network lightweight adaptation method provided by an embodiment of the present application is shown.

[0052] A terminal device deep learning network lightweight adaptation method based on dynamic coupling, specifically comprising the following steps:

[0053] Step S1, according to the hardware resource constraints of the target terminal device and the network state of the original deep learning network, a lightweight action space for the deployment of the target terminal device is constructed and an initial adaptation strategy is selected, the lightweight action space is composed of multiple lightweight actions.

[0054] In a specific embodiment, the hardware resource constraints include the upper limit of the acceptable delay of the target terminal device, the upper limit of the acceptable network parameters and the upper limit of the acceptable network calculation amount. The vectorization of hardware resource constraints is H, H = [Latency max , ModelSize max , FLOPS max ], wherein Latency max represents the upper limit of the acceptable delay of the target terminal device, ModelSize max represents the upper limit of the acceptable network parameters, and FLOPS maxrepresents an upper limit of acceptable network computation amount.

[0055] In an embodiment, the network status includes actual network parameter amount, computation amount, inference delay, and accuracy of the network on the target terminal device. The original deep learning network is represented as M0, and the vectorized representation of the network status is s0, s0 = [Params(M0), FLOPs(M0), Latency(M0), Accuracy(M0)], where Params(M0) represents the initial network parameter amount, FLOPs(M0) represents the initial network computation amount, Latency(M0) represents the initial network inference delay, and Accuracy(M0) represents the initial network accuracy.

[0056] In an embodiment, the light-weight action includes network compression operation, quantization operation, and network architecture search operation. The construction of the light-weight action space for the deployment of the target terminal device includes: defining the network compression operation, the quantization operation, and the network architecture search operation for the deployment of the target terminal device, and defining the optimization target corresponding to each light-weight action; constructing the light-weight action space adapted to the target terminal device according to the defined network compression operation, quantization operation, and network architecture search operation. In this embodiment, the definition of the light-weight action space A is: A = {C, Q, S}, where C represents the network compression operation, Q represents the quantization operation, and S represents the network architecture search operation.

[0057] For the network compression operation and the optimization target corresponding thereto, network compression is used to reduce the parameter amount of the deep network, and the compression methods that can be adopted include but are not limited to network knowledge distillation and parameter pruning. In an embodiment, the network compression operation adopts knowledge distillation, which improves the performance of a simple network by transferring knowledge from a complex teacher network to a simple student network. The reduction of the computation amount of the target terminal device is achieved by minimizing the loss of the prediction results of the teacher network and the student network, and a first loss function is constructed as:

[0058] L comp = α · L CE (O S , Y) + (1 - α) · L KL (O S / T, O T / T);

[0059] wherein, L CE is a cross-entropy loss used for the student network to directly learn the real label Y; L KLis the KL divergence loss for student network to learn the soft label of teacher network; T is the temperature parameter to adjust the smoothness of soft label; a is the weight to balance the cross-entropy loss and the KL divergence loss. The optimization objective of distillation is to reduce the distillation loss, and the optimization objective of network compression operation is to minimize the first loss function.

[0060] For the quantization operation and its corresponding optimization objective, the core of the quantization operation is to convert floating point numbers (FP32) into data types of lower precision (INT16, INT8, INT4, etc.). For quantization, a quantization function Q(x, B) can be defined, which maps a continuous value x to a discrete value represented by B bits. For example, for the weight W', the quantized weight can be represented as: W" = Q(W', B). However, different network layers have different sensitivities to precision, and it is not suitable to quantize all layers with the same bit width. In an embodiment, the quantization operation adopts adaptive post-training quantization (PTQ) to optimize the computational efficiency by automatically assigning bit widths to different network layers, and the second loss function is constructed as: L qn =∑ i C(G i ,b i ),ΔE(b i} )≤∈, where C(G i ,b i ) is the network computation cost, G i is the i-th layer network, b i is the bit width of the i-th layer network, ΔE({b i}) is the network precision loss, and ∈ is the maximum allowed precision loss. The optimization objective of the quantization operation is to minimize the second loss function.

[0061] For the network architecture search operation and its corresponding optimization objective, the network architecture search operation is used to adjust the local micro-topology of the existing lightweight network to better adapt to the hardware resource constraints of the target terminal device, i.e., to adapt to the hardware architecture of the target terminal device. The specific search dimensions include network layer number search, channel width search, connection mode search, etc. The network architecture search operation includes searching for network layer number, channel width, and / or connection mode to adjust the local micro-topology of the network to adapt to the hardware resource constraints of the target terminal device. The search space S is represented as: S = {G = (V, E, H) | G}; where G represents a candidate network structure adapted to the target terminal device; V = {v1, v2, …, v n} is the node set of the network, and each node v i represents a network layer; E = {e ij} is the connection set, where e ija connection mode representing from the i-th layer to the j-th layer; H is a hardware resource constraint of the target terminal device.

[0062] performing search selection of a candidate network structure G for a current time t t Performance evaluation can be performed using, but not limited to, full training evaluation, early stopping training evaluation, and other evaluation methods to obtain a first evaluation score R(G t The optimization target of the network architecture search operation is to maximize the first evaluation score R(G t ).

[0063] Step S2, dynamically coupling the lightweight action space, combining the optimization target of each lightweight action itself to construct a total optimization target, and iteratively optimizing the adaptation strategy.

[0064] In an embodiment, for the conflict problem between different lightweight actions, a dynamic coupling mechanism is further introduced to jointly model and cooperatively optimize multiple lightweight actions, avoid conflicts between strategies, break through the limitations of independent design and scattered optimization effect of traditional lightweight methods, realize overall lightweight effect improvement, improve lightweight efficiency and reduce strategy conflicts.

[0065] Specifically, the coupling relationship between different lightweight actions is modeled, and a third loss function is constructed, represented as:

[0066] L couple =-α cq ·CE×QE-α cn ·CE×NE-α qn ·QE×NE,

[0067] Where CE is the performance improvement value brought by the network compression operation, QE is the storage and inference delay optimization value brought by the quantization operation, NE is the performance and complexity balance improvement value brought by the network architecture search operation, and α cq , α cn , α qn are the corresponding coupling weight coefficients.

[0068] By adjusting the coupling weight coefficients, each lightweight action is cooperatively optimized, the optimization contribution of each lightweight action to the terminal resource is balanced, the lightweight action space adapts to the hardware resource constraint of the target terminal device, and the optimization target of the lightweight action space is to minimize the third loss function.

[0069] In yet another embodiment, a multi-dimensional real-time feedback mechanism integrating precision, delay and resource consumption is introduced to guide the dynamic iteration and strategy update of the lightweight process.

[0070] Specifically, the dynamic selection strategy of the lightweight action is represented as: A t~π(A t |s' t ), wherein π is an adaptation strategy, s' t is a fusion vector of hardware resource constraints of the target terminal device, network status, performance improvement of each lightweight action at time t, s' t = concat(s t , H, CE, QE, NE), s t is the network status at time t.

[0071] evaluate the network status at the current time, and construct a second evaluation score F(s t ), wherein the optimization target of the network status is to maximize the second evaluation score to improve the adaptation effect of the network on the target terminal device.

[0072] weight the optimization target of the network compression operation, the optimization target of the quantization operation, the optimization target of the network architecture search operation, the optimization target of the lightweight action space, and the optimization target of the network status, and construct a total optimization target function, wherein the total optimization target function is:

[0073]

[0074] According to the dynamic selection strategy representation, the optimal adaptation strategy that meets the preset constraint condition in the iteration process is screened out according to the total optimization target function.

[0075] In an embodiment, the adaptation strategy is iteratively optimized by using a reinforcement learning-based method, an evolutionary algorithm-based method, or a gradient optimization-based method.

[0076] Step S3: According to the optimized adaptation strategy, the original deep learning network is subjected to lightweight processing to obtain a lightweight deep learning network that meets the hardware resource constraints of the target terminal device, and the lightweight deep learning network is deployed on the target terminal device. At this time, the lightweight deep learning network meets the hardware resource constraints (such as storage capacity, computing capability, and power consumption limit) of different terminal devices.

[0077] In summary, the terminal device deep learning network lightweight adaptation method provided by the application realizes high-performance network deployment under the condition of limited computing resources of different terminal devices. According to the hardware resource constraints of the target terminal device and the network state of the original deep learning network, a lightweight action space for deployment of the target terminal device is constructed and an initial adaptation strategy is selected; the lightweight action space is dynamically coupled, and a total optimization target is constructed in combination with the optimization target of each lightweight action itself, and the adaptation strategy is iteratively optimized; the original deep learning network is lightweight processed according to the optimized adaptation strategy, and a lightweight deep learning network satisfying the hardware resource constraints of the target terminal device is obtained. The method solves the conflict problem between different lightweight actions, and realizes adaptive lightweight for diversified terminal devices, can dynamically adjust the adaptation strategy according to the change of the hardware resource constraints of the target terminal device, not only realizes high adaptability and low resource consumption between network lightweight and terminal deployment, but also significantly improves the deployment stability, inference speed and precision maintaining ability of the network in the resource limited environment, and provides support for efficient network operation in the terminal.

[0078] The following is a device embodiment corresponding to the above method embodiment, as shown in Figure 2 Figure 2 A structure schematic diagram of a terminal device deep learning network lightweight adaptation device provided by an embodiment of the application is shown. The device embodiment can be implemented in cooperation with the above-mentioned method embodiment. The related technical details mentioned in the above-mentioned method embodiment are still valid in the device embodiment. In order to reduce repetition, they will not be described here.

[0079] A terminal device deep learning network lightweight adaptation device 300 based on dynamic coupling, comprising:

[0080] An initial adaptation module 310 is used to construct a lightweight action space for deployment of the target terminal device and select an initial adaptation strategy according to the hardware resource constraints of the target terminal device and the network state of the original deep learning network, and the lightweight action space is composed of multiple lightweight actions.

[0081] A strategy optimization module 320 is used to dynamically couple the lightweight action space, and construct a total optimization target in combination with the optimization target of each lightweight action itself, and iteratively optimize the adaptation strategy.

[0082] A lightweight adaptation module 330 is used to perform lightweight processing on the original deep learning network according to the optimized adaptation strategy, and obtain a lightweight deep learning network satisfying the hardware resource constraints of the target terminal device.

[0083] ​The device embodiment can be implemented in cooperation with the implementation manners of the method embodiments. The related technical details mentioned in the implementation manners of the above embodiments are still valid in the implementation manners of the method embodiments. In order to reduce repetition, they will not be described here again.

[0084] The embodiment of the present application also provides a readable storage medium, which stores a program or instructions, and the program or instructions are executed by a processor to implement the steps of the terminal device deep learning network lightweight adaptation method and achieve the same technical effects.

[0085] The embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of the image generation method and achieve the same technical effects.

[0086] The processor is the processor in the electronic device in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0087] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles, or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing functions as shown or discussed, but can also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method can be performed in an order different from the described order, and various steps can also be applied, omitted, or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0088] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which all belong to the protection of the present application.

Claims

1. A lightweight adaptation method for deep learning networks in terminal devices, characterized in that, include: Based on the hardware resource constraints of the target terminal device and the network state of the original deep learning network, a lightweight action space is constructed for deployment on the target terminal device, and an initial adaptation strategy is selected. The lightweight action space consists of a variety of lightweight actions. The lightweight action space is dynamically coupled, and the overall optimization objective is constructed by combining the optimization objectives of each lightweight action itself, and the adaptation strategy is iteratively optimized. The original deep learning network is lightweighted according to the optimized adaptation strategy to obtain a lightweight deep learning network that meets the hardware resource constraints of the target terminal device.

2. The method according to claim 1, characterized in that, The hardware resource constraints include the acceptable latency limit, acceptable network parameter limit, and acceptable network computation limit of the target terminal device. The network status includes the actual network parameter quantity, computation quantity, inference latency, and accuracy of the network on the target terminal device.

3. The method according to claim 1, characterized in that, Constructing a lightweight action space for deployment on the target terminal device includes: Define a variety of lightweight actions to be deployed for the target terminal device and the optimization objectives corresponding to each lightweight action. The lightweight actions include network compression operation, quantization operation, and network architecture search operation. The lightweight action space adapted to the target terminal device is constructed based on the defined lightweight actions.

4. The method according to claim 3, characterized in that, The network compression operation employs knowledge distillation, which reduces the computational load on the target terminal device by minimizing the prediction result loss between the teacher network and the student network. The first loss function is constructed as follows: L comp =α·L CE (O S ,Y)+(1-α)·L KL (O S / T,O T / T); Among them, L CE It is cross-entropy loss, used for student networks to directly learn the real label Y; L KL α is the KL divergence loss, used for soft labels of the student network learning the teacher network; T is the temperature parameter, used to adjust the smoothness of the soft labels; α is the weight used to balance the cross-entropy loss and the KL divergence loss. The optimization objective of the network compression operation is to minimize the first loss function.

5. The method according to claim 4, characterized in that, The quantization operation employs adaptive post-training quantization, optimizing computational efficiency by automatically allocating bit widths for different network layers, and constructing the second loss function as follows: L qn =∑ i C(G i ,b i ),ΔE(b i} )≤∈ Where C(G) i ,b i ) is the network computing cost, G i It is the i-th layer network, b i It is the bit width of the i-th layer network, ΔE({b i }) represents the network accuracy loss, and ∈ represents the maximum allowable accuracy loss; The optimization objective of the quantization operation is to minimize the second loss function.

6. The method according to claim 5, characterized in that, The network architecture search operation includes searching for network layers, channel width, and / or connection modes to adjust the local micro-topology of the network to adapt to the hardware resource constraints of the target terminal device. The search space S is represented as: S = {G = (V, E, H) | G}; where G represents a candidate network structure adapted to the target terminal device; V = {v1, v2, ..., v n } is a set of nodes in a network, where each node v i Represents a network layer; E = {e ij } is a join set, where e ij H represents the connection method from layer i to layer j; H is the hardware resource constraint of the target terminal device. The performance of the candidate network structures selected by the search at the current moment is evaluated to obtain a first evaluation score. The optimization objective of the network architecture search operation is to maximize the first evaluation score.

7. The method according to claim 6, characterized in that, Dynamically coupling the lightweight action space includes: Model the coupling relationship between different lightweight actions and construct a third loss function, expressed as: L couple =-a cq ·CE×QE-a cn ·CE×NE-a qn ·QE×NE, Where CE is the performance improvement brought by the network compression operation, QE is the storage and inference latency optimization brought by the quantization operation, NE is the performance and complexity balance improvement brought by the network architecture search operation, and α cq α cn α qn These are the corresponding coupling weight coefficients; By adjusting the coupling weight coefficients to collaboratively optimize each lightweight action, the optimization contribution of each lightweight action to terminal resources is balanced, so that the lightweight action space adapts to the hardware resource constraints of the target terminal device. The optimization objective of the lightweight action space is to minimize the third loss function.

8. The method according to claim 7, characterized in that, The adaptation strategy is iteratively optimized by constructing a total optimization objective based on the optimization objectives of each lightweight action, including: The dynamic selection strategy for constructing lightweight actions is represented as: A t ~π(A t |s' t ), where π is the adaptation strategy, s′ t Let s′ be the fusion vector of the target terminal device's hardware resource constraints, network status, and various lightweight actions at time t. t =concat(s t ,H,CE,QE,NE),s t Network status; Assess the network status at the current moment and construct a second assessment score. The optimization objective of the network status is to maximize the second assessment score to improve the network's adaptability to the target terminal device. The overall optimization objective function is constructed by weighting the optimization objectives of the network compression operation, the quantization operation, the network architecture search operation, the lightweight action space, and the network state. Based on the dynamic selection strategy, the optimal adaptation strategy that satisfies the preset constraints during the iteration process is selected according to the overall optimization objective function.

9. The method according to claim 7, characterized in that, The adaptation strategy is iteratively optimized using reinforcement learning-based methods, evolutionary algorithm-based methods, or gradient optimization-based methods.

10. A lightweight adaptation device for deep learning networks in terminal devices, characterized in that, include: The initial adaptation module is used to construct a lightweight action space for deployment on the target terminal device and select an initial adaptation strategy based on the hardware resource constraints of the target terminal device and the network state of the original deep learning network. The lightweight action space consists of a variety of lightweight actions. The strategy optimization module is used to dynamically couple the lightweight action space, construct the overall optimization objective by combining the optimization objectives of each lightweight action, and iteratively optimize the adaptation strategy. The lightweight adaptation module is used to perform lightweight processing on the original deep learning network according to the optimized adaptation strategy, so as to obtain a lightweight deep learning network that meets the hardware resource constraints of the target terminal device.

Images