Dynamic optimization early-leaving model, method, system and equipment based on reasoning efficiency

Abstract

On the basis of an early-quit network, a bypass module is inserted into a classification branch of each stage, residual connection is carried out, after the output feature of each stage is processed by the corresponding bypass module, residual connection is carried out on the output feature of the corresponding stage, and the output feature of the corresponding stage is processed by the corresponding bypass module. Obtaining a new classification feature of each stage, and inputting the new classification feature into a corresponding classifier; multiplying an output result of each classifier by a confidence factor, obtaining a predicted classification probability value through a softmax function, and calculating a cross entropy with a label corresponding to an input image to obtain loss of each stage; taking the loss of the previous stage as a dynamic adjustment parameter and multiplying the dynamic adjustment parameter with the loss of the stage to obtain the dynamically adjusted loss of the stage; carrying out weighted summation on the dynamically adjusted loss of all stages to obtain final total loss, and training a model by using the total loss; and performing reasoning classification on the input image by adopting the trained model.

Description

Technical Field

[0001] The present invention relates to neural networks, image recognition technology, and efficient reasoning technology, and more specifically, to a dynamic optimization early retirement model, method, system and equipment based on efficient reasoning. Background Art

[0002] In recent years, large-scale deep neural networks have achieved excellent performance in the field of vision. In order to fully utilize the performance of these networks, the method of pre-training them on a larger dataset and then fine-tuning them on a downstream dataset is widely used. However, as the size of the network increases, the computational cost of fully fine-tuning the model on the downstream dataset becomes increasingly high. Therefore, in order to reduce the number of parameters for model fine-tuning and thus reduce the computational overhead of the model, parameter-efficient methods have become the focus of many studies. These methods freeze the backbone network and only train the parameters and linear classifiers of the inserted fine-tuning modules, which not only fully utilizes the knowledge of the original dataset, but also realizes knowledge transfer and saves a lot of computational overhead in the fine-tuning process. However, parameter-efficient methods maintain or increase the original inference time of the model. Although they reduce the computational overhead of training, they do not significantly improve the inference efficiency of the model.

[0003] In practical applications, computing resources are often in short supply. As the scale of the network continues to increase, the problem of model inference efficiency cannot be ignored. Therefore, many efficient inference methods have been proposed. These include further reducing the model's inference time and improving the model's inference efficiency while maintaining a small number of fine-tuned parameters. These methods often redesign the network and reduce the calculations required for samples during the inference phase, thereby improving inference efficiency without significantly different inference performance.

[0004] At present, most of the efficient reasoning methods consider that sample information in the visual field has strong spatial redundancy and cut and discard some redundant tokens contained in the intermediate features. These methods have potential information loss problems, which affects the final performance and generalization ability of the model. In addition, the network design of these methods is relatively complex and requires additional network design.

[0005] The early-retirement network is combined with an efficient parameter fine-tuning method to divide the network into different stages and set classification heads at different stages, thereby achieving efficient reasoning at the sample level. Enhancing the discriminative ability of the shallow network is crucial for the early-retirement network to ensure good image classification accuracy under incomplete reasoning. However, since in traditional networks, the shallow layer is mainly responsible for providing basic expression features for the deep layer, how to decouple the low-level expression ability of the shallow layer from the high-level discriminative ability in the early-retirement network is a problem that many existing technologies have not solved. Specifically, in order to achieve early-retirement, the shallow classifier needs to learn to classify samples, which will inevitably enhance the discriminative ability of the shallow layer, which will cause the features of the shallow layer input to the deep layer to be mixed with too much discriminative information, thereby affecting the learning of the deep layer, resulting in the contradiction that the shallow layer and the deep layer cannot have good classification capabilities at the same time, and the shallow layer cannot take into account both low-level expression capabilities and high-level discriminative capabilities. Summary of the invention

[0006] In order to improve the problem that the shallow layer of the current early-retirement network cannot decouple low-level expression ability and high-level discrimination ability, the present invention proposes a dynamic optimization early-retirement model, method, system and device based on efficient reasoning.

[0007] The purpose of the present invention can be achieved through the following technical solutions.

[0008] A dynamic optimization early-retirement model based on efficient reasoning adopts an early-retirement network and is composed of S stages, wherein the output of the previous stage is used as the input of the next stage, the classification branch of each stage is provided with a classifier, a lightweight side branch module is inserted into the classification branch of each stage and a residual connection is performed, the output of each stage is used as the input of the corresponding side branch module in its classification branch, the output of each stage and the output of the corresponding side branch module are residually connected to obtain classification features, and the classification features are used as the input of the classifier in the corresponding classification branch.

[0009] Furthermore, the backbone network of each stage is a ViT network, and the fine-tuning modules used are LoRA modules or Adapter modules.

[0010] Furthermore, the shallow classifier adopts a high-order classifier, and the deep classifier adopts a linear classifier.

[0011] Furthermore, each of the side branch modules adopts a LoRA module or an Adapter module; in each classification branch, the output features of the corresponding stage are processed by the side branch module and then residually connected with the output features of the corresponding stage to obtain new classification features for each stage, which are input into the corresponding classifier.

[0012] A dynamic optimization method based on an efficient inference dynamic optimization early retirement model comprises the following steps:

[0013] S1: The input image is processed in turn by each stage of the inference-efficient dynamic optimization early-retirement model, and the output features of each stage are input into its corresponding classification branch and the next stage respectively;

[0014] S2: In each classification branch, the output features of each stage are processed by the corresponding side branch module and then residually connected with the output features of the corresponding stage to obtain new classification features of each stage, which are input into the corresponding classifier.

[0015] S3: In each classification branch, the output of each classifier is multiplied by the confidence factor, and then passed through the softmax function to obtain the predicted classification probability value, and then the cross entropy is calculated with the label corresponding to the input image to obtain the loss of each stage;

[0016] S4: In each classification branch, the loss of the previous stage is used as a dynamic adjustment parameter and multiplied by the loss of the current stage to obtain the dynamically adjusted loss of the current stage;

[0017] S5: The weighted sum of the dynamically adjusted losses of all stages is obtained to obtain the final total loss, and the total loss is used to train the dynamic optimization early retirement model based on efficient inference;

[0018] S6: Use the trained inference-efficient dynamic optimization early-retirement model to perform inference classification on the input image.

[0019] Furthermore, the dynamically adjusted loss in step S4 is calculated according to the following formula:

[0020]

[0021] Among them, l i represents the loss of the i-th stage, represents the loss after dynamic adjustment in the i-th stage, and σ represents the sigmoid activation function.

[0022] Furthermore, the total loss in step S5 is obtained according to the following formula:

[0023]

[0024] in, represents the total loss, α i Represents the dynamic loss weight of the i-th stage. In the early stage of training, the dynamic loss weight of the shallow layer is smaller than that of the deep layer. In the later stage of training, the dynamic loss weight of the shallow layer is larger than that of the deep layer.

[0025] A dynamic optimization early exit system based on efficient reasoning, including

[0026] The input module is used to input images into the established model.

[0027] A construction module is used to establish a dynamic optimization early-retirement model based on efficient reasoning. The model adopts an early-retirement network and is composed of S stages, wherein the output of the previous stage is used as the input of the next stage, and the classification branch of each stage is provided with a classifier. A lightweight side branch module is inserted into the classification branch of each stage and a residual connection is performed. The output of each stage is used as the input of the corresponding side branch module in its classification branch. The output of each stage is residually connected with the output of the corresponding side branch module to obtain classification features, and the classification features are used as the input of the classifier in the corresponding classification branch;

[0028] The training module is used to train the established dynamic optimization early retirement model based on efficient reasoning. During the training process, in each classification branch, the output result of each classifier is multiplied by the confidence factor, and then the predicted classification probability value is obtained through the softmax function, and then the cross entropy is calculated with the label corresponding to the input image to obtain the loss of each stage. In each classification branch, the loss of the previous stage is used as a dynamic adjustment parameter and multiplied with the loss of the current stage to obtain the dynamically adjusted loss of the current stage. The weighted sum of the dynamically adjusted losses of all stages is obtained to obtain the final total loss, and the total loss is used to train the dynamic optimization early retirement model based on efficient reasoning.

[0029] The prediction module is used to perform inference classification on the input image using the trained inference-efficient dynamic optimization early-retirement model.

[0030] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the dynamic optimization method based on the above-mentioned efficient dynamic optimization early-retirement model is implemented.

[0031] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned dynamic optimization method based on the efficient inference-based dynamic optimization early-retirement model.

[0032] Compared with the prior art, in order to solve the problem that the shallow layer of the early-falling network cannot decouple the low-level expression ability and the high-level discrimination ability, the present invention makes improvements from the perspective of model architecture and optimization strategy. The beneficial effects brought by the specific technical solution are:

[0033] (1) Architecture improvement. In order to achieve the decoupling of expressive power and discriminative power, from the perspective of network architecture, it is possible to directly operate on the features of the shallow layer input into the deep layer, thereby achieving direct decoupling at the feature level. On the basis of the early-retirement network, in order to balance the ability of the shallow stage to learn low-level expressive features and the ability of high-level discriminative features, a side branch module is inserted into the classification branch of each stage, and the expressive power of the features transmitted from the shallow layer to the deep layer is maintained through residual connections, and the shallow classifier is replaced with a high-order classifier to form a multi-classifier architecture. Compared with the existing technology, it aims to achieve a solution that combines efficient training and efficient reasoning.

[0034] (2) Dynamic optimization strategy improvement perspective. After completing the architecture improvement, how to train the network is also a key issue. Through targeted improvements, the training strategy can be more in line with the design of the early retirement network. For the dynamic optimization strategy, the core point is to dynamically adjust the loss weight distribution. During the entire training process, the dynamic loss weight of the shallow layer is smaller in the early stage of training and larger in the later stage of training, and the dynamic loss weight of the deep layer is larger in the early stage of training and smaller in the later stage of training. In addition, a learnable confidence factor is introduced, which is multiplied by the classifier output to enhance the credibility of the shallow layer for the classification results and further improve the reliability of the classification effect. In order to prevent the shallow layer from affecting the deep layer while learning the discrimination ability, the loss of the shallow layer is used as a dynamic adjustment parameter to control the loss of the deep layer, thereby realizing dynamic adjustment of the deep layer update in the later stage of training.

[0035] Compared with the prior art, the present invention utilizes a multi-classifier architecture and a dynamic optimization strategy to achieve a solution that combines efficient training and efficient reasoning. The present invention solves the problem that the shallow layer of the early-retirement network cannot decouple low-level expression capabilities and high-level discrimination capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a diagram of the architecture of the early retirement model based on efficient dynamic optimization based on reasoning of the present invention.

[0037] Figure 2 The present invention is a schematic diagram of a dynamic optimization process of an early retirement model based on efficient dynamic optimization based on reasoning. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings.

[0039] like Figure 1As shown, the present invention proposes a dynamic optimization early-retirement model based on efficient reasoning, which adopts an early-retirement network and consists of S stages, wherein the output of the previous stage is used as the input of the next stage, and the classification branch of each stage is provided with a classifier, and a lightweight side branch module is inserted into the classification branch of each stage and a residual connection is performed, and the output of each stage is used as the input of the corresponding side branch module in its classification branch, and the output of each stage and the output of the corresponding side branch module are residually connected to obtain classification features, and the classification features are used as the input of the classifier in the corresponding classification branch.

[0040] In the above model, the backbone network of each stage is a ViT network, and the fine-tuning modules used are LoRA modules or Adapter modules.

[0041] In the above model, the shallow classifier can be a high-order classifier, and the deep classifier can be a linear classifier. Note that the deep and shallow layers here are relative, such as stage S-1 relative to stage S can also be said to be shallow.

[0042] In the above model, each of the side branch modules uses the same architecture as the fine-tuning module, and both use the LoRA module or the Adapter module. In each classification branch, the output features of the corresponding stage are processed by the side branch module, and then residually connected with the output features of the corresponding stage to obtain new classification features for each stage, which are input into the corresponding classifier.

[0043] The above dynamic optimization method based on the inference-efficient dynamic optimization early retirement model is as follows: Figure 2 As shown, the specific steps include:

[0044] S1: The input image is processed in turn by each stage of the inference-efficient dynamic optimization early-retirement model, and the output features of each stage are input into its corresponding classification branch and the next stage respectively.

[0045] S2: In each classification branch, the output features of each stage are processed by the corresponding side branch module and then residually connected with the output features of the corresponding stage to maintain the expressive power of the features transmitted from the shallow layer to the deep layer, obtain the new classification features of each stage, and input them into the corresponding classifier.

[0046]

[0047] in, represents the classification features obtained after residual connection in the classification branch of the i-th stage, BYP i represents the side branch module inserted in the classification branch of the i-th stage, X i represents the output features of the i-th stage.

[0048] Since LoRA has very few parameters compared to network blocks, inserting a side branch module into the classification branch at each stage will hardly increase the number of parameters, but it can achieve the encapsulation of high-level discriminant information of features, and the low-level expression features are formed by the side branch module into features with high-level discriminant information. The residual connection retains the low-level information of the input features to prevent the side branch model from directly destroying the features, and while achieving the encapsulation of high-level discriminant information of the features, the low-level expression information is input into the deep layer.

[0049] Although the discriminative ability of features is enhanced by the side branch module, it is still difficult to classify complex samples. At the same time, high-order classifiers have the potential to improve the performance of performance networks by using shallow network architectures. Inspired by this, and considering the high similarity between the shallow and shallow network architectures of the early-retirement network, high-order classifiers are introduced into the shallow layer. High-order classifiers reduce the impact on the low-level expression ability of shallow features by extracting high-level discriminative information of features. The expression of using high-order classifiers in the shallow stage is as follows:

[0050]

[0051] Among them, logits i represents the output of the classifier in the classification branch of the i-th stage, HP i represents the high-order classifier in the classification branch of the i-th stage, i = 1, 2, 3, …, S.

[0052] If it is a deep stage, the expression of its classifier is as follows:

[0053]

[0054] Among them, logits i represents the output of the classifier in the classification branch of the i-th stage, LP i Represents the linear classifier in the classification branch of the i-th stage. Similar to the use of high-order classifiers of different dimensions for different depths in the shallow layer, the deep layer has a stronger high-level discrimination ability, and the use of linear classifiers can well implement the stage training. If a high-order classifier is also used, it will not only increase the number of parameters, but also have the risk of overfitting.

[0055] S3: In each classification branch, the output result of each classifier is multiplied by the confidence factor, and then passed through the softmax function to obtain the predicted classification probability value, and then the cross entropy is calculated with the label corresponding to the input image to obtain the loss of each stage.

[0056] Considering that samples exit based on confidence during the inference process, in order to enhance the confidence of the classifier in the classification result, especially for shallow classifiers that are not suitable for classification, a learnable confidence factor is added to make the final classification prediction result more prominent. The confidence factor is related to the output logits. i Multiply them together, and the predicted classification probability obtained after multiplication is:

[0057]

[0058] Among them, β i is the confidence factor of the i-th stage, represents the predicted classification probability of the i-th stage, Calculate the cross entropy loss with the label Y corresponding to the input image l i represents the loss of the i-th stage.

[0059] S4: In each classification branch, the loss of the previous stage is used as a dynamic adjustment parameter and multiplied by the loss of this stage to obtain the dynamically adjusted loss of this stage.

[0060] Since the present invention increases the loss weight of the shallow layer in the later stage of training, when the shallow layer learns the discriminative ability, the features input to the deep layer will inevitably have more discriminative information that is not needed by the deep layer. Therefore, in order to reduce the impact of the shallow layer on the deep layer, a dynamic adjustment parameter is set to achieve the dynamic update amplitude of the deep layer. When the shallow stage provides high-level discriminative features suitable for classification, the control parameters help to reduce the update of features in the deep stage. In contrast, if the shallow stage provides low-level expression features that are not suitable for classification, the control parameters will relax the restrictions on the update of features in the deep stage. Therefore, the loss of the shallow layer can be used as a parameter to control the final loss of the deep layer. When the loss of the shallow layer is relatively large, the shallow layer will pass more expression features to the deep layer, and the update amplitude of the deep layer can be larger. When the loss of the shallow layer is relatively small, the shallow layer will pass more discriminative features to the deep layer, and the update amplitude of the deep layer should be smaller. The final dynamically adjusted loss is expressed as follows:

[0061]

[0062] Among them, l i represents the loss of the i-th stage, l i-1 represents the loss of the i-1th stage, Represents the loss after dynamic adjustment in the i-th stage. σ represents the sigmoid activation function, which maps the loss of the previous layer to the range of 0 to 1. Specifically, for the sample images of the same batch, when the loss of a certain sample at the same shallow layer is larger than that of other samples, it can be considered that the features of the sample at the shallow layer have more low-level expression information, and the mapping value is larger than that of other samples, that is, the influence of the shallow layer on the deep layer is weakened; when the loss of the sample at the same shallow layer is smaller than that of other samples, it can be considered that the features of the sample at the shallow layer have more high-level discriminant information, and the mapping value is smaller than that of other samples, that is, the influence of the shallow layer on the deep layer is enhanced.

[0063] S5: The weighted sum of the dynamically adjusted losses in all stages is obtained to obtain the final total loss, which is used to train the inference-efficient dynamic optimization early-retirement model.

[0064] The final total loss is expressed as follows:

[0065]

[0066] in, Represents the total loss. i Represents the dynamic loss weight of the i-th stage. Different loss weight allocation strategies are used for shallow and deep layers at different stages of training. The traditional weight allocation strategy generally fixes the loss weights of each stage, and the weight of the shallow layer is usually smaller than the loss weight of the deep layer, which often makes it difficult to learn the discriminative ability of the shallow layer, or it is impossible to keep the deep layer stable when learning the shallow layer. In order to solve this problem, in the early stage of training, the shallow layer should focus on learning low-level expression ability, and give priority to learning high-level discriminative ability in the deep layer. Once the discriminative ability is fully learned in the deep stage, the goal is to gradually transfer this ability to the shallow stage as much as possible. Therefore, in the early stage of training, the dynamic loss weight of the shallow layer is smaller than the dynamic loss weight of the deep layer. In the later stage of training, the dynamic loss weight of the shallow layer is greater than the dynamic loss weight of the deep layer, so as to achieve the learning of the discriminative ability of the deep layer before learning the shallow layer.

[0067] S6: Use the trained dynamic optimization early exit model based on inference efficiency to perform inference classification on the input images. The image can be exited at which stage according to the different difficulty levels of its classification, avoiding all images going through the complete model architecture, thereby achieving efficient inference at the sample level.

[0068] Based on the above principles, the present invention also proposes a dynamic optimization early exit system based on efficient reasoning, which mainly includes:

[0069] An input module, used to input images into the established model;

[0070] A construction module is used to establish a dynamic optimization early-leaving model based on efficient reasoning. The model adopts an early-leaving network and consists of S stages, in which the output of the previous stage serves as the input of the next stage, and the classification branch of each stage is provided with a classifier. A lightweight side branch module is inserted into the classification branch of each stage and is residually connected. The output of each stage serves as the input of the corresponding side branch module in its classification branch, and the output of each stage is residually connected with the output of the corresponding side branch module to obtain classification features, which are used as the input of the classifier in the corresponding classification branch.

[0071] The training module is used to train the established dynamic optimization early retirement model based on efficient reasoning. During the training process, in each classification branch, the output result of each classifier is multiplied by the confidence factor, and then the predicted classification probability value is obtained through the softmax function, and then the cross entropy is calculated with the label corresponding to the input image to obtain the loss of each stage. In each classification branch, the loss of the previous stage is used as a dynamic adjustment parameter and multiplied with the loss of the current stage to obtain the dynamically adjusted loss of the current stage. The weighted sum of the dynamically adjusted losses of all stages is obtained to obtain the final total loss, and the total loss is used to train the dynamic optimization early retirement model based on efficient reasoning.

[0072] The prediction module is used to perform inference classification on the input image using the trained inference-efficient dynamic optimization early-retirement model.

[0073] Based on the above principles, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the dynamic optimization method based on the efficient dynamic optimization early-retirement model as described above.

[0074] Based on the above principles, the present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned dynamic optimization method of the dynamic optimization early-retirement model based on efficient reasoning is implemented.

[0075] Example

[0076] In this embodiment, the present invention is evaluated using dataset image classification (VTAB-1K, CIFAR-100, CUB-200-2011, NABirds, Oxford Flowers, Stanford Cars, and Stanford Dogs) and few-shot learning (Food-101, Stanford Cars, Oxford Flower, FGVCAircraft, and OxfordPets).

[0077] The backbone network is ViT-B / 16, and the fine-tuning module used is LoRA. The model network is divided into 4 stages (S=4), each stage has 3 ViT network blocks and each stage has a classifier, using the AdamW optimization algorithm and cosine annealing training strategy. The lightweight side branch module is consistent with the fine-tuning module used by the network, and also uses the LoRA module. Considering that high-order classifiers have the potential to improve the performance of performance networks by using shallow network architectures, high-order classifiers are only used in the first two stages, and the high-order classifiers use the Moment Probing (MP) method. In addition, considering that different stages in the shallow layer also have relative depths, and the discriminative ability of the stage increases with the increase in depth. Therefore, the shallower stages use high-dimensional high-order classifiers with stronger discriminative information extraction capabilities, while the deeper stages that have better advanced discriminative capabilities themselves use high-order classifiers with smaller dimensions.

[0078] At the beginning of training, the dynamic loss weights of the four stages are 0.01, 0.01, 1.0, and 2.0, respectively.

[0079] At the end of training, the dynamic loss weights of the four stages are 1.5, 1.0, 0.1, and 0.1, respectively, and the dynamic loss weights of each stage are linearly transformed. The confidence factors of the four stages are all initialized to 1.

[0080] All programs are implemented in PyTorch and run on 8 RTX 3090 GPUs.

[0081] Although the functions and working processes of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific functions and working processes, and the above-mentioned specific implementation methods are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all within the protection of the present invention.

Claims

1. A dynamic optimization early-retirement model based on efficient reasoning, using an early-retirement network, consisting of S stages, where the output of the previous stage is used as the input of the next stage, and the classification branch of each stage is provided with a classifier, characterized in that: A lightweight side branch module is inserted into the classification branch of each stage and a residual connection is performed. The output of each stage is used as the input of the corresponding side branch module in its classification branch. The output of each stage is residually connected with the output of the corresponding side branch module to obtain classification features, which are used as the input of the classifier in the corresponding classification branch.

2. The dynamic optimization early retirement model based on efficient reasoning according to claim 1 is characterized in that: The backbone network of each stage is a ViT network, and the fine-tuning modules used are LoRA modules or Adapter modules.

3. The dynamic optimization early retirement model based on efficient reasoning according to claim 1 is characterized in that: The shallow classifier uses a high-order classifier, and the deep classifier uses a linear classifier.

4. The dynamic optimization early retirement model based on efficient reasoning according to claim 1 is characterized in that: Each of the side branch modules adopts a LoRA module or an Adapter module; in each classification branch, the output features of the corresponding stage are processed by the side branch module and then residually connected with the output features of the corresponding stage to obtain new classification features for each stage, which are input into the corresponding classifier.

5. A dynamic optimization method based on an efficient reasoning dynamic optimization early retirement model, characterized in that: The following steps are involved: S1: The input image is processed in turn by each stage of the inference-efficient dynamic optimization early-retirement model, and the output features of each stage are input into its corresponding classification branch and the next stage respectively; S2: In each classification branch, the output features of each stage are processed by the corresponding side branch module and then residually connected with the output features of the corresponding stage to obtain new classification features of each stage, which are input into the corresponding classifier. S3: In each classification branch, the output of each classifier is multiplied by the confidence factor, and then passed through the softmax function to obtain the predicted classification probability value, and then the cross entropy is calculated with the label corresponding to the input image to obtain the loss of each stage; S4: In each classification branch, the loss of the previous stage is used as a dynamic adjustment parameter and multiplied by the loss of the current stage to obtain the dynamically adjusted loss of the current stage; S5: The weighted sum of the dynamically adjusted losses of all stages is obtained to obtain the final total loss, and the total loss is used to train the dynamic optimization early retirement model based on efficient inference; S6: Use the trained inference-efficient dynamic optimization early-retirement model to perform inference classification on the input image.

6. The dynamic optimization method based on the dynamic optimization early retirement model with efficient reasoning according to claim 5 is characterized in that: The dynamically adjusted loss in step S4 is calculated according to the following formula: Among them, l i represents the loss of the i-th stage, represents the loss after dynamic adjustment in the i-th stage, and σ represents the sigmoid activation function.

7. The dynamic optimization method based on the dynamic optimization early-retirement model with efficient reasoning according to claim 5 is characterized in that: The total loss in step S5 is obtained according to the following formula: in, represents the total loss, α i Represents the dynamic loss weight of the i-th stage. In the early stage of training, the dynamic loss weight of the shallow layer is smaller than that of the deep layer. In the later stage of training, the dynamic loss weight of the shallow layer is larger than that of the deep layer.

8. A dynamic optimization early exit system based on efficient reasoning, characterized in that: include The input module is used to input images into the established model. A construction module is used to establish a dynamic optimization early-retirement model based on efficient reasoning. The model adopts an early-retirement network and is composed of S stages, wherein the output of the previous stage is used as the input of the next stage, and the classification branch of each stage is provided with a classifier. A lightweight side branch module is inserted into the classification branch of each stage and a residual connection is performed. The output of each stage is used as the input of the corresponding side branch module in its classification branch. The output of each stage is residually connected with the output of the corresponding side branch module to obtain a classification feature, and the classification feature is used as the input of the classifier in the corresponding classification branch; A training module is used to train the established early retirement model based on efficient dynamic optimization based on reasoning; During the training process, in each classification branch, the output of each classifier is multiplied by the confidence factor, and then the predicted classification probability value is obtained through the softmax function, and then the cross entropy is calculated with the label corresponding to the input image to obtain the loss of each stage; in each classification branch, the loss of the previous stage is used as a dynamic adjustment parameter and multiplied with the loss of this stage to obtain the dynamically adjusted loss of this stage; The weighted sum of the dynamically adjusted losses in all stages is used to obtain the final total loss, which is used to train the inference-efficient dynamic optimization early-retirement model. The prediction module is used to perform inference classification on the input image using the trained inference-efficient dynamic optimization early-retirement model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the dynamic optimization method based on the inference-efficient dynamic optimization early-retirement model as described in any one of claims 5 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the dynamic optimization method based on the inference-efficient dynamic optimization early-retirement model described in any one of claims 5 to 7 is implemented.

Images