Small sample density prediction method based on stable diffusion model and meta-routing network

By adopting a combination method of stable diffusion model and meta-routing network in dense prediction tasks, a dense prediction network with strong generalization is built, and the model fine-tuning is carried out through fragmented meta-learning, the traditional small sample learning method is solved in generalization and transfer. The lack of efficiency improves the accuracy of image density prediction and migration efficiency.

CN119091276APending Publication Date: 2024-12-06UNIV OF SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411282427.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The traditional small sample learning method relies on task-specific prior knowledge in model structure and training framework design, and is difficult to generalize to arbitrary dense prediction tasks. Transferring by modifying model weights may destroy the knowledge acquired by the model during the training stage. , the lack of mining of key parameters of the model leads to poor migration efficiency.

Method used

Using a small sample dense prediction method based on stable diffusion model and meta-routing network, a dense prediction network with strong generalization is constructed by building a hybrid expert model and meta-routing network, and fine-tuning the model through fragmented meta-learning , avoid destroying pre-training knowledge and mining key parameters of the model.

Benefits of technology

It improves the accuracy of image dense prediction in the scene of scarce training samples, maintains the migration efficiency and generalization ability of the model, and can quickly adapt to any new tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091276A_ABST
    Figure CN119091276A_ABST
Patent Text Reader

Abstract

The invention discloses a small sample dense prediction method based on a stable diffusion model and a meta-routing network, and the method comprises the steps: 1, building a dense prediction network based on the stable diffusion model, 2, building a basic dense prediction task data set, 3, training the dense prediction network based on the basic dense prediction task data set, 4, building a fragment data set, and 5, training the dense prediction network based on the fragment data set. 5, training a dense prediction model controlled by the element routing network based on the fragment data set; and 6, finely tuning the element routing network to realize migration of the dense prediction model controlled by the element routing network to a small sample dense prediction new task. According to the method, the image dense prediction network with strong generalization is constructed by means of the pre-training knowledge contained in the stable diffusion model, and key parameters of the image dense prediction network are effectively mined, so that the image dense prediction accuracy is improved under the scene that training samples are scarce.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of computer vision, and specifically is a small sample dense prediction method based on a stable diffusion model and a meta-routing network. Background Art

[0002] Dense prediction tasks are an important type of task in the field of computer vision, and their goal is to learn a mapping from an input image to a pixel-level label. Such tasks include semantic segmentation and depth estimation, and have important application value in scenarios such as autonomous driving. Since the pixel-level data annotation process of dense prediction tasks requires a lot of manpower and material resources, there is an urgent need for a small sample learning method that can quickly learn any dense prediction task based on a small number of samples. Traditional small sample learning methods are oriented towards a single dense prediction task. The design of the model structure and training framework usually relies on task-specific prior knowledge or assumptions, and it is difficult to generalize to any dense prediction task. In addition, most traditional small sample learning methods achieve the migration of test tasks by modifying the model weights. This method may destroy the knowledge acquired by the model during the training phase and lacks the ability to mine the key parameters of the model, so it has poor migration efficiency. Summary of the invention

[0003] In order to solve the shortcomings of the above-mentioned prior art, the present invention proposes a small sample dense prediction method based on a stable diffusion model and a meta-routing network, in order to build a dense prediction network with strong generalization with the help of the pre-training knowledge contained in the stable diffusion model, and effectively mine the key parameters of the dense prediction network, so as to improve the accuracy of image dense prediction in scenarios where training samples are scarce.

[0004] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0005] The small sample dense prediction method based on the stable diffusion model and the meta-routing network of the present invention is characterized in that it comprises the following steps:

[0006] Step 1: Build a dense prediction network based on the stable diffusion model, including: encoder , Hybrid Expert Model , Decoder , Output convolutional layer ;in, represents the i-th output convolutional layer;

[0007] The hybrid expert model It is the UNet noise prediction network of the stable diffusion model All feed-forward layers in Replaced with a randomly initialized mixture of experts layer Then we get, among which, represents the pth feed-forward network layer, represents the p-th mixed expert layer, Represents the number of feed-forward network layers;

[0008] The pth mixed expert layer Includes: N F feed-forward network layers, Basic Routing Network ,in, Indicates The i-th basic routing network of the hybrid expert layer;

[0009] The encoder is the encoder of the stable diffusion model;

[0010] Decoder It is obtained by removing the last convolutional layer from the decoder of the stable diffusion model;

[0011] Step 2: Construct the i-th basic dense prediction task Image sample set And pixel-level label sets ;

[0012] Step 3: Based on the basic dense prediction task Image sample set Train the dense prediction network to obtain a trained dense prediction model;

[0013] Step 4: Randomly select a task from the base dense prediction tasks And sample several image samples respectively to construct a fragment dataset ;

[0014] Step 5: After improving the trained dense prediction model, the training network is obtained and based on the fragment dataset , perform episodic meta-training on the training network to obtain a dense prediction model controlled by the meta-routing network;

[0015] Step 6: New tasks based on dense prediction of samples, fine-tuning the dense prediction model controlled by the meta-routing network to obtain the migrated dense prediction model for new dense prediction tasks We perform dense prediction on any new sample and obtain the pixel-level prediction label of the new sample.

[0016] The small sample dense prediction method based on the stable diffusion model and the meta-routing network described in the present invention is also characterized in that the step 2 includes the following steps:

[0017] Step 2.1. Select N B Basic dense prediction tasks ;in, represents the i-th basic dense prediction task;

[0018] Step 2.2: Construct the i-th basic dense prediction task The image sample set is denoted as ,in, Represents the i-th basic dense prediction task The jth image sample of for The total number of image samples; H represents the height of the image sample, W represents the width of the image sample, and 3 represents the number of channels of the image sample;

[0019] Step 2.3: Construct the i-th basic dense prediction task The pixel-level label set is denoted as ;in, express Pixel-level labels; express The number of channels.

[0020] Furthermore, the step 3 comprises the following steps:

[0021] Step 3.1 Encoder The i-th basic dense prediction task The jth image sample Convert to latent space and get latent space features :

[0022] Step 3.2: Input to the Hybrid Expert Model and use the first Basic Routing Network Respectively control the related feedforward network layers in each mixed expert layer Calculate and get the mixed expert features ;in, express The number of channels;

[0023] Step 3.3: Decoder Mixed Expert Features Upsample to get the output features ;in, Represents output features The number of channels;

[0024] Step 3.4: Input to the i-th output convolutional layer After processing in, the pixel-level prediction label finally output by the dense prediction network is obtained. ;in, express The number of channels;

[0025] Step 3.5: Use formula (1) to construct the i-th basic dense prediction task The loss function :

[0026] (1)

[0027] Step 3.6: Use formula (2) to construct the basic task total loss function of the dense prediction network :

[0028] (2)

[0029] In formula (2), for Hyperparameters of

[0030] Step 3.7: Based on loss function , the dense prediction network is trained using the gradient descent method until the maximum number of training times is reached, thereby obtaining a trained dense prediction model.

[0031] Further, the step 4 comprises the following steps:

[0032] Step 4.1: Randomly select a task from the basic dense prediction tasks And sampling samples, construct the lth support set ,in, express The Support samples, express Pixel-level labels; represents the number of samples in the support set; express The number of channels;

[0033] Step 4.2: From the basic dense prediction task Medium Sampling samples, construct the lth query set ,in, express The A sample of queries, express Pixel-level labels; Indicates the number of samples in the query set;

[0034] Step 4.3: and Construct the lth fragment dataset , thereby constructing a fragment dataset ;in, Represents the total number of fragment datasets.

[0035] Further, the step 5 comprises the following steps:

[0036] Step 5.1: The pth mixed expert layer of the trained dense prediction model middle Basic Routing Network Replaced with a meta-routing network ; thus completing Meta-routing Network Replacement of; the dense prediction model after training Output convolutional layer Replace it with a convolutional layer ; thus constructing the Training Network ;

[0037] Step 5.2: Enter to Training Network Processed in, get the training network Final output pixel-level prediction label ;

[0038] Step 5.3: Use formula (3) to build a training network The inner training loss function is :

[0039] (3)

[0040] Step 5.4: Fix the training network In-meta routing network and convolutional layers , and based on the loss function , to train the network After the sub-gradient update, the training network after inner layer training is obtained , and the meta-routing network after inner layer training is recorded as , the convolutional layer after inner layer training is recorded as ;in, represents the pth mixed expert layer Meta-routing network after training the inner layers;

[0041] Step 5.5: Input to the training network after inner layer training Processed in to obtain the optimal pixel-level prediction label ;

[0042] Step 5.6: Use formula (4) to construct the training network after inner layer training The outer layer training loss function is :

[0043] (4)

[0044] Step 5.7: Based on loss function , the training network after inner layer training Perform a gradient update to obtain the training network after outer layer training ; and the meta-routing network after outer layer training is recorded as , the convolutional layer after outer layer training is recorded as ;in, represents the pth mixed expert layer Meta-routing network after training in the middle and outer layers;

[0045] Step 5.8: The inner layer trained meta-routing network The parameters are recorded as ; The outer layer trained meta-routing network The parameters are recorded as ; The meta-routing network The parameters are recorded as ;in, express Parameters, express Parameters, express The parameters of the meta-routing network are obtained using formula (5) Updated parameters :

[0046] (5)

[0047] Step 5.9: Training Network The convolutional layer in Replace it with a convolutional layer , thus constructing the +1 training network ;

[0048] Step 5.10: +1 assigned to Then return to step 5.2 and execute sequentially until > So far, we get Training Network and as a dense prediction model for meta-routing network control.

[0049] Further, the step 6 comprises the following steps:

[0050] Step 6.1: Construct a new dense prediction task The training set ,in, express No. training samples, express Pixel-level labels; express The number of channels;

[0051] Step 6.2: Dense prediction model controlled by the meta-routing network The convolutional layer in Replace it with a convolutional layer , thereby building a dense prediction model for new tasks ;

[0052] Step 6.3: Dense prediction model input to new tasks Processed in to obtain a dense prediction model for the new task Final output pixel-level prediction label ;

[0053] Step 6.4: Use formula (6) to construct the loss function for dense prediction of new tasks :

[0054] (6)

[0055] Step 6.5: Based on loss function , a dense prediction model for new tasks Meta-routing network in and convolutional layers conduct Sub-gradient updates are performed to obtain the final migrated dense prediction model.

[0056] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the small sample dense prediction method, and the processor is configured to execute the program stored in the memory.

[0057] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the small sample dense prediction method when the computer program is executed by a processor. Compared with the prior art, the present invention has the following advantages:

[0058] 1. The present invention constructs a hybrid expert dense prediction network based on the stable diffusion model. The network makes full use of the pre-training knowledge of the stable diffusion model and has good generalization ability for dense prediction tasks.

[0059] 2. The present invention uses a fragmented meta-learning method to train a meta-routing network, and fine-tunes the hybrid expert dense prediction network based on the meta-routing network. This enables the model to migrate by adjusting the calculation path while retaining the weight, avoiding destroying the pre-training knowledge of the model and effectively mining the key parameters of the model, thereby improving the accuracy of image dense prediction in scenarios where training samples are scarce. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of the calculation process of the dense prediction network proposed in the present invention. DETAILED DESCRIPTION

[0061] In this embodiment, a small sample dense prediction method based on a stable diffusion model and a meta-routing network includes the following steps:

[0062] Step 1: Build a dense prediction network based on the stable diffusion model, including: encoder , Hybrid Expert Model , Decoder , Output convolutional layer ;in, represents the i-th output convolutional layer;

[0063] Mixed Expert Model It is the UNet noise prediction network of the stable diffusion model All feed-forward layers in Replaced with a randomly initialized mixture of experts layer Then we get, among which, represents the pth feed-forward network layer, represents the p-th mixed expert layer, Represents the number of feedforward network layers; the addition of the mixed expert layer gives the mixed expert model The ability to control model computation paths through routing networks;

[0064] The pth mixed expert layer Includes: N F Feed-forward layers , Basic Routing Network ,in, Indicates The i-th basic routing network of the hybrid expert layer;

[0065] Encoder is the encoder of the stable diffusion model;

[0066] Decoder It is obtained by removing the last convolutional layer from the decoder of the stable diffusion model;

[0067] The stable diffusion model uses a generative objective function for pre-training on a large-scale image-text dataset. Its pre-training parameters contain rich fine-grained visual knowledge required for dense prediction tasks. Therefore, the dense prediction network built based on the stable diffusion model has strong generalization for tasks such as dense prediction.

[0068] Step 2: Build a basic dense prediction task dataset;

[0069] Step 2.1. Select N B Basic dense prediction tasks ;in, represents the i-th basic dense prediction task;

[0070] Step 2.2: Construct the i-th basic dense prediction task The image sample set is denoted as ,in, Represents the i-th basic dense prediction task The jth image sample of for The total number of image samples; H represents the height of the image sample, W represents the width of the image sample, and 3 represents the number of channels of the image sample;

[0071] Step 2.3: Construct the i-th basic dense prediction task The pixel-level label set is denoted as ;in, express Pixel-level labels; express The number of channels;

[0072] Step 3: Train the dense prediction network based on the basic dense prediction task dataset;

[0073] Step 3.1 Encoder Will Convert to latent space and get latent space features :

[0074] Step 3.2: Input to the Hybrid Expert Model and use the first Basic Routing Network Separately control the relevant Feed-forward network layer pairs Calculate, such as Figure 1 As shown, the calculation process is as follows: For the p-th mixed expert layer Input feature map , the pth mixed expert layer Using formula (1) and formula (2) to input feature map Each location Features ( ) for processing:

[0075]

[0076]

[0077] Formula (1) represents the basic routing network The calculation process, yes The output gating weight; in formula (1) and Is the basic routing network The learnable parameters of is a smooth approximation of the ReLU function: , represents a random sample of a value from a normal distribution. The function retains the largest value, and set all other values ​​in the input vector to 0; in formula (2) represent The value at the kth position, if , you can skip , thereby reducing computing consumption;

[0078] The pth mixed expert layer For the input feature map The features of all positions are processed to obtain the output features of the pth mixed expert layer , and input into the next network layer; finally, the mixed expert features are obtained ;in, express The number of channels;

[0079] Step 3.3: Decoder Mixed Expert Features Upsample to get the output features ;in, Represents output features The number of channels;

[0080] Step 3.4: Input to the i-th output convolutional layer After processing in, the pixel-level prediction label finally output by the dense prediction network is obtained. ;in, express The number of channels;

[0081] Step 3.5: Use formula (3) to construct the i-th basic dense prediction task The loss function :

[0082] (3)

[0083] Step 3.6: Use formula (4) to construct the basic task total loss function of the dense prediction network :

[0084] (4)

[0085] In formula (4), for Hyperparameters of

[0086] Step 3.7: Based on loss function , the dense prediction network is trained using the gradient descent method until the maximum number of training times is reached, thereby obtaining a trained dense prediction model; training on basic dense prediction tasks enables the trained dense prediction model to have general knowledge in the field of dense prediction, thereby providing a guarantee for the rapid migration of the model to any new dense prediction task;

[0087] Step 4: Build the snippet dataset ;

[0088] Step 4.1: Randomly select a task from the basic dense prediction tasks And sampling samples, construct the lth support set ,in, express The Support samples, express Pixel-level labels; represents the number of samples in the support set;

[0089] Step 4.2: From the basic dense prediction task Medium Sampling samples, construct the lth query set ,in, express The A sample of queries, express Pixel-level labels; Indicates the number of samples in the query set;

[0090] Step 4.3: and Construct the lth fragment dataset , thereby constructing a fragment dataset ;in, Represents the total number of fragment datasets; each fragment dataset simulates a For small sample dense prediction tasks with 10 training samples, training the dense prediction model on the fragment dataset can enhance the fast learning ability of the dense prediction model for small sample dense prediction tasks;

[0091] Step 5: Based on the fragment dataset Perform episodic meta-training to obtain a dense prediction model for meta-routing network control;

[0092] Step 5.1: The pth mixed expert layer of the trained dense prediction model middle Basic Routing Network Replaced with a meta-routing network ; thus completing Meta-routing Network Replacement of; the dense prediction model after training Output convolutional layer Replace it with a convolutional layer , the number of output channels of this convolutional layer is , and all parameters are initialized to 0; thus constructing the Training Network ;

[0093] Step 5.2: Enter to Training Network Processed in, get the training network Final output pixel-level prediction label ;

[0094] Step 5.3: Use formula (5) to build a training network The inner training loss function is :

[0095] (5)

[0096] Step 5.4: Fix the training network In-meta routing network and convolutional layers , and based on the loss function , to train the network After the sub-gradient update, the training network after inner layer training is obtained , and the meta-routing network after inner layer training is recorded as , the convolutional layer after inner layer training is recorded as ;in, represents the pth mixed expert layer Meta-routing network after training the inner layers;

[0097] Step 5.5: Input to the training network after inner layer training Processed in to obtain the optimal pixel-level prediction label ;

[0098] Step 5.6: Use formula (6) to construct the training network after inner layer training The outer layer training loss function is :

[0099] (6)

[0100] Step 5.7: Based on loss function , the training network after inner layer training Perform a gradient update to obtain the training network after outer layer training ; and the meta-routing network after outer layer training is recorded as , the convolutional layer after outer layer training is recorded as ;in, represents the pth mixed expert layer Meta-routing network after training in the middle and outer layers;

[0101] Step 5.8: The inner layer trained meta-routing network The parameters are recorded as ; The outer layer trained meta-routing network The parameters are recorded as ; The meta-routing network The parameters are recorded as ;in, express Parameters, express Parameters, express The updated parameters of the meta-routing network are obtained by using formula (7): :

[0102] (7)

[0103] Formula (7) updates the parameters of the meta-routing network based on the idea of ​​meta-learning. Different from the traditional supervised learning training paradigm in which gradient updates are performed directly based on the objective function, the present invention adopts a double-layer training method including inner-layer training and outer-layer training. The model parameters are updated based on the parameter changes of the training network after inner-layer training in outer-layer training. This training method can enhance the generalization of the model.

[0104] Step 5.9: Training Network The convolutional layer in Replace it with a convolutional layer , the number of output channels of this convolutional layer is , and all parameters are initialized to 0; thus constructing the +1 training network ;

[0105] Step 5.10: After assigning l+1 to l, return to step 5.2 and execute sequentially until l> So far, we get Training Network and as a dense prediction model for meta-routing network control;

[0106] Step 6: Dense prediction of new tasks The dense prediction model controlled by the meta-routing network is fine-tuned using samples to obtain the migrated dense prediction model.

[0107] Step 6.1: Construct a new dense prediction task The training set ,in, express No. training samples, express Pixel-level labels; express The number of channels;

[0108] Step 6.2: Dense prediction model controlled by the meta-routing network The convolutional layer in Replace it with a convolutional layer , the number of output channels of this convolutional layer is , and all parameters are initialized to 0; thus constructing a dense prediction model for the new task ;

[0109] Step 6.3: Dense prediction model input to new tasks Processed in to obtain a dense prediction model for the new task Final output pixel-level prediction label ;

[0110] Step 6.4: Use formula (8) to construct the loss function for dense prediction of new tasks :

[0111] (8)

[0112] Step 6.5: Based on loss function , a dense prediction model for new tasks Meta-routing network in and convolutional layers conduct Sub-gradient update to obtain the final migrated dense prediction model for new dense prediction tasks We perform dense prediction on any new sample and obtain the pixel-level prediction label of the new sample.

[0113] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0114] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

Claims

1. A small sample dense prediction method based on a stable diffusion model and a meta-routing network, characterized in that: The following steps are involved: Step 1: Build a dense prediction network based on the stable diffusion model, including: encoder , Hybrid Expert Model , Decoder , Output convolutional layer ;in, represents the i-th output convolutional layer; The hybrid expert model It is the UNet noise prediction network of the stable diffusion model All feed-forward layers in Replaced with a randomly initialized mixture of experts layer Then we get, among which, represents the pth feed-forward network layer, represents the p-th mixed expert layer, Represents the number of feed-forward network layers; The pth mixed expert layer Includes: N F feed-forward network layers, Basic Routing Network ,in, Indicates The i-th basic routing network of the hybrid expert layer; The encoder is the encoder of the stable diffusion model; Decoder It is obtained by removing the last convolutional layer from the decoder of the stable diffusion model; Step 2: Construct the i-th basic dense prediction task Image sample set And pixel-level label sets ; Step 3: Based on the basic dense prediction task Image sample set Train the dense prediction network to obtain a trained dense prediction model; Step 4: Randomly select a task from the base dense prediction tasks And sample several image samples respectively to construct a fragment dataset ; Step 5: After improving the trained dense prediction model, the training network is obtained and based on the fragment dataset , perform episodic meta-training on the training network to obtain a dense prediction model controlled by the meta-routing network; Step 6: New tasks based on dense prediction of samples, fine-tuning the dense prediction model controlled by the meta-routing network to obtain the migrated dense prediction model for new dense prediction tasks We perform dense prediction on any new sample and obtain the pixel-level prediction label of the new sample.

2. According to claim 1, a small sample dense prediction method based on a stable diffusion model and a meta-routing network is characterized in that: The step 2 comprises the following steps: Step 2.

1. Select N B Basic dense prediction tasks ;in, represents the i-th basic dense prediction task; Step 2.2: Construct the i-th basic dense prediction task The image sample set is denoted as ,in, Represents the i-th basic dense prediction task The jth image sample of for The total number of image samples; H represents the height of the image sample, W represents the width of the image sample, and 3 represents the number of channels of the image sample; Step 2.3: Construct the i-th basic dense prediction task The pixel-level label set is denoted as ;in, express Pixel-level labels; express The number of channels.

3. According to claim 1, a small sample dense prediction method based on a stable diffusion model and a meta-routing network is characterized in that: The step 3 comprises the following steps: Step 3.1 Encoder The i-th basic dense prediction task The jth image sample Convert to latent space and get latent space features : Step 3.2: Input to the Hybrid Expert Model and use the first Basic Routing Network Respectively control the related feedforward network layers in each mixed expert layer Calculate and get the mixed expert features ;in, express The number of channels; Step 3.3: Decoder Mixed Expert Features Upsample to get the output features ;in, Represents output features The number of channels; Step 3.4: Input to the i-th output convolutional layer After processing in, the pixel-level prediction label finally output by the dense prediction network is obtained. ;in, express The number of channels; Step 3.5: Use formula (1) to construct the i-th basic dense prediction task The loss function : (1) Step 3.6: Use formula (2) to construct the basic task total loss function of the dense prediction network : (2) In formula (2), for Hyperparameters of Step 3.7: Based on loss function , the dense prediction network is trained using the gradient descent method until the maximum number of training times is reached, thereby obtaining a trained dense prediction model.

4. According to claim 1, a small sample dense prediction method based on a stable diffusion model and a meta-routing network is characterized in that: The step 4 comprises the following steps: Step 4.1: Randomly select a task from the basic dense prediction tasks And sampling samples, construct the lth support set ,in, express The Support samples, express Pixel-level labels; represents the number of samples in the support set; express The number of channels; Step 4.2: From basic dense prediction tasks Medium Sampling samples, construct the lth query set ,in, express The A sample of queries, express Pixel-level labels; Indicates the number of samples in the query set; Step 4.3: and Construct the lth fragment dataset , thereby constructing a fragment dataset ;in, Represents the total number of fragment datasets.

5. According to claim 4, a small sample dense prediction method based on a stable diffusion model and a meta-routing network is characterized in that: The step 5 comprises the following steps: Step 5.1: The pth mixed expert layer of the trained dense prediction model middle Basic Routing Network Replaced with a meta-routing network ; thus completing Meta-routing Network Replacement of; replace the trained dense prediction model with Output convolutional layer Replace it with a convolutional layer ; thus constructing the Training Network ; Step 5.2: Enter to Training Network Processed in, get the training network Final output pixel-level prediction label ; Step 5.3: Use formula (3) to build a training network The inner training loss function is : (3) Step 5.4: Fix the training network In-meta routing network and convolutional layers , and based on the loss function , to train the network After the sub-gradient update, the training network after inner layer training is obtained , and the meta-routing network after inner layer training is recorded as , the convolutional layer after inner layer training is recorded as ;in, represents the pth mixed expert layer Meta-routing network after training the inner layers; Step 5.5: Input to the training network after inner layer training Processed in to obtain the optimal pixel-level prediction label ; Step 5.6: Use formula (4) to construct the training network after inner layer training The outer layer training loss function is : (4) Step 5.7: Based on loss function , the training network after inner layer training Perform a gradient update to obtain the training network after outer layer training ; and the meta-routing network after outer layer training is recorded as , the convolutional layer after outer layer training is recorded as ;in, represents the pth mixed expert layer Meta-routing network after training in the middle and outer layers; Step 5.8: The inner layer trained meta-routing network The parameters are recorded as ; The outer layer trained meta-routing network The parameters are recorded as ; The meta-routing network The parameters are recorded as ;in, express Parameters, express Parameters, express The parameters of the meta-routing network are obtained using formula (5) Updated parameters : (5) Step 5.9: Training Network The convolutional layer in Replace it with a convolutional layer , thus constructing the +1 training network ; Step 5.10: +1 assigned to Then return to step 5.2 and execute sequentially until > So far, we get Training Network and as a dense prediction model for meta-routing network control.

6. The small sample dense prediction method based on stable diffusion model and meta-routing network according to claim 5 is characterized in that: The step 6 comprises the following steps: Step 6.1: Construct a new dense prediction task The training set ,in, express No. training samples, express Pixel-level labels; express The number of channels; Step 6.2: Dense prediction model controlled by the meta-routing network The convolutional layer in Replace it with a convolutional layer , thereby building a dense prediction model for new tasks ; Step 6.3: Dense prediction model input to new tasks Processed in to obtain a dense prediction model for the new task Final output pixel-level prediction label ; Step 6.4: Use formula (6) to construct the loss function for dense prediction of new tasks : (6) Step 6.5: Based on loss function , a dense prediction model for new tasks Meta-routing network in and convolutional layers conduct Sub-gradient updates are performed to obtain the final migrated dense prediction model.

7. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute any one of the small sample dense prediction methods in claims 1-6, and the processor is configured to execute the program stored in the memory.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the small sample dense prediction method according to any one of claims 1 to 6 are executed.