A diffusion model architecture search method and system based on attention mechanism
By introducing attention mechanisms into the diffusion model, calculating and selecting important feature maps and candidate operations, the diffusion model manual design problems and time-consuming training problems are solved, and efficient diffusion model architecture search and better performance are achieved.
Patent Information
- Application Number
- CN202411415408.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2044-10-11
AI Technical Summary
Existing diffusion models are difficult to manually design network architectures, and the training process is time-consuming and the search for gradient descent neural architectures may lead to suboptimal networks, and candidate operation evaluation indicators do not match actual performance.
Using the diffusion model architecture search method based on attention mechanism, by constructing the diffusion model backbone architecture supernet, adding the attention mechanism to calculate the feature map and the attention weight of candidate operations, selecting important feature maps and optimization subnets, reducing the calculation of the search process volume and improve stability.
It realizes more efficient diffusion model architecture search, avoids the generation of suboptimal networks, improves model performance and training stability, and reduces search time consumption.
Smart Images
Figure CN118917389B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep neural network technology, and in particular to a diffusion model architecture search method and system based on an attention mechanism. Background Art
[0002] The diffusion model was proposed in 2015. Due to its innovative ideas, the diffusion model has had a profound impact on deep learning and generative model research. The diffusion model simulates the forward diffusion process of data from the original distribution to the noisy distribution and learns the inverse operation of the process, that is, gradually denoising from the noise to reconstruct high-quality data samples. The diffusion model has many advantages, making it an important tool in the field of generative models in deep learning. First, the diffusion model is not limited to a single neighborhood and can be used to generate multimodal data, such as from text to image and from image to text. Second, the diffusion model is highly flexible in structure and can adapt to various complex data sets and scenarios. At the same time, its training process is relatively stable and easy to adjust and optimize. This allows researchers to adjust model parameters according to specific needs to obtain better performance. Finally, the diffusion model implements precise mathematical operations, allowing researchers to conduct in-depth mathematical analysis and theoretical research based on these models. However, the diffusion model still faces some challenges. For example, the network architecture of the traditional diffusion model uses the U-Net network designed manually by humans, and the performance of the U-Net network depends largely on the design of the network architecture. If the network architecture is not well designed, there is a risk of overfitting. In addition, the network architecture of the U-Net network is relatively complex and time-consuming during training. Therefore, how to solve these problems to improve the performance of the diffusion model is a very meaningful research direction.
[0003] In response to the problems that existing diffusion models are difficult to manually design and model training is time-consuming, researchers proposed using neural network architecture search technology to automatically design diffusion model network architectures. Neural architecture search is proposed to automatically discover high-performance deep neural network architectures. The three most widely used categories for neural network architecture search are evolutionary algorithms, reinforcement learning, and gradient descent algorithms. Gradient descent neural architecture search has become a topic of extensive research because its implementation does not require a lot of prior knowledge about the problem and can be used directly. However, gradient descent neural architecture search may involve the problem that the size of the candidate operation evaluation indicator does not match the actual performance of the candidate operation, resulting in the searched network being suboptimal.
[0004] The attention mechanism focuses on modeling the importance of each feature map, thereby making more effective use of information in the feature extraction process. By dynamically adjusting the weights of different channels, important features can be highlighted and unimportant features can be suppressed, thereby improving the representation ability of features. Therefore, in order to solve the problem that the existing diffusion model is difficult to manually design and the model training is time-consuming, the neural network architecture search technology can be used to search for the diffusion model architecture, and the attention mechanism can be used to guide the search process to select a sub-network with fewer parameters and better performance. Therefore, it is necessary to propose a diffusion model architecture search method based on the attention mechanism. Summary of the invention
[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a diffusion model architecture search method and system based on the attention mechanism in view of the shortcomings of the prior art. The method comprises the following steps:
[0006] Step 1, construct a diffusion model backbone architecture supernet;
[0007] Step 2: Add an attention mechanism to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node;
[0008] Step 3: Select a feature map from each input node to participate in the forward propagation according to the attention weight calculated in step 2;
[0009] Step 4: Add an attention mechanism after the candidate operation of each edge of the diffusion model backbone architecture supernet to calculate the attention weights of the output feature maps of all candidate operations in each edge;
[0010] Step 5, training the diffusion model backbone architecture supernet until the objective function reaches convergence;
[0011] Step 6: Accumulate the attention weights calculated in step 4 to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and construct the optimal subnet based on the operation.
[0012] In step 1, the diffusion model backbone architecture supernet includes 1 fully connected layer, 4 downsampling layers, 4 upsampling layers and 9 convolution modules;
[0013] Each downsampling layer contains one global maximum pooling operation; each upsampling layer contains one transposed convolution operation; each convolution module contains three nodes and two edges, where the three nodes represent three groups of feature maps, and the two edges represent two groups of convolution operations, each group of convolution operations contains two ordinary convolution operations and three depth-separable convolution operations;
[0014] The fully connected layer, down-sampling layer and up-sampling layer are fixed layers and do not participate in the search. The convolution module is the search layer. The search process selects an optimal operation from each edge of the convolution module as the operation of the final optimal subnet.
[0015] Among them, the edges connecting different intermediate nodes perform certain operations, process the data of the input nodes, and finally the output nodes output the results, including: 3*3 convolution, 5*5 convolution, 3*3 separable convolution, 5*5 separable convolution and 7*7 separable convolution operations, to achieve effective feature extraction, information fusion and resolution adjustment, so as to improve the performance and expression of the model.
[0016] In order to more clearly express the network architecture of the diffusion model, the present invention uses a 2*5*9 matrix to express a diffusion model architecture, 2 represents 2 edges in each convolution module, 5 represents 5 candidate operations in each edge, and 9 represents that the diffusion model supernet architecture has 9 convolution modules.
[0017] In step 2, the traditional gradient-based diffusion model architecture search will retain all feature maps in the input node during the search process and use them as input for each candidate operation. Although this method retains all the features of the input image, it has huge computational consumption, resulting in excessively high search costs. Moreover, among all the feature maps of a node, not all feature maps contain detailed features, but more than half of the feature maps contain only a single feature or no features, which will affect the generation of image detail features by the diffusion model. Therefore, these feature maps can be eliminated during the forward propagation process, making the model training more stable and efficient.
[0018] The present invention adopts the attention mechanism as a tool for feature map selection. The attention mechanism is a mechanism widely used in deep learning, especially in computer vision tasks. The core idea is to enable the model to automatically learn and pay attention to the importance of different feature maps in the input data, so as to more effectively use this information for feature extraction and decision-making. The attention mechanism adaptively adjusts the weights of different feature maps by modeling the importance of each feature map. In a convolutional neural network, each feature map represents a specific feature, such as edge, texture, color, etc. The attention mechanism enhances or suppresses the expression of these features by learning the importance of each feature map for the current task, thereby improving the performance of the model. Therefore, it is adopted as a tool for feature map selection.
[0019] Step 2 includes: each convolutional module contains 3 nodes, the first 2 nodes are regarded as input nodes, there are 9 convolutional modules in total, so a total of 18 attention mechanisms need to be added to the supernet;
[0020] The attention mechanism is added to the input node of each convolution module by the following method: first, a global compression operation is performed on the feature map in the input node to obtain the global information of each feature map. The compression operation includes global average pooling and global maximum pooling. The compression operation calculates the average value and maximum value of each feature map respectively, compresses the spatial dimension of the feature map to 1, and thus obtains a vector containing only channel information;
[0021] Then, a fully connected layer is used to transform the vector to learn the attention weight of each channel, and a nonlinear activation function ReLU is used to increase the nonlinear ability of the fully connected layer, and a Sigmoid function is used to normalize the weight to between [0,1];
[0022] Finally, the learned weights are multiplied with each channel of the original feature map to achieve channel-level weighting, so that each feature map adjusts its contribution according to the attention weight, thereby enhancing important features and suppressing unimportant features.
[0023] Step 3 includes: in order to improve the stability of model training and reduce the amount of calculation in the search process, first, sort the attention weights calculated in step 2 from large to small, and select the attention weights with larger values in the first 1 / 4; then, loop through the weighted feature graph of the input node, if the attention weight corresponding to the weighted feature graph belongs to the selected first 1 / 4 attention weight, then mark the weighted feature graph as 1, otherwise mark it as 0; finally, select the weighted feature graph marked as 1 and splice it as the input feature graph of each edge in the convolution module.
[0024] Step 4 includes:
[0025] Step 4.1, input the weighted feature map selected in step 3 into the candidate operation in each edge of the convolution module;
[0026] Step 4.2, concatenate the output feature maps of the candidate operations in each edge and input them into the attention mechanism to calculate the attention weight;
[0027] In step 4.3, the feature maps to which the attention mechanism is applied are added.
[0028] In step 4.1, because the optimal subnet needs to be selected from the supernet, the pros and cons of each candidate operation in each edge of the convolution module need to be verified. The weighted feature map selected in step 3 is simultaneously input into the five candidate operations in each edge of the convolution module, expressed as:
[0029] (1),
[0030] in, represents the weighted feature map selected in step 3, represents the candidate operation in the i-th edge of the convolutional module, represents the output feature map of the i-th candidate operation, ;
[0031] After receiving the input feature map, the five candidate operations will output five sets of corresponding feature maps, and then judge the importance of the five candidate operations based on the importance of the five sets of corresponding feature maps output.
[0032] Step 4.2 includes: concatenating the five sets of corresponding feature maps, and then inputting the concatenated feature maps into the attention mechanism to calculate the attention weight of each feature map, expressed as:
[0033] (2),
[0034] in, represents the calculated attention weight, represents the Sigmoid function, and represents the fully connected layer, represents the activation function, A feature map representing the concatenation;
[0035] Finally, the concatenated feature map is multiplied by the attention weight to obtain a weighted feature map, so as to highlight important features and suppress unimportant features.
[0036] Step 4.3 includes: the number of weighted feature maps obtained in step 4.2 is the sum of the number of output feature maps of the five candidate operations, so the number of weighted feature maps is large, resulting in too high a computational cost in the search process; in order to reduce the amount of computation, first, the weighted feature maps corresponding to different operations are separated to obtain five groups of weighted feature maps; then, the five groups of weighted feature maps are looped through, and one feature map is taken out from each group each time and added together, and finally the five groups of weighted feature maps are added together into one group of weighted feature maps.
[0037] Step 5 includes: after completing step 1, step 2 and step 4, a constructed diffusion model backbone architecture supernet is obtained, and the attention mechanism has been added to the supernet, and the supernet is trained: first, the cross entropy loss function is selected as the objective function for training the diffusion model; then, the hyperparameters in the training process are set, the learning rate is set to 0.0001, the optimizer is set to Adam optimizer, the number of training cycles is set to 100, and the batch size is set to 64; finally, the diffusion model is trained until the objective function converges;
[0038] Step 6 includes: after completing the training of the diffusion model in step 5, step 4 will also obtain the attention weights corresponding to the output feature graphs of the candidate operations. At this time, the attention weights need to be accumulated to calculate the importance of each operation. Separate the attention weights corresponding to the output feature graphs of different operations; accumulate the separated attention weights separately, and the accumulated value at this time represents the importance of each operation; select the operation with the largest accumulated value from each edge of the convolution module to build the optimal subnet.
[0039] The present invention also provides a diffusion model architecture search system based on an attention mechanism, comprising:
[0040] A construction module is used to construct a diffusion model backbone architecture supernet; an attention mechanism is added to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node; a feature map is selected from each input node to participate in forward propagation according to the calculated attention weight; an attention mechanism is added after the candidate operation of each edge of the diffusion model backbone architecture supernet to calculate the attention weight of all the candidate operation output feature maps in each edge;
[0041] The training module is used to train the diffusion model backbone architecture supernet until the objective function reaches convergence;
[0042] The optimization module is used to accumulate the calculated attention weights to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and to construct the optimal subnet based on the operation.
[0043] The present invention effectively alleviates the problem that the diffusion model is difficult to design manually by introducing a method of neural network architecture search. In order to be able to search for a more stable diffusion model architecture from multiple angles, the present invention designs a gradient-based search algorithm by introducing an attention mechanism. At the same time, in order to reduce the time consumption of the search process, the present invention designs a feature map selection strategy based on the attention mechanism to reduce the amount of calculation in the search process. This method achieves better results than existing methods.
[0044] The beneficial effects of the present invention are:
[0045] The method of the present invention proposes a neural network architecture search method based on an attention mechanism for a diffusion model. The method searches for the optimal subnet from a supernet by gradient descent. In this method, we propose a feature map selection strategy based on an attention mechanism, thereby avoiding complex calculation processes in the search and improving the stability of the search. At the same time, a method for calculating the importance of candidate operations through an attention mechanism is also proposed, which solves the problem of mismatch between the importance index of candidate operations and the actual performance of candidate operations in traditional gradient-based neural network architecture search methods. This diffusion model architecture search method based on an attention mechanism brings new methods and ideas to the field of diffusion model architecture search, and has high practical value and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is the architecture diagram of the hypernet convolution module of the diffusion model in the present invention.
[0047] Figure 2 It is the basic framework diagram of the diffusion model supernet in the present invention.
[0048] Figure 3 It is a structural diagram of the attention mechanism in the present invention.
[0049] Figure 4 It is the calculation flow chart of each edge of the convolution module.
[0050] Figure 5 It is a schematic diagram of feature map addition. DETAILED DESCRIPTION
[0051] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.
[0052] In a specific embodiment of the present invention, a diffusion model architecture search method based on an attention mechanism is provided, comprising the following steps:
[0053] Step 1: Figure 1 As shown, the diffusion model architecture search space proposed by the present invention is Figure 1 The internal structure of a convolution module in the diffusion model supernet is described, which consists of 3 groups of feature maps and 2 groups of candidate operations. The first two groups of feature maps are input nodes, and the candidate operations include 5 operations, namely 3×3 convolution, 5×5 convolution, 3×3 depth-wise separable convolution, 5×5 depth-wise separable convolution and 7×7 depth-wise separable convolution. Figure 2 It is the basic framework diagram of the diffusion model supernet, which is composed of 9 convolution modules stacked by downsampling and upsampling. The green arrow represents the downsampling operation, which is composed of pooling operations, and the red arrow represents the upsampling operation, which is composed of transposed convolution operations. Figure 2The white arrow in the middle is a jump connection, which combines the features of the first half of the network with the features of the second half, effectively solving the problem of feature loss. The entire search process is to select the best operation from the candidate operations in the 9 convolution modules as the final operation.
[0054] Step 2: After the diffusion model hypernet is built, an attention mechanism needs to be added after the input node of the hypernet convolution module to select feature maps. Figure 3 The structure diagram of the attention mechanism is shown in Figure 1. Its input is a set of feature maps, which correspond to the input nodes in the convolution module in the supernet. First, the input feature maps need to be subjected to global average pooling and global maximum pooling, that is, the feature maps are compressed in two ways. Global average pooling is to average all features in a feature map and use the average value to represent the features of the entire feature map. Global maximum pooling is to take the maximum value of all features in a feature map and use the maximum value to represent the features of the entire feature map. The purpose of compressing the feature maps in two ways is to calculate the importance of each feature map from multiple dimensions. After the feature is compressed, two vectors will be obtained. The two vectors are then input into the fully connected layer for feature extraction. A fully connected layer is used to transform the compressed vectors to learn the attention weight of each channel, and the nonlinear activation function ReLU is used to increase the nonlinear ability of the fully connected layer. The output of the fully connected layer is also two vectors, denoted as vector A and vector B. Vector A corresponds to the attention weight of each feature map calculated by average pooling, and vector B corresponds to the attention weight of each feature map calculated by maximum pooling. Then the two attention weights are added together to obtain the fused attention weight. Finally, the final attention weight is normalized by the Sigmoid function, and the weight value is normalized to the range of [0,1]. Figure 4 As shown, the green part is the specific process of step 2. Represents an input node in the convolution module, which is first input into the attention mechanism to obtain the attention weight , then and Multiply to get the weighted feature map .
[0055] Step 3: After obtaining the weighted feature map, we need to select the weighted feature map. First, Select the larger first 1 / 4 attention weights, and then loop through them , if the traversed attention weight belongs to the larger 1 / 4 weight, then record The position of the corresponding weighted feature map in After that, from The positions of the weighted feature maps with the largest attention weight values in the first 1 / 4 are found. These feature maps are more important than other feature maps, so Select these feature maps to form new feature maps .
[0056] Step 4: Add an attention mechanism after the candidate operation of each edge of the supernet to calculate the attention weights of the output feature maps of all candidate operations in each edge. The specific process includes the following steps:
[0057] Step 4.1: After step 3, select After that, you need to As the input of each candidate operation. Figure 4 As shown, Input into 3×3 convolution, 5×5 convolution, 3×3 separable convolution, 5×5 separable convolution and 7×7 separable convolution respectively to obtain 5 sets of feature maps .
[0058] Step 4.2: After obtaining 5 sets of feature maps through step 4.1, these 5 sets of feature maps are concatenated together as the input of the attention mechanism. Figure 4 As shown, the five groups of feature maps are spliced together to obtain the spliced feature map , then Input to the attention mechanism to get the attention weight , then and Multiply to get the weighted feature map .
[0059] Step 4.3: After step 4.2, Afterwards, you need to Perform feature fusion, that is, fuse the features extracted by different operations together. Figure 5 The figure shows the schematic diagram of adding feature maps. Figure 5 Suppose there are 3 groups of feature maps, each group of feature maps contains 2 feature maps. First separate the 3 groups of feature maps, and then separate the feature maps in the same group, such as Figure 5 Each group of feature maps is separated into two feature maps, and finally the feature maps of different groups are added together. Because each group of feature maps contains two feature maps, the feature map obtained by the final addition also contains two feature maps. Figure 4 middle, After adding the feature maps, a set of feature maps is obtained.
[0060] Step 5: After the diffusion model supernet is constructed and the attention mechanism is added to the corresponding position of the supernet, the supernet can be trained. First, select the cross entropy loss function as the objective function for training the diffusion model, and build a training data set and a validation data set. Then, set the hyperparameters in the training process, set the learning rate to 0.0001, the optimizer to Adam optimizer, the number of training cycles to 100, and the batch size to 64. Finally, train the diffusion model until the objective function converges.
[0061] In step 6, after completing step 5, step 4 will also obtain the attention weight corresponding to the output feature map of the candidate operation, such as Figure 4 As shown, That is, the attention weight corresponding to the output feature map of different candidate operations on each edge when the supernet converges. Accumulate to calculate the importance of each operation. First, separate the attention weights corresponding to the output feature maps of different operations. Then, accumulate the separated attention weights separately, and the accumulated value at this time represents the importance of each operation. Finally, select the operation with the largest accumulated value from each edge of the supernet convolution module to build the optimal subnet. The final network architecture obtained by this method is: [[5×5 convolution, 3×3 separable convolution], [3×3 convolution, 5×5 separable convolution], [5×5 separable convolution, 3×3 separable convolution], [3×3 convolution, 7×7 separable convolution], [3×3 separable convolution, 3×3 separable convolution], [5×5 convolution, 7×7 separable convolution], [5×5 convolution, 7×7 separable convolution], [3×3 convolution, 3×3 convolution], [5×5 separable convolution, 3×3 convolution]]. The shape of this two-dimensional list is 9×2, indicating that there are 9 convolution modules and each convolution module has two edges. The list shows the operation finally selected for each edge.
[0062] Compared with the prior art, the present invention searches for diffusion model architectures through a neural network architecture, which can alleviate the problem that diffusion model architectures are difficult to design manually. In addition, the feature map selection strategy proposed in the present invention greatly reduces the resources and energy required for the search. The performance of the method of the present invention is tested on the public dataset CIFAR-10, which contains 10 categories and 60,000 pictures, of which 50,000 are used for training and 10,000 are used for testing. The size of pictures of all categories is 32×32 and the number is equal. The present invention can search for a network architecture that meets the task requirements in only 20 hours on a GPU 2080ti, and the Fréchet index distance of the final network architecture is 9.28, and the image quality score is 9.05.
[0063] The present invention provides a diffusion model architecture search method and system based on an attention mechanism. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.
Claims
1. A diffusion model architecture search method based on attention mechanism, characterized in that: The method takes an image as input and comprises the following steps: Step 1, construct a diffusion model backbone architecture supernet; Step 2: Add an attention mechanism to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node; Step 3: Select a feature map from each input node to participate in the forward propagation according to the attention weight calculated in step 2; Step 4: Add an attention mechanism after the candidate operation of each edge of the diffusion model backbone architecture supernet to calculate the attention weights of the output feature maps of all candidate operations in each edge; Step 5, training the diffusion model backbone architecture supernet until the objective function reaches convergence; Step 6: Accumulate the attention weights calculated in step 4 to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and construct the optimal subnet based on the operation; In step 1, the diffusion model backbone architecture supernet includes 1 fully connected layer, 4 downsampling layers, 4 upsampling layers and 9 convolution modules; Each downsampling layer contains one global maximum pooling operation; each upsampling layer contains one transposed convolution operation; each convolution module contains three nodes and two edges, where the three nodes represent three groups of feature maps, and the two edges represent two groups of convolution operations, each group of convolution operations contains two ordinary convolution operations and three depth-separable convolution operations; The fully connected layer, down-sampling layer and up-sampling layer are fixed layers and do not participate in the search. The convolution module is the search layer. The search process is to select an optimal operation from each edge of the convolution module as the operation of the final optimal subnet. Step 2 includes: each convolutional module contains 3 nodes, the first 2 nodes are regarded as input nodes, there are 9 convolutional modules in total, so a total of 18 attention mechanisms need to be added to the supernet; The attention mechanism is added to the input node of each convolution module by the following method: first, a global compression operation is performed on the feature map in the input node to obtain the global information of each feature map. The compression operation includes global average pooling and global maximum pooling. The compression operation calculates the average value and maximum value of each feature map respectively, compresses the spatial dimension of the feature map to 1, and thus obtains a vector containing only channel information; Then, a fully connected layer is used to transform the vector to learn the attention weight of each channel, and a nonlinear activation function ReLU is used to increase the nonlinear ability of the fully connected layer, and a Sigmoid function is used to normalize the weight to between [0,1]; Finally, the learned weights are multiplied by each channel of the original feature map to achieve channel-level weighting, so that each feature map adjusts its contribution according to the attention weight; Step 3 includes: first, sorting the attention weights calculated in step 2 from large to small, and selecting the attention weights with larger values in the first 1 / 4; then, looping through the weighted feature graphs of the input nodes, if the attention weight corresponding to the weighted feature graph belongs to the selected first 1 / 4 attention weights, then marking the weighted feature graph as 1, otherwise marking it as 0; finally, selecting the weighted feature graph marked as 1, and splicing it as the input feature graph of each edge in the convolution module; Step 4 includes: Step 4.1, input the weighted feature map selected in step 3 into the candidate operation in each edge of the convolution module; Step 4.2, concatenate the output feature maps of the candidate operations in each edge and input them into the attention mechanism to calculate the attention weight; Step 4.3, adding feature maps to which the attention mechanism is applied; In step 4.1, the weighted feature map selected in step 3 is simultaneously input to the five candidate operations in each edge of the convolution module, expressed as: F i =O i (F select ) (1) Among them, F select represents the weighted feature map selected in step 3, O i represents the candidate operation in the i-th edge of the convolutional module, F i represents the output feature map of the i-th candidate operation, i=1,…,5; After receiving the input feature map, the five candidate operations will output five sets of corresponding feature maps, and then judge the importance of the five candidate operations based on the importance of the five sets of corresponding feature maps output; Step 4.2 includes: concatenating the five sets of corresponding feature maps, and then inputting the concatenated feature maps into the attention mechanism to calculate the attention weight of each feature map, expressed as: Atten=σ(F2(Relu(F1(F con )))) (2) Among them, Atten represents the calculated attention weight, σ represents the Sigmoid function, F2 and F1 represent the fully connected layer, Relu represents the activation function, and F con A feature map representing the concatenation; Finally, the concatenated feature map is multiplied by the attention weight to obtain a weighted feature map, so as to highlight important features and suppress unimportant features. Step 4.3 includes: first, separating the weighted feature maps corresponding to different operations to obtain 5 groups of weighted feature maps; then, looping through the 5 groups of weighted feature maps, taking out one feature map from each group each time and adding them together, and finally adding the 5 groups of weighted feature maps into one group of weighted feature maps; Step 5 includes: after completing step 1, step 2 and step 4, a diffusion model backbone architecture supernet is constructed, and the attention mechanism has been added to the supernet, and the supernet is trained: first, the cross entropy loss function is selected as the objective function for training the diffusion model; then, the hyperparameters in the training process are set; finally, the diffusion model is trained until the objective function reaches convergence; Step 6 includes: separating the attention weights corresponding to the output feature maps of different operations; accumulating the separated attention weights separately, and the accumulated value at this time represents the importance of each operation; selecting the operation with the largest accumulated value from each edge of the convolution module to construct the optimal subnet.
2. The diffusion model architecture search system based on the attention mechanism implemented by the method according to claim 1 is characterized in that: include: A building block for building a diffusion model backbone architecture supernet; Add an attention mechanism to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node; Select feature maps from each input node to participate in forward propagation according to the calculated attention weights; An attention mechanism is added after the candidate operation of each edge of the diffusion model backbone architecture supernet to calculate the attention weights of the output feature maps of all candidate operations in each edge; The training module is used to train the diffusion model backbone architecture supernet until the objective function reaches convergence; The optimization module is used to accumulate the calculated attention weights to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and to construct the optimal subnet based on the operation.