Attention mechanism-based diffusion model architecture search method and system
By introducing attention mechanisms into the diffusion model, calculating the attention weights of feature maps and candidate operations, and selecting important features and advantages and disadvantages, the problem of diffusion model being difficult to manually design and train time-consuming, and improving model performance and search stability , achieving more efficient feature extraction and training.
Patent Information
- Application Number
- CN202411415408.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-10-11
AI Technical Summary
Existing diffusion models are difficult to manually design network architectures, and the training process is time-consuming and the search for gradient descent neural architectures may lead to suboptimal networks, and the candidate operation evaluation indicators do not match actual performance.
Using the diffusion model architecture search method based on attention mechanism, by constructing the diffusion model backbone architecture supernet, adding the attention mechanism to calculate the attention weight of the feature map and candidate operations, select important feature maps and candidate operations for advantages and disadvantages, and construct the optimal sub network, reduce the amount of calculation in the search process and improve stability.
It effectively alleviates the problem of difficult to manually design and train diffusion models, improves model performance and search stability, avoids the problem of mismatch between candidate operational importance indicators and actual performance, and achieves more efficient feature extraction and model training.
Smart Images

Figure CN118917389A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep neural network technology, and in particular to a diffusion model architecture search method and system based on an attention mechanism. Background Art
[0002] The diffusion model, proposed in 2015, has had a profound impact on deep learning and generative model research due to its innovative concept. Diffusion models simulate the forward diffusion process of data from its original distribution to a noisy distribution and learn the inverse of this process, gradually denoising the noise to reconstruct high-quality data samples. Diffusion models offer several advantages, making them a valuable tool in the field of generative modeling in deep learning. First, they are not restricted to a single neighborhood and can be used to generate multimodal data, such as from text to images and vice versa. Second, diffusion models are structurally flexible, adapting to a variety of complex datasets and scenarios. Furthermore, their training process is relatively stable and easy to tune and optimize. This allows researchers to adjust model parameters to achieve optimal performance based on specific needs. Finally, diffusion models implement precise mathematical operations, enabling researchers to conduct in-depth mathematical analysis and theoretical research based on these models. However, diffusion models still face several challenges. For example, the traditional diffusion model uses a manually designed U-Net network. The performance of the U-Net network depends heavily on the design of the network architecture. A poorly designed network architecture can lead to overfitting. Furthermore, the U-Net network architecture is complex and time-consuming to train. Therefore, addressing these issues to improve the performance of diffusion models is a highly significant research direction.
[0003] In response to the problems that existing diffusion models are difficult to manually design and time-consuming to train, researchers have proposed using neural network architecture search technology to automatically design diffusion model network architectures. Neural architecture search is proposed to automatically discover high-performance deep neural network architectures. The three most widely used categories for neural network architecture search are evolutionary algorithms, reinforcement learning, and gradient descent algorithms. Gradient descent neural architecture search has become a topic of extensive research because its implementation does not require a lot of prior knowledge about the problem and can be used directly. However, gradient descent neural architecture search may involve the problem of mismatch between the size of the candidate operation evaluation indicator and the actual performance of the candidate operation, resulting in the searched network being suboptimal.
[0004] The attention mechanism focuses on modeling the importance of each feature map, thereby more effectively utilizing information during feature extraction. By dynamically adjusting the weights of different channels, it can highlight important features and suppress unimportant ones, thereby improving the representational power of features. Therefore, to address the difficulties of manual design and time-consuming model training in existing diffusion models, neural network architecture search techniques can be used to search for diffusion model architectures. The attention mechanism can then guide the search process to select sub-networks with fewer parameters and better performance. Therefore, it is necessary to propose a diffusion model architecture search method based on the attention mechanism. Summary of the Invention
[0005] Purpose of the invention: The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and provide a diffusion model architecture search method and system based on the attention mechanism. The method comprises the following steps: Step 1: Build a diffusion model backbone architecture supernet; Step 2: Add an attention mechanism to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node; Step 3: Select a feature map from each input node to participate in the forward propagation according to the attention weight calculated in step 2; Step 4: Add an attention mechanism after each candidate operation on each edge of the diffusion model backbone architecture supernet to calculate the attention weights of the output feature maps of all candidate operations on each edge; Step 5: Train the diffusion model backbone architecture supernet until the objective function reaches convergence; In step 6, the attention weights calculated in step 4 are accumulated to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and the optimal subnet is constructed based on the operation.
[0006] In step 1, the diffusion model backbone architecture supernet includes 1 fully connected layer, 4 downsampling layers, 4 upsampling layers and 9 convolution modules; Each downsampling layer contains one global maximum pooling operation; each upsampling layer contains one transposed convolution operation; each convolution module contains three nodes and two edges, where the three nodes represent three groups of feature maps, and the two edges represent two groups of convolution operations, each group of convolution operations contains two ordinary convolution operations and three depth-separable convolution operations; The fully connected layer, downsampling layer and upsampling layer are fixed layers and do not participate in the search. The convolution module is the search layer. The search process selects an optimal operation from each edge of the convolution module as the operation of the final optimal subnet.
[0007] Among them, the edges connecting different intermediate nodes perform certain operations, process the data of the input nodes, and finally the output nodes output the results, including: 3*3 convolution, 5*5 convolution, 3*3 separable convolution, 5*5 separable convolution and 7*7 separable convolution operations, to achieve effective feature extraction, information fusion and resolution adjustment, so as to improve the performance and performance of the model.
[0008] In order to more clearly express the network architecture of the diffusion model, the present invention uses a 2*5*9 matrix to express a diffusion model architecture, 2 represents 2 edges in each convolution module, 5 represents 5 candidate operations in each edge, and 9 represents that the diffusion model supernet architecture has 9 convolution modules.
[0009] In step 2, the traditional gradient-based diffusion model architecture search retains all feature maps from the input node during the search process and uses them as input for each candidate operation. While this approach retains all features of the input image, it incurs significant computational overhead, resulting in excessively high search costs. Furthermore, not all feature maps from a node contain detailed features; instead, more than half contain only a single feature or no features at all. These feature maps can affect the diffusion model's ability to generate detailed image features. Therefore, these feature maps can be removed during the forward propagation process, making model training more stable and efficient.
[0010] The present invention uses the attention mechanism as a tool for feature map selection. The attention mechanism is a mechanism widely used in deep learning, especially in computer vision tasks. Its core idea is to enable the model to automatically learn and pay attention to the importance of different feature maps in the input data, so as to more effectively use this information for feature extraction and decision-making. The attention mechanism adaptively adjusts the weights of different feature maps by modeling the importance of each feature map. In a convolutional neural network, each feature map represents a specific feature, such as edges, textures, colors, etc. The attention mechanism enhances or suppresses the expression of these features by learning the importance of each feature map for the current task, thereby improving the performance of the model. Therefore, it is used as a tool for feature map selection.
[0011] Step 2 includes: each convolutional module contains 3 nodes, the first 2 nodes are considered as input nodes, there are 9 convolutional modules in total, so a total of 18 attention mechanisms need to be added to the supernet; The attention mechanism is added to the input node of each convolutional module using the following method: First, a global compression operation is performed on the feature map in the input node to obtain the global information of each feature map. The compression operation includes global average pooling and global maximum pooling. The compression operation calculates the average and maximum value of each feature map respectively, compressing the spatial dimension of the feature map to 1, thereby obtaining a vector containing only channel information; Then, a fully connected layer is used to transform the vector to learn the attention weight of each channel, and a nonlinear activation function ReLU is used to increase the nonlinear ability of the fully connected layer, and the Sigmoid function is used to normalize the weight to between [0, 1]; Finally, the learned weights are multiplied with each channel of the original feature map to achieve channel-level weighting, so that each feature map adjusts its contribution according to the attention weight, thereby enhancing important features and suppressing unimportant features.
[0012] Step 3 includes: in order to improve the stability of model training and reduce the amount of calculation in the search process, first, sort the attention weights calculated in step 2 from large to small, and select the attention weights with larger values in the first 1 / 4; then, loop through the weighted feature graph of the input node, if the attention weight corresponding to the weighted feature graph belongs to the selected first 1 / 4 attention weights, then mark the weighted feature graph as 1, otherwise mark it as 0; finally, select the weighted feature graph marked as 1 and splice it as the input feature graph of each edge in the convolution module.
[0013] Step 4 includes: Step 4.1: Input the weighted feature map selected in step 3 into the candidate operation of each edge of the convolution module; Step 4.2: Concatenate the output feature maps of the candidate operations in each edge and input them into the attention mechanism to calculate the attention weight. In step 4.3, the feature maps to which the attention mechanism is applied are added.
[0014] In step 4.1, because we need to select the optimal subnet from the supernet, we need to verify the quality of each candidate operation in each edge of the convolution module. The weighted feature map selected in step 3 is simultaneously input to the five candidate operations in each edge of the convolution module, expressed as: (1), in, represents the weighted feature map selected in step 3, represents the candidate operation in the i-th edge of the convolution module, represents the output feature map of the i-th candidate operation, ; After receiving the input feature map, the five candidate operations will output five sets of corresponding feature maps, and then judge the importance of the five candidate operations based on the importance of the five sets of corresponding feature maps output.
[0015] Step 4.2 includes: concatenating the five corresponding feature maps, and then inputting the concatenated feature maps into the attention mechanism to calculate the attention weight of each feature map, expressed as: (2), in, represents the calculated attention weight, represents the Sigmoid function, and represents the fully connected layer, represents the activation function, Represents the concatenated feature map; Finally, the concatenated feature map is multiplied by the attention weight to obtain a weighted feature map, so as to highlight important features and suppress unimportant features.
[0016] Step 4.3 includes: the number of weighted feature maps obtained in step 4.2 is the sum of the number of output feature maps of the five candidate operations, so the number of weighted feature maps is large, resulting in too high a computational cost in the search process; in order to reduce the computational cost, first, the weighted feature maps corresponding to different operations are separated to obtain five groups of weighted feature maps; then, the five groups of weighted feature maps are looped through, and one feature map is taken out from each group each time and added together, and finally the five groups of weighted feature maps are added together into one group of weighted feature maps.
[0017] Step 5 includes: After completing steps 1, 2, and 4, a diffusion model backbone architecture supernet is constructed, and the attention mechanism has been added to the supernet. The supernet is trained: first, the cross entropy loss function is selected as the objective function for training the diffusion model; then, the hyperparameters of the training process are set, including the learning rate to 0.0001, the optimizer to the Adam optimizer, the number of training cycles to 100, and the batch size to 64; finally, the diffusion model is trained until the objective function converges; Step 6 includes: After completing the training of the diffusion model in step 5, the attention weights corresponding to the output feature maps of the candidate operations are obtained in step 4. At this time, the attention weights need to be accumulated to calculate the importance of each operation. The attention weights corresponding to the output feature maps of different operations are separated; the separated attention weights are accumulated separately, and the accumulated value at this time represents the importance of each operation; the operation with the largest accumulated value is selected from each edge of the convolution module to construct the optimal subnet.
[0018] The present invention also provides a diffusion model architecture search system based on an attention mechanism, comprising: A construction module is used to construct a diffusion model backbone architecture supernet; add an attention mechanism to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node; select a feature map from each input node to participate in the forward propagation based on the calculated attention weight; add an attention mechanism after the candidate operation of each edge of the diffusion model backbone architecture supernet to calculate the attention weight of the output feature map of all candidate operations in each edge; The training module is used to train the diffusion model backbone architecture supernet until the objective function reaches convergence; The optimization module is used to accumulate the calculated attention weights to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and to construct the optimal subnet based on the operation.
[0019] This paper introduces a neural network architecture search method to effectively alleviate the difficulty of manually designing diffusion models. To search for more stable diffusion model architectures from multiple perspectives, the present invention incorporates an attention mechanism to design a gradient-based search algorithm. Furthermore, to reduce the time consumption of the search process, the present invention designs a feature map selection strategy based on the attention mechanism to reduce the computational complexity of the search process. This approach achieves superior results compared to existing methods.
[0020] The beneficial effects of the present invention are: The method of the present invention proposes a neural network architecture search method based on an attention mechanism for diffusion models. This method searches for the optimal subnet from a supernet through gradient descent. In this method, we propose a feature map selection strategy based on the attention mechanism, thereby avoiding the complex calculation process in the search and improving the stability of the search. At the same time, we also propose a method for calculating the importance of candidate operations through the attention mechanism, which solves the problem of mismatch between the candidate operation importance index and the actual performance of the candidate operation in traditional gradient-based neural network architecture search methods. This diffusion model architecture search method based on the attention mechanism brings new methods and ideas to the field of diffusion model architecture search, with high practical value and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 This is the architecture diagram of the supernet convolution module of the diffusion model in the present invention.
[0022] Figure 2 This is the basic framework diagram of the diffusion model supernet in the present invention.
[0023] Figure 3 This is the structural diagram of the attention mechanism in the present invention.
[0024] Figure 4 It is the calculation flow chart of each edge of the convolution module.
[0025] Figure 5 It is a schematic diagram of feature map addition. DETAILED DESCRIPTION
[0026] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.
[0027] In a specific embodiment of the present invention, a diffusion model architecture search method based on an attention mechanism is provided, comprising the following steps: Step 1: If Figure 1 As shown, the diffusion model architecture search space proposed in the present invention is Figure 1 The internal structure of a convolutional module in the diffusion model supernet is described, which consists of 3 groups of feature maps and 2 groups of candidate operations. The first two groups of feature maps are input nodes, and the candidate operations include 5 operations, namely 3×3 convolution, 5×5 convolution, 3×3 depth-wise separable convolution, 5×5 depth-wise separable convolution and 7×7 depth-wise separable convolution. Figure 2 This is the basic framework diagram of the diffusion model supernet, which is composed of 9 convolution modules stacked by downsampling and upsampling. The green arrow represents the downsampling operation, which is composed of pooling operations, and the red arrow represents the upsampling operation, which is composed of transposed convolution operations. Figure 2 The white arrows in the middle represent skip connections, which fuse the features of the first half of the network with those of the second half, effectively solving the problem of feature loss. The entire search process involves selecting the optimal operation from the candidate operations in the nine convolutional modules as the final operation.
[0028] Step 2: After the diffusion model supernet is built, an attention mechanism needs to be added after the input node of the supernet convolution module to select the feature map. Figure 3 The following is the structural diagram of the attention mechanism. Its input is a set of feature maps, which correspond to the input nodes of the convolutional module in the hypernet. First, the input feature maps are subjected to global average pooling and global max pooling, respectively. These two methods compress the feature maps. Global average pooling averages all features in a feature map, using the average value to represent the characteristics of the entire feature map. Global max pooling takes the maximum value of all features in a feature map, using the maximum value to represent the characteristics of the entire feature map. These two methods of feature map compression are used to calculate the importance of each feature map from multiple dimensions. After feature compression, two vectors are generated. These two vectors are then fed into a fully connected layer for feature extraction. A fully connected layer transforms the compressed vectors to learn the attention weights for each channel. A nonlinear activation function, ReLU, is used to increase the nonlinearity of the fully connected layer. The output of the fully connected layer is also two vectors, denoted as vector A and vector B. Vector A corresponds to the attention weights for each feature map calculated by average pooling, and vector B corresponds to the attention weights for each feature map calculated by max pooling. The two attention weights are then summed to obtain the fused attention weights. Finally, the final attention weight is normalized by the Sigmoid function, and the weight value is normalized to the range of [0,1]. Figure 4 As shown, the green part is the specific process of step 2. Represents an input node in the convolution module, which is first input into the attention mechanism to obtain the attention weight , then and Multiply to get the weighted feature map .
[0029] Step 3: After obtaining the weighted feature map, we need to select the weighted feature map. First, we select Select the larger attention weight of the first 1 / 4 and then loop through If the traversed attention weight belongs to the larger 1 / 4 weight, then record The position of the corresponding weighted feature map in After that, from The positions of the weighted feature maps with the largest attention weight values in the first 1 / 4 are found. These feature maps are more important than other feature maps, so Select these feature maps to form a new feature map .
[0030] Step 4: Add an attention mechanism after the candidate operation of each edge in the supernet to calculate the attention weights of the output feature maps of all candidate operations in each edge. The specific process includes the following steps: Step 4.1: After step 3, select Afterwards, you need to As the input of each candidate operation. Figure 4 As shown, Input into 3×3 convolution, 5×5 convolution, 3×3 separable convolution, 5×5 separable convolution and 7×7 separable convolution respectively to obtain 5 sets of feature maps .
[0031] Step 4.2: After obtaining 5 sets of feature maps through step 4.1, these 5 sets of feature maps are concatenated together as the input of the attention mechanism. Figure 4 As shown, the five groups of feature maps are spliced together to obtain the spliced feature map , then Input to the attention mechanism to get the attention weight , then and Multiply to get the weighted feature map .
[0032] Step 4.3: After step 4.2, Afterwards, you need to Perform feature fusion, that is, fuse the features extracted by different operations together. Figure 5 The figure shows the addition of feature maps. Figure 5Suppose there are 3 groups of feature maps, each group of feature maps contains 2 feature maps. First separate the 3 groups of feature maps, and then separate the feature maps in the same group, such as Figure 5 Each set of feature maps is separated into two feature maps, and finally the feature maps of different groups are added together. Because each set of feature maps contains two feature maps, the feature map obtained by the final addition also contains two feature maps. Figure 4 middle, After adding the feature maps, a set of feature maps is obtained.
[0033] Step 5: Once the diffusion model supernet is constructed and the attention mechanism has been added to the corresponding positions in the supernet, you can begin training the supernet. First, select the cross-entropy loss function as the objective function for training the diffusion model and construct the training and validation datasets. Then, set the training hyperparameters: the learning rate is set to 0.0001, the optimizer is the Adam optimizer, the number of training cycles is set to 100, and the batch size is set to 64. Finally, train the diffusion model until the objective function converges.
[0034] In step 6, after completing step 5, step 4 will also obtain the attention weight corresponding to the candidate operation output feature map, such as Figure 4 As shown, That is, the attention weight corresponding to the output feature map of each edge of the candidate operation when the super network converges. At this time, The importance of each operation is calculated by accumulation. First, the attention weights corresponding to the output feature maps of different operations are separated. Then, the separated attention weights are accumulated separately, and the accumulated value represents the importance of each operation. Finally, the operation with the largest accumulated value is selected from each edge of the supernet convolution module to construct the optimal subnet. The final network architecture obtained by this method is: [[5×5 convolution, 3×3 separable convolution], [3×3 convolution, 5×5 separable convolution], [5×5 separable convolution, 3×3 separable convolution], [3×3 convolution, 7×7 separable convolution], [3×3 separable convolution, 3×3 separable convolution], [5×5 convolution, 7×7 separable convolution], [5×5 convolution, 7×7 separable convolution], [3×3 convolution, 3×3 convolution], [5×5 separable convolution, 3×3 convolution]]. The shape of this two-dimensional list is 9×2, indicating that there are 9 convolution modules and each convolution module has two edges. The list shows the operation finally selected for each edge.
[0035] Compared with the existing technology, the present invention searches for diffusion model architectures through a neural network architecture, which can alleviate the problem that diffusion model architectures are difficult to design manually. In addition, the feature map selection strategy proposed in the present invention greatly reduces the resources and energy required for the search. The performance of the method of the present invention was tested on the public dataset CIFAR-10, which contains 10 categories and 60,000 images, of which 50,000 are used for training and 10,000 are used for testing. The size of images of all categories is 32×32 and the number is equal. The present invention can search for a network architecture that meets the task requirements in just 20 hours on a GPU 2080ti, and the Fréchet index distance of the final network architecture is 9.28, and the image quality score is 9.05.
[0036] The present invention provides a method and system for searching for diffusion model architectures based on an attention mechanism. There are many methods and approaches for implementing this technical solution. The above is only a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.
Claims
1. A diffusion model architecture search method based on attention mechanism, It is characterized in that The following steps are involved: Step 1, construct a diffusion model backbone architecture supernet; Step 2: Add an attention mechanism to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node; Step 3: Select a feature map from each input node to participate in the forward propagation according to the attention weight calculated in step 2; Step 4: Add an attention mechanism after the candidate operation of each edge of the diffusion model backbone architecture supernet to calculate the attention weights of the output feature maps of all candidate operations in each edge; Step 5, training the diffusion model backbone architecture supernet until the objective function reaches convergence; Step 6: Accumulate the attention weights calculated in step 4 to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and construct the optimal subnet based on the operation.
2. The method according to claim 1, It is characterized in that In step 1, the diffusion model backbone architecture supernet includes 1 fully connected layer, 4 downsampling layers, 4 upsampling layers and 9 convolution modules; Each downsampling layer contains one global maximum pooling operation; each upsampling layer contains one transposed convolution operation; each convolution module contains three nodes and two edges, where the three nodes represent three groups of feature maps, and the two edges represent two groups of convolution operations, each group of convolution operations contains two ordinary convolution operations and three depth-separable convolution operations; The fully connected layer, down-sampling layer and up-sampling layer are fixed layers and do not participate in the search. The convolution module is the search layer. The search process selects an optimal operation from each edge of the convolution module as the operation of the final optimal subnet.
3. The method according to claim 2, It is characterized in that Step 2 includes: each convolutional module contains 3 nodes, the first 2 nodes are regarded as input nodes, there are 9 convolutional modules in total, so a total of 18 attention mechanisms need to be added to the supernet; The attention mechanism is added to the input node of each convolution module by the following method: first, a global compression operation is performed on the feature map in the input node to obtain the global information of each feature map. The compression operation includes global average pooling and global maximum pooling. The compression operation calculates the average value and maximum value of each feature map respectively, compresses the spatial dimension of the feature map to 1, and thus obtains a vector containing only channel information; Then, a fully connected layer is used to transform the vector to learn the attention weight of each channel, and a nonlinear activation function ReLU is used to increase the nonlinear ability of the fully connected layer, and the Sigmoid function is used to normalize the weight to [0,1]; Finally, the learned weights are multiplied by each channel of the original feature map to achieve channel-level weighting, so that each feature map adjusts its contribution according to the attention weight.
4. The method according to claim 3, It is characterized in that Step 3 includes: first, sorting the attention weights calculated in step 2 from large to small, and selecting the attention weights with larger values in the first 1 / 4; then, looping through the weighted feature graph of the input node, if the attention weight corresponding to the weighted feature graph belongs to the selected first 1 / 4 attention weights, then marking the weighted feature graph as 1, otherwise marking it as 0; finally, selecting the weighted feature graph marked as 1, and splicing it as the input feature graph of each edge in the convolution module.
5. The method according to claim 4, It is characterized in that Step 4 includes: Step 4.1, input the weighted feature map selected in step 3 into the candidate operation in each edge of the convolution module; Step 4.2, concatenate the output feature maps of the candidate operations in each edge and input them into the attention mechanism to calculate the attention weight; In step 4.3, the feature maps to which the attention mechanism is applied are added.
6. The method according to claim 5, It is characterized in that In step 4.1, the weighted feature map selected in step 3 is simultaneously input to the five candidate operations in each edge of the convolution module, expressed as: (1), in, represents the weighted feature map selected in step 3, represents the candidate operation in the i-th edge of the convolutional module, represents the output feature map of the i-th candidate operation, ; After receiving the input feature map, the five candidate operations will output five sets of corresponding feature maps, and then judge the importance of the five candidate operations based on the importance of the five sets of corresponding feature maps output.
7. The method according to claim 6, It is characterized in that Step 4.2 includes: concatenating the five sets of corresponding feature maps, and then inputting the concatenated feature maps into the attention mechanism to calculate the attention weight of each feature map, expressed as: (2), in, represents the calculated attention weight, represents the Sigmoid function, and represents the fully connected layer, represents the activation function, A feature map representing the concatenation; Finally, the concatenated feature map is multiplied by the attention weight to obtain a weighted feature map, so as to highlight important features and suppress unimportant features.
8. The method according to claim 7, It is characterized in that Step 4.3 includes: first, separating the weighted feature maps corresponding to different operations to obtain 5 groups of weighted feature maps; then, looping through the 5 groups of weighted feature maps, taking out 1 feature map from each group each time and adding them together, and finally adding the 5 groups of weighted feature maps into 1 group of weighted feature maps.
9. The method according to claim 8, It is characterized in that Step 5 includes: after completing step 1, step 2 and step 4, a diffusion model backbone architecture supernet is constructed, and the attention mechanism has been added to the supernet, and the supernet is trained: first, the cross entropy loss function is selected as the objective function for training the diffusion model; then, the hyperparameters in the training process are set; finally, the diffusion model is trained until the objective function reaches convergence; Step 6 includes: separating the attention weights corresponding to the output feature maps of different operations; accumulating the separated attention weights separately, and the accumulated value at this time represents the importance of each operation; selecting the operation with the largest accumulated value from each edge of the convolution module to construct the optimal subnet.
10. A diffusion model architecture search system based on an attention mechanism implemented by the method according to any one of claims 1 to 9, It is characterized in that include: A building block for building a diffusion model backbone architecture supernet; Add an attention mechanism to the diffusion model backbone architecture supernet to calculate the attention weight of each feature map in the input node; Select feature maps from each input node to participate in forward propagation according to the calculated attention weights; An attention mechanism is added after the candidate operation of each edge of the diffusion model backbone architecture supernet to calculate the attention weights of the output feature maps of all candidate operations in each edge; The training module is used to train the diffusion model backbone architecture supernet until the objective function reaches convergence; The optimization module is used to accumulate the calculated attention weights to obtain the operation with the largest attention weight corresponding to each edge when the supernet of the diffusion model backbone architecture converges, and to construct the optimal subnet based on the operation.
Citation Information
Patent Citations
Lightweight self-attention module, neural network model and search method of neural network framework
CN111539524A
Benign and malignant lung nodule discrimination method based on neural network architecture search and attention mechanism
CN112053351A
Rapid attention neural network architecture search method based on evolutionary method
CN112465120A
Neural network unit structure searching method and system based on attention mechanism
CN114626506A
Neural network macro architecture searching method and system based on attention mechanism
CN114742199A
Cited By
Complex neural network miniaturized high-voltage transmission line monitoring method and system
CN119598342A
Neural architecture searching method based on diffusion evolutionary algorithm
CN119886226A
