Bionic model for photovoltaic cell defect detection based on mimic vision

Abstract

The present invention relates to the field of photovoltaic cell technology, namely, a bionic model for photovoltaic cell defect detection based on mimetic vision. It comprises three parts: (1) backbone network: the backbone network is used to extract effective information from the image and map the original data into the required space. The features extracted by the backbone network are passed to the neck network; (2) neck network: the neck network is used to fuse features of different scales and levels and enrich the expression capability of the features. The features fused by the neck network are passed to the detection head; (3) detection head: the detection head is mainly responsible for predicting the features, generating bounding boxes and predicting categories. The backbone network based on human receptive field and visual mechanism has a stronger abstraction capability for features and can efficiently extract effective features from photovoltaic cell electroluminescent images with single color and complex texture. The proposed neck network can effectively enhance the expression capability of defect characteristics in photovoltaic cell electroluminescent images.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic cells, namely, a bionic model for photovoltaic cell defect detection based on mimicry vision. Background Art

[0002] In existing technologies, photovoltaic cell quality issues are prone to occur during production, transportation, and operation. During the production phase, photovoltaic modules can be damaged by manufacturing faults such as improper welding or defective wiring. These external stresses can cause defects such as cracks in the modules, affecting power output. Cell cracking is one of the most common causes of power loss in photovoltaic modules. Therefore, photovoltaic modules must be closely monitored throughout the entire manufacturing process. An intelligent photovoltaic cell defect detection method is urgently needed to improve the stability of photovoltaic cell quality and meet the needs of clean energy development.

[0003] Photovoltaic cell defect images have complex textures, single colors, blurred defect boundaries, and unclear defect features. Traditional target detection models can easily miss or misdetect defects, causing significant losses.

[0004] In recent years, methods based on electroluminescence (EL) imaging have become the main method for photovoltaic cell defect detection due to their superior performance. EL imaging is a mature non-destructive and non-contact defect detection technology with remarkably high resolution, which can detect defects such as microcracks that are difficult to identify with the naked eye. However, visual evaluation of EL images is not only time-consuming and expensive, but also requires skilled experts. Manual inspection methods waste a lot of human resources, and the inspection structure is also affected by human factors. In addition, visual inspection is only feasible on a small scale. For large-scale inspection, automated inspection methods are crucial.

[0005] In photovoltaic systems, defect detection technologies based on computer vision and artificial intelligence algorithms are gaining increasing attention. Deep learning-based object detection models, in particular, offer unlimited possibilities due to their advantages in eliminating the need for manually designed feature extraction and their flexible and adaptable architecture. These models effectively address the challenges of photovoltaic cell defect detection, such as small, unclear features, monotonous colors, and high background noise, meeting the demands of industrial applications. However, the numerous impurities and wrinkles on photovoltaic cell surfaces, known as pseudo-defects, are often difficult to identify. These artifacts, which appear diffusely across cluttered industrial surface images, resemble real defects in appearance, making them difficult for conventional models to distinguish.

[0006] In recent years, methods based on convolutional neural networks have received increasing attention in defect detection tasks. Generally speaking, most convolutional neural network (CNN)-based models attempt to better distinguish salient objects by interleaving multiple two-dimensional convolution-like structures to obtain high-level semantic knowledge. However, the effectiveness of these methods is limited because the empirical receptive field in CNN is smaller than the theoretical one. Therefore, many techniques focus on using large kernels, no convolution, attention mechanisms, etc. to model more global context dependencies in CNN. Although advantageous, the inherent locality of two-dimensional convolution still limits access to the global context of all pixels in the entire image. Typical CNN-based methods achieve better results than traditional methods, but still produce misjudgments in some images with extremely complex backgrounds. Therefore, performance needs to be further improved.

[0007] Recently, with the remarkable progress in natural language processing (NLP), Visual Transformers (ViT) have demonstrated for the first time that self-attention alone can surpass existing image classification tasks when pre-trained on large-scale training data. This is based on the superior ability of self-attention in explicitly modeling long-term contextual interactions and learning abstract semantic information. However, these convolution-free transformers view images as a series of visual patches (tokens) and lack the inductive bias of images when modeling local structures such as edges and corners. Therefore, large-scale datasets and longer training times are required to implicitly learn this inductive bias, which is contrary to the limited sample industrial imaging applications.

[0008] Existing research has mostly focused on single-target classification or detection. Compared to detection, defect image classification lacks crucial information such as defect location and size. Furthermore, in practical applications, it is often necessary to detect multiple common defect types, which greatly challenges model performance. Furthermore, defect scale varies significantly. Many defect types, such as broken fingers and cracks, are relatively small. Models based solely on convolution often perform poorly on multiple small object detection, and the models as a whole lack generalization and robustness.

[0009] Bioinspiration has a profound and huge impact on deep learning. Whether it is a neural network based on biological nerves or a spiking neural network based on synapses, it has undoubtedly proved that bioinspiration can bring new vitality and research directions to artificial intelligence. Summary of the Invention

[0010] The present invention addresses these shortcomings by providing a biomimetic model for photovoltaic cell defect detection based on mimetic vision. This model, inspired by biology, offers enhanced biological interpretability. This model, inspired by biological principles, combines biological characteristics with the receptive field of human peripheral vision and enhanced feature fusion capabilities, offering significant performance advantages.

[0011] The technical solution of the present invention is: a bionic model for photovoltaic cell defect detection based on mimic vision, which is characterized by comprising three parts:

[0012] (1) Backbone network: The backbone network is used to extract effective information from the image and map the original data into the required space. The features extracted by the backbone network will be passed to the neck network.

[0013] (2) Neck network: The neck network is used to fuse features of different scales and levels and enrich the expressive power of features. The features fused by the neck network will be passed to the detection head.

[0014] (3) Detection head: The detection head is mainly responsible for predicting features, generating bounding boxes and predicting categories.

[0015] The processing flow of the model is:

[0016] First, the image of the photovoltaic cell to be inspected is input into the backbone network to extract the defect features in the image.

[0017] The different-scale features extracted by the backbone network are then fed into the model's neck network. In the bottom-up path, the neck network transmits precise positional signals from the underlying features to the top layer via the position information transfer module. In the top-down path, the neck network transmits rich semantic information from the top layer to the bottom layer via the semantic information transfer module.

[0018] Finally, the fused features are adaptively weighted and fused, and then input into the separate detection head. The detection head calculates the above information to obtain the location and category information of the defect.

[0019] The above scheme also includes:

[0020] The backbone network is composed of a stack of bionic feature extraction modules and mimetic visual attention mechanism modules, and is divided into four stages; the bionic feature extraction module is mainly composed of a spatial aggregation feature extraction module SA-Block and a self-calibrated convolution module SC-Conv; the mimetic visual attention mechanism module contains two sub-modules: a deep mining attention module and a mimetic vision module; in each stage, the image is first output to the bionic feature extraction module, where the image is downsampled through convolution with scale factors of [1 / 4, 1 / 8, 1 / 16, 1 / 32] respectively, and the number of channels of the three-channel RGB image is expanded to [64, 128, 256, 512] to increase the information capacity; then, the spatial aggregation feature extraction module SA-Block, which imitates biological vision, is used to extract features; finally, the self-calibrated convolution module SC-Conv is used to aggregate target feature information across channels; in the last two stages, a mimetic visual attention module is added to capture rich dynamic context and imitate the unique peripheral vision mechanism of humans.

[0021] The semantic information transfer module is added between every two layers in the top-down channel of the feature pyramid to fuse feature layers of different scales and levels and transfer effective semantic information of the target. The specific approach is as follows:

[0022]

[0023] The feature layer of the i-th layer is represented as First, global average pooling is applied to adjacent layers, and carafe() is an upsampling operator to obtain the channel vector of each channel. then:

[0024]

[0025] Channel attention is obtained through outer product and normalization operations Each row in M ​​represents the importance distribution of the jth channel on the i+1th layer relative to the i-th layer. This attention map is then applied to the feature map obtained after upsampling on the i+1th layer to obtain the reweighted version Y of the i+1th layer. i+1 .

[0026]

[0027] Finally, by adding Y i+1 Add as residual to On, with A i After splicing, the convolution kernel is 1 and the channel mixing is performed, where λ i is the gating parameter.

[0028] The spatial information transfer module is added between adjacent feature layers in the bottom-up path to pass accurate position signals to the top layer. The specific approach is as follows:

[0029]

[0030] First, the adjacent feature layer A i ,A i-1 To perform convolution operation, in order to control the computational complexity of the module, the convolution adopts a bottleneck structure, so the step size of the convolution kernel is selected as a, and the output dimension is 1, so Transpose and inner product it to get M′, then:

[0031]

[0032] Normalize M′ to get the spatial attention map, and i Similarly, perform convolution with a step size of a to obtain A′ i Operation alignment scale, A i Then perform convolution with a step size of 1 to get A i A downsampled version of .

[0033]

[0034] Then, spatial attention is applied to A′ i After upsampling, the upsampling operator is applied to the scale, and the upsampling result is used as the residual and gated with weight α i Add to A i A downsampled version of i Finally, the output of the previous step is combined with A i-1 After splicing, the channel mixing operation is performed through convolution with a convolution kernel size of 1 to obtain the final output.

[0035] The detection head introduces a separate detection head. Referring to the partitioning mechanism of the human brain, it aims to make the detection head focus on different content through different branches. The three detection heads process the different scale features output by the neck respectively and output the final results. The specific approach is as follows: For the features output by the neck, two convolution mappings are used in the three branches, where the final convolution kernel is 1 to compress the number of channels, which replaces the fully connected layer and compresses the channels to [num, 1, 4]. The branch with n channels outputs categories, where n is the number of categories. The branch with 1 output channel outputs the IoU threshold, and the branch with 4 output channels outputs the position and size [x, y, h, w] of the target detection box. It is expressed as:

[0036]

[0037]

[0038]

[0039]

[0040] The advantages of the present invention are:

[0041] 1. The backbone network based on human receptive field and visual mechanism has a stronger ability to abstract features, which is superior to most mainstream backbone networks. It can efficiently extract effective features from photovoltaic cell electroluminescence images with single color and complex texture.

[0042] 2. The proposed neck network is different from traditional neck networks. Similar to biological organisms, it transmits information in a more logical way. Depending on the different functions of the features being fused and transmitted, it adopts completely different fusion methods, which can effectively enhance the ability to express defect characteristics in electroluminescent images of photovoltaic cells.

[0043] 3. Based on the human brain's partitioning mechanism, a separate detection head is proposed. Similar to the neck, for head detection, this invention selects scale monitoring, scale fusion, and separate detection for multi-scale features. Scale monitoring is responsible for adaptively adjusting the effectiveness of features at three scales for detecting the current input image. Scale fusion integrates features at three scales to ensure information integrity. The separate detection head decouples classification and localization tasks based on the different feature focus of different information streams.

[0044] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a partial EL imaging photovoltaic cell defect dataset.

[0046] Figure 2 is the distribution of defect data of different categories of photovoltaic cells.

[0047] Figure 3 It is a data distribution graph.

[0048] Figure 4 This is a diagram of a mimetic visual bionic model.

[0049] Figure 5 It is a diagram of the spatial aggregation feature extraction module.

[0050] Figure 6 It is the self-calibrated convolution module diagram.

[0051] Figure 7 This is a module diagram of the mimetic visual attention mechanism.

[0052] Figure 8It is a separated feature pyramid diagram.

[0053] Figure 9 is the Semantic Information Passing Module (SIF) graph.

[0054] Figure 10 It is the spatial information passing module (PIF) diagram.

[0055] Figure 11 It is a separate detection head image of adaptive fusion.

[0056] Figure 12 It is a diagram of the adaptive scale monitoring module.

[0057] Figure 13 This is a diagram of a detachable detection head.

[0058] Figure 14 It is the experimental result.

[0059] Figure 15 It is a curve chart showing the changes in loss and various evaluation indicators during the experiment.

[0060] Figure 16 It is a visualization diagram of the mimic vision module.

[0061] Figure 17 is the feature map of the neck network. DETAILED DESCRIPTION

[0062] Principle of electroluminescence imaging

[0063] Faults such as cracks, fingers, and black cores on photovoltaic cells cannot be seen with the naked eye, but can be detected through EL imaging. In electroluminescence (EL) imaging, photovoltaic cells emit radiation due to the recombination of electrons and holes under forward bias conditions. The emitted radiation is in the near-infrared range.

[0064] Equation 1 gives the electron-hole recombination rate per energy for nondegenerate electron and hole concentrations.

[0065]

[0066] In the above formula, g r (x, hω) is the spontaneous recombination rate, α(hω) is the absorption coefficient, h is the reduced Planck constant, k is the Boltzmann constant, k is the electron charge, and V is the voltage. If a crack exists in a photovoltaic cell, the current path will be reduced or blocked depending on the type and severity of the defect and the forward bias conditions at that location. Because the defect region experiences reduced or zero electron-hole recombination, it emits less or no near-infrared radiation. Therefore, in EL images, the defective region emits less radiation and appears darker. Regions with defects are generally darker in color.

[0067] PVEL-AD dataset

[0068] like Figure 1 As shown, the present invention adopts the PVEL-AD photovoltaic cell defect EL imaging dataset. The PVEL-AD dataset has near-infrared images of various internal defects and uneven backgrounds, including 12 different types of abnormal defects, such as cracks (lines, stars), finger interruptions, black cores, dislocations, thick lines, scratches, fragments, bad corners, printing errors, horizontal dislocations, vertical dislocations and short circuit defects. Figure 1 Among them, star crack, line crack, black core, finger interruption, horizontal dislocation, vertical dislocation, short circuit, fragment, thick line, scratch, comer, defect-free. Different types of data distribution are as follows Figure 2 shown.

[0069] Some defect categories are rare and therefore present in very small numbers. Using all categories would result in a highly unbalanced dataset and be meaningless. Therefore, this paper removes some of these rare defect categories, retaining only five common defect categories: finger interruptions (1901 images), cracks (1279 images), thick lines (1031 images), misalignments (1279 images), and black cores (777 images).

[0070] Data augmentation

[0071] The effectiveness of neural network models is strongly correlated with the quantity and quality of datasets. During network training, the model's weights tend to favor targets with a larger number of categories. Without proper data processing, the data can easily overfit or underfit, making it difficult for the model to converge. The categories in this data are unbalanced, and the quantity doesn't meet the requirements for model training. Therefore, this paper leverages existing technologies to enhance the data.

[0072] First, we flip, rotate, and crop the image, preserving its linearity and relative position while changing its size, orientation, and location. We also triple the number of image samples through affine transformations and enrich the spatial variation of the samples.

[0073] Secondly, we perform mosaic data augmentation on the data. We remove a batch of data from the total dataset, randomly select four images each time, crop and splice them at random positions, and synthesize a new image. This is repeated batch size times, resulting in a batch size of new data after mosaic data augmentation. This is then passed to the neural network for training. This increases data diversity and the number of objects, effectively increasing the batch size. This improves the mean and variance during batch normalization, improving data quality for defects such as severed fingers (multiple small objects in a single image).

[0074] Then, we use Gaussian blur to enhance the images showing dendrites and deep cracks to obtain new images of the same label with different information. The details of the dark image are reduced, providing different effective information. Then, we use contrast enhancement on the crack image to make the model more tolerant to lighting conditions. It is worth noting that the number of images for each class is kept equal to help the stability of training. The data distribution after enhancement is as follows: Figure 3 shown. Figure 3 Chinese: Bionics Attention (mimetic attention mechanism), SC-Conv (self-calibrating convolution), SA-Block (bionic feature extraction module), CBL (representative convolution, normalization, activation function), Focus (slicing), SIF (semantic feature aggregation module), PIF (spatial feature aggregation module), Carafe (lightweight upsampling operator), Adaptive Scale adjustment (adaptive scale monitoring), C3 (convolution bottleneck module), Decoupled Head (separate detection head), DWConv (depth separable convolution), Concat (tensor splicing), BottleNeck (bottleneck layer), BN (batch normalization), Leaky ReLU (activation function), SiLU (activation function).

[0075] Bionic model for photovoltaic cell defect detection based on mimic vision

[0076] The structure of the bionic model for photovoltaic cell defect detection based on mimic vision is as follows: Figure 4 As shown, it mainly contains three parts:

[0077] (1) Backbone network: The backbone network is used to extract effective information from the image and map the original data to the required space. The features extracted by the backbone network will be passed to the neck network. The backbone network consists of four modules, namely the convolution downsampling module for reducing the image size and expanding the number of channels, the self-calibration convolution module, the mimetic visual attention mechanism module and the bionic feature extraction module.

[0078] (2) Neck Network: The neck network is used to fuse features of different scales and levels and enrich the expressive power of features. The features fused by the neck network are then transferred to the detection head. The neck network consists of a lightweight upsampling operator, a semantic feature transfer module, a spatial feature transfer module, and a C3 module.

[0079] (3) Detection Head: The detection head is responsible for predicting features, generating bounding boxes, and predicting categories. The detection head consists of an adaptive scale monitoring module, an adaptive scale adjustment module, and a separate detection head.

[0080] The processing flow of the model is;

[0081] First, the image of the photovoltaic cell to be inspected is input into the backbone network to extract the defect features in the image.

[0082] The different-scale features extracted by the backbone network are then fed into the model's neck network. In the bottom-up path, the neck network transmits precise positional signals from the underlying features to the top layer via the position information transfer module. In the top-down path, the neck network transmits rich semantic information from the top layer to the bottom layer via the semantic information transfer module.

[0083] Finally, the fused features are adaptively weighted and fused, and then input into the separate detection head. The detection head calculates the above information to obtain the location and category information of the defect.

[0084] A. Bio-inspired biomimetic backbone network

[0085] The bionic backbone network consists of a stacked bionic feature extraction module and a mimic visual attention mechanism module, divided into four stages. In each stage, the image is first input to the bionic feature extraction module, where it is downsampled using convolution with scaling factors of [1 / 4, 1 / 8, 1 / 16, 1 / 32] and the number of channels of the three-channel RGB image is expanded to [64, 128, 256, 512] to increase information capacity. Next, a spatial aggregation feature extraction module (SA-Block) that mimics biological vision is used to extract features. Finally, a self-calibrated convolution module (SC-Conv) is used to aggregate target feature information across channels. In the final two stages, a mimic visual attention module is added to capture rich dynamic context and mimic the unique human peripheral vision mechanism.

[0086] a. Bionic feature extraction module

[0087] The bionic feature extraction module mainly consists of a spatial aggregation feature extraction module (SA-Block) and a self-calibration convolution module (SC-Conv).

[0088] (1) Spatial Aggregation Feature Extraction Module (SA-Block)

[0089] The spatial aggregation feature extraction module is placed in each stage of the backbone network to extract the features of the defects. Feature integration theory shows that human vision perceives targets by extracting basic contextual features and associating individual features with attention. Traditional convolutional neural networks tend to perform low-order or high-order information interactions, leaving out the richest mid-order interactions, which conflicts with human visual patterns. Therefore, it is particularly important to reasonably capture multi-order display interactions in local contexts for the extraction of effective features. To this end, the present invention uses a spatial aggregation feature extraction module (SA-Block) in the backbone network to capture multi-order contextual information, such as Figure 5 As shown, Figure 5 Middle: Norm (normalization), GAP (global average pooling), Subtract (tensor subtraction), GELU (activation function), SILU (activation function), Split (tensor segmentation), Concatanate (tensor concatenation), Conv (convolution), DWConv (depth-wise separable convolution), Dialation (dilation rate). This block consists of two cascaded components:

[0090] z=x+Moga(FD(Norm(x)))

[0091] Among them, FD() is a low-level information suppression module, and Moga() is a gated context information aggregation module to extract multi-level features with static and adaptive region awareness. FD() is expressed as:

[0092]

[0093] Z=GeLU(Y+γ s (Y-AugPool(Y)))

[0094] in is a depthwise separable convolution, k is the convolution kernel size, n represents the nth group of convolutions, FD() models the 0th and 1st order interaction information through convolution with a kernel size of 1 and global average pooling Augpool(). And because features such as cracks in photovoltaic cells are easily lost as low-order information such as texture, this low-order interaction information needs to be gated by the parameter γ s Suppress or retain.

[0095] Then, the multi-order features are modeled in the context branch of Moga(), and depth-wise separable convolutions with different dilation rates are used to capture low-, medium-, and high-order interactions. Given an input feature x∈R bs,c,h,w, first use convolution with a dilation rate of 1 and a convolution kernel size of 5 to capture low-order features, and then split the input feature x along the channel dimension:

[0096]

[0097] in Represents concatenating matrices along the channel dimension.

[0098] Finally, the features of the branches are aggregated by the SiLU activation function with a gating effect to obtain the output Z∈R of the spatial aggregation feature extraction module. bs,c,h,w :

[0099] Z=SiLU(f k＝1 (x))⊙SiLU(f1(Y C ))

[0100] SiLU=x·σ(x)

[0101] Where σ() represents the sigmoid activation function and ⊙ represents the Hadamard product.

[0102] (2) Self-calibrated convolution module (SC-Conv)

[0103] The self-calibrated convolution module (SC-Conv) is placed after the spatial aggregation feature extraction module. It expands the field of view of each convolution layer through internal communication, enriches the output characteristics, and adaptively establishes long-range spatial and channel dependencies around each spatial position, which can explicitly merge richer feature information. The specific structure is as follows: Figure 6 As shown, Figure 6 Middle: Up (up sampling), Down (down sampling), Sigmoid (activation function),

[0104] First, given the output x∈R bs,c,h,w , divide x into two parts

[0105] Then x1 and x2 are input into different branches respectively, [u2,u3,u4] is used to collect different types of information y1, and the output is obtained after a self-calibration operation in the first branch. The purpose of the self-calibration operation is to effectively collect information at each spatial position. The method is to perform convolutional feature transformation in two different scale spaces, obtain the latent space by downsampling the input of the original resolution, and use the latent space as a reference after feature mapping to guide the feature transformation in the original resolution feature space.

[0106] The specific approach is as follows: given an input x1, use average pooling with a pooling kernel size and a step size of r: Then perform feature mapping and upsampling operations on T1: Where Up() is a bilinear interpolation operator that maps the latent space from the reduced scale to the original resolution. Then the calibration operation is performed: Use x′1 as the residual to form the calibration weight, and finally perform the calibrated convolution mapping operation to obtain the final output of the first branch In the second branch, a simple convolution operation is performed to obtain the output Its goal is to preserve the original spatial information. Then [y1,y2] are connected together to get the output of the module

[0107] b. Mimic visual attention mechanism module

[0108] Traditional attention mechanisms achieve self-attention calculations by applying them to features through fully connected layers, without considering the surrounding context of pixels, resulting in isolated operations between queries and keys. Furthermore, global attention mechanisms are not compatible with the human visual system; noise and redundant features in global information can easily interfere with the model's final detection results. Finally, compared to fully convolutional modules, attention mechanisms implemented through full connectivity lose the advantageous properties of convolution, such as local connections and translational variability.

[0109] Therefore, from the perspective of bionics, this paper proposes a mimetic visual attention mechanism module implemented by full convolution based on the idea of ​​attention mechanism and the human peripheral vision system, which improves the interaction mode of query, key, and value in the attention mechanism, and improves performance and biological interpretability.

[0110] The mimic visual attention mechanism module contains two submodules: deep mining attention module and peripheral vision module (see Figure 7 ), Figure 7 : input, Output, RBPjeck (representing the channel mixing operation of features through two deconvolutions with a kernel size of 1), RBPjeck&softmax (representing the channel mixing operation of features through two deconvolutions with a kernel size of 1, and then normalizing the tensor through softmax), Matmul (matrix multiplication), Sum&mean&sofmax (representing summation, averaging, and normalization operations), Peripheral Initialization (initialization of peripheral vision), Peripheral projection (peripheral projection), In&GeLU (instantiation and activation function operations). The deep mining attention module deeply mines the semantic information of features through the multi-interactive self-attention mechanism of full convolution. The peripheral vision module is responsible for forcing the deep mining attention module to learn the human peripheral vision mechanism through position encoding, and outputs a position encoding wp ,The outputs of the two modules are mixed through Hadamard product and linear projection to obtain the final output result of the module.

[0111] (1) Deeply explore the attention submodule

[0112] In the deep mining attention submodule, the main calculations are query, value, and key. Given an input x∈R b×c×h×w , respectively obtained through convolution mapping:

[0113] query=g n k＝1 (x), key = g n k＝3 (x), value = g n k＝1 (x)

[0114] where g n k () is a ghost convolution, k is the size of the convolution kernel, and n represents the nth group of convolutions. The query and key are calculated using a ghost convolution with a kernel size of 1, replacing the fully connected layer. This suppresses model overfitting without destroying the spatial structure of the image, reducing parameters and computation at a low cost. The key is calculated using a convolution with a kernel size of 3, extracting and aggregating contextual information around the key. In traditional attention mechanisms, query, key, and value are the result of convolution mapping of input features. This mapping process may lose some valid information, so this invention designs a new interaction method without increasing the number of parameters to ensure that the image does not lose texture details and fine-grained features as much as possible.

[0115] First, concatenate the input x and key, and perform two ghost convolutions through the anti-bottleneck structure to fully interact with the original features and key channels:

[0116]

[0117] Where σ is the Softmax activation function, which is used to compensate for nonlinear expression capabilities, g n k,d (), g n k,d () are two sets of ghost convolutions, d is the number of channels of the convolution output, and 1×1 convolution is used to map the same feature several times the number of channels. Through channel expansion, an effect similar to the multi-head attention mechanism is achieved, and finally the key after context information aggregation is obtained:

[0118] Then calculate the value matrix, concatenate the key and value, and use the anti-bottleneck layer to interact and aggregate the information of the two to get the output

[0119]

[0120] After the key and value are concatenated, the channel dimension summation and global average pooling operation are performed to obtain the spatial weight matrix w weight :

[0121]

[0122] Through convolution mapping and multiple matrix multiplications, qkv and input x are interacted more flexibly and deeply, which fully aggregates contextual information without introducing too much computation and parameters, and compensates for some texture information lost in the traditional attention mechanism.

[0123] Finally, w k2 ,w q Perform matrix multiplication to get the result of qk interaction

[0124] (2) Peripheral Vision Submodule

[0125] The interaction mode of the deep mining attention module belongs to the global attention mechanism, and the receptive field is redundant, resulting in the model's weak perception and focusing ability on some details, and unable to control the interaction between query and key. In order to enable deeper and more effective interaction between query and key, the present invention introduces a peripheral vision mechanism into the module, which limits the interaction strength between query and key through Euclidean distance, allowing the network to learn to divide the field of view into different peripheral areas based on the given training data, forcing the model to learn this unique human visual pattern.

[0126] The human visual field is divided into several regions based on the Euclidean distance from the center of gaze: central, relatively central, middle, and peripheral regions, each of which captures different visual information; the closer to the gaze, the more complex the features processed, and the farther from the gaze, the simpler the perceived visual features.

[0127] To mimic the aforementioned mechanism, we choose to use the query location as the gaze center. The query's local area is divided into four sub-regions: center, quasi-center, periphery, and far periphery. The query no longer interacts with all keys globally, but rather with keys within four sub-regions. The interaction strength is limited by varying Euclidean distances. Specifically, we introduce peripheral vision projection:

[0128]

[0129] in The set of normalized weights of the peripheral vision representing the interaction strength between all queries and keys, h is the number of heads of the multi-head attention mechanism, and the input R is defined as Among them D r Represents D r The Euclidean distance between the query and the key, ω r is a set of learnable parameters shared across layers and heads, Represents the Euclidean distance between the query position q and the key position k. Re LU(·) and σ(·) are nonlinear activation functions, the latter of which is used to normalize the weights. and is the parameter of linear projection. The specific projection method is to obtain a non-Gaussian attention map through a multi-layer perceptron and use a convolution with a convolution kernel size of K to perform a local K×K area. By performing linear projection on the image, the resolution of the transformation parameters is reduced by aggregating local information, so that the model can adaptively capture diverse peripheral visual attention maps according to the input image.

[0130] After the above calculations, a layer of instance normalization is needed to stabilize the training process:

[0131]

[0132]

[0133] in is the instance normalization of weights and biases, and MAP(·) is the peripheral vision projection.

[0134] In order to facilitate the training of the network, it is necessary to inject the peripheral vision attribute at the beginning of the training phase, so it is necessary to initialize δ p The Euclidean distance is used to limit the interaction strength between the query and the key. The farther the distance, the lower the attention weight, similar to the characteristics of peripheral vision. Specifically, for all layers l∈[N l ] and headh∈[N h ]have:

[0135] Where c1, c2 are positive real numbers J N,M ∈R N×M Refers to an all-one matrix of size N×M. The above initialization can effectively provide local attention, that is, given Deviation in the second instance norm and weights We control the size and strength of local attention respectively by setting their initial values ​​to Initialization of simulated peripheral vision, where s l and v l is the set of initial values ​​of attention size and intensity, s l ∈[-5.0,4,0],v l ∈[3.0,0.01], where s l-1 <s l And v l-1 >v l , and set c1, c2 = 0.02.

[0136] Finally, the outputs of the peripheral vision submodule and the deep mining attention submodule are mixed through matrix multiplication to obtain the mimic visual attention mechanism, which is specifically defined as:

[0137]

[0138] The outputs of the two submodules are mixed through matrix multiplication and Haffmar product.

[0139] Separate feature fusion pyramid

[0140] Among the defect features of photovoltaic cells, cracks and other features have large scale variations, and the feature expressions at different levels contain different meanings. The feature pyramid is used to enrich the representation capabilities of the above features. The separated spatial semantic fusion pyramid of the present invention refers to the information transmission method of the human cerebral cortex and designs two feature transmission and fusion modules according to the different requirements of information flow: semantic information transmission module and position information transmission module. The structure of the separated feature fusion pyramid is as follows: Figure 8 As shown in the figure (for the convenience of expression, the convolution blocks, upsampling operators and C3 modules used to align the scale and number of channels are omitted in the figure), it can fuse the features of different scales and numbers of channels in the three stages after the backbone network. It uses the semantic information transfer module (SIF) in the top-down path to transfer semantic information, and uses the position information transfer module (PIF) with a bottleneck structure in the bottom-up path to transfer the precise position signal to the top layer. Figure 8 Chinese: SIF (Semantic information fusion) stands for semantic information fusion, Local Information fusion (PIF) stands for spatial information fusion.

[0141] (1) Semantic Information Transfer Module (SIF)

[0142] The semantic information transfer module is added between every two layers in the top-down channel of the feature pyramid to fuse feature layers of different scales and levels and transfer effective semantic information of the target. Figure 9 , Figure 9Chinese: CARAFE (lightweight upsampling operator), Global avg.pool (global average pooling), Matmul (matrix multiplication), Add (matrix bitwise addition), Concat (matrix concatenation).

[0143] The specific steps are as follows:

[0144]

[0145] The feature layer of the i-th layer is represented as First, global average pooling is applied to adjacent layers, and carafe() is an upsampling operator to obtain the channel vector of each channel. then:

[0146]

[0147] Channel attention is obtained through outer product and normalization operations Each row in M ​​represents the importance distribution of the jth channel on the i+1th layer relative to the i-th layer. This attention map is then applied to the feature map obtained after upsampling on the i+1th layer to obtain the reweighted version Y of the i+1th layer. i+1 .

[0148]

[0149] Finally, by adding Y i+1 Add as residual to On, with A i After splicing, the convolution kernel is 1 and the channel mixing is performed, where λ i is the gating parameter.

[0150] (2) Spatial Information Transfer Module (PIF)

[0151] The spatial information transfer module is added between adjacent feature layers in the bottom-up path to transfer the precise location signal to the top layer (such as Figure 10 ), the specific steps are as follows:

[0152]

[0153] First, the adjacent feature layer A i ,A i-1 To perform convolution operation, in order to control the computational complexity of the module, the convolution adopts a bottleneck structure, so the step size of the convolution kernel is selected as a, and the output dimension is 1, so Transpose and inner product it to get M′, then:

[0154]

[0155] Normalize M′ to get the spatial attention map, and i Similarly, perform convolution with a step size of a to obtain A′ i Operation alignment scale, A i Then perform convolution with a step size of 1 to get A i A downsampled version of .

[0156]

[0157] Then, spatial attention is applied to A′ i After upsampling, the upsampling operator is applied to the scale, and the upsampling result is used as the residual and gated with weight α i Add to A i A downsampled version of i Finally, the output of the previous step is combined with A i-1 After splicing, the channel mixing operation is performed through convolution with a convolution kernel size of 1 to obtain the final output.

[0158] Separate detection head with adaptive fusion

[0159] The scale of photovoltaic cell defects varies greatly. Defects such as black cores usually occupy the entire image, while defects such as broken fingers and cracks are very small. This leads to large differences in the impact of features of different scales on the detection results. Therefore, the present invention designs a separate detection head (such as Figure 11 ), all aiming to adapt the model to tasks with widely varying scales and detection difficulty. First, pyramid pooling is used to aggregate global information from feature layers at three different scales. This is used to learn and monitor the importance of feature layers at different scales and adaptively weight them. Subsequently, features from the three scales are aligned and fused to enhance their expressiveness. Finally, the features are input into the decoupled detection head, where they are used to predict defect category and location information within different branches.

[0160] (1) Adaptive scale monitoring

[0161] For photovoltaic cell defect detection, due to the difference in scale, features of different scales should have different importance. Therefore, the present invention introduces an adaptive scale adjustment factor into the model, uses global average pooling to aggregate the global information of features of different scales, and adaptively adjusts the feature information flow of different scales (such as Figure 12 ), Figure 12 Middle: GAP (global average pooling),

[0162] The scale adjustment factor η can be specifically expressed as:

[0163]

[0164] where gi () is the global average pooling, i is the pooling kernel size, c dim / dim′ is a fully connected mapping. After obtaining the scale adjustment factor, the output O is:

[0165]

[0166] (2) Adaptive scale fusion

[0167] The adaptive fusion module adaptively learns the fusion spatial weights of each scale feature map. This is done in two steps: constant scaling and adaptive fusion. It specifically operates on the three scaled features output by Neck, which are the features that were originally input to the three detection heads, performing constant scaling and adaptive fusion. The specific steps are as follows:

[0168] a. Identical scaling: Features of different scales output by Neck need to be aligned, and features of different scales are represented as x t (t∈[1,2,3]), t=1 is the minimum scale feature layer, and vice versa. t , it is necessary to adjust the features of adjacent scales to the same size. For upsampling, the resolution is increased by convolution with a stride of 1 and a kernel size of 1 and bilinear interpolation:

[0169]

[0170] For downsampling with a scaling factor of 2, a convolutional layer with a stride of 2 and a kernel size of 3 is used to reduce the resolution:

[0171]

[0172] For downsampling with a scaling factor of 4, a max pooling layer with a pooling kernel size of 2 is added before the convolutional layer.

[0173]

[0174] b. Adaptive fusion: Indicates that the t-layer features are fused to the l-layer, and (i, j) represents the position of the feature vector. : The vector at the spatial position (i, j) after fusion is a weighted fusion of the vectors at (i, j) of the three feature maps before fusion. Specifically, these features are fused on the corresponding level l feature layer:

[0175]

[0176] in Represents the features output after fusion, which will be input into the detection head. is the spatial importance weight of the feature maps at three different levels, which is adaptively learned by the network and shared across all channels. With constraints and and define The feature layers of the three scales are calculated using a convolution layer with a convolution kernel of 1, and the results are normalized by the softmax function to meet the constraints:

[0177]

[0178] Finally, the adaptively fused features are input into the head for prediction.

[0179] (3) Separate detection head

[0180] Since classification and positioning focus on different points, classification focuses more on which category the extracted features are closest to the existing categories, that is, the texture content and semantic information of the target, while positioning focuses more on the position coordinates of the target detection box to correct the bounding box parameters, that is, the edge information of the target. Therefore, the present invention introduces a separate detection head (such as Figure 13 , Class (category), IoU (confidence), Boxes (detection box), referring to the partitioning mechanism of the human brain, it is intended to make the detection head focus on different content through different branches. The three detection heads process different scale features output by the neck (such as Figure 4 、 13 ), output the final result. The specific steps are as follows:

[0181] For the features output by the neck, two convolutions are performed in each of the three branches. The final convolution with a kernel of 1 is used to compress the number of channels, acting as a replacement for the fully connected layer. The channels are compressed to [num, 1, 4], respectively. The branch with n channels outputs the category, where n is the number of categories. The branch with 1 output channel outputs the IoU threshold, and the branch with 4 output channels outputs the position and size of the object detection box [x, y, h, w]. This is expressed as:

[0182]

[0183]

[0184]

[0185]

[0186] experiment

[0187] (1) Experimental setup

[0188] To fully exploit the model's performance, this paper employed a transfer learning strategy. First, the model was pre-trained on the large COCO2017 dataset, creating a pre-trained model. Subsequently, 80% of the photovoltaic cell defect dataset was selected as the training set and 20% as the test set. The original data was randomly partitioned using stratified sampling, preserving the sample distribution of samples within different categories in the training and test sets. The pre-trained model was then loaded and used for model training.

[0189] The initial learning rate of the experimental setting is 0.01, the SGD optimizer is used, the batch size is 32, and a total of 300 rounds of training are carried out. L2 regularization is introduced to alleviate overfitting.

[0190] The experiments were run on a Windows 10 computer with an AMD 5600X GPU, two RTX8000 GPUs, and 32G of memory, using the Pycharm platform.

[0191] (2) Evaluation indicators

[0192] The present invention uses two commonly used indicators to evaluate the performance of the model, namely average precision AP and recall R. A higher evaluation value indicates a higher performance, and vice versa.

[0193]

[0194]

[0195]

[0196]

[0197] Target detection models produce four types of detection results: true positive (TP), true negative (TN), false positive (FP), and false negative (FN). Precision is used to evaluate the accuracy of a detection model. Recall is used to assess the detection model's ability to identify all positive samples. mAP is the average precision across all categories. A higher mAP value indicates better overall model performance, where n is the number of categories and n is the total number of categories. Average precision (AP) is used to evaluate the accuracy of a detection model for a specific target type, where p is precision and r is recall.

[0198] (3) Model performance verification

[0199] In order to verify the idea of ​​the present invention, the performance of the model was verified using the test set, and the results were analyzed and evaluated. Figure 14The defect characteristics of black cores and faults are very clear and simple, achieving a nearly 100% detection success rate, with no false positives or missed detections for these two types of defects. For smaller defects such as broken fingers and cracks, the model effectively detected the vast majority of defect targets with relatively high accuracy, demonstrating that the improvements presented in this paper have effectively enhanced the model's detection capabilities for small, less distinct targets, essentially meeting the requirements of industrial applications.

[0200] The training loss curve of the model during training is as follows Figure 15 As shown in the figure, the experimental results show that the loss of the improved model steadily decreases. In the first 20 epochs, the loss drops rapidly, and the accuracy improves rapidly. After 20 epochs of training, the rate of loss decreases gradually, but it still decreases significantly. After 40 epochs of training, the loss decreases very slowly, with slight fluctuations within a certain range, but the overall downward trend remains. This trend continues until 200 epochs of training, when it stabilizes. Finally, after 250 epochs, the loss decrease gradually stabilizes, and the loss curve is almost flat, indicating that the loss has reached its minimum value and the model has basically converged and performed well.

[0201] The trend of the test set loss curve of the model is basically the same as the training loss, but there are also slight differences. Figure 15 It can be seen that the model's test loss, especially the category loss, experienced significant fluctuations early on. Analysis revealed that this was due to the overly pronounced defects of black cores and faults in the detection categories, coupled with overly simple texture structures, affecting the test set loss. In the few rounds where the fluctuations subsided, the test set loss trended roughly parallel to that of the training set, declining rapidly in the first 20 rounds and slowing after 40 rounds, though with some fluctuation. After 100 rounds of training, the training loss essentially stabilized, fluctuating only within a very small range. This indicates that the model has reached its performance ceiling.

[0202] The accuracy change curve of the model is as follows Figure 15As shown, unlike the training and test losses, the model's accuracy fluctuates significantly. During the first 20 epochs, the model's accuracy fluctuates dramatically, and the AP curve exhibits an interesting phenomenon early on. During the early stages of training, AP improves significantly, even reaching around 80%, but the recall and mAP are low. This is because black core and fault detection are too simple, while other defect types are too difficult to detect. This directly leads to a significant imbalance in the model's capabilities in the early stages. In other words, the model can detect the vast majority of black cores and faults with relatively high accuracy, but it is largely unable to detect other defects. This is the fundamental reason for the sharp fluctuations in the model's accuracy curve in the early stages. From epochs 20 to 100 during training, the rate of increase in the model's precision and recall gradually slows, and the fluctuations gradually decrease, but they still fluctuate significantly. After 100 epochs of training, the model's precision and recall barely improve, but remain unstable. After 200 epochs of training, the precision and recall gradually stabilize, and the curves become essentially flat, indicating that the model has converged.

[0203] (4) Ablation experiment of the model

[0204] This section quantitatively analyzes the effectiveness of each improvement, analyzing the contribution of different improvements to model performance through ablation experiments. To control for variation, we divide the ablation experiments into three parts: the backbone, the neck, and the head. We then analyze how improvements in each part contribute to the overall performance of the model.

[0205] a. Backbone network ablation experiment

[0206]

[0207] Table 1 Ablation comparison experimental results of backbone network

[0208] As can be seen from Table 1, the performance of the backbone network proposed in this paper is better than the current mainstream model on this task. And each improvement of the backbone network can bring effective performance improvement to the model.

[0209] The present invention visualizes a three-dimensional image of the peripheral vision interaction intensity matrix, wherein the results tend to be more like Figure 16 The larger the three-dimensional image, the more human-like the module's interaction method is. The plane of the x and y axes is a k×k matrix, k=h s ×w s where h s ,w sis the feature map size of the feature layer where the mimic visual attention mechanism module resides, z∈[0,1] is the weight of each point in the matrix, and the weight of the ρth row and γth column in the matrix represents the interaction strength between the query at the ρth pixel and the key at the γth pixel. Converting the above matrix into a three-dimensional interaction strength image resembles a valley, where the diagonal line represents the interaction strength between each query and its own position. The farther away from the diagonal line, the lower the correlation between the query and the key. There are many cliff-like weight drops in the valley. This is because the key gradually exceeds the peripheral field of view of the query, and the interaction strength decreases precipitously.

[0210] b. Neck network ablation experiment

[0211]

[0212] Table 2 Ablation comparison experiment of neck network

[0213] The information transmitted by the spatial information transmission module and the semantic transmission module is as follows Figure 17 As shown in the figure, it is clear that the semantic information, that is, the feature map on the left, contains more texture and detail information of the target. In other words, the target category information can be identified through these features. Similarly, the feature map on the right is the position information. It can be seen that the position information contains the target center point position and target size, and basically loses the texture and semantic details. Therefore, the target category information cannot be calculated based on this part of the features.

[0214] In summary, the semantic transfer module and the spatial transfer module successfully separate the information flow fused by the neck network and select different information transfer methods according to the requirements of different information flows. As shown in Table 2, this structure effectively improves the performance of the model.

[0215] c. Ablation experiment of the detection head and its parts

[0216]

[0217] Table 3 Ablation experiments of various parts of the model

[0218] As shown in Table 3, the separate detection head effectively decouples the classification and positioning tasks, and adaptively integrates and monitors multi-scale features, effectively improving the detection capability of small targets.

[0219] (5) Model comparison experiment

[0220]

[0221] Table 4 Comparative test between the model and mainstream models

[0222] To evaluate the advantages of our proposed model, we compared its detection performance with six currently popular object detection models: DETR, Deformable-DETR, YOLOv3, YOLOv4, YOLOv5, and PicoDet. We trained all seven models on the same dataset and evaluated their performance on the same test set.

[0223] Under the same conditions, each detection model has different degrees of missed detection. The model of the present invention misses the least number of targets and has better detection effect when facing small targets. Data shows that the evaluation indicators AP and Recall of the model of the present invention are better than those of several other detection models, indicating that the model has better performance. For industrial defect detection, the recall rate indicator is more important. Compared with: DETR, Deformable-DETR, YOLOv3, YOLOv4, YOLOv5, and PicoDet, the model of the present invention has achieved a greater advantage in recall rate, proving that the model has a low missed detection rate for targets. This is because different detection models use different feature extraction networks and fusion methods. Therefore, the information of key targets learned by different networks is also different. The model of the present invention has biological characteristics and has the human peripheral visual receptive field and stronger feature fusion ability, which is also the main reason for its performance advantage. At the same time, the image sizes input to the training network of each model are different, and the environment configuration of the detection examples and model training in the present invention is relatively high. The indicators are for reference only.

[0224] The key points of the present invention are as follows:

[0225] 1) Based on biological inspiration, high-level information with biological characteristics is extracted from three perspectives: human visual receptive field, peripheral visual pattern, and human eye focusing method. This invention imitates the biological receptive field through efficient spatial aggregation blocks, learns semantic features and multi-order interactions, and uses self-calibrated convolution to guide which area of ​​the display space features should be mapped to, similar to the focus of the human eye. Next, a mimetic visual attention mechanism is proposed to fully aggregate dynamic and static contextual information, control the interaction of information in different visual areas, and imitate the human peripheral visual pattern.

[0226] 2) Based on the way the human brain transmits information, a separated spatial semantic fusion is designed. Two lightweight modules are used to transmit position and semantic information according to the needs of different information flows. This enhances the expressiveness of multi-scale features, enriches the information contained in features at different levels, and improves the detection effect of small targets.

[0227] 3) Adaptively fuse multi-scale features and weight features at multiple levels by extracting global information. Furthermore, inspired by how the human cortex processes information, and considering the different focuses of classification and localization tasks, the algorithm decouples localization from classification tasks and employs different branches for regression calculation, improving both detection accuracy and recall.

[0228] The above description is only a specific embodiment of the present invention, and the various examples do not limit the essential content of the present invention.

Claims

1. A bionic model for photovoltaic cell defect detection based on mimicry vision, characterized by It consists of three parts: (1) Backbone network: The backbone network is used to extract effective information from the image and map the original data into the required space. The features extracted by the backbone network are passed to the neck network; (2) Neck network: The neck network is used to fuse features of different scales and levels and enrich the expressiveness of features. The features fused by the neck network are passed to the detection head; (3) Detection head: The detection head is responsible for predicting features, generating bounding boxes and predicting categories. The detection head introduces a separate detection head, referring to the partitioning mechanism of the human brain, in order to make the detection head focus on different content through different branches. The three detection heads process the different scale features output by the neck respectively and output the final results. The specific approach is as follows: For the features output by the neck, two convolution mappings are used in the three branches, where the final convolution kernel is 1 and the number of channels is compressed to [num, 1, 4], where the branch with n channels outputs the category, where n is the number of categories. The branch with 1 output channel outputs the IoU threshold, and the branch with 4 output channels outputs the position and size [x, y, h, w] of the target detection box. Expressed as: The processing flow of the model is: First, the image of the photovoltaic cell to be inspected is input into the backbone network to extract the defect features in the image; Subsequently, the different scale features extracted by the backbone network will be sent to the neck network of the model; in the bottom-up path, the neck network transmits the precise position signal of the bottom-level features to the top layer through the position information transmission module, and in the top-down path, the semantic information transmission module transmits the rich semantic information of the top layer to the bottom layer; the backbone network is composed of a stack of bionic feature extraction modules and mimetic attention mechanism modules, which is divided into four stages; the bionic feature extraction module is composed of a spatial aggregation feature extraction module SA-Block and a self-calibration convolution module SC-Conv; the mimetic visual attention mechanism module contains two sub-modules: a deep mining attention module and a mimetic vision module; in In each stage, the image is first output to the biomimetic feature extraction module, which downsamples the image through convolution with scale factors of [1 / 4, 1 / 8, 1 / 16, 1 / 32] and expands the number of channels of the three-channel RGB image to [64, 128, 256, 512] to increase information capacity. Next, the spatial aggregation feature extraction module SA-Block, which mimics biological vision, extracts features. Finally, the self-calibrating convolution module SC-Conv aggregates target feature information across channels. In the last two stages, a mimic visual attention module is added to capture rich dynamic context and imitate the unique peripheral vision mechanism of humans. Finally, the fused features are adaptively weighted and fused, and then input into the separate detection head. The detection head calculates the above information to obtain the location and category information of the defect.

2. The bionic model for photovoltaic cell defect detection based on mimicry vision according to claim 1 is characterized in that The semantic information transfer module is added between every two layers in the top-down channel of the feature pyramid to fuse feature layers of different scales and levels and transfer effective semantic information of the target. The specific approach is as follows: The feature layer of the i-th layer is represented as First, global average pooling is applied to adjacent layers, and carafe() is an upsampling operator to obtain the channel vector of each channel. then: Channel attention is obtained through outer product and normalization operations Each row in M ​​represents the importance distribution of the jth channel on the i+1th layer relative to the i-th layer. This attention map is then applied to the feature map obtained after upsampling on the i+1th layer to obtain the reweighted version Y of the i+1th layer. i+1 ; Finally, by adding Y i+1 Add as residual to On, with A i After splicing, the convolution kernel is 1 and the channel mixing is performed, where λ i is the gating parameter.

3. The bionic model for photovoltaic cell defect detection based on mimicry vision according to claim 2 is characterized in that The spatial information transfer module is added between adjacent feature layers in the bottom-up path to pass accurate position signals to the top layer. The specific approach is as follows: First, the adjacent feature layer A i ,A i-1 To perform convolution operation, in order to control the computational complexity of the module, the convolution adopts a bottleneck structure, so the step size of the convolution kernel is selected as a, and the output dimension is 1, so Transpose and inner product it to get M′, then: Normalize M′ to get the spatial attention map, and i Similarly, perform convolution with a step size of a to obtain A i ′Operation alignment scale, A i Then perform convolution with a step size of 1 to get A i A downsampled version of Then, spatial attention is applied to A i ′After applying the upsampling operator to it, the upsampling result is used as the residual and gated with weight α i Add to A i A downsampled version of i Finally, the output of the previous step is combined with A i-1 After splicing, the channel mixing operation is performed through convolution with a convolution kernel size of 1 to obtain the final output.

Images